突发奇想：如果舍弃网络接受部分的软中断处理？

nhuczp

多核CPU并不能真正并行，看INTEL下一代能不能做到独占式。

dreamice

原帖由 nhuczp 于 2009-2-18 15:37 发表
多核CPU并不能真正并行，看INTEL下一代能不能做到独占式。

此言差矣！可能你针对的是Intel的CPU吧，如果现今的网络处理器不能并行，那性能如何能上得去？

platinum

期待成果！

dreamice

原帖由 platinum 于 2009-2-18 21:30 发表
期待成果！

这个东西，看来感兴趣的人很多啊。

skybyte

问下这里的高手呢，我在网络技术版问过了，在这贴里面再问下：Ｄ



双光纤1g的网关服务器 流量上了600M 就扛不住了掉包严重,该怎么优化
我在论坛上问了一，有的人说要升级网卡驱动，不过现在用的是　OPENSUSE 10.3 bigsmp内核，没提供kernel-headers
bigsmp的包　结果编译不起新的驱动

有一个人回复
--------------------------
可以尝试：
1）用最新的驱动
2）确保NAPI开启
3）增大RX Ring size

另外，研究一下ethtool的输出中，这几个不太好的数字是怎么产生的。
rx_no_buffer_count: 170718107
rx_missed_errors: 83998346
tx_restart_queue: 290668
rx_csum_offload_errors: 290884
-------------------
RX Ring size 这个我增加了，　没有变化　NAPI我不知道怎么开启　是不是  modprobe ioatdma &&
ifcnfig eth2 down && ufconfig eth2 up　这样就可以了，我执行了，没变化
所以来请教一下您。


详细情况如下

===================================
cat /proc/sys/net/netfilter/nf_conntrack_count
184337

ping 掉包严重超过20%
日志出现很多下面的信息

avahi-daemon[5619]: wide-area.c: Ignoring invalid response for wide
area datagram.

syslog-ng[5410]: last message repeated 5 times

avahi-daemon[5619]: wide-area.c: Ignoring invalid response for wide
area datagram.

kernel: printk: 11 messages suppressed.

kernel: UDP: bad checksum. From xxx.xxx.xxx.xxx:18369 to
xxx.xxx.xxx.xxx:64771 ulen
1069

avahi-daemon[5619]: wide-area.c: Ignoring invalid response for wide
area datagram.



snat代码是
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
SNAT all -- anywhere anywhere
toxx.xxx.xxx.1-xxx.xxx.xxx.254



服务器是两个4核 2G的CPU 4G内存
系统是 Linux linux-NAT 2.6.22.19-0.3-bigsmp #1 SMP 2009-05-27 10:35:34
+0200 i686 i686
i386 GNU/Linux
内存还很空
# free
total used free shared buffers cached
Mem: 4148208 2049220 2098988 0 126008 1528504
-/+ buffers/cache: 394708 3753500
Swap: 2103480 0 2103480

vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
3 0 0 2099764 126088 1528516 0 0 3 17 36 58 11 17 72 1
5 0 0 2098576 126088 1528516 0 0 0 0 0 33710 16 30 53 0
4 0 0 2099448 126088 1528516 0 0 0 0 0 33512 17 31 53 0
3 1 0 2098480 126092 1528512 0 0 0 148 7 33325 16 30 52 1
4 1 0 2098996 126096 1528516 0 0 0 32 7 32348 15 30 50 5
4 0 0 2099272 126096 1528520 0 0 0 0 2 33821 15 31 53 1
4 0 0 2099696 126096 1528520 0 0 0 0 0 33726 15 31 53 0
5 0 0 2099640 126096 1528520 0 0 0 0 0 33887 16 31 52 0
3 1 0 2099956 126096 1528520 0 0 0 104 25 33596 15 31 53 1
4 1 0 2099556 126096 1528520 0 0 0 32 7 33329 15 30 50 5
4 0 0 2099592 126096 1528520 0 0 0 0 2 34066 15 31 53 0
4 0 0 2099500 126096 1528520 0 0 0 100 6 33541 17 30 51 2
4 0 0 2099420 126096 1528520 0 0 0 0 0 33474 16 31 53 0
2 1 0 2100996 126096 1528520 0 0 0 124 10 33655 16 31 49 4
4 1 0 2099764 126096 1528520 0 0 0 132 12 33813 17 30 44 8
4 0 0 2100208 126096 1528520 0 0 0 4 3 33498 17 31 52 0
4 0 0 2099828 126096 1528520 0 0 0 0 0 33850 16 31 53 0
3 0 0 2100376 126096 1528520 0 0 0 0 0 34126 17 30 53 0


top 里面
top - 10:05:49 up 15:24, 4 users, load average: 4.37, 4.23, 4.19
Tasks: 128 total, 4 running, 122 sleeping, 2 stopped, 0 zombie
Cpu(s): 15.3%us, 6.0%sy, 0.0%ni, 52.1%id, 1.7%wa, 0.0%hi, 25.0%si, 0.0%st
Mem: 4148208k total, 2048140k used, 2100068k free, 126124k buffers
Swap: 2103480k total, 0k used, 2103480k free, 1528512k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND


6 root 39 19 0 0 0 R 100 0.0 25:23.91 ksoftirqd/1


18 root 39 19 0 0 0 R 100 0.0 25:28.52 ksoftirqd/7


3273 ulogd 15 0 6000 688 292 S 72 0.0 389:24.25 ulogd


5946 mysql 15 0 102m 15m 4356 S 8 0.4 63:52.65 mysqld

脚本里面有
ulimit 0
ulimit -f 10240000000
modprobe ip_conntrack hashsize=1048576
echo 3600 > /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_established
echo 1048576 > /proc/sys/net/netfilter/nf_conntrack_count
echo "1024 65000" > /proc/sys/net/ipv4/ip_local_port_range
echo "100 1200 128 512 15 5000 500 1884 2">/proc/sys/vm/bdflush
echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses
echo "1048576" > /proc/sys/net/netfilter/nf_conntrack_max
echo "1048576" > /proc/sys/net/ipv4/tcp_max_tw_buckets
echo "1" > /proc/sys/net/ipv4/ip_forward
echo "268435456" >/proc/sys/kernel/shmall
echo "536870912" >/proc/sys/kernel/shmmax
echo "1024" > /proc/sys/net/ipv4/neigh/default/gc_thresh1
echo "2048" > /proc/sys/net/ipv4/neigh/default/gc_thresh2
echo "4096" > /proc/sys/net/ipv4/neigh/default/gc_thresh3
echo "52428800" > /proc/sys/net/ipv4/route/max_size
echo "1" > /proc/sys/net/ipv4/conf/all/proxy_arp
echo "1" > /proc/sys/net/ipv4/tcp_window_scaling


iptables -A INPUT -p tcp --dport 3306 -j DROP
iptables -A INPUT -p tcp --dport 111 -j DROP
iptables -A INPUT -p tcp --dport 631 -j DROP
iptables -A INPUT -p tcp --dport 6010 -j DROP
iptables -A INPUT -p tcp --dport 6011 -j DROP
iptables -A INPUT -p tcp --dport 6012 -j DROP
iptables -A INPUT -p udp --dport 111 -j DROP
iptables -A INPUT -p udp --dport 631 -j DROP
#echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_all
iptables -A INPUT -p udp -j DROP
iptables -A INPUT -p tcp --dport ! 22 -j DROP
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT


网卡的信息。
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2258934208 errors:5 dropped:53075393 overruns:0 frame:5
TX packets:2044851900 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:240973845 (229.8 Mb) TX bytes:1154965411 (1101.4 Mb)
Base address:0x6000 Memory:c7ee0000-c7f00000


ethtool -S eth2
NIC statistics:
rx_packets: 2454702026
tx_packets: 2236369668
rx_bytes: 1973021291410
tx_bytes: 709236276521
rx_broadcast: 6
tx_broadcast: 14198060
rx_multicast: 0
tx_multicast: 579
rx_errors: 5
tx_errors: 0
tx_dropped: 0
multicast: 0
collisions: 0
rx_length_errors: 1
rx_over_errors: 0
rx_crc_errors: 4
rx_frame_errors: 0
rx_no_buffer_count: 170718107
rx_missed_errors: 83998346
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_window_errors: 0
tx_abort_late_coll: 0
tx_deferred_ok: 0
tx_single_coll_ok: 0
tx_multi_coll_ok: 0
tx_timeout_count: 0
tx_restart_queue: 290668
rx_long_length_errors: 0
rx_short_length_errors: 1
rx_align_errors: 0
tx_tcp_seg_good: 667
tx_tcp_seg_failed: 0
rx_flow_control_xon: 0
rx_flow_control_xoff: 0
tx_flow_control_xon: 23669502
tx_flow_control_xoff: 105039312
rx_long_byte_count: 1973021291410
rx_csum_offload_good: 2421425478
rx_csum_offload_errors: 290884
rx_header_split: 0
alloc_rx_buff_failed: 0
tx_smbus: 0
rx_smbus: 0
dropped_smbus: 0