精通polling参数调优的进来帮帮忙吧

xie_minix

阁下的机器是否是AMD 8111芯片+AMD 8131 PCI-X.如果是这样.
他的A边是最大6.4G,B边是3.2G左右每秒在AMD 8131上.
在AMD 8111上只是通常的100M卡.

xfsoul

是单独买的Intel的千兆双端口网卡。还有两块Intel光口网卡。

xie_minix

哦,不是的,你dmesg下,贴出来看看,我的意思想看看芯片组.
我有几台机器和你的差不多.是SUN V20Z的.bge的卡在PCI-X
槽上.理论速度是.AMD 8131 A 边通道可以达到6G每秒.B边的
也能跑到3G(因为B边通道是8位传输,A是16位穿输).所以总线
并不成为瓶颈(可参考AMD 8131芯片说明,既文献24637.pdf).

[ 本帖最后由 xie_minix 于 2006-6-1 09:32 编辑 ]

xfsoul

[root@shyfzx ~/em/em-5.1.5]# dmesg
Copyright (c) 1992-2006 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD 6.1-RELEASE #4: Wed May 31 09:12:54 UTC 2006
    root@shyfzx.star-net.cn:/usr/obj/usr/src/sys/GENERIC
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Opteron(tm) Processor 250 (2392.56-MHz 686-class CPU)
  Origin = "AuthenticAMD"  Id = 0xf5a  Stepping = 10
  Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
  AMD Features=0xe0500800<SYSCALL,NX,MMX+,LM,3DNow+,3DNow>
real memory  = 1073152000 (1023 MB)
avail memory = 1041059840 (992 MB)
ACPI APIC Table: <PTLTD          APIC  >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-23 on motherboard
ioapic1 <Version 1.1> irqs 24-27 on motherboard
ioapic2 <Version 1.1> irqs 28-31 on motherboard
kbd1 at kbdmux0
acpi0: <PTLTD    XSDT> on motherboard
acpi0: Power Button (fixed)
unknown: I/O range not supported
unknown: I/O range not supported
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x8008-0x800b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff,0x8000-0x807f,0x8080-0x80ff iomem 0xd8000-0xdbfff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci1: <ACPI PCI bus> on pcib1
ohci0: <OHCI (generic) USB controller> mem 0xfc100000-0xfc100fff irq 19 at device 0.0 on pci1
ohci0: [GIANT-LOCKED]
usb0: OHCI version 1.0, legacy support
usb0: SMM does not respond, resetting
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 3 ports with 3 removable, self powered
ohci1: <OHCI (generic) USB controller> mem 0xfc101000-0xfc101fff irq 19 at device 0.1 on pci1
ohci1: [GIANT-LOCKED]
usb1: OHCI version 1.0, legacy support
usb1: SMM does not respond, resetting
usb1: <OHCI (generic) USB controller> on ohci1
usb1: USB revision 1.0
uhub1: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 3 ports with 3 removable, self powered
atapci0: <SiI 3114 SATA150 controller> port 0x2420-0x2427,0x2414-0x2417,0x2418-0x241f,0x2410-0x2413,0x2400-0x240f mem 0xfc103000-0xfc1033ff irq 16 at device 5.0 on pci1
ata2: <ATA channel 0> on atapci0
ata3: <ATA channel 1> on atapci0
ata4: <ATA channel 2> on atapci0
ata5: <ATA channel 3> on atapci0
pci1: <display, VGA> at device 7.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci1: <AMD 8111 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1000-0x100f at device 7.1 on pci0
ata0: <ATA channel 0> on atapci1
ata1: <ATA channel 1> on atapci1
pci0: <serial bus, SMBus> at device 7.2 (no driver attached)
pci0: <bridge> at device 7.3 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 10.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pci0: <base peripheral, interrupt controller> at device 10.1 (no driver attached)
pcib3: <ACPI PCI-PCI bridge> at device 11.0 on pci0
pci3: <ACPI PCI bus> on pcib3
em0: <Intel(R) PRO/1000 Network Connection Version - 3.2.18> port 0x3000-0x303f mem 0xfe0c0000-0xfe0dffff,0xfe040000-0xfe07ffff irq 28 at device 1.0 on pci3
em0: Ethernet address: 00:04:23:b3:cb:76
em1: <Intel(R) PRO/1000 Network Connection Version - 3.2.18> port 0x3040-0x307f mem 0xfe0e0000-0xfe0fffff,0xfe080000-0xfe0bffff irq 29 at device 1.1 on pci3
em1: Ethernet address: 00:04:23:b3:cb:77
bge0: <Broadcom BCM5704C Dual Gigabit Ethernet, ASIC rev. 0x2003> mem 0xfe010000-0xfe01ffff,0xfe000000-0xfe00ffff irq 29 at device 2.0 on pci3
miibus0: <MII bus> on bge0
brgphy0: <BCM5704 10/100/1000baseTX PHY> on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto
bge0: Ethernet address: 00:11:09:15:1b:ce
bge1: <Broadcom BCM5704C Dual Gigabit Ethernet, ASIC rev. 0x2003> mem 0xfe030000-0xfe03ffff,0xfe020000-0xfe02ffff irq 30 at device 2.1 on pci3
miibus1: <MII bus> on bge1
brgphy1: <BCM5704 10/100/1000baseTX PHY> on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto
bge1: Ethernet address: 00:11:09:15:1d:05
pci0: <base peripheral, interrupt controller> at device 11.1 (no driver attached)
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ppc0: <Standard parallel printer port> port 0x378-0x37f,0x778-0x77f irq 7 on acpi0
ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode
ppbus0: <Parallel port bus> on ppc0
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc97ff,0xc9800-0xcafff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 8250 or not responding
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
ad0: 78167MB <Maxtor 6L080L0 BAJ41G20> at ata0-master UDMA133
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/ad0s2a
WARNING: / was not properly dismounted

xfsoul

原帖由 xie_minix 于 2006-6-1 09:25 发表
哦,不是的,你dmesg下,贴出来看看,我的意思想看看芯片组.
我有几台机器和你的差不多.是SUN V20Z的.bge的卡在PCI-X
槽上.理论速度是.AMD 8131 A 边通道可以达到6G每秒.B边的
也能跑到3G(因为B边通道是8位传输,A是 ...

你说的A边 B边是什么意思？

xie_minix

呵呵,简单画个图给你看看.

               |-------通道----------|
HOST<----|   A 边           B 边| ------>其他下游设备
          |----------------------|
    16位带宽  |                  |  8位带宽
             |                  |
             PCI-X桥A       PCI-X桥B
              |     |     |       |    |     |
          一些PCI槽....
所以在A边PCI槽传送数据到CPU中比B边快一倍,但无所谓了,G兆的卡跑不到那么快.
10G的卡又跑不起来.

[ 本帖最后由 xie_minix 于 2006-6-1 14:51 编辑 ]

雨丝风片

原帖由 xfsoul 于 2006-5-31 15:07 发表
我怎么看过去一段时间，两个网卡各收到多少包呢？
我想测试一下没有网桥时候 FreeBSD收包性能。
netstat -w 10 -i em0
看到的结果根本不对！我用smartbit发了大量的包，netstat都没有统计到。

有一个疑问请教一下。

看到xfsoul提到netstat对收包数目的统计不对，就去初步看了一下netstat的源码。以入包为例，netstat在每次时间间隔到达的时候，打印的是接口结构ifnet中的if_ipackets字段的变化：

1. 
   
2.     ....
   
3.                         show_stat("lu", 10, ifnet.if_ipackets - ip->ift_ip, 1);
   
4.     ....
   
5.                 ip->ift_ip = ifnet.if_ipackets;
   
6.     ....
   

复制代码

但接口中的这个字段对于桥转发流程来说，仅在if_bridge.c中的bridge_forward()函数和bridge_input()函数中有增加的操作，但在bridge.c中却看不到对这个字段的赋值。结论似乎是netstat只对if_bridge处理的包进行计数，而不对bridge处理的包进行计数。我对这个流程不熟，不知上述分析是否有误？

xie_minix

在驱动程序中会对他进行处理,如:vr_rxeof中
ifp->if_ipackets++;
你图中的代码部分的IFNET应该是单个桥自己的.
网卡进入的if_ipackets和桥进入的if_ipackets应该是不一样的.

[ 本帖最后由 xie_minix 于 2006-6-1 17:22 编辑 ]

雨丝风片

原帖由 xie_minix 于 2006-6-1 17:08 发表
在驱动程序中会对他进行处理,如:vr_rxeof中
ifp->if_ipackets++;

我也曾怀疑过这个方向，但由于代码不熟，就只能做点简单的“考证”了：

毕竟if_bridge.c中对其加1，bridge.c中没有对其加1。如果if_xxxx.c和桥代码是前后关系，就能解释bridge.c的情况，但不能解释if_bridge.c的情况。如果if_xxxx.c和桥代码是二选一的关系，就能解释if_bridge.c的情况，但不能解释bridge.c的情况。除非if_bridge.c和bridge.c并不是简单的替换关系，它们和驱动的处理接口有所不同？

查看了一下桥代码和以太网代码的接口，是在if_ethersubr.c文件的ether_input()函数中：

1. 
   
2.         /*
   
3.          * Tap the packet off here for a bridge.  bridge_input()
   
4.          * will return NULL if it has consumed the packet, otherwise
   
5.          * it gets processed as normal.  Note that bridge_input()
   
6.          * will always return the original packet if we need to
   
7.          * process it locally.
   
8.          */
   
9.         if (ifp->if_bridge) {
   
10.                 BRIDGE_INPUT(ifp, m);
    
11.                 if (m == NULL)
    
12.                         return;
    
13.         }
    
14. 
    
15.         /* Check for bridging mode */
    
16.         if (BDG_ACTIVE(ifp) )
    
17.                 if ((m = bridge_in_ptr(ifp, m)) == NULL)
    
18.                         return;
    

复制代码

其中，后面的bridge_in_ptr()调用就是原来的bridge的处理，而前面的BRIDGE_INPUT()调用则是（2005年6月5号）新加的if_bridge的处理。从这儿看来，bridge和if_bridge的处理代码从以太网处理流程中“截获”的地点是一样的。因此似乎没有道理if_bridge对if_ipackets的加一操作在其自身代码内部进行，而bridge对if_ipackets的加1操作则放到其它地方或是根本就没有进行呢？

[ 本帖最后由 雨丝风片 于 2006-6-1 17:37 编辑 ]

雨丝风片

原帖由 xie_minix 于 2006-6-1 17:08 发表
你图中的代码部分的IFNET应该是单个桥自己的.
网卡进入的if_ipackets和桥进入的if_ipackets应该是不一样的.

这个我也心存疑虑，就怕两个if_ipackets不是同一个东西，但我暂时还搞不清楚，所以就先拿出来让你指点一下了！