请教:solaris 连接 emc 存储速度极慢
大家好,我的系统环境:
一台sun solaris 主机 (SunOS XXX 5.10 Generic_144488-04 sun4u sparc SUNW,SPARC-Enterprise)
一台 hp linux 安腾 小机 2660 ,一台 emc nas 存储。
两台小机都通过 nfs 方式连接到 emc 的同一空间,当然是通过不同的ip挂载(mount)上的。
主机和存贮之间通过Hua3千兆交换机连接。
solairs上运行着一个机构仓储(olap)系统,管理的都是几百兆至2G的大文件,大量的客户端会从这里下载文件,
问题:linux 连接emc没有问题,速度正常,但是 solaris连接 emc时就非常慢,几乎连接不上,现象如下:
1、如果 在solaris上执行df -h 就会发现 原本是要 挂载的几个 存储卷出现的速度非常慢, 比如正常情况下应该有如下信息:
root@digital # df -h
Filesystem size usedavail capacityMounted on
/dev/md/dsk/d0 20G 7.5G 12G 39% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 108G 1.9M 108G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/dev/md/dsk/d20 9.9G 3.9G 5.8G 41% /usr
fd 0K 0K 0K 0% /dev/fd
swap 108G 896K 108G 1% /tmp
swap 108G 88K 108G 1% /var/run
/dev/md/dsk/d40 42G 1.1G 41G 3% /export/home
/dev/dsk/c5t6006016008D028000D9C3D7B9120E111d0s6
1.6T 337M 1.6T 1% /ebook2
运行df后本地设备很快就看到了,但红色部分(emc设备)就要等很久很久才能出现,df的输出就卡在那儿,不动了。
2、从solaris 主机ping emc 存储,结果如下:
64 bytes from 192.168.1.90: icmp_seq=0. time=0.251 ms
64 bytes from 192.168.1.90: icmp_seq=1. time=0.240 ms
64 bytes from 192.168.1.90: icmp_seq=2. time=0.241 ms
64 bytes from 192.168.1.90: icmp_seq=3. time=0.228 ms
64 bytes from 192.168.1.90: icmp_seq=4. time=0.214 ms
64 bytes from 192.168.1.90: icmp_seq=5. time=0.202 ms
64 bytes from 192.168.1.90: icmp_seq=6. time=0.203 ms
64 bytes from 192.168.1.90: icmp_seq=7. time=0.186 ms
64 bytes from 192.168.1.90: icmp_seq=8. time=0.173 ms
64 bytes from 192.168.1.90: icmp_seq=9. time=0.281 ms
64 bytes from 192.168.1.90: icmp_seq=10. time=0.268 ms
64 bytes from 192.168.1.90: icmp_seq=11. time=0.247 ms
64 bytes from 192.168.1.90: icmp_seq=12. time=0.267 ms
64 bytes from 192.168.1.90: icmp_seq=13. time=0.240 ms
64 bytes from 192.168.1.90: icmp_seq=14. time=0.257 ms
3、我查过主机的负载,诸如 mpstat ,vmstat ,iostat等,貌似都很正常,一点都不高,
4、运行 netstat -s :
TCP tcpRtoAlgorithm = 4 tcpRtoMin = 400
tcpRtoMax = 60000 tcpMaxConn = -1
tcpActiveOpens =625995 tcpPassiveOpens =653332
tcpAttemptFails =240240 tcpEstabResets =100845
tcpCurrEstab = 141 tcpOutSegs =176425096
tcpOutDataSegs =221848300tcpOutDataBytes =1164290171
tcpRetransSegs =8454951 tcpRetransBytes =3622524478
tcpOutAck =46609139 tcpOutAckDelayed =1921528
tcpOutUrg = 126 tcpOutWinUpdate = 19405
tcpOutWinProbe =5762 tcpOutControl =2498101
tcpOutRsts =289327 tcpOutFastRetrans = 112
tcpInSegs =187196237
tcpInAckSegs =108991448tcpInAckBytes =172433623
tcpInDupAck =9538040 tcpInAckUnsent = 13
tcpInInorderSegs =253961330tcpInInorderBytes =2151919823
tcpInUnorderSegs =219678 tcpInUnorderBytes =1633190326
tcpInDupSegs = 44673 tcpInDupBytes =4308560
tcpInPartDupSegs = 16267 tcpInPartDupBytes =9297984
tcpInPastWinSegs = 89 tcpInPastWinBytes =2044903847
tcpInWinProbe = 11 tcpInWinUpdate =3902
tcpInClosed =2630 tcpRttNoUpdate =77906834
tcpRttUpdate =30544671 tcpTimRetrans =2771191
tcpTimRetransDrop =2133 tcpTimKeepalive = 18977
tcpTimKeepaliveProbe=6773 tcpTimKeepaliveDrop = 2
tcpListenDrop = 0 tcpListenDropQ0 = 0
tcpHalfOpenDrop = 0 tcpOutSackRetrans =2817688
IPv4 ipForwarding = 2 ipDefaultTTL = 255
ipInReceives =361884451ipInHdrErrors = 7
ipInAddrErrors = 0 ipInCksumErrs = 0
ipForwDatagrams = 0 ipForwProhibits = 12960
ipInUnknownProtos = 0 ipInDiscards = 41
ipInDelivers =365653531ipOutRequests =272316826
ipOutDiscards =1479 ipOutNoRoutes = 0
ipReasmTimeout = 60 ipReasmReqds = 0
ipReasmOKs = 0 ipReasmFails = 0
ipReasmDuplicates = 0 ipReasmPartDups = 0
ipFragOKs = 0 ipFragFails = 0
ipFragCreates = 0 ipRoutingDiscards = 0
tcpInErrs = 1 udpNoPorts =5161846
udpInCksumErrs = 0 udpInOverflows = 0
rawipInOverflows = 0 ipsecInSucceeded = 426
ipsecInFailed = 0 ipInIPv6 = 0
ipOutIPv6 = 0 ipOutSwitchIPv6 = 0
tcpRetransBytes/ tcpOutDataBytes 的值 大的有点离谱。
5、 solarismount连接 emc 上的某些逻辑卷正常,mount 某些应用系统经常使用的就不正常,很慢。
6、重起 solaris , storage 和 network 就都正常了,但过一段后,上面的问题又会出现。
已经咨询过emc的技术支持,他们认定 存储是没有问题的,觉得是 交换机和网线的问题,对这个答案不满意,我也换过网线,甚至用了直连的方式,也不行。
也咨询过 sun的厂商技术支持(现在是oracle了,blalalala),他们一口咬叮 操作系统,网卡都是没有问题的。
我的软件开发商也很强势,几乎每提供什么帮助,唉,技不如人就要被欺负阿。
我想请教大家的是,如果想优化这个系统,我应该从那里入手,是应用系统的问题?把某些资源耗光了?
还是存储或nfs的问题,请大家多多指点,我对网络了解的有限,请各位不吝赐教
先谢谢了。
solaris 连接 emc超级慢
我在网络版发过这个帖子,大家认为不是网络问题:大家好,
我的系统环境:
一台sun solaris 主机 (SunOS XXX 5.10 Generic_144488-04 sun4u sparc SUNW,SPARC-Enterprise)
一台 hp linux 安腾 小机 2660 ,一台 emc nas 存储。
两台小机都通过 nfs 方式连接到 emc 的同一空间,当然是通过不同的ip挂载(mount)上的。
主机和存贮之间通过Hua3千兆交换机连接。
solairs上运行着一个机构仓储(olap)系统,管理的都是几百兆至2G的大文件,大量的客户端会从这里下载文件,
问题:linux 连接emc没有问题,速度正常,但是 solaris连接 emc时就非常慢,几乎连接不上,现象如下:
1、如果 在solaris上执行df -h 就会发现 原本是要 挂载的几个 存储卷出现的速度非常慢, 比如正常情况下应该有如下信息:
root@digital # df -h
Filesystem size usedavail capacityMounted on
/dev/md/dsk/d0 20G 7.5G 12G 39% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 108G 1.9M 108G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/dev/md/dsk/d20 9.9G 3.9G 5.8G 41% /usr
fd 0K 0K 0K 0% /dev/fd
swap 108G 896K 108G 1% /tmp
swap 108G 88K 108G 1% /var/run
/dev/md/dsk/d40 42G 1.1G 41G 3% /export/home
/dev/dsk/c5t6006016008D028000D9C3D7B9120E111d0s6
1.6T 337M 1.6T 1% /ebook2
运行df后本地设备很快就看到了,但红色部分(emc设备)就要等很久很久才能出现,df的输出就卡在那儿,不动了。
2、从solaris 主机ping emc 存储,结果如下:
64 bytes from 192.168.1.90: icmp_seq=0. time=0.251 ms
64 bytes from 192.168.1.90: icmp_seq=1. time=0.240 ms
64 bytes from 192.168.1.90: icmp_seq=2. time=0.241 ms
64 bytes from 192.168.1.90: icmp_seq=3. time=0.228 ms
64 bytes from 192.168.1.90: icmp_seq=4. time=0.214 ms
64 bytes from 192.168.1.90: icmp_seq=5. time=0.202 ms
64 bytes from 192.168.1.90: icmp_seq=6. time=0.203 ms
64 bytes from 192.168.1.90: icmp_seq=7. time=0.186 ms
64 bytes from 192.168.1.90: icmp_seq=8. time=0.173 ms
64 bytes from 192.168.1.90: icmp_seq=9. time=0.281 ms
64 bytes from 192.168.1.90: icmp_seq=10. time=0.268 ms
64 bytes from 192.168.1.90: icmp_seq=11. time=0.247 ms
64 bytes from 192.168.1.90: icmp_seq=12. time=0.267 ms
64 bytes from 192.168.1.90: icmp_seq=13. time=0.240 ms
64 bytes from 192.168.1.90: icmp_seq=14. time=0.257 ms
3、我查过主机的负载,诸如 mpstat ,vmstat ,iostat等,貌似都很正常,一点都不高,
4、运行 netstat -s :
TCP tcpRtoAlgorithm = 4 tcpRtoMin = 400
tcpRtoMax = 60000 tcpMaxConn = -1
tcpActiveOpens =625995 tcpPassiveOpens =653332
tcpAttemptFails =240240 tcpEstabResets =100845
tcpCurrEstab = 141 tcpOutSegs =176425096
tcpOutDataSegs =221848300tcpOutDataBytes =1164290171
tcpRetransSegs =8454951 tcpRetransBytes =3622524478
tcpOutAck =46609139 tcpOutAckDelayed =1921528
tcpOutUrg = 126 tcpOutWinUpdate = 19405
tcpOutWinProbe =5762 tcpOutControl =2498101
tcpOutRsts =289327 tcpOutFastRetrans = 112
tcpInSegs =187196237
tcpInAckSegs =108991448tcpInAckBytes =172433623
tcpInDupAck =9538040 tcpInAckUnsent = 13
tcpInInorderSegs =253961330tcpInInorderBytes =2151919823
tcpInUnorderSegs =219678 tcpInUnorderBytes =1633190326
tcpInDupSegs = 44673 tcpInDupBytes =4308560
tcpInPartDupSegs = 16267 tcpInPartDupBytes =9297984
tcpInPastWinSegs = 89 tcpInPastWinBytes =2044903847
tcpInWinProbe = 11 tcpInWinUpdate =3902
tcpInClosed =2630 tcpRttNoUpdate =77906834
tcpRttUpdate =30544671 tcpTimRetrans =2771191
tcpTimRetransDrop =2133 tcpTimKeepalive = 18977
tcpTimKeepaliveProbe=6773 tcpTimKeepaliveDrop = 2
tcpListenDrop = 0 tcpListenDropQ0 = 0
tcpHalfOpenDrop = 0 tcpOutSackRetrans =2817688
IPv4 ipForwarding = 2 ipDefaultTTL = 255
ipInReceives =361884451ipInHdrErrors = 7
ipInAddrErrors = 0 ipInCksumErrs = 0
ipForwDatagrams = 0 ipForwProhibits = 12960
ipInUnknownProtos = 0 ipInDiscards = 41
ipInDelivers =365653531ipOutRequests =272316826
ipOutDiscards =1479 ipOutNoRoutes = 0
ipReasmTimeout = 60 ipReasmReqds = 0
ipReasmOKs = 0 ipReasmFails = 0
ipReasmDuplicates = 0 ipReasmPartDups = 0
ipFragOKs = 0 ipFragFails = 0
ipFragCreates = 0 ipRoutingDiscards = 0
tcpInErrs = 1 udpNoPorts =5161846
udpInCksumErrs = 0 udpInOverflows = 0
rawipInOverflows = 0 ipsecInSucceeded = 426
ipsecInFailed = 0 ipInIPv6 = 0
ipOutIPv6 = 0 ipOutSwitchIPv6 = 0
tcpRetransBytes/ tcpOutDataBytes 的值 大的有点离谱。
5、 solarismount连接 emc 上的某些逻辑卷正常,mount 某些应用系统经常使用的就不正常,很慢。
6、重起 solaris , storage 和 network 就都正常了,但过一段后,上面的问题又会出现。
已经咨询过emc的技术支持,他们认定 存储是没有问题的,觉得是 交换机和网线的问题,对这个答案不满意,我也换过网线,甚至用了直连的方式,也不行。
也咨询过 sun的厂商技术支持(现在是oracle了,blalalala),他们一口咬叮 操作系统,网卡都是没有问题的。
我的软件开发商也很强势,几乎每提供什么帮助,唉,技不如人就要被欺负阿。
我想请教大家的是,如果想优化这个系统,我应该从那里入手,是应用系统的问题?把某些资源耗光了?
还是存储或nfs的问题,请大家多多指点,我对网络了解的有限,请各位不吝赐教
先谢谢了。 这个算solaris的问题吧,我帮你转过去把
这个问题有点难,需要点综合实力。 各位指点指点阿 你说的红色部分是 /dev/dsk/c5t6006016008D028000D9C3D7B9120E111d0s6 1.6T 337M 1.6T 1% /ebook2?
是iscsi方式挂载的吗?
我给你支一招。用tcpdump抓包,看卡在哪一步了?
按理来说EMC or Oracle都有应该有能力独立解决此问题。 我总感觉是 应用系统把 某些 io或网络资源 抓着不释放,才导致这种情况的,但我不知道如何才能确定是哪一块的问题。 多谢 nimysun 的指点,
红色部分就是 /dev/dsk/c5t6006016008D028000D9C3D7B9120E111d0s6 1.6T 337M 1.6T 1% /ebook2
是 nfs 方式挂载的 ,该存储用于 大文件的 存储(repository)
按照 nimysun 的方法 ,我用 snoop 试了一下 ,我对网络不熟,把结构贴出来,请大家帮我分析分析
其中 192.168.1.70 是 solaris , 192.168.1.90 是 emc
# snoop 192.168.1.70 192.168.1.90
Using device bge2 (promiscuous mode)
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598224942 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598226402 Seq=689416401 Len=0 Win=49640 Options=<nop,nop,sack 2598234127-2598234>
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598226402 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598227862 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598227862 Seq=689416401 Len=0 Win=49640 Options=<nop,nop,sack 2598234127-2598234>
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598229322 Seq=689416401 Len=0 Win=49640 Options=<nop,nop,sack 2598234127-2598234>
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598229322 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598230782 Seq=689416401 Len=0 Win=49640 Options=<nop,nop,sack 2598234127-2598234>
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598230782 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598232242 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598232242 Seq=689416401 Len=0 Win=49640 Options=<nop,nop,sack 2598234127-2598234>
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598233702 Seq=689416401 Len=0 Win=49640 Options=<nop,nop,sack 2598234127-2598234>
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598233702 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598235162 Seq=689416401 Len=0 Win=49640
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598235162 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598236622 Len=547 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598237169 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598238629 Len=515 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598239144 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598240604 Len=490 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598237169 Seq=689416401 Len=0 Win=47633
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598239144 Seq=689416401 Len=0 Win=45658
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598241094 Seq=689416401 Len=0 Win=43708
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598241094 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598242554 Len=470 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598243024 Seq=689416401 Len=0 Win=49170
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598243024 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598244484 Len=1000 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416401 Seq=2598245484 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598245484 Seq=689416401 Len=0 Win=46710
192.168.1.70 -> 192.168.1.90 NFS C READ3 FH=E2E8 at 23330816 for 32768
192.168.1.70 -> 192.168.1.90 NFS C READ3 FH=E2E8 at 23330816 for 32768 (retransmit)
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416525 Seq=2598257335 Len=0 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416525 Seq=2598246944 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598248404 Seq=689416525 Len=0 Win=49640
192.168.1.70 -> 192.168.1.90 NFS C READ3 FH=74B7 at 71335936 for 32768
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416525 Seq=2598248404 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416525 Seq=2598249864 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598251324 Seq=689416649 Len=0 Win=46720
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598251324 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598252784 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598254244 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598255704 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598254244 Seq=689416649 Len=0 Win=43800
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598257164 Seq=689416649 Len=0 Win=48180
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598257164 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598258624 Seq=689416649 Len=0 Win=49640
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598258624 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598260084 Seq=689416649 Len=0 Win=49640
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598260084 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598261544 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598263004 Seq=689416649 Len=0 Win=46720
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598263004 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598264464 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598265924 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598267384 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598265924 Seq=689416649 Len=0 Win=43800
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598268844 Seq=689416649 Len=0 Win=48180
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598268844 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598270304 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598271764 Len=547 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598272311 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598273771 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598275231 Len=515 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598271764 Seq=689416649 Len=0 Win=45260
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598273771 Seq=689416649 Len=0 Win=49640
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598275746 Seq=689416649 Len=0 Win=47665
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598275746 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598277206 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598278666 Len=490 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598279156 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598280616 Len=1017 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598281633 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598278666 Seq=689416649 Len=0 Win=44745
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598280616 Seq=689416649 Len=0 Win=49640
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598283093 Seq=689416649 Len=0 Win=47163
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598283093 Len=968 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598284061 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598285521 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598286981 Len=438 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598287419 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598288879 Len=915 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598289794 Len=1460 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598285521 Seq=689416649 Len=0 Win=44735
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598287419 Seq=689416649 Len=0 Win=49202
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598289794 Seq=689416649 Len=0 Win=46827
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598291254 Len=1431 Win=12288
192.168.1.70 -> 192.168.1.90 TCP D=2049 S=639 Ack=2598292685 Seq=689416649 Len=0 Win=43936
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598292685 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598294145 Len=1372 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598295517 Len=1460 Win=12288
192.168.1.90 -> 192.168.1.70 TCP D=639 S=2049 Ack=689416649 Seq=2598296977 Len=833 Win=12288
我发现里面有很多个 192.168.1.70 -> 192.168.1.90 NFS C READ3 FH=E2E8 at 23330816 for 32768 (retransmit)