- 论坛徽章:
- 0
|
20080605_固網.zip
(46.38 KB, 下载次数: 32)
系統緩慢時好時壞,請大家幫忙看看問題在那裡
廠商(Sun與HDS)都說他們的部份沒問題,但就是緩慢
OS: Solaris 8在 Sun Fine 15K, 20 個CPU, Memeory:32G, File system: vxfs ------> san switch ----->HDS的HBS(Switch)-->HDS storage
Storage: HDS raid 1
DB:Oracle 8.1.7.3
我看到iostat中 device busy 100%的幾個read & write每秒次數不多,但是read & write 速度很低,asvc_t很長.又看不到wait(%w)狀況.
另外附上當日的2個小時的iostat & vmstat log(行網)與另一個系統(固網)的iostat & vmstat log作比較(在相同Sun Fine 15k不同domain,統樣使用HDS stoarge, Solaris 8, 16 CPU, Mem 20G, file system :vxfs)同樣有deivcs忙碌但是read & write 次數多,速度快.
iostat:
Thu Jun 5 14:01:20 2008
cpu
us sy wt id
20 4 39 37
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.2 0.0 1.6 0.0 0.0 0.0 8.5 0 0 c0t1d0
0.0 0.2 0.0 1.6 0.0 0.0 0.0 8.6 0 0 c0t2d0
0.4 1.2 3.2 9.6 0.0 4.8 0.0 3008.1 0 100 c6t60060E80047DA50000007DA50000027Fd0
3.4 1.2 27.2 8.1 0.0 4.5 0.0 989.0 0 100 c6t60060E80047DA50000007DA50000025Ad0
2.4 1.0 20.8 14.4 0.0 3.1 0.0 922.7 0 100 c6t60060E80047DA50000007DA500000248d0
0.4 0.4 3.2 3.2 0.0 2.1 0.0 2628.6 0 100 c6t60060E80047DA50000007DA500000236d0
26.6 0.2 622.4 1.6 0.0 1.2 0.0 45.7 0 78 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:01:25 2008
cpu
us sy wt id
33 5 33 29
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
1.6 1.0 12.8 8.0 0.0 2.9 0.0 1118.3 0 100 c6t60060E80047DA50000007DA50000027Fd0
8.0 1.2 142.4 8.3 0.0 4.6 0.0 504.9 0 100 c6t60060E80047DA50000007DA50000025Ad0
5.4 0.8 43.2 219.2 0.0 3.1 0.0 493.5 0 100 c6t60060E80047DA50000007DA500000248d0
1.0 0.6 8.0 4.8 0.0 1.3 0.0 817.5 0 89 c6t60060E80047DA50000007DA500000236d0
0.8 0.0 6.4 0.0 0.0 0.0 0.0 9.3 0 1 c6t60060E80047DA50000007DA500000224d0
0.4 0.0 3.2 0.0 0.0 0.0 0.0 8.4 0 0 c6t60060E80047DA50000007DA500000212d0
122.8 0.0 3025.6 0.0 0.0 1.3 0.0 10.4 0 46 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:01:30 2008
cpu
us sy wt id
24 3 34 39
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.2 0.0 3.2 0.0 0.0 0.0 5.9 0 0 c0t1d0
0.0 0.2 0.0 3.2 0.0 0.0 0.0 12.5 0 0 c0t2d0
0.6 1.4 4.8 14.4 0.0 3.7 0.0 1831.9 0 100 c6t60060E80047DA50000007DA50000027Fd0
1.0 2.8 8.0 24.4 0.0 5.5 0.0 1454.3 0 100 c6t60060E80047DA50000007DA50000025Ad0
2.4 1.6 19.2 217.6 0.0 3.0 0.0 754.6 0 100 c6t60060E80047DA50000007DA500000248d0
0.6 0.6 4.8 4.8 0.0 1.5 0.0 1281.2 0 100 c6t60060E80047DA50000007DA500000236d0
0.0 0.2 0.0 1.6 0.0 0.0 0.0 8.4 0 0 c6t60060E80047DA50000007DA500000212d0
58.8 0.2 1318.4 1.6 0.0 0.9 0.0 15.2 0 31 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:01:35 2008
cpu
us sy wt id
18 2 45 34
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
1.0 2.0 8.0 19.2 0.0 6.0 0.0 2008.5 0 100 c6t60060E80047DA50000007DA50000027Fd0
0.2 0.6 1.6 3.4 0.0 4.0 0.0 5010.1 0 100 c6t60060E80047DA50000007DA50000025Ad0
3.4 1.4 65.6 22.4 0.0 3.8 0.0 791.0 0 100 c6t60060E80047DA50000007DA500000248d0
0.8 0.4 6.4 3.2 0.0 1.0 0.0 853.9 0 100 c6t60060E80047DA50000007DA500000236d0
69.8 0.0 1500.8 0.0 0.0 1.0 0.0 14.3 0 41 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:01:40 2008
cpu
us sy wt id
18 3 36 44
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.8 0.0 6.4 0.0 3.4 0.0 4305.5 0 100 c6t60060E80047DA50000007DA50000027Fd0
0.0 3.6 0.0 222.8 0.0 4.7 0.0 1319.3 0 100 c6t60060E80047DA50000007DA50000025Ad0
2.2 2.0 17.6 230.4 0.0 4.8 0.0 1137.6 0 100 c6t60060E80047DA50000007DA500000248d0
0.8 0.4 6.4 3.2 0.0 2.0 0.0 1671.6 0 100 c6t60060E80047DA50000007DA500000236d0
75.8 0.6 1478.4 4.8 0.0 1.0 0.0 12.6 0 51 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:01:45 2008
cpu
us sy wt id
21 4 41 34
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 1.8 0.0 14.4 0.0 3.9 0.0 2190.2 0 100 c6t60060E80047DA50000007DA50000027Fd0
1.8 1.8 25.6 13.0 0.0 7.1 0.0 1983.0 0 100 c6t60060E80047DA50000007DA50000025Ad0
2.0 2.2 16.0 246.4 0.0 7.3 0.0 1733.1 0 100 c6t60060E80047DA50000007DA500000248d0
3.6 1.6 28.8 12.8 0.0 1.8 0.0 349.5 0 100 c6t60060E80047DA50000007DA500000236d0
4.6 0.0 86.4 0.0 0.0 0.0 0.0 9.1 0 3 c6t60060E80047DA50000007DA500000224d0
19.4 0.4 417.6 3.2 0.0 1.0 0.0 50.4 0 85 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:01:50 2008
cpu
us sy wt id
18 3 36 43
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.2 0.0 3.2 0.0 0.0 0.0 10.3 0 0 c0t1d0
0.0 0.2 0.0 3.2 0.0 0.0 0.0 10.6 0 0 c0t2d0
1.0 2.0 8.0 16.0 0.0 4.2 0.0 1403.3 0 100 c6t60060E80047DA50000007DA50000027Fd0
0.4 1.8 3.2 217.6 0.0 3.4 0.0 1533.9 0 100 c6t60060E80047DA50000007DA50000025Ad0
2.8 2.6 22.4 230.4 0.0 6.2 0.0 1157.0 0 100 c6t60060E80047DA50000007DA500000248d0
1.8 1.0 14.4 8.0 0.0 1.7 0.0 617.4 0 100 c6t60060E80047DA50000007DA500000236d0
81.4 0.0 1398.4 0.0 0.0 1.2 0.0 14.2 0 34 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:01:55 2008
cpu
us sy wt id
18 3 33 46
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 2.4 0.0 19.2 0.0 0.1 0.0 30.4 0 1 c0t1d0
0.0 2.4 0.0 19.2 0.0 0.1 0.0 36.7 0 2 c0t2d0
0.0 1.4 0.0 11.2 0.0 4.9 0.0 3478.4 0 100 c6t60060E80047DA50000007DA50000027Fd0
0.0 1.0 0.0 8.0 0.0 4.3 0.0 4294.2 0 100 c6t60060E80047DA50000007DA50000025Ad0
1.8 2.2 14.4 35.2 0.0 5.0 0.0 1249.2 0 100 c6t60060E80047DA50000007DA500000248d0
0.2 0.4 1.6 3.2 0.0 0.3 0.0 531.0 0 32 c6t60060E80047DA50000007DA500000236d0
0.0 0.2 0.0 0.2 0.0 0.0 0.0 6.9 0 0 c6t60060E80047DA50000007DA500000212d0
85.0 0.2 1448.0 0.2 0.0 1.5 0.0 17.8 0 47 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:02:00 2008
cpu
us sy wt id
34 7 30 28
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.2 0.0 3.2 0.0 0.0 0.0 11.1 0 0 c0t1d0
0.0 0.2 0.0 3.2 0.0 0.0 0.0 11.2 0 0 c0t2d0
0.8 2.6 6.4 20.8 0.0 3.0 0.0 886.7 0 100 c6t60060E80047DA50000007DA50000027Fd0
0.2 2.0 1.6 219.2 0.0 4.4 0.0 2002.2 0 100 c6t60060E80047DA50000007DA50000025Ad0
6.8 26.4 54.4 500.8 0.0 5.6 0.0 169.7 0 100 c6t60060E80047DA50000007DA500000248d0
0.6 0.2 4.8 1.6 0.0 1.3 0.0 1589.4 0 73 c6t60060E80047DA50000007DA500000236d0
91.6 0.0 1664.0 0.0 0.0 1.3 0.0 13.8 0 35 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:02:05 2008
cpu
us sy wt id
32 6 35 27
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 6.6 0.0 32.4 0.0 0.0 0.0 4.5 0 3 c0t1d0
0.0 6.6 0.0 32.4 0.0 0.0 0.0 4.9 0 3 c0t2d0
0.8 2.0 6.4 16.0 0.0 2.9 0.0 1041.6 0 100 c6t60060E80047DA50000007DA50000027Fd0
6.2 2.2 49.6 216.4 0.0 4.9 0.0 585.9 0 100 c6t60060E80047DA50000007DA50000025Ad0
2.2 6.6 17.6 262.4 0.0 5.8 0.0 659.8 0 100 c6t60060E80047DA50000007DA500000248d0
1.2 1.0 9.6 8.0 0.0 1.2 0.0 543.2 0 100 c6t60060E80047DA50000007DA500000236d0
0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0 45 c6t60060E80047DA50000007DA500000224d0
0.0 0.2 0.0 1.6 0.0 0.0 0.0 41.9 0 1 c6t60060E80047DA50000007DA500000212d0
121.8 0.0 2454.4 0.0 0.0 1.0 0.0 8.6 0 33 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:02:10 2008
cpu
us sy wt id
22 3 40 34
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.2 0.0 1.6 0.0 0.0 0.0 7.8 0 0 c0t1d0
0.0 0.2 0.0 1.6 0.0 0.0 0.0 5.3 0 0 c0t2d0
0.0 1.8 0.0 14.4 0.0 3.5 0.0 1925.3 0 100 c6t60060E80047DA50000007DA50000027Fd0
0.2 2.2 1.6 17.6 0.0 4.7 0.0 1956.0 0 100 c6t60060E80047DA50000007DA50000025Ad0
1.8 4.4 14.4 260.8 0.0 7.7 0.0 1244.1 0 100 c6t60060E80047DA50000007DA500000248d0
2.2 0.6 28.8 4.8 0.0 0.5 0.0 160.8 0 43 c6t60060E80047DA50000007DA500000236d0
4.6 0.6 36.8 4.8 0.0 0.2 0.0 36.9 0 18 c6t60060E80047DA50000007DA500000224d0
0.2 0.0 1.6 0.0 0.0 0.5 0.0 2553.9 0 51 c6t60060E80047DA50000007DA500000212d0
117.6 0.0 2329.6 0.0 0.0 1.3 0.0 11.3 0 40 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:02:15 2008
cpu
us sy wt id
20 5 35 40
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.8 0.0 7.0 0.0 0.0 0.0 9.9 0 1 c0t1d0
0.0 0.8 0.0 7.0 0.0 0.0 0.0 13.1 0 1 c0t2d0
0.6 2.4 4.8 18.2 0.0 3.3 0.0 1097.1 0 100 c6t60060E80047DA50000007DA50000027Fd0
2.2 2.0 17.6 216.3 0.0 3.9 0.0 918.9 0 100 c6t60060E80047DA50000007DA50000025Ad0
1.6 3.8 12.8 41.6 0.0 4.2 0.0 777.8 0 100 c6t60060E80047DA50000007DA500000248d0
4.6 0.6 36.8 4.8 0.0 1.1 0.0 202.5 0 100 c6t60060E80047DA50000007DA500000236d0
2.0 0.0 16.0 0.0 0.0 0.0 0.0 9.4 0 2 c6t60060E80047DA50000007DA500000224d0
0.0 0.6 0.0 4.8 0.0 0.3 0.0 535.5 0 32 c6t60060E80047DA50000007DA500000212d0
33.8 0.0 625.6 0.0 0.0 0.2 0.0 7.2 0 10 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:02:20 2008
cpu
us sy wt id
18 3 37 42
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.2 0.0 1.6 0.0 0.0 0.0 9.6 0 0 c0t1d0
0.0 0.2 0.0 1.6 0.0 0.0 0.0 9.5 0 0 c0t2d0
0.2 1.4 1.6 11.2 0.0 3.0 0.0 1904.7 0 99 c6t60060E80047DA50000007DA50000027Fd0
1.2 1.6 9.6 214.5 0.0 3.6 0.0 1300.4 0 100 c6t60060E80047DA50000007DA50000025Ad0
3.8 2.8 30.4 260.8 0.0 5.2 0.0 793.7 0 100 c6t60060E80047DA50000007DA500000248d0
6.4 0.4 94.4 3.2 0.0 0.7 0.0 102.2 0 65 c6t60060E80047DA50000007DA500000236d0
3.6 0.0 28.8 0.0 0.0 0.0 0.0 7.2 0 3 c6t60060E80047DA50000007DA500000224d0
1.2 0.0 9.6 0.0 0.0 0.0 0.0 8.9 0 1 c6t60060E80047DA50000007DA500000212d0
9.8 0.0 124.8 0.0 0.0 0.1 0.0 10.1 0 7 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:02:25 2008
cpu
us sy wt id
26 6 32 36
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 9.4 0.0 70.5 0.0 0.8 0.0 84.3 0 6 c0t1d0
0.0 9.4 0.0 70.5 0.0 0.9 0.0 100.7 0 6 c0t2d0
0.0 1.6 0.0 12.8 0.0 4.5 0.0 2832.3 0 100 c6t60060E80047DA50000007DA50000027Fd0
5.8 2.2 47.9 17.6 0.0 5.3 0.0 662.1 0 100 c6t60060E80047DA50000007DA50000025Ad0
11.6 33.5 493.4 592.4 0.0 4.3 0.0 95.3 0 100 c6t60060E80047DA50000007DA500000248d0
8.2 0.6 119.8 4.8 0.0 0.2 0.0 25.9 0 21 c6t60060E80047DA50000007DA500000236d0
4.4 0.0 87.8 0.0 0.0 0.1 0.0 14.7 0 5 c6t60060E80047DA50000007DA500000224d0
13.6 0.4 170.9 3.2 0.0 0.2 0.0 12.3 0 14 c6t60060E80047DA50000007DA500000200d0
Thu Jun 5 14:02:30 2008
cpu
us sy wt id
21 4 57 19
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.8 0.0 7.4 0.0 0.0 0.0 11.2 0 1 c0t1d0
0.0 0.8 0.0 7.4 0.0 0.0 0.0 11.1 0 1 c0t2d0
0.8 3.4 6.4 19.2 0.0 4.8 0.0 1136.1 0 100 c6t60060E80047DA50000007DA50000027Fd0
0.0 5.6 0.0 243.5 0.0 6.3 0.0 1132.5 0 100 c6t60060E80047DA50000007DA50000025Ad0
3.2 8.2 432.0 284.8 0.0 8.5 0.0 746.5 0 100 c6t60060E80047DA50000007DA500000248d0
0.0 1.0 0.0 8.0 0.0 1.2 0.0 1170.9 0 87 c6t60060E80047DA50000007DA500000236d0
0.0 0.4 0.0 3.2 0.0 0.0 0.0 8.4 0 0 c6t60060E80047DA50000007DA500000224d0
0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0 29 c6t60060E80047DA50000007DA500000212d0
14.6 0.6 324.8 4.8 0.0 0.4 0.0 27.3 0 31 c6t60060E80047DA50000007DA500000200d0
我看到vmstat上看到b部份排序準備進入CPU運算的數量有超過4,並很頻繁.disk部份s1 & s2時有數字並不常歸零,cpu idle高表示cpu使用不高
vmstat:
Start Time : 2008/06/05 14:00:00
procs memory page disk faults cpu
r b w swap free re mf pi po fr de sr s0 s1 s2 sd in sy cs us sy id
0 1 0 31805536 20385056 0 0 0 0 0 0 0 0 3 3 0 4294967196 0 0 -92 -19 -247
0 0 0 30465296 19521680 6202 12185 324 4 4 0 0 0 7 7 0 1763 89421 7382 43 15 42
0 0 0 30439544 19498920 8592 6644 352 3 3 0 0 0 0 0 0 1473 75884 3813 52 12 35
0 3 0 30434080 19503408 10459 7390 1017 3 3 0 0 0 0 0 0 2841 85612 7870 48 17 35
0 12 0 30425656 19502496 7312 3200 246 3 3 0 0 0 0 0 0 2190 74648 4980 31 9 60
0 7 0 30444184 19517600 10265 5574 664 6 6 0 0 0 1 1 0 3482 81150 9076 43 14 43
0 12 0 30438512 19514176 9439 4333 510 1 1 0 0 0 8 8 0 2976 81010 6830 40 12 48
0 8 0 30444496 19517096 9473 4103 572 3 3 0 0 0 0 0 0 3366 81081 8459 39 13 49
0 1 0 30470480 19532288 7713 5932 1456 0 0 0 0 0 0 0 0 3265 84458 8972 43 13 44
0 0 0 30463152 19529088 7713 6217 1403 3 3 0 0 0 0 0 0 3320 87919 9501 45 13 42
0 0 0 30464416 19527304 7856 5797 1716 4 4 0 0 0 0 0 0 3273 83082 8511 35 13 52
0 0 0 30462832 19523536 7732 5421 1369 0 0 0 0 0 0 0 0 3152 80540 7910 28 12 60
0 1 0 30464024 19520128 8088 8048 1176 4 4 0 0 0 1 1 0 2261 80306 6117 34 13 53
0 6 0 30474520 19519576 7927 5221 518 3 3 0 0 0 8 8 0 1585 77929 4957 31 11 58
0 12 0 30458000 19499080 8256 8298 976 6 6 0 0 0 2 2 0 1624 87033 6855 35 13 52
0 14 0 30437184 19486232 7475 5071 1080 48 48 0 0 0 8 8 0 1405 76172 4380 31 9 59
0 14 0 30435152 19486672 472 2317 665 0 0 0 0 0 0 0 0 1189 67296 2622 21 4 76
0 11 0 30428592 19481280 1532 3061 2857 0 0 0 0 0 0 0 0 1798 72586 4902 33 5 62
0 13 0 30407656 19462872 471 1997 1696 3 3 0 0 0 0 0 0 1281 69766 3055 24 3 73
procs memory page disk faults cpu
r b w swap free re mf pi po fr de sr s0 s1 s2 sd in sy cs us sy id
0 12 0 30409088 19463800 445 1419 1756 0 0 0 0 0 0 0 0 891 63448 2277 19 2 79
0 11 0 30405672 19461944 374 1832 1492 0 0 0 0 0 0 0 0 1021 65626 2919 17 3 80
0 15 0 30402776 19461488 908 3252 580 0 0 0 0 0 0 0 0 1091 65595 3358 22 4 75
0 12 0 30420272 19470016 476 2016 1446 3 3 0 0 0 0 0 0 976 63496 2930 18 3 80
0 10 0 30419272 19470112 552 2667 1464 0 0 0 0 0 2 2 0 926 66145 2700 19 3 78
0 12 0 30439312 19481248 1270 6376 1729 3 3 0 0 0 0 0 0 2327 76739 6202 35 7 58
0 14 0 30414832 19469464 1279 6250 2659 3 3 0 0 0 7 7 0 1689 72004 4075 32 6 62
0 16 0 30379224 19450128 485 2112 2312 0 0 0 0 0 0 0 0 870 63114 2350 22 3 75
0 12 0 30372384 19447000 688 3729 766 4 4 0 0 0 1 1 0 1322 71611 4015 21 5 75
0 10 0 30345224 19432096 196 1051 224 0 0 0 0 0 0 0 0 1163 65986 2881 18 3 79
0 12 0 30358736 19440384 733 3861 516 1 1 0 0 0 9 9 0 2209 71286 5529 26 6 68
0 20 0 30359176 19440800 797 3346 353 4 4 0 0 0 1 1 0 1255 66852 2909 20 4 76
0 22 0 30344480 19432568 663 3164 920 0 0 0 0 0 0 0 0 1238 67636 3026 19 3 78
0 11 0 30338784 19428016 488 2593 649 3 3 0 0 0 0 0 0 1138 65115 2828 17 3 80
0 10 0 30341152 19428824 487 2568 448 0 0 0 0 0 0 0 0 1058 64755 2963 15 3 82
0 12 0 30323344 19418720 528 2662 412 0 0 0 0 0 0 0 0 1275 66438 3197 16 3 81
0 12 0 30308056 19410808 501 2661 174 3 3 0 0 0 0 0 0 1250 64781 3044 17 3 80
0 10 0 30315120 19414360 1036 5288 940 1 1 0 0 0 0 0 0 2202 76050 6493 28 7 65
0 9 0 30276296 19386416 726 4757 270 3 3 0 0 0 2 2 0 1108 65647 2425 19 4 77
procs memory page disk faults cpu
r b w swap free re mf pi po fr de sr s0 s1 s2 sd in sy cs us sy id
0 9 0 30243008 19358208 3215 4826 2324 3 3 0 0 0 1 1 0 1686 70409 4858 21 7 72
0 12 0 30225088 19349016 3006 4233 2336 0 0 0 0 0 0 0 0 1226 71019 4217 22 7 72
0 5 0 30219752 19346416 1498 2250 2040 4 4 0 0 0 1 1 0 1495 69534 4256 21 5 74
0 17 0 30228616 19350712 607 3175 1240 0 0 0 0 0 1 1 0 1563 70297 4134 34 5 61
0 16 0 30214440 19342880 428 2217 360 0 0 0 0 0 1 1 0 890 66750 2080 28 3 69
0 12 0 30255184 19381952 565 3058 217 4 4 0 0 0 7 7 0 1218 69060 2759 19 4 77
0 12 0 30257640 19385248 451 2430 16 0 0 0 0 0 2 2 0 1037 64313 2282 19 3 78
0 16 0 30289048 19409360 1012 5478 238 3 3 0 0 0 0 0 0 2249 73877 6815 29 7 64
0 14 0 30266968 19398704 505 2745 214 0 0 0 0 0 0 0 0 1187 66793 2636 26 3 71
0 12 0 30260768 19395520 846 4579 52 4 4 0 0 0 0 0 0 1248 68261 3040 19 4 77
0 10 0 30256400 19392960 312 1694 51 0 0 0 0 0 0 0 0 1056 66613 2702 22 3 75
0 16 0 30281960 19406096 1035 5206 404 3 3 0 0 0 1 1 0 1900 74975 5150 38 6 56
0 8 0 30273120 19400840 774 4127 179 0 0 0 0 0 0 0 0 1304 71827 3204 26 5 70
0 5 0 30279296 19403824 439 2256 699 3 3 0 0 0 0 0 0 1277 71202 3812 20 4 76
0 2 0 30307344 19417416 875 4331 1014 0 0 0 0 0 0 0 0 1434 68600 3519 22 6 72
0 6 0 30304440 19415440 1236 6474 851 3 3 0 0 0 1 1 0 2201 77866 6040 29 7 63
0 7 0 30282512 19404224 685 3389 502 3 3 0 0 0 8 8 0 1373 69880 2939 22 5 73
0 6 0 30276984 19402176 587 3128 395 0 0 0 0 0 0 0 0 1099 65852 2523 23 3 74
0 8 0 30284808 19406104 610 3222 328 3 3 0 0 0 0 0 0 1209 68807 2739 21 4 75
End Time : 2008/06/05 14:04:56 |
|