i/o error

wwszadu

环境：
BL25P两台（双端口光纤通道适配器）
MSA1000（双控制器）
2/16san交换机两台
redhatas4(2.6.9-34-ELsmp)
redhat cluster suite u3
HBA信息：

主机1

firmware: 3.03.18 IPX
driver: 8.01.02-d4

主机2

firmware: 3.03.11 IPX
driver: 8.00.02-fo

问题：
安装了PSP包后，配置一下FCA2214，把failover的功能，选择为QLogicFailover。用ACU工具在MSA1000上划了两个逻辑卷分别映射两台主机。重启机器后，其中一台机器会报一下错误：
i/O error on device sdc, logical block 66
Buffer I/O error on device sdc, logical block 67
Buffer I/O error on device sdc, logical block 68
Buffer I/O error on device sdc, logical block 69
Buffer I/O error on device sdc, logical block 70
Buffer I/O error on device sdc, logical block 71
Device sdc not ready.
end_request: I/O error, dev sdc, sector 512
Buffer I/O error on device sdc, logical block 64
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
mtrr: type mismatch for f6000000,800000 old: uncachable new: write-combining
mtrr: type mismatch for f6000000,800000 old: uncachable new: write-combining
NET: Unregistered protocol family 30
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Buffer I/O error on device sdc, logical block 1
Buffer I/O error on device sdc, logical block 2
Buffer I/O error on device sdc, logical block 3
Buffer I/O error on device sdc, logical block 4
Buffer I/O error on device sdc, logical block 5
Buffer I/O error on device sdc, logical block 6
Buffer I/O error on device sdc, logical block 7
Buffer I/O error on device sdc, logical block 8
Buffer I/O error on device sdc, logical block 9
Buffer I/O error on device sdc, logical block 10
Buffer I/O error on device sdc, logical block 11
Buffer I/O error on device sdc, logical block 12
Buffer I/O error on device sdc, logical block 13
Buffer I/O error on device sdc, logical block 14
Buffer I/O error on device sdc, logical block 15
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 96255352
Buffer I/O error on device sdc, logical block 12031919
Device sdc not ready.
end_request: I/O error, dev sdc, sector 96255352
Buffer I/O error on device sdc, logical block 12031919
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
Buffer I/O error on device sdd, logical block 1
Buffer I/O error on device sdd, logical block 2
Buffer I/O error on device sdd, logical block 3
Buffer I/O error on device sdd, logical block 4
Buffer I/O error on device sdd, logical block 5
Buffer I/O error on device sdd, logical block 6
Buffer I/O error on device sdd, logical block 7
Buffer I/O error on device sdd, logical block 8
Buffer I/O error on device sdd, logical block 9
Buffer I/O error on device sdd, logical block 10
Buffer I/O error on device sdd, logical block 11
Buffer I/O error on device sdd, logical block 12
Buffer I/O error on device sdd, logical block 13
Buffer I/O error on device sdd, logical block 14
Buffer I/O error on device sdd, logical block 15
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
Device sdd not ready.
end_request: I/O error, dev sdd, sector 88168792
Buffer I/O error on device sdd, logical block 11021099
Device sdd not ready.
end_request: I/O error, dev sdd, sector 88168792
Buffer I/O error on device sdd, logical block 11021099
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
Device sdd not ready.
end_request: I/O error, dev sdd, sector 0
Buffer I/O error on device sdd, logical block 0
usb 2-1: USB disconnect, address 2
usb 2-2: USB disconnect, address 3
ohci_hcd 0000:01:00.1: wakeup
usb 2-1: new low speed USB device using address 4
input: USB HID v1.00 Mouse [413c:3010] on usb-0000:01:00.1-1
usb 2-2: new low speed USB device using address 5
input: USB HID v1.10 Keyboard [CHICONY HP Basic USB Keyboard] on usb-0000:01:00.1-

lsw

没记错的话，MSA1000有专门针对非WINDOWS平台的firmware

ljhb

你的failover功能在哪里配置的？要在加载模块的时候使用参数ql2xfailover=1

wwszadu

谢谢回复,在安装PSP包的时候配置的QLOGICFAILOVER.在/etc/modprobe.conf里面有的是ql2xfailover=1,有的是ql2xfailover=0,还有就是完全没有这个.

nntp

1. 不要安装psp包.

2. 断开MSA,  单独把每个blade装好后， configure好OS之后，从 hp download site ,search FCA2214, -> linux -> redhat EL xxxx -> 一个8MB以上的文件 (tar.gz)

3. blade 如果network load比较高，可以把RHEL bundle的 driver(tg3 or e1000)用hp download site的 driver换掉，这个我就不多说了.

4.  tar zxvf 那个dl下来的包之后， INSTALL -f  安装, 然后reboot

5. MSA上进入 ACU -> enable SSP -> 选择好LUN ,  connection 从default 修改成 Linux (否则 path不会failover)

6. 你的这些故障信息出现的原因就是第5步没有做.


btw:你是HP得人，还是HP的代理还是最终用户?

wwszadu

第五步我是做了的，不过我在做SSP时一直都开着MSA1000，会不会是这个原因呢？只有等五一过后再去弄了。但是12台机子中，有2对是正常的（未报错）。下次再去试试，快放假了，这段时间时间好像有点心慌似的，一直没静下心来做这件事。做技术的是不是不该这样哟：）

nntp

connection 真的确定改成Linux了么? 你要100%确认的,因为你的故障信息十有八九是这个没有弄.

即便是你的LUN两个node都要看到，SSP也一定要enable,然后手工assign到两个node的WWN上去.

你会 acu CLI么？ 如果会的话，在linux上把 acu cli 装好，然后连上去配，可以有比ACU 图形更多的控制，很方便的. 不要用附带的线直接连MSA配，速度龟慢.

wwszadu

是的  我是通过ACU进去配置的 HP的做小机和存储的工程师都看过了,没问题的.但是他搞不懂LINUX.有些机子我是用超级终端进去配置的.我发现有些机子安装了PSP包后无法使用ACU工具.ACU只检测到本地的阵列,看不到MSA1000.

nntp

原帖由 wwszadu 于 2006-4-29 21:54 发表
是的  我是通过ACU进去配置的 HP的做小机和存储的工程师都看过了,没问题的.但是他搞不懂LINUX.有些机子我是用超级终端进去配置的.我发现有些机子安装了PSP包后无法使用ACU工具.ACU只检测到本地的阵列,看不到MSA1000.

你要用MSA1000 Support CD.

说老实话，你这个问题如果抛开硬件故障，基本上就是SSP范围的东西.   ACU GUI看到的东西太少，我基本不用GUI来配的，因为node还需要重新启动，不太现实.  ACU CLI可以online配，很方便，最主要是可以做成脚本.

wwszadu

谢谢你nntp，非常感谢你给我的建议。