光纤链路问题 求解

lzp_0004

Jun 27 16:10:16 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 16:10:16 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 16:10:16 OBB VCS_AGFW[1318]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 Thread(11)

Resource(disk6540) - monitor procedure did not complete within the expected time.
Jun 27 16:10:16 OBB Had[1243]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 (OBB) Resource(disk6540)

- monitor procedure did not complete within the expected time.
Jun 27 16:17:15 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 16:17:15 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 16:17:16 OBB VCS_AGFW[1318]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 Thread(12)

Resource(disk6540) - monitor procedure did not complete within the expected time.
Jun 27 16:17:16 OBB Had[1243]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 (OBB) Resource(disk6540)

- monitor procedure did not complete within the expected time.
Jun 27 16:19:16 OBB scsi: [ID 107833 kern.warning] WARNING:

/scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7):
Jun 27 16:19:16 OBB         SCSI transport failed: reason 'tran_err': retrying command
Jun 27 16:26:15 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 16:26:15 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 16:26:16 OBB VCS_AGFW[1318]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 Thread( Resource

(disk6540) - monitor procedure did not complete within the expected time.
Jun 27 16:26:16 OBB Had[1243]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 (OBB) Resource(disk6540)

- monitor procedure did not complete within the expected time.
Jun 27 16:29:15 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 16:29:15 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 16:29:16 OBB VCS_AGFW[1318]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 Thread(16)

Resource(disk6540) - monitor procedure did not complete within the expected time.
Jun 27 16:29:16 OBB Had[1243]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 (OBB) Resource(disk6540)

- monitor procedure did not complete within the expected time.
Jun 27 16:35:15 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 16:35:15 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 16:35:15 OBB scsi: [ID 107833 kern.warning] WARNING:

/scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7):
Jun 27 16:35:15 OBB         SCSI transport failed: reason 'timeout': retrying command
Jun 27 16:41:15 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 16:41:15 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 16:43:15 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 16:43:15 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 16:43:15 OBB VCS_AGFW[1318]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 Thread(7) Resource

(disk6540) - monitor procedure did not complete within the expected time.
Jun 27 16:43:15 OBB Had[1243]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 (OBB) Resource(disk6540)

- monitor procedure did not complete within the expected time.
Jun 27 16:45:15 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 16:45:15 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 16:47:53 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 16:47:53 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 16:48:15 OBB scsi: [ID 107833 kern.warning] WARNING:

/scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7):
Jun 27 16:48:15 OBB         SCSI transport failed: reason 'tran_err': retrying command
Jun 27 16:51:15 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 16:51:15 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 16:51:15 OBB VCS_AGFW[1318]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 Thread(10)

Resource(disk6540) - monitor procedure did not complete within the expected time.
Jun 27 16:51:15 OBB Had[1243]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 (OBB) Resource(disk6540)

- monitor procedure did not complete within the expected time.
Jun 27 17:01:15 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 17:01:15 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 17:02:15 OBB VCS_AGFW[1318]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 Thread(10)

Resource(disk6540) - monitor procedure did not complete within the expected time.
Jun 27 17:02:15 OBB Had[1243]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13027 (OBB) Resource(disk6540)

- monitor procedure did not complete within the expected time.
Jun 27 17:04:15 OBB scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jun 27 17:04:15 OBB         /scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7): Command Timeout on

path /pci@8,700000/SUNW,qlc@4,1/fp@0,0 (fp2)
Jun 27 17:04:15 OBB scsi: [ID 107833 kern.warning] WARNING:

/scsi_vhci/ssd@g600a0b80004726400000019a4833bd74 (ssd7):
Jun 27 17:04:15 OBB         SCSI transport failed: reason 'timeout': retrying command


本机为备机，V890

doging

/pci@8,700000/SUNW,qlc@4,1/fp@0,0 这一条链路出问题了

lzp_0004

额  知道是他的问题
求解决办法
有一段时间没有报这个错  今天重启了下又开始重新报错

dfck001

问问题要说明清楚环境V890+存储?+vcs
给LZ看个判断链路问题的实例(sun 6140的存储)
在操作系统中确认故障端口的WWN

root@xxx # luxadm -e port

Found path to 3 HBA ports

/devices/pci@9,600000/SUNW,qlc@2/fp@0,0:devctl                     CONNECTED
/devices/pci@8,600000/SUNW,emlxs@1/fp@0,0:devctl                   CONNECTED
/devices/pci@8,600000/SUNW,emlxs@2/fp@0,0:devctl                   CONNECTED
root@xxx# luxadm -e dump_map /devices/pci@8,600000/SUNW,emlxs@1/fp@0,0:devctl
Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type
0    10000   0         201400a0b82a214c 200400a0b82a214c 0x0  (Disk device)
1    10100   0         201500a0b82a214c 200400a0b82a214c Failed to get the type.
2    10200   0         10000000c9b23937 20000000c9b23937 0x1f (Unknown Type)
3    10300   0         10000000c9b23655 20000000c9b23655 0x1f (Unknown Type)
4    10600   0         5006016946e05aa4 50060160c6e05aa4 0x0  (Disk device)
5    10700   0         5006016146e05aa4 50060160c6e05aa4 0x0  (Disk device)
6    10800   0         10000000c9b85229 20000000c9b85229 0x1f (Unknown Type)
7    10900   0         10000000c9b85274 20000000c9b85274 0x1f (Unknown Type)
8    10a00   0         10000000c9b8522e 20000000c9b8522e 0x1f (Unknown Type)
9    10d00   0         10000000c96760e7 20000000c96760e7 0x1f (Unknown Type)
10   10e00   0         10000000c967b42e 20000000c967b42e 0x1f (Unknown Type)
11   10f00   0         10000000c967b24f 20000000c967b24f 0x1f (Unknown Type)
12   10c00   0         10000000c967b454 20000000c967b454 0x1f (Unknown Type,Host Bus Adapter)
root@xxxx # luxadm -e dump_map /devices/pci@8,600000/SUNW,emlxs@2/fp@0,0:devctl
Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type
0    10000   0         202400a0b82a214c 200400a0b82a214c 0x0  (Disk device)
1    10100   0         202500a0b82a214c 200400a0b82a214c 0x0  (Disk device)
2    10200   0         10000000c9b22d41 20000000c9b22d41 0x1f (Unknown Type)
3    10300   0         10000000c9b231a5 20000000c9b231a5 0x1f (Unknown Type)
4    10600   0         5006016046e05aa4 50060160c6e05aa4 0x0  (Disk device)
5    10700   0         5006016846e05aa4 50060160c6e05aa4 0x0  (Disk device)
6    10800   0         10000000c9b851db 20000000c9b851db 0x1f (Unknown Type)
7    10900   0         10000000c9b852b4 20000000c9b852b4 0x1f (Unknown Type)
8    10a00   0         10000000c9b8520a 20000000c9b8520a 0x1f (Unknown Type)
9    10d00   0         10000000c967615f 20000000c967615f 0x1f (Unknown Type)
10   10e00   0         10000000c967b43a 20000000c967b43a 0x1f (Unknown Type)
11   10f00   0         10000000c96db917 20000000c96db917 0x1f (Unknown Type)
12   10c00   0         10000000c967b34d 20000000c967b34d 0x1f (Unknown Type,Host Bus Adapter)

root@xxx # luxadm probe
No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):
  Node WWN:20000011c6e385a3  Device Typeisk device
    Logical Path:/dev/rdsk/c1t0d0s2
  Node WWN:20000011c6e38909  Device Typeisk device
    Logical Path:/dev/rdsk/c1t1d0s2
  Node WWN:200400a0b82a214c  Device Typeisk device
    Logical Path:/dev/rdsk/c4t600A0B80002A214A0000090C47DF34DBd0s2
  Node WWN:200400a0b82a214c  Device Typeisk device
    Logical Path:/dev/rdsk/c4t600A0B80002A214C00000A9647DF3620d0s2
root@xxxx # luxadm disp /dev/rdsk/c4t600A0B80002A214A0000090C47DF34DBd0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c4t600A0B80002A214A0000090C47DF34DBd0s2
  Vendor:               SUN   
  Product ID:           CSM200_R      
  Revision:             0619
  Serial Num:           SG71303394
  Unformatted capacity: 499712.000 MBytes
  Write Cache:          Enabled
  Read Cache:           Enabled
    Minimum prefetch:   0x1
    Maximum prefetch:   0x1
  Device Type:          Disk device
  Path(s):

  /dev/rdsk/c4t600A0B80002A214A0000090C47DF34DBd0s2
  /devices/scsi_vhci/ssd@g600a0b80002a214a0000090c47df34db:c,raw
   Controller           /devices/pci@8,600000/SUNW,emlxs@1/fp@0,0
    Device Address              201400a0b82a214c,0
    Host controller port WWN    10000000c967b454
    Class                       secondary
    State                       STANDBY
   Controller           /devices/pci@8,600000/SUNW,emlxs@2/fp@0,0
    Device Address              202400a0b82a214c,0
    Host controller port WWN    10000000c967b34d
    Class                       secondary
    State                       STANDBY
   Controller           /devices/pci@8,600000/SUNW,emlxs@2/fp@0,0
    Device Address              202500a0b82a214c,0
    Host controller port WWN    10000000c967b34d
    Class                       primary
    State                       ONLINE

通过上面的命令发现通往WWN 201500a0b82a214c的链路异常，该链路对应端口B/1

在另外一台主机上出现问题的路径完全相同，分析主机HBA卡到交换机的链路有3条正常，因此主机到存储交换机的链路应该没有问题，否则会丢失两条链路。故只应该是B/1端口到存储交换机链路有问题。

3）     解决问题

通过重新插拔B/1光纤模块并检查光纤解决问题

lzp_0004

luxadm -e port

Found path to 5 HBA ports

/devices/pci@8,700000/SUNW,qlc@4/fp@0,0:devctl                     NOT CONNECTED
/devices/pci@8,700000/SUNW,qlc@4,1/fp@0,0:devctl                   CONNECTED
/devices/pci@8,700000/SUNW,qlc@5/fp@0,0:devctl                     NOT CONNECTED
/devices/pci@8,700000/SUNW,qlc@5,1/fp@0,0:devctl                   CONNECTED
/devices/pci@8,600000/SUNW,qlc@2/fp@0,0:devctl                     CONNECTED

luxadm -e dump_map /devices/pci@8,700000/SUNW,qlc@4,1/fp@0,0:devctl
Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type
0    10300   0         202200a0b8472640 200200a0b8472640 0x0  (Disk device)
1    10700   0         201300a0b8472640 200200a0b8472640 0x0  (Disk device)
2    10d00   0         210100e08ba75485 200100e08ba75485 0x1f (Unknown Type)
3    10a00   0         210100e08ba644ca 200100e08ba644ca 0x1f (Unknown Type,Host Bus Adapter)

luxadm probe
Found Enclosure:
SUNWGS INT FCBPL   Name:FCloop   Node WWN:5080020000224ea0   Logical Path:/dev/es/ses0
Jun 29 13:13:07 OBB genunix: WARNING: Unexpected token '1' on line 27 of /kernel/drv/st.conf
Jun 29 13:13:07 OBB genunix: WARNING: Unexpected token 'STK' on line 68 of /kernel/drv/st.conf
Jun 29 13:13:07 OBB genunix: WARNING: Unexpected token '1' on line 68 of /kernel/drv/st.conf
Jun 29 13:13:07 OBB genunix: WARNING: add_spec: No major number for qla2300
Jun 29 13:13:07 OBB last message repeated 5 times
Jun 29 13:13:07 OBB genunix: WARNING: Unexpected token '1' on line 27 of /kernel/drv/st.conf
Jun 29 13:13:07 OBB genunix: WARNING: Unexpected token 'STK' on line 68 of /kernel/drv/st.conf
Jun 29 13:13:07 OBB genunix: WARNING: Unexpected token '1' on line 68 of /kernel/drv/st.conf
Jun 29 13:13:07 OBB genunix: WARNING: add_spec: No major number for qla2300

Found Fibre Channel device(s):
  Node WWN:200200a0b8472640  Device Typeisk device
    Logical Path:/dev/rdsk/c6t600A0B80004726400000019A4833BD74d0s2


luxadm disp /dev/rdsk/c6t600A0B80004726400000019A4833BD74d0s2
Jun 29 13:16:18 OBB genunix: WARNING: Unexpected token '1' on line 27 of /kernel/drv/st.conf
Jun 29 13:16:18 OBB genunix: WARNING: Unexpected token 'STK' on line 68 of /kernel/drv/st.conf
Jun 29 13:16:18 OBB genunix: WARNING: Unexpected token '1' on line 68 of /kernel/drv/st.conf
Jun 29 13:16:18 OBB genunix: WARNING: add_spec: No major number for qla2300
Jun 29 13:16:18 OBB last message repeated 5 times
Jun 29 13:16:18 OBB genunix: WARNING: Unexpected token '1' on line 27 of /kernel/drv/st.conf
Jun 29 13:16:18 OBB genunix: WARNING: Unexpected token 'STK' on line 68 of /kernel/drv/st.conf
DEVICE PROPERTIES for disk: /dev/rdsk/c6t600A0B80004726400000019A4833BD74d0s2
  Vendor:               STK     
  Product ID:           FLEXLINE 380   
  Revision:             0619
  Serial Num:           SP80455634  
  Unformatted capacity: 30720.000 MBytes
  Write Cache:          Enabled
  Read Cache:           Enabled
    Minimum prefetch:   0x0
    Maximum prefetch:   0x0
  Device Type:          Disk device
  Path(s):

  /dev/rdsk/c6t600A0B80004726400000019A4833BD74d0s2
  /devices/scsi_vhci/ssd@g600a0b80004726400000019a4833bd74:c,raw
   Controller           /devices/pci@8,700000/SUNW,qlc@4,1/fp@0,0
    Device Address              201300a0b8472640,0

    Host controller port WWN    210100e08ba644ca
    Class                       secondary
    State                       STANDBY
   Controller           /devices/pci@8,700000/SUNW,qlc@4,1/fp@0,0
    Device Address              202200a0b8472640,0
    Host controller port WWN    210100e08ba644ca
    Class                       primary
    State                       ONLINE
   Controller           /devices/pci@8,700000/SUNW,qlc@5,1/fp@0,0
    Device Address              201200a0b8472640,0
    Host controller port WWN    210100e08ba63dcc
    Class                       primary
    State                       ONLINE
   Controller           /devices/pci@8,700000/SUNW,qlc@5,1/fp@0,0
    Device Address              202300a0b8472640,0
    Host controller port WWN    210100e08ba63dcc
    Class                       secondary
    State                       STANDBY


机器是v890  存储STK6540  vcs群集的备机