help！！sun cluster3.0 did共享盘不一致

abbend

两台sun 480+2台sun 3510，solaris8、cluster3.0、veritas2.2
今天一台3510挂了，重建了logic-driver，db2主机使用devfsadm就识别出来了，而db1一直不能识别阵列，直到关机重启后才识别。但是新问题又出现了，执行scdidadm -r和scgdevs后发现新识别的阵列硬盘在两台主机上did分别为
3        db2:/dev/rdsk/c7t40d0     /dev/did/rdsk/d3
5        db1:/dev/rdsk/c7t44d0     /dev/did/rdsk/d5  

而另一台识别是正常的
4        db1:/dev/rdsk/c6t46d0     /dev/did/rdsk/d4     
4        db2:/dev/rdsk/c6t40d0     /dev/did/rdsk/d4
请问各位大虾，这个是什么原因啊？如何修改？

部分信息如下：
root@db1 # scdidadm -r
root@db1 # scdidadm -L
1        db1:/dev/rdsk/c0t0d0      /dev/did/rdsk/d1     
2        db1:/dev/rdsk/c1t1d0      /dev/did/rdsk/d2     
3        db2:/dev/rdsk/c7t40d0     /dev/did/rdsk/d3     
4        db1:/dev/rdsk/c6t46d0     /dev/did/rdsk/d4     
4        db2:/dev/rdsk/c6t40d0     /dev/did/rdsk/d4     
5        db1:/dev/rdsk/c7t44d0     /dev/did/rdsk/d5     
6        db2:/dev/rdsk/c0t0d0      /dev/did/rdsk/d6     
7        db2:/dev/rdsk/c1t1d0      /dev/did/rdsk/d7     
8        db2:/dev/rdsk/c1t0d0      /dev/did/rdsk/d8     
10       db1:/dev/rdsk/c1t0d0      /dev/did/rdsk/d10   
8190     db2:/dev/rmt/0            /dev/did/rmt/2      
8191     db1:/dev/rmt/0            /dev/did/rmt/1      
root@db1 # scdidadm -l
1        db1:/dev/rdsk/c0t0d0      /dev/did/rdsk/d1     
2        db1:/dev/rdsk/c1t1d0      /dev/did/rdsk/d2     
4        db1:/dev/rdsk/c6t46d0     /dev/did/rdsk/d4     
5        db1:/dev/rdsk/c7t44d0     /dev/did/rdsk/d5     
10       db1:/dev/rdsk/c1t0d0      /dev/did/rdsk/d10   
8191     db1:/dev/rmt/0            /dev/did/rmt/1      
root@db1 # scgdevs
Configuring DID devices
Could not open /dev/rdsk/c0t0d0s2 to verify device id.
Device busy
Configuring the /dev/global directory (global devices)
obtaining access to all attached disks

东方蜘蛛

回复 1# abbend


# devfsadm -C
# scdidadm -C
# devfsadm
# scdidadm -r
# scgdevs
按照这个步骤看看结果

东方蜘蛛

在两个节点都要执行以上命令！

wait空白

关注一下。

wait空白

楼主回复一下信息啊。。。


你这不一致，是不是在执行什么命令的时候中断掉导致的？

abbend

# devfsadm -C
# scdidadm -C
# devfsadm
# scdidadm -r
# scgdevs

这些命令分别在两个节点都执行过了，情况还是这样。
当时就是删除了一台3510的logic-driver，然后重建的，db2主机使用devfsadm就识别出来了，而db1一直不能识别阵列，直到关机重启后才识别。

主机没有重启前的部分信息如下：
root@db1 #
root@db1 # devfsadm -Cv
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:a
devfsadm[19686]: verbose: removing file: /dev/dsk/c7t44d0s0
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:b
devfsadm[19686]: verbose: removing file: /dev/dsk/c7t44d0s1
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:c
devfsadm[19686]: verbose: removing file: /dev/dsk/c7t44d0s2
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:d
devfsadm[19686]: verbose: removing file: /dev/dsk/c7t44d0s3
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:e
devfsadm[19686]: verbose: removing file: /dev/dsk/c7t44d0s4
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:f
devfsadm[19686]: verbose: removing file: /dev/dsk/c7t44d0s5
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:g
devfsadm[19686]: verbose: removing file: /dev/dsk/c7t44d0s6
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:h
devfsadm[19686]: verbose: removing file: /dev/dsk/c7t44d0s7
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:a,raw
devfsadm[19686]: verbose: removing file: /dev/rdsk/c7t44d0s0
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:b,raw
devfsadm[19686]: verbose: removing file: /dev/rdsk/c7t44d0s1
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:c,raw
devfsadm[19686]: verbose: removing file: /dev/rdsk/c7t44d0s2
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:d,raw
devfsadm[19686]: verbose: removing file: /dev/rdsk/c7t44d0s3
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:e,raw
devfsadm[19686]: verbose: removing file: /dev/rdsk/c7t44d0s4
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:f,raw
devfsadm[19686]: verbose: removing file: /dev/rdsk/c7t44d0s5
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:g,raw
devfsadm[19686]: verbose: removing file: /dev/rdsk/c7t44d0s6
devfsadm[19686]: verbose: removing file: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0/ssd@w256000c0ffc82499,0:h,raw
devfsadm[19686]: verbose: removing file: /dev/rdsk/c7t44d0s7
root@db1 #
root@db1 #
root@db1 # format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
          /pci@9,600000/SUNW,qlc@2/fp@0,0/ssd@w500000e01086bae1,0
       1. c1t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
          /pci@9,600000/SUNW,qlc@2/fp@0,0/ssd@w21000000876d1851,0
       2. c6t46d0 <SUN-StorEdge3510-327R cyl 61269 alt 2 hd 64 sec 255>
          /pci@8,700000/SUNW,qlc@2/fp@0,0/ssd@w266000c0ffe7d2b5,0
Specify disk (enter its number): ^D
root@db1 #
root@db1 #
root@db1 # cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t0d0                 CD-ROM       connected    configured   unknown
c1                             fc-private   connected    configured   unknown
c1::21000000876d1851           disk         connected    configured   unknown
c1::500000e01086bae1           disk         connected    configured   unknown
c2                             scsi-bus     connected    configured   unknown
c2::rmt/0                      tape         connected    configured   unknown
c3                             scsi-bus     connected    unconfigured unknown
c6                             fc-private   connected    configured   unknown
c6::266000c0ffe7d2b5           disk         connected    configured   unknown
c7                             fc-private   connected    configured   unknown
c7::256000c0ffc82499           disk         connected    configured   unknown
root@db1 # cfgadm -c configure c7
root@db1 #luxadm -e forcelip c7
root@db1 # format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
          /pci@9,600000/SUNW,qlc@2/fp@0,0/ssd@w500000e01086bae1,0
       1. c1t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
          /pci@9,600000/SUNW,qlc@2/fp@0,0/ssd@w21000000876d1851,0
       2. c6t46d0 <SUN-StorEdge3510-327R cyl 61269 alt 2 hd 64 sec 255>
          /pci@8,700000/SUNW,qlc@2/fp@0,0/ssd@w266000c0ffe7d2b5,0
Specify disk (enter its number): ^D
root@db1 #
root@db1 #
root@db1 # luxadm probe
No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):
  Node WWN:500000e01086bae0  Device Typeisk device
    Logical Path:/dev/rdsk/c1t0d0s2
  Node WWN:20000000876d1851  Device Typeisk device
    Logical Path:/dev/rdsk/c1t1d0s2
  Node WWN:206000c0ff07d2b5  Device Typeisk device
    Logical Path:/dev/rdsk/c6t46d0s2
root@db1 #
root@db1 #
root@db1 # luxadm -e port

Found path to 3 HBA ports

/devices/pci@8,700000/SUNW,qlc@2/fp@0,0:devctl                     CONNECTED
/devices/pci@8,700000/SUNW,qlc@3/fp@0,0:devctl                     CONNECTED
/devices/pci@9,600000/SUNW,qlc@2/fp@0,0:devctl                     CONNECTED

zyl555

正常的是t40和t46，不正常的是t40和t44，看看映射

abbend

zyl555 发表于 2012-05-25 23:48
正常的是t40和t46，不正常的是t40和t44，看看映射

映射怎么看？能手动修改吗？

zyl555

从默认的SE3510来看，t40和t44对应的是host0和host4，而你有问题的Lun映射到了t46（这个应该是host5）。
你要登录到存储上，把有问题的lun重新映射到t44上，也就是host4。

回复 8# abbend


   

abbend

有问题的3510配置，0、4口分别接2台主机，map都是重新建的，原来的因为重新建logic-driver已经unmap了

* channels

Ch  Type    Media   Speed   Width  PID / SID
--------------------------------------------
0  Host    FC(L)   2G      Serial  40 / N/A
1  Host    FC(L)   2G      Serial  42 / N/A
2  DRV+RCC FC(L)   2G      Serial  14 / 15
3  DRV+RCC FC(L)   2G      Serial  14 / 15
4  Host    FC(L)   2G      Serial  44 / N/A
5  Host    FC(L)   2G      Serial  N/A / 46
6  Host    LAN     N/A     Serial  N/A / N/A


* lun-maps

Ch Tgt LUN   ld/lv   ID-Partition  Assigned  Filter Map
---------------------------------------------------------------------
0  40   0   ld0     73F15B57-00   Primary   
1  42   0   ld0     73F15B57-00   Primary   
4  44   0   ld0     73F15B57-00   Primary   

* port-WWNs

Ch  Id   WWPN
-------------------------
0  40   216000C0FF882499
1  42   226000C0FFA82499
4  44   256000C0FFC82499
5  46   266000C0FFE82499

root@db1 # luxadm probe
No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):
  Node WWN:500000e01086bae0  Device Typeisk device
    Logical Path:/dev/rdsk/c1t0d0s2
  Node WWN:20000000876d1851  Device Typeisk device
    Logical Path:/dev/rdsk/c1t1d0s2
  Node WWN:206000c0ff07d2b5  Device Typeisk device
    Logical Path:/dev/rdsk/c6t46d0s2
  Node WWN:206000c0ff082499  Device Typeisk device
    Logical Path:/dev/rdsk/c7t44d0s2