M5000突然宕机，请大侠帮忙分析分析！ZFS文件系统。
root@ngsetapp3 # cat /var/adm/messagesMay 13 12:55:43 ngsetapp3 ip: ipsec_check_inbound_policy: Policy Failure for the incoming packet (not secure); Source 192.168.224.001, Destination 192.168.224.002.
May 13 16:05:42 ngsetapp3 ip: ipsec_check_inbound_policy: Policy Failure for the incoming packet (not secure); Source 192.168.224.001, Destination 192.168.224.002.
May 14 14:19:12 ngsetapp3 su: 'su isma' failed for zhjs on /dev/pts/62
May 14 21:20:33 ngsetapp3 in.mpathd: Cannot meet requested failure detection time of 10000 ms on (inet e1000g4) new failure detection time for group "sc_ipmp0" is 79548 ms
May 14 21:21:34 ngsetapp3 in.mpathd: Improved failure detection time 39774 ms on (inet e1000g4) for group "sc_ipmp0"
May 14 21:21:35 ngsetapp3 in.mpathd: Improved failure detection time 19887 ms on (inet e1000g4) for group "sc_ipmp0"
May 14 21:21:37 ngsetapp3 in.mpathd: Improved failure detection time 10000 ms on (inet e1000g4) for group "sc_ipmp0"
May 14 21:56:08 ngsetapp3 in.mpathd: All Interfaces in group sc_ipmp0 have failed
May 14 21:56:08 ngsetapp3 Cluster.PNM: sc_ipmp0: state transition from OK to DOWN.
May 14 21:56:19 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs status on node ngsetapp3 change to R_FM_DEGRADED
May 14 21:56:19 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs status msg on node ngsetapp3 change to <IPMP Failure.>
May 14 21:56:19 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs in resource group zfs-isma-rg has requested failover of the resource group on ngsetapp3.
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource group zfs-isma-rg state on node ngsetapp3 change to RG_PENDING_OFFLINE
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: launching method <hafoip_monitor_stop> for resource <logicalhost-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, timeout <300> seconds
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: launching method <hastorageplus_monitor_stop> for resource <zfs-muissp-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, timeout <90> seconds
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: launching method <hastorageplus_monitor_stop> for resource <zfs-muis-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, timeout <90> seconds
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: launching method <hastorageplus_monitor_stop> for resource <zfs-muis-bak>, resource group <zfs-isma-rg>, node <ngsetapp3>, timeout <90> seconds
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: launching method <hastorageplus_monitor_stop> for resource <zfs-isma-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, timeout <90> seconds
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: method <hafoip_monitor_stop> completed successfully for resource <logicalhost-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, time used: 0% of timeout <300 seconds>
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs state on node ngsetapp3 change to R_ONLINE_UNMON
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs state on node ngsetapp3 change to R_STOPPING
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: launching method <hafoip_stop> for resource <logicalhost-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, timeout <300> seconds
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs status on node ngsetapp3 change to R_FM_UNKNOWN
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs status msg on node ngsetapp3 change to <Stopping>
May 14 21:56:20 ngsetapp3 ip: TCP_IOC_ABORT_CONN: local = 010.154.051.101:0, remote = 000.000.000.000:0, start = -2, end = 6
May 14 21:56:20 ngsetapp3 ip: TCP_IOC_ABORT_CONN: aborted 116 connections
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs status on node ngsetapp3 change to R_FM_OFFLINE
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs status msg on node ngsetapp3 change to <LogicalHostname offline.>
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: method <hafoip_stop> completed successfully for resource <logicalhost-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, time used: 0% of timeout <300 seconds>
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs state on node ngsetapp3 change to R_OFFLINE
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: method <hastorageplus_monitor_stop> completed successfully for resource <zfs-muis-bak>, resource group <zfs-isma-rg>, node <ngsetapp3>, time used: 0% of timeout <90 seconds>
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak state on node ngsetapp3 change to R_ONLINE_UNMON
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: method <hastorageplus_monitor_stop> completed successfully for resource <zfs-muis-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, time used: 0% of timeout <90 seconds>
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: method <hastorageplus_monitor_stop> completed successfully for resource <zfs-muissp-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, time used: 0% of timeout <90 seconds>
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: method <hastorageplus_monitor_stop> completed successfully for resource <zfs-isma-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, time used: 0% of timeout <90 seconds>
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak status on node ngsetapp3 change to R_FM_UNKNOWN
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak status msg on node ngsetapp3 change to <Stopping>
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: launching method <hastorageplus_postnet_stop> for resource <zfs-muis-bak>, resource group <zfs-isma-rg>, node <ngsetapp3>, timeout <1800> seconds
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs state on node ngsetapp3 change to R_ONLINE_UNMON
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs state on node ngsetapp3 change to R_ONLINE_UNMON
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: launching method <hastorageplus_postnet_stop> for resource <zfs-isma-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, timeout <1800> seconds
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: launching method <hastorageplus_postnet_stop> for resource <zfs-muissp-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, timeout <1800> seconds
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs status on node ngsetapp3 change to R_FM_UNKNOWN
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs status msg on node ngsetapp3 change to <Stopping>
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs status on node ngsetapp3 change to R_FM_UNKNOWN
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs status msg on node ngsetapp3 change to <Stopping>
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs state on node ngsetapp3 change to R_ONLINE_UNMON
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: launching method <hastorageplus_postnet_stop> for resource <zfs-muis-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, timeout <1800> seconds
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs status on node ngsetapp3 change to R_FM_UNKNOWN
May 14 21:56:20 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs status msg on node ngsetapp3 change to <Stopping>
May 14 21:56:24 ngsetapp3 in.mpathd: NIC repair detected on e1000g4 of group sc_ipmp0
May 14 21:56:24 ngsetapp3 in.mpathd: Successfully failed back to NIC e1000g4
May 14 21:56:24 ngsetapp3 in.mpathd: At least 1 interface (e1000g4) of group sc_ipmp0 has repaired
May 14 21:56:24 ngsetapp3 Cluster.PNM: sc_ipmp0: state transition from DOWN to OK.
May 14 21:56:24 ngsetapp3 in.mpathd: Successfully failed over from NIC e1000g6 to NIC e1000g4
May 14 21:56:43 ngsetapp3 Cluster.RGM.global.rgmd: method <hastorageplus_postnet_stop> completed successfully for resource <zfs-isma-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, time used: 1% of timeout <1800 seconds>
May 14 21:56:43 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs state on node ngsetapp3 change to R_OFFLINE
May 14 21:56:43 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs status on node ngsetapp3 change to R_FM_OFFLINE
May 14 21:56:43 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs status msg on node ngsetapp3 change to <>
May 14 22:00:07 ngsetapp3 Cluster.RGM.global.rgmd: method <hastorageplus_postnet_stop> completed successfully for resource <zfs-muis-bak>, resource group <zfs-isma-rg>, node <ngsetapp3>, time used: 12% of timeout <1800 seconds>
May 14 22:00:07 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak state on node ngsetapp3 change to R_OFFLINE
May 14 22:00:07 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak status on node ngsetapp3 change to R_FM_OFFLINE
May 14 22:00:07 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak status msg on node ngsetapp3 change to <>
May 14 22:05:03 ngsetapp3 Cluster.RGM.global.rgmd: method <hastorageplus_postnet_stop> completed successfully for resource <zfs-muissp-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, time used: 29% of timeout <1800 seconds>
May 14 22:05:03 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs state on node ngsetapp3 change to R_OFFLINE
May 14 22:05:03 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs status on node ngsetapp3 change to R_FM_OFFLINE
May 14 22:05:03 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs status msg on node ngsetapp3 change to <>
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: method <hastorageplus_postnet_stop> completed successfully for resource <zfs-muis-rs>, resource group <zfs-isma-rg>, node <ngsetapp3>, time used: 30% of timeout <1800 seconds>
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs state on node ngsetapp3 change to R_OFFLINE
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs status on node ngsetapp3 change to R_FM_OFFLINE
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs status msg on node ngsetapp3 change to <>
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource group zfs-isma-rg state on node ngsetapp3 change to RG_OFFLINE
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource group zfs-isma-rg state on node ngsetapp4 change to RG_PENDING_ONLINE
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs status on node ngsetapp4 change to R_FM_UNKNOWN
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs status msg on node ngsetapp4 change to <Starting>
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs status on node ngsetapp4 change to R_FM_UNKNOWN
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs status msg on node ngsetapp4 change to <Starting>
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak status on node ngsetapp4 change to R_FM_UNKNOWN
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak status msg on node ngsetapp4 change to <Starting>
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs status on node ngsetapp4 change to R_FM_UNKNOWN
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs status msg on node ngsetapp4 change to <Starting>
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs status on node ngsetapp4 change to R_FM_UNKNOWN
May 14 22:05:26 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs status msg on node ngsetapp4 change to <Starting>
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs state on node ngsetapp4 change to R_ONLINE_UNMON
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs state on node ngsetapp4 change to R_ONLINE_UNMON
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs state on node ngsetapp4 change to R_STARTING
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak state on node ngsetapp4 change to R_ONLINE_UNMON
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs state on node ngsetapp4 change to R_ONLINE_UNMON
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs status on node ngsetapp4 change to R_FM_ONLINE
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs status msg on node ngsetapp4 change to <>
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs status on node ngsetapp4 change to R_FM_ONLINE
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs status msg on node ngsetapp4 change to <>
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak status on node ngsetapp4 change to R_FM_ONLINE
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak status msg on node ngsetapp4 change to <>
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs status on node ngsetapp4 change to R_FM_ONLINE
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs status msg on node ngsetapp4 change to <>
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs status on node ngsetapp4 change to R_FM_ONLINE
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs status msg on node ngsetapp4 change to <LogicalHostname online.>
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs state on node ngsetapp4 change to R_ONLINE_UNMON
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-isma-rs state on node ngsetapp4 change to R_ONLINE
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-rs state on node ngsetapp4 change to R_ONLINE
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muis-bak state on node ngsetapp4 change to R_ONLINE
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource zfs-muissp-rs state on node ngsetapp4 change to R_ONLINE
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource logicalhost-rs state on node ngsetapp4 change to R_ONLINE
May 14 22:06:22 ngsetapp3 Cluster.RGM.global.rgmd: resource group zfs-isma-rg state on node ngsetapp4 change to RG_ONLINE 回复 1# yulemi
LZ，能否描述一下宕机的现象？
是无法登陆呢？
还是无法操作？
还是其他？
Cleo_Z 发表于 2013-05-17 09:21 static/image/common/back.gif
回复 1# yulemi
无法登陆，业务发生了切换 你IPMP到网关不通了，可不就是来回切换了。检查你网关是不是正常，是不是有人在做网络设备割接了 4L正解
页:
[1]