- 论坛徽章:
- 0
|
6个RHEL主机节点通过GFS共享2个存储,对应的2个GFS文件系统(即/SANData/Production/SANData/Backup)。6个主机节点中,其中有一个节点换完主机硬件后,重启只能挂在一个GFS文件系统(即/SANData/Backup),另一个无法挂着,通过手工mount显示收入输出错误,错误。但是6个节点的cman、clvmd和gfs都启动正常。不知道是什么问题呢?
请指点
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 28G 9.2G 18G 35% /
/dev/sda2 95G 41G 50G 46% /var
tmpfs 63G 0 63G 0% /dev/shm
df: `/SANData/Production': Input/output error
/dev/mapper/CUGFSa_Backup-KVM_Backup
10T 221G 9.8T 3% /SANData/Backup
[root@KVMHost-B8AC6FD292E0 ~]# tailf /var/log/messages
Jan 15 01:15:53 KVMHost-B8AC6FD292E0 kernel: br0: topology change detected, propagating
Jan 15 01:15:53 KVMHost-B8AC6FD292E0 kernel: br0: port 2(vnet0) entering forwarding state
Jan 15 01:15:53 KVMHost-B8AC6FD292E0 kernel: br0: port 2(vnet0) entering disabled state
Jan 15 01:15:53 KVMHost-B8AC6FD292E0 kernel: device vnet0 left promiscuous mode
Jan 15 01:15:53 KVMHost-B8AC6FD292E0 kernel: br0: port 2(vnet0) entering disabled state
Jan 15 01:15:54 KVMHost-B8AC6FD292E0 init: Id "x" respawning too fast: disabled for 5 minutes
Jan 15 01:19:05 KVMHost-B8AC6FD292E0 ntpd[14094]: synchronized to LOCAL(0), stratum 10
Jan 15 01:19:05 KVMHost-B8AC6FD292E0 ntpd[14094]: kernel time sync enabled 0001
Jan 15 01:20:06 KVMHost-B8AC6FD292E0 ntpd[14094]: synchronized to 218.75.4.130, stratum 2
Jan 15 01:20:56 KVMHost-B8AC6FD292E0 init: Id "x" respawning too fast: disabled for 5 minutes
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: INFO: task gfs_glockd:13678 blocked for more than 120 seconds.
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: gfs_glockd D ffffffff80150939 0 13678 1299 13686 13677 (L-TLB)
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: gfs_glockd D ffffffff80150939 0 13678 1299 13686 13677 (L-TLB)
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: ffff81086a135c20 0000000000000046 0000000000000001 ffff81107c303910
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: ffff81086a135c80 000000000000000a ffff81207db57820 ffff81089c23a860
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: 0000008581a6bb46 000000000000b8bf ffff81207db57a08 000000067c765250
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: Call Trace:
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8006e1d7>] do_gettimeofday+0x40/0x90
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff80028b44>] sync_page+0x0/0x43
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff800637ea>] io_schedule+0x3f/0x67
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff80028b82>] sync_page+0x3e/0x43
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff80063a16>] __wait_on_bit+0x40/0x6e
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff80035214>] wait_on_page_bit+0x6c/0x72
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff800a0b8d>] wake_bit_function+0x0/0x23
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff80047fd4>] pagevec_lookup_tag+0x1a/0x21
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8004a0fc>] wait_on_page_writeback_range+0x62/0x12e
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8005acff>] do_writepages+0x29/0x2f
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8004fb1a>] __filemap_fdatawrite_range+0x50/0x5b
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff88793c07>] :gfs:gfs_sync_buf+0x36/0x7c
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8879fe70>] :gfs:inode_go_sync+0x5b/0xc4
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8879f16e>] :gfs:gfs_glock_drop_th+0xbe/0x187
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8879d543>] :gfs:run_queue+0x12e/0x35e
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8879e0c8>] :gfs:unlock_on_glock+0x1b/0x24
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8879d389>] :gfs:gfs_reclaim_glock+0xed/0x179
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff88793218>] :gfs:gfs_glockd+0x0/0xd1
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8879322d>] :gfs:gfs_glockd+0x15/0xd1
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8003296e>] kthread+0xfe/0x132
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff80032870>] kthread+0x0/0x132
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11
Jan 15 01:23:34 KVMHost-B8AC6FD292E0 kernel: |
|