- 论坛徽章:
- 0
|
系统报软锁的问题,此问题表现为,在这台服务器中将NFS挂载的目录拷贝大文件到这服务器硬盘时,出现此错误,只能硬重启。
怀疑是否nfs的问题?- BUG: soft lockup - CPU#5 stuck for 67s! [sudo:2965]
- Nov 5 18:05:16 cc02 kernel: Modules linked in: xfs exportfs drbd(U) libcrc32c nfs lockd fscache auth_rpcgss nfs_acl sunrpc xt_CHECKSUM iptable_mangle ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT iptable_filter ip_tables bridge cpufreq_ondemand freq_table pcc_cpufreq bonding 8021q garp stp llc ipv6 openvswitch(U) vhost_net macvtap macvlan tun kvm_intel kvm microcode serio_raw power_meter sg iTCO_wdt iTCO_vendor_support hpilo hpwdt bnx2 e1000e i7core_edac edac_core shpchp ext4 jbd2 mbcache sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix hpsa radeon ttm drm_kms_helper drm i2c_algo_bit i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
- Nov 5 18:05:16 cc02 kernel: CPU 5
- Nov 5 18:05:16 cc02 kernel: Modules linked in: xfs exportfs drbd(U) libcrc32c nfs lockd fscache auth_rpcgss nfs_acl sunrpc xt_CHECKSUM iptable_mangle ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT iptable_filter ip_tables bridge cpufreq_ondemand freq_table pcc_cpufreq bonding 8021q garp stp llc ipv6 openvswitch(U) vhost_net macvtap macvlan tun kvm_intel kvm microcode serio_raw power_meter sg iTCO_wdt iTCO_vendor_support hpilo hpwdt bnx2 e1000e i7core_edac edac_core shpchp ext4 jbd2 mbcache sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix hpsa radeon ttm drm_kms_helper drm i2c_algo_bit i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
- Nov 5 18:05:16 cc02 kernel:
- Nov 5 18:05:16 cc02 kernel: Pid: 2965, comm: sudo Not tainted 2.6.32-358.123.2.openstack.el6.x86_64 #1 HP ProLiant DL380 G7
- Nov 5 18:05:16 cc02 kernel: RIP: 0010:[<ffffffff810d4737>] [<ffffffff810d4737>] audit_log_start+0xc7/0x430
- Nov 5 18:05:16 cc02 kernel: RSP: 0018:ffff880037325998 EFLAGS: 00000282
- Nov 5 18:05:16 cc02 kernel: RAX: fffffffffffb0383 RBX: ffff880037325a48 RCX: 0000000000000141
- Nov 5 18:05:16 cc02 kernel: RDX: 000000000000ea60 RSI: 000000000000ea60 RDI: 0000000000000140
- Nov 5 18:05:16 cc02 kernel: RBP: ffffffff8100bb8e R08: fffffffeff6f2970 R09: 00000000ffffffff
- Nov 5 18:05:16 cc02 kernel: R10: 0000000000000000 R11: 0000000000000000 R12: dead000000200200
- Nov 5 18:05:16 cc02 kernel: R13: 0000000000000000 R14: 0000000000000286 R15: ffff8800373258f8
- Nov 5 18:05:16 cc02 kernel: FS: 00007f48e04c57a0(0000) GS:ffff88042e440000(0000) knlGS:0000000000000000
- Nov 5 18:05:16 cc02 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
- Nov 5 18:05:16 cc02 kernel: CR2: 0000000000481000 CR3: 00000000370d0000 CR4: 00000000000007e0
- Nov 5 18:05:16 cc02 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
- Nov 5 18:05:16 cc02 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
- Nov 5 18:05:16 cc02 kernel: Process sudo (pid: 2965, threadinfo ffff880037324000, task ffff8801d06d8ae0)
- Nov 5 18:05:16 cc02 kernel: Stack:
- Nov 5 18:05:16 cc02 kernel: ffff8800373259b8 000000000000ea60 000000d000000451 0000000000000000
- Nov 5 18:05:16 cc02 kernel: <d> ffff8800373259d8 ffff8807d192e440 0000000000000000 ffff8801d06d8ae0
- Nov 5 18:05:16 cc02 kernel: <d> ffffffff81063990 dead000000100100 dead000000200200 ffff880037ae0810
- Nov 5 18:05:16 cc02 kernel: Call Trace:
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81063990>] ? default_wake_function+0x0/0x20
- Nov 5 18:05:16 cc02 kernel: [<ffffffff8114497a>] ? handle_mm_fault+0x23a/0x310
- Nov 5 18:05:16 cc02 kernel: [<ffffffff810d4caa>] ? audit_log_common_recv_msg+0x6a/0xf0
- Nov 5 18:05:16 cc02 kernel: [<ffffffff8104759c>] ? __do_page_fault+0x1ec/0x480
- Nov 5 18:05:16 cc02 kernel: [<ffffffff810d50f8>] ? audit_receive+0x3c8/0xd90
- Nov 5 18:05:16 cc02 kernel: [<ffffffff8111a377>] ? unlock_page+0x27/0x30
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81055ad3>] ? __wake_up+0x53/0x70
- Nov 5 18:05:16 cc02 kernel: [<ffffffff814748db>] ? netlink_unicast+0x2db/0x320
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81475350>] ? netlink_sendmsg+0x2c0/0x3d0
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81436b33>] ? sock_sendmsg+0x123/0x150
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
- Nov 5 18:05:16 cc02 kernel: [<ffffffff8112c093>] ? __alloc_pages_nodemask+0x113/0x8d0
- Nov 5 18:05:16 cc02 kernel: [<ffffffff811a2590>] ? mntput_no_expire+0x30/0x110
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81436e49>] ? sys_sendto+0x139/0x190
- Nov 5 18:05:16 cc02 kernel: [<ffffffff811a2590>] ? mntput_no_expire+0x30/0x110
- Nov 5 18:05:16 cc02 kernel: [<ffffffff810dcd87>] ? audit_syscall_entry+0x1d7/0x200
- Nov 5 18:05:16 cc02 kernel: [<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
- Nov 5 18:05:16 cc02 kernel: Code: 41 89 c5 48 89 d3 49 89 d6 8b 05 09 43 9d 00 8b 15 5f 43 9d 00 8b 0d 59 36 e0 00 85 c0 48 63 f2 0f 84 6c 01 00 00 41 8d 7c 05 00 <39> cf 0f 83 5f 01 00 00 45 85 ff 0f 84 98 00 00 00 85 d2 0f 84
- Nov 5 18:05:16 cc02 kernel: Call Trace:
- Nov 5 18:05:16 cc02 kernel: [<ffffffff810d47db>] ? audit_log_start+0x16b/0x430
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81063990>] ? default_wake_function+0x0/0x20
- Nov 5 18:05:16 cc02 kernel: [<ffffffff8114497a>] ? handle_mm_fault+0x23a/0x310
- Nov 5 18:05:16 cc02 kernel: [<ffffffff810d4caa>] ? audit_log_common_recv_msg+0x6a/0xf0
- Nov 5 18:05:16 cc02 kernel: [<ffffffff8104759c>] ? __do_page_fault+0x1ec/0x480
- Nov 5 18:05:16 cc02 kernel: [<ffffffff810d50f8>] ? audit_receive+0x3c8/0xd90
- Nov 5 18:05:16 cc02 kernel: [<ffffffff8111a377>] ? unlock_page+0x27/0x30
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81055ad3>] ? __wake_up+0x53/0x70
- Nov 5 18:05:16 cc02 kernel: [<ffffffff814748db>] ? netlink_unicast+0x2db/0x320
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81475350>] ? netlink_sendmsg+0x2c0/0x3d0
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81436b33>] ? sock_sendmsg+0x123/0x150
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
- Nov 5 18:05:16 cc02 kernel: [<ffffffff8112c093>] ? __alloc_pages_nodemask+0x113/0x8d0
- Nov 5 18:05:16 cc02 kernel: [<ffffffff811a2590>] ? mntput_no_expire+0x30/0x110
- Nov 5 18:05:16 cc02 kernel: [<ffffffff81436e49>] ? sys_sendto+0x139/0x190
- Nov 5 18:05:16 cc02 kernel: [<ffffffff811a2590>] ? mntput_no_expire+0x30/0x110
- Nov 5 18:05:16 cc02 kernel: [<ffffffff810dcd87>] ? audit_syscall_entry+0x1d7/0x200
- Nov 5 18:05:16 cc02 kernel: [<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
- Nov 5 18:06:34 cc02 lrmd[16463]: warning: child_timeout_callback: p_drbd_mysql_monitor_30000 process (PID 6172) timed out
- Nov 5 18:06:34 cc02 lrmd[16463]: warning: operation_finished: p_drbd_mysql_monitor_30000:6172 - timed out after 20000ms
- Nov 5 18:06:34 cc02 crmd[16466]: error: process_lrm_event: LRM operation p_drbd_mysql_monitor_30000 (76) Timed Out (timeout=20000ms)
- Nov 5 18:06:34 cc02 attrd[16464]: notice: attrd_cs_dispatch: Update relayed from cc01.ss
- Nov 5 18:06:34 cc02 attrd[16464]: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-p_drbd_mysql (1)
- Nov 5 18:06:35 cc02 attrd[16464]: notice: attrd_perform_update: Sent update 30: fail-count-p_drbd_mysql=1
- Nov 5 18:06:35 cc02 attrd[16464]: notice: attrd_cs_dispatch: Update relayed from cc01.ss
- Nov 5 18:06:35 cc02 attrd[16464]: notice: attrd_trigger_update: Sending flush op to all hosts for: last-failure-p_drbd_mysql (1415181995)
- Nov 5 18:06:35 cc02 attrd[16464]: notice: attrd_perform_update: Sent update 33: last-failure-p_drbd_mysql=1415181995
复制代码 |
|