centos6.3系统死机问题，麻烦大家帮忙看看

reblue520

操作系统是centos6.3
硬件
DELL PowerEdge R720 （2U机架式服务器）
4根8GB RDIMM, 1600MT/s,
3块300GB SAS热插拔硬盘 raid 5

服务器上面跑的是mysql数据库，用了两个月没啥问题，8月22号早上突然死机，重启后部分表损坏(修复后正常了，有一个sys_log表直接无法修复，使用备份后可以使用)
让dell售后的人看了下，就电源报错，说是代理商更换了电源，这个是dest软件的截图


下面是/var/logmessages，看的不是太明白，发上来让大家帮忙看看，能看出问题的兄弟麻烦帮忙解答下，谢谢了

==========================================
Aug 17 03:40:01 localhost rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1787" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
Aug 17 23:49:01 localhost auditd[1771]: Audit daemon rotating log files
Aug 20 15:38:57 localhost proftpd[41893]: localhost (192.168.3.11[192.168.3.11]) - FTP session opened.
Aug 20 15:38:59 localhost proftpd[41894]: localhost (192.168.3.11[192.168.3.11]) - FTP session opened.
Aug 21 04:09:02 localhost auditd[1771]: Audit daemon rotating log files
Aug 21 14:59:09 localhost proftpd[9544]: localhost (192.168.3.11[192.168.3.11]) - FTP session opened.
Aug 21 16:49:55 localhost proftpd[10791]: localhost (192.168.3.11[192.168.3.11]) - FTP session opened.
Aug 21 16:50:36 localhost proftpd[10803]: localhost (192.168.3.11[192.168.3.11]) - FTP session opened.
Aug 22 09:33:47 localhost kernel: imklog 5.8.10, log source = /proc/kmsg started.
Aug 22 09:33:47 localhost rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1830" x-info="http://www.rsyslog.com"] start
Aug 22 09:33:47 localhost kernel: Initializing cgroup subsys cpuset
Aug 22 09:33:47 localhost kernel: Initializing cgroup subsys cpu
Aug 22 09:33:47 localhost kernel: Linux version 2.6.32-279.el6.x86_64 (mockbuild@c6b9.bsys.dev.centos.org) (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Fri Jun 22 12:19:21 UTC 2012
Aug 22 09:33:47 localhost kernel: Command line: ro root=UUID=cf5eecde-0866-4e33-b3bf-62c29d831aa2 rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_MD crashkernel=auto LANG=zh_CN.UTF-8 rd_NO_LVM rd_NO_DM rhgb quiet
Aug 22 09:33:47 localhost kernel: KERNEL supported cpus:
Aug 22 09:33:47 localhost kernel:  Intel GenuineIntel
Aug 22 09:33:47 localhost kernel:  AMD AuthenticAMD
Aug 22 09:33:47 localhost kernel:  Centaur CentaurHauls
Aug 22 09:33:47 localhost kernel: BIOS-provided physical RAM map:
Aug 22 09:33:47 localhost kernel: BIOS-e820: 0000000000000000 - 000000000009c000 (usable)
Aug 22 09:33:47 localhost kernel: BIOS-e820: 0000000000100000 - 00000000cd2f0000 (usable)
Aug 22 09:33:47 localhost kernel: BIOS-e820: 00000000cd2f0000 - 00000000cd31c000 (reserved)
Aug 22 09:33:47 localhost kernel: BIOS-e820: 00000000cd31c000 - 00000000cd35b000 (ACPI data)
Aug 22 09:33:47 localhost kernel: BIOS-e820: 00000000cd35b000 - 00000000d0000000 (reserved)
Aug 22 09:33:47 localhost kernel: BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
Aug 22 09:33:47 localhost kernel: BIOS-e820: 00000000fe000000 - 0000000100000000 (reserved)
Aug 22 09:33:47 localhost kernel: BIOS-e820: 0000000100000000 - 0000000830000000 (usable)
Aug 22 09:33:47 localhost kernel: DMI 2.7 present.
Aug 22 09:33:47 localhost kernel: SMBIOS version 2.7 @ 0xFCFB0
Aug 22 09:33:47 localhost kernel: last_pfn = 0x830000 max_arch_pfn = 0x400000000
Aug 22 09:33:47 localhost kernel: x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
Aug 22 09:33:47 localhost kernel: last_pfn = 0xcd2f0 max_arch_pfn = 0x400000000
Aug 22 09:33:47 localhost kernel: Using GB pages for direct mapping
Aug 22 09:33:47 localhost kernel: init_memory_mapping: 0000000000000000-00000000cd2f0000
Aug 22 09:33:47 localhost kernel: init_memory_mapping: 0000000100000000-0000000830000000
Aug 22 09:33:47 localhost kernel: RAMDISK: 3737e000 - 37fef0e9
Aug 22 09:33:47 localhost kernel: ACPI: RSDP 00000000000f0c70 00024 (v02 DELL  )
Aug 22 09:33:47 localhost kernel: ACPI: XSDT 00000000000f0db8 000A4 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: FACP 00000000cd33534c 000F4 (v03 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: DSDT 00000000cd31c000 08029 (v01 DELL   PE_SC3   00000001 INTL 20110211)
Aug 22 09:33:47 localhost kernel: ACPI: FACS 00000000cd337000 00040
Aug 22 09:33:47 localhost kernel: ACPI: APIC 00000000cd334478 001EA (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: SPCR 00000000cd334664 00050 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: HPET 00000000cd3346b8 00038 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: DMAR 00000000cd3346f4 00160 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: MCFG 00000000cd334a10 0003C (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: WD__ 00000000cd334a50 00134 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: SLIC 00000000cd334b88 00024 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: ERST 00000000cd3241ec 00270 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: HEST 00000000cd32445c 005B0 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: BERT 00000000cd32402c 00030 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: EINJ 00000000cd32405c 00190 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: TCPA 00000000cd3352e4 00064 (v02 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: PC__ 00000000cd335274 0006E (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: SRAT 00000000cd334db0 004C0 (v01 DELL   PE_SC3   00000001 DELL 00000001)
Aug 22 09:33:47 localhost kernel: ACPI: SSDT 00000000cd338000 06A9C (v01 INTEL  PPM RCM  80000001 INTL 20061109)
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 0 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 32 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 2 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 34 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 4 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 36 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 6 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 38 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 8 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 40 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 10 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 42 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 1 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 33 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 3 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 35 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 5 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 37 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 7 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 39 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 9 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 41 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: PXM 1 -> APIC 11 -> Node 0
Aug 22 09:33:47 localhost kernel: SRAT: PXM 2 -> APIC 43 -> Node 1
Aug 22 09:33:47 localhost kernel: SRAT: Node 0 PXM 1 0-430000000
Aug 22 09:33:47 localhost kernel: SRAT: Node 1 PXM 2 430000000-830000000
Aug 22 09:33:47 localhost kernel: Bootmem setup node 0 0000000000000000-0000000430000000
Aug 22 09:33:47 localhost kernel:  NODE_DATA [000000000000b140 - 000000000003f13f]
Aug 22 09:33:47 localhost kernel:  bootmap [0000000000100000 -  0000000000185fff] pages 86
Aug 22 09:33:47 localhost kernel: (9 early reservations) ==> bootmem [0000000000 - 0430000000]
Aug 22 09:33:47 localhost kernel:  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
Aug 22 09:33:47 localhost kernel:  #1 [0000006000 - 0000008000]       TRAMPOLINE ==> [0000006000 - 0000008000]
Aug 22 09:33:47 localhost kernel:  #2 [0001000000 - 0002012024]    TEXT DATA BSS ==> [0001000000 - 0002012024]
Aug 22 09:33:47 localhost kernel:  #3 [003737e000 - 0037fef0e9]          RAMDISK ==> [003737e000 - 0037fef0e9]
Aug 22 09:33:47 localhost kernel:  #4 [000009cc00 - 0000100000]    BIOS reserved ==> [000009cc00 - 0000100000]
Aug 22 09:33:47 localhost kernel:  #5 [0002013000 - 000201372d]              BRK ==> [0002013000 - 000201372d]
Aug 22 09:33:47 localhost kernel:  #6 [0000008000 - 000000a000]          PGTABLE ==> [0000008000 - 000000a000]
Aug 22 09:33:47 localhost kernel:  #7 [000000a000 - 000000b000]          PGTABLE ==> [000000a000 - 000000b000]
Aug 22 09:33:47 localhost kernel:  #8 [000000b000 - 000000b140]       MEMNODEMAP ==> [000000b000 - 000000b140]
Aug 22 09:33:47 localhost kernel: Bootmem setup node 1 0000000430000000-0000000830000000
Aug 22 09:33:47 localhost kernel:  NODE_DATA [0000000430000040 - 000000043003403f]
Aug 22 09:33:47 localhost kernel:  bootmap [0000000430035000 -  00000004300b4fff] pages 80
Aug 22 09:33:47 localhost kernel: (9 early reservations) ==> bootmem [0430000000 - 0830000000]
Aug 22 09:33:47 localhost kernel:  #0 [0000000000 - 0000001000]   BIOS data page
Aug 22 09:33:47 localhost kernel:  #1 [0000006000 - 0000008000]       TRAMPOLINE
Aug 22 09:33:47 localhost kernel:  #2 [0001000000 - 0002012024]    TEXT DATA BSS
Aug 22 09:33:47 localhost kernel:  #3 [003737e000 - 0037fef0e9]          RAMDISK
Aug 22 09:33:47 localhost kernel:  #4 [000009cc00 - 0000100000]    BIOS reserved
Aug 22 09:33:47 localhost kernel:  #5 [0002013000 - 000201372d]              BRK
Aug 22 09:33:47 localhost kernel:  #6 [0000008000 - 000000a000]          PGTABLE
Aug 22 09:33:47 localhost kernel:  #7 [000000a000 - 000000b000]          PGTABLE
Aug 22 09:33:47 localhost kernel:  #8 [000000b000 - 000000b140]       MEMNODEMAP
Aug 22 09:33:47 localhost kernel: found SMP MP-table at [ffff8800000fe710] fe710
Aug 22 09:33:47 localhost kernel: Reserving 131MB of memory at 48MB for crashkernel (System RAM: 33536MB)
Aug 22 09:33:47 localhost kernel: [ffffea000ea80000-ffffea000ebfffff] potential offnode page_structs
Aug 22 09:33:47 localhost kernel: Zone PFN ranges:
Aug 22 09:33:47 localhost kernel:  DMA      0x00000001 -> 0x00001000
Aug 22 09:33:47 localhost kernel:  DMA32    0x00001000 -> 0x00100000
Aug 22 09:33:47 localhost kernel:  Normal   0x00100000 -> 0x00830000
Aug 22 09:33:47 localhost kernel: Movable zone start PFN for each node
Aug 22 09:33:47 localhost kernel: early_node_map[4] active PFN ranges
Aug 22 09:33:47 localhost kernel:    0: 0x00000001 -> 0x0000009c
Aug 22 09:33:47 localhost kernel:    0: 0x00000100 -> 0x000cd2f0
Aug 22 09:33:47 localhost kernel:    0: 0x00100000 -> 0x00430000
Aug 22 09:33:47 localhost kernel:    1: 0x00430000 -> 0x00830000
Aug 22 09:33:47 localhost kernel: ACPI: PM-Timer IO Port: 0x808
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x02] lapic_id[0x20] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x04] lapic_id[0x22] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x05] lapic_id[0x04] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x06] lapic_id[0x24] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x07] lapic_id[0x06] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x08] lapic_id[0x26] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x09] lapic_id[0x08] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x28] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0a] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x2a] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x01] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x21] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x03] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x10] lapic_id[0x23] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x11] lapic_id[0x05] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x12] lapic_id[0x25] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x13] lapic_id[0x07] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x14] lapic_id[0x27] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x15] lapic_id[0x09] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x16] lapic_id[0x29] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x17] lapic_id[0x0b] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x18] lapic_id[0x2b] enabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x19] lapic_id[0x58] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x59] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x5a] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x5b] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x5c] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x5d] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x5e] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x20] lapic_id[0x5f] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x21] lapic_id[0x60] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x22] lapic_id[0x61] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x23] lapic_id[0x62] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x24] lapic_id[0x63] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x25] lapic_id[0x64] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x26] lapic_id[0x65] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x27] lapic_id[0x66] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x28] lapic_id[0x67] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x29] lapic_id[0x68] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x2a] lapic_id[0x69] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x2b] lapic_id[0x6a] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x2c] lapic_id[0x6b] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x2d] lapic_id[0x6c] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x2e] lapic_id[0x6d] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x2f] lapic_id[0x6e] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC (acpi_id[0x30] lapic_id[0x6f] disabled)
Aug 22 09:33:47 localhost kernel: ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
Aug 22 09:33:47 localhost kernel: ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
Aug 22 09:33:47 localhost kernel: IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
Aug 22 09:33:47 localhost kernel: ACPI: IOAPIC (id[0x01] address[0xfec3f000] gsi_base[32])
Aug 22 09:33:47 localhost kernel: IOAPIC[1]: apic_id 1, version 32, address 0xfec3f000, GSI 32-55
Aug 22 09:33:47 localhost kernel: ACPI: IOAPIC (id[0x02] address[0xfec7f000] gsi_base[64])
Aug 22 09:33:47 localhost kernel: IOAPIC[2]: apic_id 2, version 32, address 0xfec7f000, GSI 64-87
Aug 22 09:33:47 localhost kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Aug 22 09:33:47 localhost kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Aug 22 09:33:47 localhost kernel: Using ACPI (MADT) for SMP configuration information
Aug 22 09:33:47 localhost kernel: ACPI: HPET id: 0x8086a701 base: 0xfed00000
Aug 22 09:33:47 localhost kernel: SMP: Allowing 48 CPUs, 24 hotplug CPUs
Aug 22 09:33:47 localhost kernel: PM: Registered nosave memory: 000000000009c000 - 0000000000100000
Aug 22 09:33:47 localhost kernel: PM: Registered nosave memory: 00000000cd2f0000 - 00000000cd31c000
Aug 22 09:33:47 localhost kernel: PM: Registered nosave memory: 00000000cd31c000 - 00000000cd35b000
Aug 22 09:33:47 localhost kernel: PM: Registered nosave memory: 00000000cd35b000 - 00000000d0000000
Aug 22 09:33:47 localhost kernel: PM: Registered nosave memory: 00000000d0000000 - 00000000e0000000
Aug 22 09:33:47 localhost kernel: PM: Registered nosave memory: 00000000e0000000 - 00000000f0000000
Aug 22 09:33:47 localhost kernel: PM: Registered nosave memory: 00000000f0000000 - 00000000fe000000
Aug 22 09:33:47 localhost kernel: PM: Registered nosave memory: 00000000fe000000 - 0000000100000000
Aug 22 09:33:47 localhost kernel: Allocating PCI resources starting at d0000000 (gap: d0000000:10000000)
Aug 22 09:33:47 localhost kernel: Booting paravirtualized kernel on bare hardware
Aug 22 09:33:47 localhost kernel: Detected CPU family 6 model 62
Aug 22 09:33:47 localhost kernel: UNSUPPORTED HARDWARE DEVICE: CPU family 6 model > 59
Aug 22 09:33:47 localhost kernel: ------------[ cut here ]------------
Aug 22 09:33:47 localhost kernel: WARNING: at kernel/rh_taint.c:13 mark_hardware_unsupported+0x39/0x40() (Not tainted)
Aug 22 09:33:47 localhost kernel: Hardware name: PowerEdge R720
Aug 22 09:33:47 localhost kernel: Your hardware is unsupported.  Please do not report bugs, panics, oopses, etc., on this hardware.
Aug 22 09:33:47 localhost kernel: Modules linked in:
Aug 22 09:33:47 localhost kernel: Pid: 0, comm: swapper Not tainted 2.6.32-279.el6.x86_64 #1
Aug 22 09:33:47 localhost kernel: Call Trace:
Aug 22 09:33:47 localhost kernel: [<ffffffff8106b747>] ? warn_slowpath_common+0x87/0xc0
Aug 22 09:33:47 localhost kernel: [<ffffffff8106b7df>] ? warn_slowpath_fmt_taint+0x3f/0x50
Aug 22 09:33:47 localhost kernel: [<ffffffff8109a869>] ? mark_hardware_unsupported+0x39/0x40
Aug 22 09:33:47 localhost kernel: [<ffffffff81c27b5d>] ? setup_arch+0xb1f/0xb42
Aug 22 09:33:47 localhost kernel: [<ffffffff814fd223>] ? printk+0x41/0x46
Aug 22 09:33:47 localhost kernel: [<ffffffff81c21c33>] ? start_kernel+0xdc/0x430
Aug 22 09:33:47 localhost kernel: [<ffffffff81c2133a>] ? x86_64_start_reservations+0x125/0x129
Aug 22 09:33:47 localhost kernel: [<ffffffff81c21438>] ? x86_64_start_kernel+0xfa/0x109
Aug 22 09:33:47 localhost kernel: ---[ end trace a7919e7f17c0a725 ]---
Aug 22 09:33:47 localhost kernel: NR_CPUS:4096 nr_cpumask_bits:48 nr_cpu_ids:48 nr_node_ids:2
Aug 22 09:33:47 localhost kernel: PERCPU: Embedded 31 pages/cpu @ffff880036800000 s94424 r8192 d24360 u131072
Aug 22 09:33:47 localhost kernel: pcpu-alloc: s94424 r8192 d24360 u131072 alloc=1*2097152
Aug 22 09:33:47 localhost kernel: pcpu-alloc: [0] 00 02 04 06 08 10 12 14 16 18 20 22 24 26 28 30
Aug 22 09:33:47 localhost kernel: pcpu-alloc: [0] 32 34 36 38 40 42 44 46 -- -- -- -- -- -- -- --
Aug 22 09:33:47 localhost kernel: pcpu-alloc: [1] 01 03 05 07 09 11 13 15 17 19 21 23 25 27 29 31
Aug 22 09:33:47 localhost kernel: pcpu-alloc: [1] 33 35 37 39 41 43 45 47 -- -- -- -- -- -- -- --
Aug 22 09:33:47 localhost kernel: Built 2 zonelists in Zone order, mobility grouping on.  Total pages: 8259490
Aug 22 09:33:47 localhost kernel: Policy zone: Normal
Aug 22 09:33:47 localhost kernel: Kernel command line: ro root=UUID=cf5eecde-0866-4e33-b3bf-62c29d831aa2 rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_MD crashkernel=131M@0M LANG=zh_CN.UTF-8 rd_NO_LVM rd_NO_DM rhgb quiet
Aug 22 09:33:47 localhost kernel: PID hash table entries: 4096 (order: 3, 32768 bytes)
Aug 22 09:33:47 localhost kernel: xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
Aug 22 09:33:47 localhost kernel: Checking aperture...
Aug 22 09:33:47 localhost kernel: No AGP bridge found
Aug 22 09:33:47 localhost kernel: Queued invalidation will be enabled to support x2apic and Intr-remapping.
Aug 22 09:33:47 localhost kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Aug 22 09:33:47 localhost kernel: Placing 64MB software IO TLB between ffff880020000000 - ffff880024000000
Aug 22 09:33:47 localhost kernel: software IO TLB at phys 0x20000000 - 0x24000000
Aug 22 09:33:47 localhost kernel: Memory: 32810868k/34340864k available (5151k kernel code, 832980k absent, 697016k reserved, 7166k data, 1260k init)
Aug 22 09:33:47 localhost kernel: Hierarchical RCU implementation.
Aug 22 09:33:47 localhost kernel: NR_IRQS:33024 nr_irqs:1880
Aug 22 09:33:47 localhost kernel: Extended CMOS year: 2000
Aug 22 09:33:47 localhost kernel: Console: colour VGA+ 80x25
Aug 22 09:33:47 localhost kernel: console [tty0] enabled
Aug 22 09:33:47 localhost kernel: allocated 268435456 bytes of page_cgroup
Aug 22 09:33:47 localhost kernel: please try 'cgroup_disable=memory' option if you don't want memory cgroups
Aug 22 09:33:47 localhost kernel: Fast TSC calibration using PIT
Aug 22 09:33:47 localhost kernel: Detected 2100.217 MHz processor.
Aug 22 09:33:47 localhost kernel: Calibrating delay loop (skipped), value calculated using timer frequency.. 4200.43 BogoMIPS (lpj=2100217)
Aug 22 09:33:47 localhost kernel: pid_max: default: 49152 minimum: 384
Aug 22 09:33:47 localhost kernel: Security Framework initialized
Aug 22 09:33:47 localhost kernel: SELinux:  Initializing.
Aug 22 09:33:47 localhost kernel: Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
Aug 22 09:33:47 localhost kernel: Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
Aug 22 09:33:47 localhost kernel: Mount-cache hash table entries: 256

q1208c

内容有点多, 没有细看, 个人有如下建议.

1. R720 应该是 四通道内存的. 最佳的配置应该是每CPU 4条内存. 楼主的机器一共4条, 如果有两个CPU的话, 性能不会太好.
2. 无法修复的表损坏, 如果硬盘本身在没坏的话. 应该是mysql的cache在flush之前机器crash导致的. 这个问题mysql解决不了. 因为它只有一个 redo log, 好象, 还是要配置才行的.
3. 从DESET log看, 你的机器一直只有一个电源, 或者说有一个电源是坏的(或者没插线).

reblue520

回复 2# q1208c

多谢楼上的兄弟解答
1.服务器内存基本够用，经过一段时间的观察，基本上用不到swap，CPU占用率一般不会超过20%
3.其实电源报警与否我不太关心，这对硬件和应用没有太大的影响

2.这点才是最致命的，跑在上面的数据库应用无法访问，重启以后被损坏，就在找这方面的原因

从dell的硬件检测软件来看基本可以排除硬件的问题，现在在想是操作系统的问题呢？还是兼容性问题，或者是由某软件导致的问题


   

q1208c

回复 3# reblue520

我很奇怪你不关心硬件报警. 你的直接原因就是因为电源问题导致down机.

如果你觉得软件可以解决硬件down的问题, 那你可以研究一下. 有结果告诉我一下.
我替你去BS那些买双电源, RAID卡, mirror内存的人.
   

dzminglong

  dell的备用电源线被拔掉了啊，我都插了俩条电源线，虽然不是必须的，好歹是个冗余嘛，而且，有点强迫症，老是看着机箱后面的黄灯闪，别扭

reblue520

回复 4# q1208c

这个前面忘了说明，机器不是dell厂家直销的，被代理商更换了CPU、硬盘、RAID卡，还拆掉了一个电源

也就是说这个机器是单电源，应该不是电源掉电导致的系统挂掉，因为当时我去机房看了，电源并没有关闭，插上显示器显示的是乱码

当时ctrl+alt+delete重启了机器，后面mysql表损坏




   

dengbao2001

日志 中间有个Call trace, 建议你查查，这儿有个链接，参考http://lvii.github.io/system/201 ... -hardware-detected/ 下

另外，如果订阅了服务，可以看到些资料

q1208c

回复 6# reblue520

Aug 22 09:33:47 localhost kernel: Booting paravirtualized kernel on bare hardware
Aug 22 09:33:47 localhost kernel: Detected CPU family 6 model 62
Aug 22 09:33:47 localhost kernel: UNSUPPORTED HARDWARE DEVICE: CPU family 6 model > 59
Aug 22 09:33:47 localhost kernel: ------------[ cut here ]------------
Aug 22 09:33:47 localhost kernel: WARNING: at kernel/rh_taint.c:13 mark_hardware_unsupported+0x39/0x40() (Not tainted)
Aug 22 09:33:47 localhost kernel: Hardware name: PowerEdge R720
Aug 22 09:33:47 localhost kernel: Your hardware is unsupported.  Please do not report bugs, panics, oopses, etc., on this hardware.
Aug 22 09:33:47 localhost kernel: Modules linked in:
Aug 22 09:33:47 localhost kernel: Pid: 0, comm: swapper Not tainted 2.6.32-279.el6.x86_64 #1
Aug 22 09:33:47 localhost kernel: Call Trace:
Aug 22 09:33:47 localhost kernel: [<ffffffff8106b747>] ? warn_slowpath_common+0x87/0xc0
Aug 22 09:33:47 localhost kernel: [<ffffffff8106b7df>] ? warn_slowpath_fmt_taint+0x3f/0x50
Aug 22 09:33:47 localhost kernel: [<ffffffff8109a869>] ? mark_hardware_unsupported+0x39/0x40
Aug 22 09:33:47 localhost kernel: [<ffffffff81c27b5d>] ? setup_arch+0xb1f/0xb42
Aug 22 09:33:47 localhost kernel: [<ffffffff814fd223>] ? printk+0x41/0x46
Aug 22 09:33:47 localhost kernel: [<ffffffff81c21c33>] ? start_kernel+0xdc/0x430
Aug 22 09:33:47 localhost kernel: [<ffffffff81c2133a>] ? x86_64_start_reservations+0x125/0x129
Aug 22 09:33:47 localhost kernel: [<ffffffff81c21438>] ? x86_64_start_kernel+0xfa/0x109
Aug 22 09:33:47 localhost kernel: ---[ end trace a7919e7f17c0a725 ]---



你的机器上看来改了不少东西呀. 已经变成 UNSUPPORTED 了.     

reblue520

感谢 7楼的 dengbao2001 指出问题
也感谢 q1208c 一直热心关注

问题应该出在了兼容性上，CPU和centos内核不兼容

dengbao2001

问题修正了，麻烦告诉下大家的步骤，以便我们学习学习