免费注册 查看新帖 |

Chinaunix

  平台 论坛 博客 文库
› 论坛 › 程序设计 › 内核源码 › 求助：Linux系统ebtables配合iptables总死机
最近访问板块 发新帖
查看: 3411 | 回复: 4

[网络子系统] 求助：Linux系统ebtables配合iptables总死机 [复制链接]

iceblood

论坛徽章:
1
金牛座 日期:2014-05-29 15:55:47
发表于 2015-08-21 13:23 |显示全部楼层
环境：使用Linux做了一个网桥，p1p1和p1p2建立网桥br1，以及独立网卡em1。流经p1p2的数据会用ebtables强制转发，然后使用iptables做NAT。
情况：该构架已经稳定运行1年多了，很多服务器甚至2~3个月几乎没碰过，也就是说没修改过配置，但是从这周1开始大批服务器突然死机。原来以为是硬件问题，但后来发现硬件更换后故障依旧，现象也一样。
错误信息：死机前我们抓到的信息如下：

  2. Aug 20 20:33:53 br1-7 kernel: IP: [<ffffffff8128e532>] memmove+0x42/0x1a0
  3. Aug 20 20:33:53 br1-7 kernel: PGD 1a86063 PUD 8067 PMD 9067 PTE 0
  4. Aug 20 20:33:53 br1-7 kernel: Oops: 0000 [#1] SMP
  5. Aug 20 20:33:53 br1-7 kernel:
  6. Aug 20 20:33:53 br1-7 kernel: last sysfs file: /sys/devices/system/cpu/online
  7. Aug 20 20:33:53 br1-7 kernel: CPU 15
  8. Aug 20 20:33:53 br1-7 kernel: Modules linked in:
  9. Aug 20 20:33:53 br1-7 kernel: ebtable_filter iptable_raw httpcontentchange(U) xt_state xt_physdev iptable_mangle xt_MARK ebt_redirect ebt_ip ebtable_nat ebtables bridge ipt_LOG act_police cls_u32 sch_ingress sch_prio sch_sfq sch_htb ipip tunnel4 ip_gre
  10. Aug 20 20:33:53 br1-7 kernel: ip_tunnel xt_TCPMSS xt_tcpmss nf_nat_ftp nf_conntrack_ftp iptable_nat iptable_filter ip_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 8021q garp stp llc ipv6 dm_mod power_meter iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core ixgbe dca ptp pps_core mdio i7core_edac edac_core sg bnx2 ext4 jbd2 mbcache sd_mod crc_t10dif pata_acpi ata_generic ata_piix mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
  11. Aug 20 20:33:53 br1-7 kernel:
  12. Aug 20 20:33:53 br1-7 kernel: Pid: 0, comm: swapper Not tainted 2.6.32-431.el6.x86_64 #1 Dell Inc. PowerEdge R410/0N83VF
  13. Aug 20 20:33:53 br1-7 kernel: RIP: 0010:[<ffffffff8128e532>]  [<ffffffff8128e532>] memmove+0x42/0x1a0
  14. Aug 20 20:33:53 br1-7 kernel: RSP: 0018:ffff8800282e3608  EFLAGS: 00010202
  15. Aug 20 20:33:53 br1-7 kernel: RAX: ffff8800cf405017 RBX: ffff880126770980 RCX: 0000000000000000
  16. Aug 20 20:33:53 br1-7 kernel: RDX: 00000000fd2ae4fb RSI: ffff8800cf678fe4 RDI: ffff8800cf678fd7
  17. Aug 20 20:33:53 br1-7 kernel: RBP: ffff8800282e3650 R08: 3d4e49203a524544 R09: 7031703d4e495359
  18. Aug 20 20:33:53 br1-7 kernel: R10: 485020313531312e R11: 316d653d54554f20 R12: 000000000000000d
  19. Aug 20 20:33:53 br1-7 kernel: R13: 0000000000000000 R14: 0000000002addfbf R15: ffff8800cc927058
  20. Aug 20 20:33:53 br1-7 kernel: FS:  0000000000000000(0000) GS:ffff8800282e0000(0000) knlGS:0000000000000000
  21. Aug 20 20:33:53 br1-7 kernel: CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
  22. Aug 20 20:33:53 br1-7 kernel: CR2: ffff8800cf679000 CR3: 000000020b691000 CR4: 00000000000007e0
  23. Aug 20 20:33:53 br1-7 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  24. Aug 20 20:33:53 br1-7 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  25. Aug 20 20:33:53 br1-7 kernel: Process swapper (pid: 0, threadinfo ffff88012e2ae000, task ffff88022e12a080)
  26. Aug 20 20:33:53 br1-7 kernel: Stack:
  27. Aug 20 20:33:53 br1-7 kernel: ffffffffa0213781 ffffffffa06d97d0 0000000000000000 ffff8800282e3640
  28. Aug 20 20:33:53 br1-7 kernel: <d> ffff880126770980 ffff8800cc927030 ffff8800ca93fb98 ffff8800cc927044
  29. Aug 20 20:33:53 br1-7 kernel: <d> 000000000000000d ffff8800282e36d0 ffffffffa0213c6b 000000000000000d
  30. Aug 20 20:33:53 br1-7 kernel: Call Trace:
  31. Aug 20 20:33:53 br1-7 kernel: <IRQ>
  32. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa0213781>] ? mangle_contents+0x81/0x1d0 [nf_nat]
  33. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa0213c6b>] nf_nat_mangle_tcp_packet+0xbb/0x550 [nf_nat]
  34. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa02d4eab>] myhook_func+0x7cb/0x930 [httpcontentchange]
  35. Aug 20 20:33:53 br1-7 kernel: [<ffffffff814893a9>] nf_iterate+0x69/0xb0
  36. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa029aba0>] ? br_nf_forward_finish+0x0/0x140 [bridge]
  37. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81489566>] nf_hook_slow+0x76/0x120
  38. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa029aba0>] ? br_nf_forward_finish+0x0/0x140 [bridge]
  39. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa029b126>] br_nf_forward_ip+0x1d6/0x360 [bridge]
  40. Aug 20 20:33:53 br1-7 kernel: [<ffffffff814893a9>] nf_iterate+0x69/0xb0
  41. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa02948d0>] ? br_forward_finish+0x0/0x60 [bridge]
  42. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81489566>] nf_hook_slow+0x76/0x120
  43. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa02948d0>] ? br_forward_finish+0x0/0x60 [bridge]
  44. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa02949ae>] __br_forward+0x7e/0xd0 [bridge]
  45. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81489566>] ? nf_hook_slow+0x76/0x120
  46. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa0294a5d>] br_forward+0x5d/0x70 [bridge]
  47. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa0295a6b>] br_handle_frame_finish+0x17b/0x2a0 [bridge]
  48. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa029b938>] br_nf_pre_routing_finish+0x238/0x350 [bridge]
  49. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa029bedb>] br_nf_pre_routing+0x48b/0x7b0 [bridge]
  50. Aug 20 20:33:53 br1-7 kernel: [<ffffffff814893a9>] nf_iterate+0x69/0xb0
  51. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa02958f0>] ? br_handle_frame_finish+0x0/0x2a0 [bridge]
  52. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81489566>] nf_hook_slow+0x76/0x120
  53. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa02958f0>] ? br_handle_frame_finish+0x0/0x2a0 [bridge]
  54. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa0295d1c>] br_handle_frame+0x18c/0x250 [bridge]
  55. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81500220>] ? vlan_hwaccel_do_receive+0x70/0x130
  56. Aug 20 20:33:53 br1-7 kernel: [<ffffffff8145b5c9>] __netif_receive_skb+0x529/0x750
  57. Aug 20 20:33:53 br1-7 kernel: [<ffffffff8145f1b8>] netif_receive_skb+0x58/0x60
  58. Aug 20 20:33:53 br1-7 kernel: [<ffffffff8145f3a8>] napi_gro_complete+0xc8/0x150
  59. Aug 20 20:33:53 br1-7 kernel: [<ffffffff8145f652>] dev_gro_receive+0x222/0x350
  60. Aug 20 20:33:53 br1-7 kernel: [<ffffffff815006e0>] vlan_gro_common+0x190/0x260
  61. Aug 20 20:33:53 br1-7 kernel: [<ffffffff8150098a>] vlan_gro_receive+0x7a/0xa0
  62. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa013ae68>] ixgbe_poll+0xce8/0x12c0 [ixgbe]
  63. Aug 20 20:33:53 br1-7 kernel: [<ffffffffa0295d1c>] ? br_handle_frame+0x18c/0x250 [bridge]
  64. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81500220>] ? vlan_hwaccel_do_receive+0x70/0x130
  65. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81460b43>] net_rx_action+0x103/0x2f0
  66. Aug 20 20:33:53 br1-7 kernel: [<ffffffff8107a8e1>] __do_softirq+0xc1/0x1e0
  67. Aug 20 20:33:53 br1-7 kernel: [<ffffffff810e6ec0>] ? handle_IRQ_event+0x60/0x170
  68. Aug 20 20:33:53 br1-7 kernel: [<ffffffff8100c30c>] call_softirq+0x1c/0x30
  69. Aug 20 20:33:53 br1-7 kernel: [<ffffffff8100fa75>] do_softirq+0x65/0xa0
  70. Aug 20 20:33:53 br1-7 kernel: [<ffffffff8107a795>] irq_exit+0x85/0x90
  71. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81530fe5>] do_IRQ+0x75/0xf0
  72. Aug 20 20:33:53 br1-7 kernel: [<ffffffff8100b9d3>] ret_from_intr+0x0/0x11
  73. Aug 20 20:33:53 br1-7 kernel: <EOI>
  74. Aug 20 20:33:53 br1-7 kernel: [<ffffffff812e09be>] ? intel_idle+0xde/0x170
  75. Aug 20 20:33:53 br1-7 kernel: [<ffffffff812e09a1>] ? intel_idle+0xc1/0x170
  76. Aug 20 20:33:53 br1-7 kernel: [<ffffffff814266f7>] cpuidle_idle_call+0xa7/0x140
  77. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81009fc6>] cpu_idle+0xb6/0x110
  78. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81520e13>] start_secondary+0x2ac/0x2ef
  79. Aug 20 20:33:53 br1-7 kernel: Code: d0 49 39 f8 0f 8f 9f 00 00 00 48 81 fa a8 02 00 00 72 05 40 38 fe 74 41 48 83 ea 20 48 83 ea 20 4c 8b 1e 4c 8b 56 08 4c 8b 4e 10 <4c> 8b 46 18 48 8d 76 20 4c 89 1f 4c 89 57 08 4c 89 4f 10 4c 89
  80. Aug 20 20:33:53 br1-7 kernel: RIP  [<ffffffff8128e532>] memmove+0x42/0x1a0
  81. Aug 20 20:33:53 br1-7 kernel: RSP <ffff8800282e3608>
  82. Aug 20 20:33:53 br1-7 kernel: CR2: ffff8800cf679000
  83. Aug 20 20:33:53 br1-7 kernel: BUG: scheduling while atomic: swapper/0/0x10000100
  84. Aug 20 20:33:53 br1-7 kernel: Modules linked in: ebtable_filter iptable_raw httpcontentchange(U) xt_state xt_physdev iptable_mangle xt_MARK ebt_redirect ebt_ip ebtable_nat ebtables bridge ipt_LOG act_police cls_u32 sch_ingress sch_prio sch_sfq sch_htb ipip tunnel4 ip_gre ip_tunnel xt_TCPMSS xt_tcpmss nf_nat_ftp nf_conntrack_ftp iptable_nat iptable_filter ip_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 8021q garp stp llc ipv6 dm_mod power_meter iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core ixgbe dca ptp pps_core mdio i7core_edac edac_core sg bnx2 ext4 jbd2 mbcache sd_mod crc_t10dif pata_acpi ata_generic ata_piix mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
  85. Aug 20 20:33:53 br1-7 kernel: CPU 15
  86. Aug 20 20:33:53 br1-7 kernel: Modules linked in: ebtable_filter iptable_raw httpcontentchange(U) xt_state xt_physdev iptable_mangle xt_MARK ebt_redirect ebt_ip ebtable_nat ebtables bridge ipt_LOG act_police cls_u32 sch_ingress sch_prio sch_sfq sch_htb ipip tunnel4 ip_gre ip_tunnel xt_TCPMSS xt_tcpmss nf_nat_ftp nf_conntrack_ftp iptable_nat iptable_filter ip_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 8021q garp stp llc ipv6 dm_mod power_meter iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core ixgbe dca ptp pps_core mdio i7core_edac edac_core sg bnx2 ext4 jbd2 mbcache sd_mod crc_t10dif pata_acpi ata_generic ata_piix mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
  87. Aug 20 20:33:53 br1-7 kernel:
  88. Aug 20 20:33:53 br1-7 kernel: Pid: 0, comm: swapper Not tainted 2.6.32-431.el6.x86_64 #1 Dell Inc. PowerEdge R410/0N83VF
  89. Aug 20 20:33:53 br1-7 kernel: RIP: 0010:[<ffffffff812e09be>]  [<ffffffff812e09be>] intel_idle+0xde/0x170
  90. Aug 20 20:33:53 br1-7 kernel: RSP: 0018:ffff88012e2afe68  EFLAGS: 00000202
  91. Aug 20 20:33:53 br1-7 kernel: RAX: 0000000000000000 RBX: ffff88012e2afed8 RCX: 0000000000000000
  92. Aug 20 20:33:53 br1-7 kernel: RDX: 000000000000006d RSI: 0000000000000000 RDI: 000000000001aa57
  93. Aug 20 20:33:53 br1-7 kernel: RBP: ffffffff8100b9ce R08: 0000000000000002 R09: 0000000000000050
  94. Aug 20 20:33:53 br1-7 kernel: R10: 000010906c507ee3 R11: 0000000000000000 R12: 0000000000000086
  95. Aug 20 20:33:53 br1-7 kernel: R13: ffff88012e2afdd8 R14: ffff8800282f15e0 R15: 0000000000000000
  96. Aug 20 20:33:53 br1-7 kernel: FS:  0000000000000000(0000) GS:ffff8800282e0000(0000) knlGS:0000000000000000
  97. Aug 20 20:33:53 br1-7 kernel: CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
  98. Aug 20 20:33:53 br1-7 kernel: CR2: ffff8800cf679000 CR3: 000000020b691000 CR4: 00000000000007e0
  99. Aug 20 20:33:53 br1-7 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  100. Aug 20 20:33:53 br1-7 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  101. Aug 20 20:33:53 br1-7 kernel: Process swapper (pid: 0, threadinfo ffff88012e2ae000, task ffff88022e12a080)
  102. Aug 20 20:33:53 br1-7 kernel: Stack:
  103. Aug 20 20:33:53 br1-7 kernel: ffff8800282ee100 0000000000000282 0000000000000000 000000000000006d
  104. Aug 20 20:33:53 br1-7 kernel: <d> 0000000000000000 000000000000006d 0000000000000000 0000000f00cb791a
  105. Aug 20 20:33:53 br1-7 kernel: <d> 0000000000000000 ffff8800282fe040 ffff8800282fe0b0 0000000000000000
  106. Aug 20 20:33:53 br1-7 kernel: Call Trace:
  107. Aug 20 20:33:53 br1-7 kernel: [<ffffffff814266f7>] ? cpuidle_idle_call+0xa7/0x140
  108. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81009fc6>] ? cpu_idle+0xb6/0x110
  109. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81520e13>] ? start_secondary+0x2ac/0x2ef
  110. Aug 20 20:33:53 br1-7 kernel: Code: 58 dc ff 4c 29 e8 48 89 c7 e8 bf 88 d9 ff 49 89 c4 49 89 d5 48 89 45 a0 48 89 55 a8 48 89 45 b0 48 89 55 b8 fb 66 0f 1f 44 00 00 <85> 1d 24 57 81 00 75 0e 48 8d 75 cc bf 05 00 00 00 e8 1c a7 dc
  111. Aug 20 20:33:53 br1-7 kernel: Call Trace:
  112. Aug 20 20:33:53 br1-7 kernel: [<ffffffff812e09a1>] ? intel_idle+0xc1/0x170
  113. Aug 20 20:33:53 br1-7 kernel: [<ffffffff814266f7>] ? cpuidle_idle_call+0xa7/0x140
  114. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81009fc6>] ? cpu_idle+0xb6/0x110
  115. Aug 20 20:33:53 br1-7 kernel: [<ffffffff81520e13>] ? start_secondary+0x2ac/0x2ef
  116. Aug 20 20:33:53 br1-7 kernel: Clocksource tsc unstable (delta = -8589919981 ns).  Enable clocksource failover by adding clocksource_failover kernel parameter.
  117. Aug 20 20:35:02 br1-7 kernel: ------------[ cut here ]------------
  118. Aug 20 20:35:02 br1-7 kernel: WARNING: at net/sched/sch_generic.c:261 dev_watchdog+0x26b/0x280() (Not tainted)
  119. Aug 20 20:35:02 br1-7 kernel: Hardware name: PowerEdge R410
  120. Aug 20 20:35:02 br1-7 kernel: NETDEV WATCHDOG: p1p1 (ixgbe): transmit queue 22 timed out
  121. Aug 20 20:35:02 br1-7 kernel: Modules linked in: ebtable_filter iptable_raw httpcontentchange(U) xt_state xt_physdev iptable_mangle xt_MARK ebt_redirect ebt_ip ebtable_nat ebtables bridge ipt_LOG act_police cls_u32 sch_ingress sch_prio sch_sfq sch_htb ipip tunnel4 ip_gre ip_tunnel xt_TCPMSS xt_tcpmss nf_nat_ftp nf_conntrack_ftp iptable_nat iptable_filter ip_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 8021q garp stp llc ipv6 dm_mod power_meter iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core ixgbe dca ptp pps_core mdio i7core_edac edac_core sg bnx2 ext4 jbd2 mbcache sd_mod crc_t10dif pata_acpi ata_generic ata_piix mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
  122. Aug 20 20:35:02 br1-7 kernel: Pid: 0, comm: swapper Not tainted 2.6.32-431.el6.x86_64 #1
  123. Aug 20 20:35:02 br1-7 kernel: Call Trace:
  124. Aug 20 20:35:02 br1-7 kernel: <IRQ>  [<ffffffff81071e27>] ? warn_slowpath_common+0x87/0xc0
  125. Aug 20 20:35:02 br1-7 kernel: [<ffffffff81071f16>] ? warn_slowpath_fmt+0x46/0x50
  126. Aug 20 20:35:02 br1-7 kernel: [<ffffffff8147b74b>] ? dev_watchdog+0x26b/0x280
  127. Aug 20 20:35:02 br1-7 kernel: [<ffffffff813c6160>] ? rh_timer_func+0x0/0x10
  128. Aug 20 20:35:02 br1-7 kernel: [<ffffffff813c5890>] ? usb_hcd_poll_rh_status+0x140/0x180
  129. Aug 20 20:35:02 br1-7 kernel: [<ffffffff8105dd5c>] ? scheduler_tick+0xcc/0x260
  130. Aug 20 20:35:02 br1-7 kernel: [<ffffffff8147b4e0>] ? dev_watchdog+0x0/0x280
  131. Aug 20 20:35:02 br1-7 kernel: [<ffffffff81084b07>] ? run_timer_softirq+0x197/0x340
  132. Aug 20 20:35:02 br1-7 kernel: [<ffffffff810ac8f5>] ? tick_dev_program_event+0x65/0xc0
  133. Aug 20 20:35:02 br1-7 kernel: [<ffffffff8107a8e1>] ? __do_softirq+0xc1/0x1e0
  134. Aug 20 20:35:02 br1-7 kernel: [<ffffffff810ac9ca>] ? tick_program_event+0x2a/0x30
  135. Aug 20 20:35:02 br1-7 kernel: [<ffffffff8100c30c>] ? call_softirq+0x1c/0x30
  136. Aug 20 20:35:02 br1-7 kernel: [<ffffffff8100fa75>] ? do_softirq+0x65/0xa0
  137. Aug 20 20:35:02 br1-7 kernel: [<ffffffff8107a795>] ? irq_exit+0x85/0x90
  138. Aug 20 20:35:02 br1-7 kernel: [<ffffffff815310aa>] ? smp_apic_timer_interrupt+0x4a/0x60
  139. Aug 20 20:35:02 br1-7 kernel: [<ffffffff8100bb93>] ? apic_timer_interrupt+0x13/0x20
  140. Aug 20 20:35:02 br1-7 kernel: <EOI>  [<ffffffff812e09be>] ? intel_idle+0xde/0x170
  141. Aug 20 20:35:02 br1-7 kernel: [<ffffffff812e09a1>] ? intel_idle+0xc1/0x170
  142. Aug 20 20:35:02 br1-7 kernel: [<ffffffff814266f7>] ? cpuidle_idle_call+0xa7/0x140
  143. Aug 20 20:35:02 br1-7 kernel: [<ffffffff81009fc6>] ? cpu_idle+0xb6/0x110
  144. Aug 20 20:35:02 br1-7 kernel: [<ffffffff8150cbea>] ? rest_init+0x7a/0x80
  145. Aug 20 20:35:02 br1-7 kernel: [<ffffffff81c26f8f>] ? start_kernel+0x424/0x430
  146. Aug 20 20:35:02 br1-7 kernel: [<ffffffff81c2633a>] ? x86_64_start_reservations+0x125/0x129
  147. Aug 20 20:35:02 br1-7 kernel: [<ffffffff81c26453>] ? x86_64_start_kernel+0x115/0x124
  148. Aug 20 20:35:02 br1-7 kernel: ---[ end trace c8c3564c120a86fd ]---
  149. Aug 20 20:35:02 br1-7 kernel: ixgbe 0000:03:00.0: p1p1: initiating reset due to tx timeout
  150. Aug 20 20:35:36 br1-7 kernel: ixgbe 0000:03:00.0: p1p1: initiating reset due to tx timeout
  151. Aug 20 20:35:49 br1-7 kernel: ixgbe 0000:03:00.0: p1p1: Reset adapter
  152. Aug 20 20:36:57 br1-7 kernel: BUG: soft lockup - CPU#5 stuck for 71s! [ksoftirqd/5:25]
  153. Aug 20 20:36:57 br1-7 kernel: Modules linked in: ebtable_filter iptable_raw httpcontentchange(U) xt_state xt_physdev iptable_mangle xt_MARK ebt_redirect ebt_ip ebtable_nat ebtables bridge ipt_LOG act_police cls_u32 sch_ingress sch_prio sch_sfq sch_htb ipip tunnel4 ip_gre ip_tunnel xt_TCPMSS xt_tcpmss nf_nat_ftp nf_conntrack_ftp iptable_nat iptable_filter ip_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 8021q garp stp llc ipv6 dm_mod power_meter iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core ixgbe dca ptp pps_core mdio i7core_edac edac_core sg bnx2 ext4 jbd2 mbcache sd_mod crc_t10dif pata_acpi ata_generic ata_piix mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
  154. Aug 20 20:36:57 br1-7 kernel: CPU 5
  155. Aug 20 20:36:57 br1-7 kernel: Modules linked in: ebtable_filter iptable_raw httpcontentchange(U) xt_state xt_physdev iptable_mangle xt_MARK ebt_redirect ebt_ip ebtable_nat ebtables bridge ipt_LOG act_police cls_u32 sch_ingress sch_prio sch_sfq sch_htb ipip tunnel4 ip_gre ip_tunnel xt_TCPMSS xt_tcpmss nf_nat_ftp nf_conntrack_ftp iptable_nat iptable_filter ip_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 8021q garp stp llc ipv6 dm_mod power_meter iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core ixgbe dca ptp pps_core mdio i7core_edac edac_core sg bnx2 ext4 jbd2 mbcache sd_mod crc_t10dif pata_acpi ata_generic ata_piix mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
  156. Aug 20 20:36:57 br1-7 kernel:
  157. Aug 20 20:36:57 br1-7 kernel: Pid: 25, comm: ksoftirqd/5 Tainted: G        W  ---------------    2.6.32-431.el6.x86_64 #1 Dell Inc. PowerEdge R410/0N83VF
  158. Aug 20 20:36:57 br1-7 kernel: RIP: 0010:[<ffffffff8152a42b>]  [<ffffffff8152a42b>] _spin_lock_bh+0x2b/0x40
  159. Aug 20 20:36:57 br1-7 kernel: RSP: 0018:ffff880028243598  EFLAGS: 00000293
  160. Aug 20 20:36:57 br1-7 kernel: RAX: 0000000000001544 RBX: ffff8800282435a8 RCX: 0000000000000000
  161. Aug 20 20:36:57 br1-7 kernel: RDX: 0000000000001542 RSI: 0000000000000004 RDI: ffffffffa028f430
  162. Aug 20 20:36:57 br1-7 kernel: RBP: ffffffff8100bb93 R08: ffff88022a4f2020 R09: ffff880028243608
  163. Aug 20 20:36:57 br1-7 kernel: R10: ffff88010aba61e0 R11: ffff88012619a020 R12: ffff880028243510
  164. Aug 20 20:36:57 br1-7 kernel: R13: ffffffffa028f430 R14: ffff880028243500 R15: ffffffff815310a5
  165. Aug 20 20:36:57 br1-7 kernel: FS:  0000000000000000(0000) GS:ffff880028240000(0000) knlGS:0000000000000000
  166. Aug 20 20:36:57 br1-7 kernel: CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
  167. Aug 20 20:36:57 br1-7 kernel: CR2: 0000000000be0e80 CR3: 000000020b691000 CR4: 00000000000007e0
  168. Aug 20 20:36:57 br1-7 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  169. Aug 20 20:36:57 br1-7 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  170. Aug 20 20:36:57 br1-7 kernel: Process ksoftirqd/5 (pid: 25, threadinfo ffff88022e7fe000, task ffff88022e7f8aa0)
  171. Aug 20 20:36:57 br1-7 kernel: Stack:
  172. Aug 20 20:36:57 br1-7 kernel: 12bd146f00022168 ffff88022a4f2020 ffff8800282435e8 ffffffffa028e9f7
  173. Aug 20 20:36:57 br1-7 kernel: <d> ffff88012e1cc000 ffff88010aba6170 ffff8800cd077380 ffff880028243708
  174. Aug 20 20:36:57 br1-7 kernel: <d> ffff8800ca253430 ffff88010aba6370 ffff880028243618 ffffffffa028ebca
  175. Aug 20 20:36:57 br1-7 kernel: Call Trace:
  176. Aug 20 20:36:57 br1-7 kernel: <IRQ>
  177. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa028e9f7>] ? ipt_log_packet+0x47/0x1d0 [ipt_LOG]
  178. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa028ebca>] ? log_tg+0x4a/0x54 [ipt_LOG]
  179. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa023220b>] ? ipt_do_table+0x3cb/0x678 [ip_tables]
  180. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8144f912>] ? kfree_skb+0x42/0x90
  181. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa02320d5>] ? ipt_do_table+0x295/0x678 [ip_tables]
  182. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa023c224>] ? nf_nat_rule_find+0x24/0x80 [iptable_nat]
  183. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa023c4c1>] ? nf_nat_fn+0x111/0x260 [iptable_nat]
  184. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa023c738>] ? nf_nat_out+0x48/0xd0 [iptable_nat]
  185. Aug 20 20:36:57 br1-7 kernel: [<ffffffff814893a9>] ? nf_iterate+0x69/0xb0
  186. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81499f90>] ? ip_finish_output+0x0/0x310
  187. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81489566>] ? nf_hook_slow+0x76/0x120
  188. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81499f90>] ? ip_finish_output+0x0/0x310
  189. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8149a344>] ? ip_output+0xa4/0xc0
  190. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8149562f>] ? ip_forward_finish+0x4f/0x60
  191. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8149583c>] ? ip_forward+0x1fc/0x430
  192. Aug 20 20:36:57 br1-7 kernel: [<ffffffff814939cd>] ? ip_rcv_finish+0x12d/0x440
  193. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81493f55>] ? ip_rcv+0x275/0x350
  194. Aug 20 20:36:57 br1-7 kernel: [<ffffffff814893a9>] ? nf_iterate+0x69/0xb0
  195. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145b54b>] ? __netif_receive_skb+0x4ab/0x750
  196. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145f160>] ? netif_receive_skb+0x0/0x60
  197. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145f1b8>] ? netif_receive_skb+0x58/0x60
  198. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa0295ad8>] ? br_handle_frame_finish+0x1e8/0x2a0 [bridge]
  199. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa029b938>] ? br_nf_pre_routing_finish+0x238/0x350 [bridge]
  200. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa029bedb>] ? br_nf_pre_routing+0x48b/0x7b0 [bridge]
  201. Aug 20 20:36:57 br1-7 kernel: [<ffffffff814893a9>] ? nf_iterate+0x69/0xb0
  202. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa02958f0>] ? br_handle_frame_finish+0x0/0x2a0 [bridge]
  203. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81489566>] ? nf_hook_slow+0x76/0x120
  204. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa02958f0>] ? br_handle_frame_finish+0x0/0x2a0 [bridge]
  205. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa0295d1c>] ? br_handle_frame+0x18c/0x250 [bridge]
  206. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81500220>] ? vlan_hwaccel_do_receive+0x70/0x130
  207. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145b5c9>] ? __netif_receive_skb+0x529/0x750
  208. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145f1b8>] ? netif_receive_skb+0x58/0x60
  209. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145f2c0>] ? napi_skb_finish+0x50/0x70
  210. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81500994>] ? vlan_gro_receive+0x84/0xa0
  211. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa013ae68>] ? ixgbe_poll+0xce8/0x12c0 [ixgbe]
  212. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa01ecbe9>] ? nf_conntrack_free+0x59/0x70 [nf_conntrack]
  213. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa01ed876>] ? destroy_conntrack+0xd6/0x150 [nf_conntrack]
  214. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81460b43>] ? net_rx_action+0x103/0x2f0
  215. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8107a8e1>] ? __do_softirq+0xc1/0x1e0
  216. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8100c30c>] ? call_softirq+0x1c/0x30
  217. Aug 20 20:36:57 br1-7 kernel: <EOI>
  218. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8100fa75>] ? do_softirq+0x65/0xa0
  219. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8107a4a0>] ? ksoftirqd+0x80/0x110
  220. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8107a420>] ? ksoftirqd+0x0/0x110
  221. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8109aef6>] ? kthread+0x96/0xa0
  222. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8100c20a>] ? child_rip+0xa/0x20
  223. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8109ae60>] ? kthread+0x0/0xa0
  224. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8100c200>] ? child_rip+0x0/0x20
  225. Aug 20 20:36:57 br1-7 kernel: Code: 55 48 89 e5 53 48 83 ec 08 0f 1f 44 00 00 48 89 fb e8 8a f8 b4 ff b8 00 00 01 00 f0 0f c1 03 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 13 eb f5 83 3b 00 75 f4 eb df 48 83 c4 08 5b c9 c3 66 90
  226. Aug 20 20:36:57 br1-7 kernel: Call Trace:
  227. Aug 20 20:36:57 br1-7 kernel: <IRQ>  [<ffffffff8152a416>] ? _spin_lock_bh+0x16/0x40
  228. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa028e9f7>] ? ipt_log_packet+0x47/0x1d0 [ipt_LOG]
  229. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa028ebca>] ? log_tg+0x4a/0x54 [ipt_LOG]
  230. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa023220b>] ? ipt_do_table+0x3cb/0x678 [ip_tables]
  231. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8144f912>] ? kfree_skb+0x42/0x90
  232. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa02320d5>] ? ipt_do_table+0x295/0x678 [ip_tables]
  233. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa023c224>] ? nf_nat_rule_find+0x24/0x80 [iptable_nat]
  234. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa023c4c1>] ? nf_nat_fn+0x111/0x260 [iptable_nat]
  235. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa023c738>] ? nf_nat_out+0x48/0xd0 [iptable_nat]
  236. Aug 20 20:36:57 br1-7 kernel: [<ffffffff814893a9>] ? nf_iterate+0x69/0xb0
  237. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81499f90>] ? ip_finish_output+0x0/0x310
  238. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81489566>] ? nf_hook_slow+0x76/0x120
  239. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81499f90>] ? ip_finish_output+0x0/0x310
  240. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8149a344>] ? ip_output+0xa4/0xc0
  241. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8149562f>] ? ip_forward_finish+0x4f/0x60
  242. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8149583c>] ? ip_forward+0x1fc/0x430
  243. Aug 20 20:36:57 br1-7 kernel: [<ffffffff814939cd>] ? ip_rcv_finish+0x12d/0x440
  244. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81493f55>] ? ip_rcv+0x275/0x350
  245. Aug 20 20:36:57 br1-7 kernel: [<ffffffff814893a9>] ? nf_iterate+0x69/0xb0
  246. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145b54b>] ? __netif_receive_skb+0x4ab/0x750
  247. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145f160>] ? netif_receive_skb+0x0/0x60
  248. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145f1b8>] ? netif_receive_skb+0x58/0x60
  249. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa0295ad8>] ? br_handle_frame_finish+0x1e8/0x2a0 [bridge]
  250. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa029b938>] ? br_nf_pre_routing_finish+0x238/0x350 [bridge]
  251. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa029bedb>] ? br_nf_pre_routing+0x48b/0x7b0 [bridge]
  252. Aug 20 20:36:57 br1-7 kernel: [<ffffffff814893a9>] ? nf_iterate+0x69/0xb0
  253. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa02958f0>] ? br_handle_frame_finish+0x0/0x2a0 [bridge]
  254. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81489566>] ? nf_hook_slow+0x76/0x120
  255. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa02958f0>] ? br_handle_frame_finish+0x0/0x2a0 [bridge]
  256. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa0295d1c>] ? br_handle_frame+0x18c/0x250 [bridge]
  257. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81500220>] ? vlan_hwaccel_do_receive+0x70/0x130
  258. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145b5c9>] ? __netif_receive_skb+0x529/0x750
  259. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145f1b8>] ? netif_receive_skb+0x58/0x60
  260. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8145f2c0>] ? napi_skb_finish+0x50/0x70
  261. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81500994>] ? vlan_gro_receive+0x84/0xa0
  262. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa013ae68>] ? ixgbe_poll+0xce8/0x12c0 [ixgbe]
  263. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa01ecbe9>] ? nf_conntrack_free+0x59/0x70 [nf_conntrack]
  264. Aug 20 20:36:57 br1-7 kernel: [<ffffffffa01ed876>] ? destroy_conntrack+0xd6/0x150 [nf_conntrack]
  265. Aug 20 20:36:57 br1-7 kernel: [<ffffffff81460b43>] ? net_rx_action+0x103/0x2f0
  266. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8107a8e1>] ? __do_softirq+0xc1/0x1e0
  267. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8100c30c>] ? call_softirq+0x1c/0x30
  268. Aug 20 20:36:57 br1-7 kernel: <EOI>  [<ffffffff8100fa75>] ? do_softirq+0x65/0xa0
  269. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8107a4a0>] ? ksoftirqd+0x80/0x110
  270. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8107a420>] ? ksoftirqd+0x0/0x110
  271. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8109aef6>] ? kthread+0x96/0xa0
  272. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8100c20a>] ? child_rip+0xa/0x20
  273. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8109ae60>] ? kthread+0x0/0xa0
  274. Aug 20 20:36:57 br1-7 kernel: [<ffffffff8100c200>] ? child_rip+0x0/0x20
  275. Aug 20 20:36:57 br1-7 kernel: RES=0x00 SYN URGP=0 MARK=0xa
  276. Aug 20 20:37:04 br1-7 kernel: Clocksource tsc unstable (delta = -8589922803 ns).  Enable clocksource failover by adding clocksource_failover kernel parameter.
  277. Aug 20 20:37:04 br1-7 kernel: PHYSIN=p1p2.1017 SRC=106.74.24.238 DST=58.251.139.210 LEN=64 TOS=0x00 PREC=0x00 TTL=56 ID=44707 DF PROTO=TCP SPT=51061 DPT=443 WINDOW=65535 RES=0x00 SYN URGP=0 MARK=0xa
  278. Aug 20 20:37:18 br1-7 kernel: Clocksource tsc unstable (delta = -8589923406 ns).  Enable clocksource failover by adding clocksource_failover kernel parameter.
  279. Aug 20 20:37:18 br1-7 kernel: PHYSIN=p1p2.1017 SRC=106.74.24.250 DST=1.28.145.96 LEN=116 TOS=0x00 PREC=0x00 TTL=120 ID=65088 PROTO=UDP SPT=19107 DPT=17788 LEN=96 MARK=0xa
  280. Aug 20 20:37:57 br1-7 kernel: BUG: soft lockup - CPU#8 stuck for 65s! [ksoftirqd/8:37]
  281. Aug 20 20:37:57 br1-7 kernel: Modules linked in: ebtable_filter iptable_raw httpcontentchange(U) xt_state xt_physdev iptable_mangle xt_MARK ebt_redirect ebt_ip ebtable_nat ebtables bridge ipt_LOG act_police cls_u32 sch_ingress sch_prio sch_sfq sch_htb ipip tunnel4 ip_gre ip_tunnel xt_TCPMSS xt_tcpmss nf_nat_ftp nf_conntrack_ftp iptable_nat iptable_filter ip_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 8021q garp stp llc ipv6 dm_mod power_meter iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core ixgbe dca ptp pps_core mdio i7core_edac edac_core sg bnx2 ext4 jbd2 mbcache sd_mod crc_t10dif pata_acpi ata_generic ata_piix mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
  282. Aug 20 20:37:57 br1-7 kernel: CPU 8
  283. Aug 20 20:37:57 br1-7 kernel: Modules linked in: ebtable_filter iptable_raw httpcontentchange(U) xt_state xt_physdev iptable_mangle xt_MARK ebt_redirect ebt_ip ebtable_nat ebtables bridge ipt_LOG act_police cls_u32 sch_ingress sch_prio sch_sfq sch_htb ipip tunnel4 ip_gre ip_tunnel xt_TCPMSS xt_tcpmss nf_nat_ftp nf_conntrack_ftp iptable_nat iptable_filter ip_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 8021q garp stp llc ipv6 dm_mod power_meter iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core ixgbe dca ptp pps_core mdio i7core_edac edac_core sg bnx2 ext4 jbd2 mbcache sd_mod crc_t10dif pata_acpi ata_generic ata_piix mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
  284. Aug 20 20:37:57 br1-7 kernel:
  285. Aug 20 20:37:57 br1-7 kernel: Pid: 37, comm: ksoftirqd/8 Tainted: G        W  ---------------    2.6.32-431.el6.x86_64 #1 Dell Inc. PowerEdge R410/0N83VF
  286. Aug 20 20:37:57 br1-7 kernel: RIP: 0010:[<ffffffff8152a42b>]  [<ffffffff8152a42b>] _spin_lock_bh+0x2b/0x40
  287. Aug 20 20:37:57 br1-7 kernel: RSP: 0018:ffff880133c83598  EFLAGS: 00000297
  288. Aug 20 20:37:57 br1-7 kernel: RAX: 00000000000017c3 RBX: ffff880133c835a8 RCX: 0000000000000000
  289. Aug 20 20:37:57 br1-7 kernel: RDX: 00000000000017c2 RSI: 0000000000000004 RDI: ffffffffa028f430
  290. Aug 20 20:37:57 br1-7 kernel: RBP: ffffffff8100bb93 R08: ffff88022a4f2020 R09: ffff880133c83608
  291. Aug 20 20:37:57 br1-7 kernel: R10: ffff88020c4049e0 R11: ffff88012619a020 R12: ffff880133c83510
  292. Aug 20 20:37:57 br1-7 kernel: R13: ffffffffa028f430 R14: ffff880133c83500 R15: ffffffff815310a5
  293. Aug 20 20:37:57 br1-7 kernel: FS:  0000000000000000(0000) GS:ffff880133c80000(0000) knlGS:0000000000000000
  294. Aug 20 20:37:57 br1-7 kernel: CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
  295. Aug 20 20:37:57 br1-7 kernel: CR2: 00007fee4063fc7f CR3: 000000022adf0000 CR4: 00000000000007e0
  296. Aug 20 20:37:57 br1-7 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  297. Aug 20 20:37:57 br1-7 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  298. Aug 20 20:37:57 br1-7 kernel: Process ksoftirqd/8 (pid: 37, threadinfo ffff88022e074000, task ffff88022e073540)
  299. Aug 20 20:37:57 br1-7 kernel: Stack:
  300. Aug 20 20:37:57 br1-7 kernel: ffff880133c83608 ffff88022a4f2020 ffff880133c835e8 ffffffffa028e9f7
  301. Aug 20 20:37:57 br1-7 kernel: <d> ffff880133c83628 ffff88020c404970 ffff88020d123e80 ffff880133c83708
  302. Aug 20 20:37:57 br1-7 kernel: <d> ffff8801f3194030 ffff88020c404b70 ffff880133c83618 ffffffffa028ebca
  303. Aug 20 20:37:57 br1-7 kernel: Call Trace:
  304. Aug 20 20:37:57 br1-7 kernel: <IRQ>
  305. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa028e9f7>] ? ipt_log_packet+0x47/0x1d0 [ipt_LOG]
  306. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa028ebca>] ? log_tg+0x4a/0x54 [ipt_LOG]
  307. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa023220b>] ? ipt_do_table+0x3cb/0x678 [ip_tables]
  308. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8144f7f7>] ? __kfree_skb+0x47/0xa0
  309. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8144f912>] ? kfree_skb+0x42/0x90
  310. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8147bdaf>] ? __qdisc_run+0x3f/0xe0
  311. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa02320d5>] ? ipt_do_table+0x295/0x678 [ip_tables]
  312. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa023c224>] ? nf_nat_rule_find+0x24/0x80 [iptable_nat]
  313. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa023c4c1>] ? nf_nat_fn+0x111/0x260 [iptable_nat]
  314. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa023c738>] ? nf_nat_out+0x48/0xd0 [iptable_nat]
  315. Aug 20 20:37:57 br1-7 kernel: [<ffffffff814893a9>] ? nf_iterate+0x69/0xb0
  316. Aug 20 20:37:57 br1-7 kernel: [<ffffffff81499f90>] ? ip_finish_output+0x0/0x310
  317. Aug 20 20:37:57 br1-7 kernel: [<ffffffff81489566>] ? nf_hook_slow+0x76/0x120
  318. Aug 20 20:37:57 br1-7 kernel: [<ffffffff81499f90>] ? ip_finish_output+0x0/0x310
  319. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8149a344>] ? ip_output+0xa4/0xc0
  320. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8149562f>] ? ip_forward_finish+0x4f/0x60
  321. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8149583c>] ? ip_forward+0x1fc/0x430
  322. Aug 20 20:37:57 br1-7 kernel: [<ffffffff814939cd>] ? ip_rcv_finish+0x12d/0x440
  323. Aug 20 20:37:57 br1-7 kernel: [<ffffffff81493f55>] ? ip_rcv+0x275/0x350
  324. Aug 20 20:37:57 br1-7 kernel: [<ffffffff814893a9>] ? nf_iterate+0x69/0xb0
  325. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8145b54b>] ? __netif_receive_skb+0x4ab/0x750
  326. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8145f160>] ? netif_receive_skb+0x0/0x60
  327. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8145f1b8>] ? netif_receive_skb+0x58/0x60
  328. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa0295ad8>] ? br_handle_frame_finish+0x1e8/0x2a0 [bridge]
  329. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa029b938>] ? br_nf_pre_routing_finish+0x238/0x350 [bridge]
  330. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa029bedb>] ? br_nf_pre_routing+0x48b/0x7b0 [bridge]
  331. Aug 20 20:37:57 br1-7 kernel: [<ffffffff814893a9>] ? nf_iterate+0x69/0xb0
  332. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa02958f0>] ? br_handle_frame_finish+0x0/0x2a0 [bridge]
  333. Aug 20 20:37:57 br1-7 kernel: [<ffffffff81489566>] ? nf_hook_slow+0x76/0x120
  334. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa02958f0>] ? br_handle_frame_finish+0x0/0x2a0 [bridge]
  335. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa0295d1c>] ? br_handle_frame+0x18c/0x250 [bridge]
  336. Aug 20 20:37:57 br1-7 kernel: [<ffffffff81500220>] ? vlan_hwaccel_do_receive+0x70/0x130
  337. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8145b5c9>] ? __netif_receive_skb+0x529/0x750
  338. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8145f1b8>] ? netif_receive_skb+0x58/0x60
  339. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8145f2c0>] ? napi_skb_finish+0x50/0x70
  340. Aug 20 20:37:57 br1-7 kernel: [<ffffffff81500994>] ? vlan_gro_receive+0x84/0xa0
  341. Aug 20 20:37:57 br1-7 kernel: [<ffffffffa013ae68>] ? ixgbe_poll+0xce8/0x12c0 [ixgbe]
  342. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8145b5c9>] ? __netif_receive_skb+0x529/0x750
  343. Aug 20 20:37:57 br1-7 kernel: [<ffffffff81460b43>] ? net_rx_action+0x103/0x2f0
  344. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8107a8e1>] ? __do_softirq+0xc1/0x1e0
  345. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8100c30c>] ? call_softirq+0x1c/0x30
  346. Aug 20 20:37:57 br1-7 kernel: <EOI>
  347. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8100fa75>] ? do_softirq+0x65/0xa0
  348. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8107a4a0>] ? ksoftirqd+0x80/0x110
  349. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8107a420>] ? ksoftirqd+0x0/0x110
  350. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8109aef6>] ? kthread+0x96/0xa0
  351. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8100c20a>] ? child_rip+0xa/0x20
  352. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8109ae60>] ? kthread+0x0/0xa0
  353. Aug 20 20:37:57 br1-7 kernel: [<ffffffff8100c200>] ? child_rip+0x0/0x20
  354. Aug 20 20:37:57 br1-7 kernel: Code: 55 48 89 e5 53 48 83 ec 08 0f 1f 44 00 00 48 89 fb e8 8a f8 b4 ff b8 00 00 01 00 f0 0f c1 03 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 13 eb f5 83 3b 00 75 f4 eb df 48 83 c4 08 5b c9 c3 66 90


复制代码
devices, 服务器, system, Linux, 网桥
iceblood

论坛徽章:
1
金牛座 日期:2014-05-29 15:55:47
发表于 2015-08-21 13:30 |显示全部楼层
完整错误信息见附件。

系统错误信息.zip

31.93 KB, 下载次数: 10
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
philarlala

论坛徽章:
1
2017金鸡报晓 日期:2017-01-10 15:13:29
发表于 2015-08-21 16:44 |显示全部楼层
哇，跟我现在做的东西一样
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
philarlala

论坛徽章:
1
2017金鸡报晓 日期:2017-01-10 15:13:29
发表于 2015-08-31 15:22 |显示全部楼层
我不能回复站内信息，你贴出来的死机信息是一次的死机信息，还是很多次的，内核是什么版本的回复 2# iceblood


   
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
jhl19880722

论坛徽章:
0
发表于 2015-10-09 21:51 |显示全部楼层
本帖最后由 jhl19880722 于 2015-10-09 22:29 编辑

nf_nat_mangle_tcp_packet,里面有没有可能收到的skb不是线性的数据，但是在nf_nat_mangle_tcp_packet和mangle_contents中都有判断线性的bug_on，不知道内核中有没有把debug打开，瞎猜的。
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
返回列表 发新帖
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

  

北京盛拓优讯信息技术有限公司. 版权所有 京ICP备16024965号-6 北京市公安局海淀分局网监中心备案编号：11010802020122 niuxiaotong@pcpop.com 17352615567
未成年举报专区
中国互联网协会会员  联系我们：huangweiwei@itpub.net
感谢所有关心和支持过ChinaUnix的朋友们 转载本站内容请注明原作者名及出处

清除 Cookies - ChinaUnix - Archiver - WAP - TOP