求助T4-2自动重启原因

laiqifei

最近公司一台T4-2老是自动重启，求助重启真正原因：
疑点如下：
1.上面部署的是应用是tocmat和apache,每天应用都有自动重启，但是从性能图上看，内存没有真正的释放，标注的4,3,2 应用重启，内存虽然有释放，但是
整个系统的内存使用率比上次应用重启的高 如下图

2.每次系统DOWN机的时候，内存使用率都100%，但是以SOLARIS的机制来看，当内存用完，还会使用SWAP，也就是应用变慢，不会使机器重启
3.系统日志重启日志如下


Jan 29 15:39:23 msuu067 unix: [ID 330770 kern.notice] send mondo timeout [retries: 0x49355]  cpuids:
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x38
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x39
Jan 29 15:39:23 msuu067 unix: [ID 330770 kern.notice] send mondo timeout [retries: 0x48e62]  cpuids:
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3a
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x38
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3b
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x39
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3c
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3a
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3d
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3b
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3e
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3c
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3f
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3d
Jan 29 15:39:23 msuu067 unix: [ID 100000 kern.notice]
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3e
Jan 29 15:39:23 msuu067 unix: [ID 152697 kern.notice]  0x3f
Jan 29 15:39:23 msuu067 unix: [ID 100000 kern.notice]
Jan 29 15:39:23 msuu067 unix: [ID 350512 kern.notice] panic: failed to stop cpu56
Jan 29 15:39:23 msuu067 unix: [ID 350512 kern.notice] panic: failed to stop cpu57
Jan 29 15:39:23 msuu067 unix: [ID 350512 kern.notice] panic: failed to stop cpu58
Jan 29 15:39:23 msuu067 unix: [ID 350512 kern.notice] panic: failed to stop cpu59
Jan 29 15:39:23 msuu067 unix: [ID 350512 kern.notice] panic: failed to stop cpu60
Jan 29 15:39:23 msuu067 unix: [ID 350512 kern.notice] panic: failed to stop cpu61
Jan 29 15:39:23 msuu067 unix: [ID 350512 kern.notice] panic: failed to stop cpu62
Jan 29 15:39:23 msuu067 unix: [ID 350512 kern.notice] panic: failed to stop cpu63
Jan 29 15:39:23 msuu067 unix: [ID 836849 kern.notice]
Jan 29 15:39:23 msuu067 ^Mpanic[cpu65]/thread=3000cc67400:
Jan 29 15:39:23 msuu067 unix: [ID 152620 kern.notice] send_mondo_set: timeout
Jan 29 15:39:23 msuu067 unix: [ID 100000 kern.notice]
Jan 29 15:39:23 msuu067 genunix: [ID 723222 kern.notice] 000002a107170ec0 unix:send_mondo_set+4a4 (8, 10, 1d709fdb78d62, 77, 3000b4f
6600, 49355)
Jan 29 15:39:23 msuu067 genunix: [ID 179002 kern.notice]   %l0-3: 000000000183c730 0000000000000008 000000000000ffff 0001d709fdb78f7
4
Jan 29 15:39:23 msuu067   %l4-7: 0000000001dc7ac2 0000000000000000 00000000010b8120 0000000000000000
Jan 29 15:39:23 msuu067 genunix: [ID 723222 kern.notice] 000002a107170fa0 unixt_some+194 (2a107171210, 2a107171090, ffffffffffffff
ff, fffffffffffffff8, 2a107171058, 0)
Jan 29 15:39:23 msuu067 genunix: [ID 179002 kern.notice]   %l0-3: 00000000018919c4 000002a107171090 0000000000000002 000000000000000
1
Jan 29 15:39:23 msuu067   %l4-7: fffffffffffffffd 000002a1071710d0 000002a107171098 0000000000000008
Jan 29 15:39:23 msuu067 genunix: [ID 723222 kern.notice] 000002a107171110 unixt_sync+148 (2a107171520, ffffffffffffffff, fffffffff
ffffff8, 1043c00, 3000b4f6000, 2a10717121
Jan 29 15:39:23 msuu067 genunix: [ID 179002 kern.notice]   %l0-3: 0000000000000008 0001d709c2282f72 000000000000007f 000002a10717156
0
Jan 29 15:39:23 msuu067   %l4-7: 0000000000000001 000000000000007f 000000000000003f 0000000000000001
Jan 29 15:39:23 msuu067 genunix: [ID 723222 kern.notice] 000002a107171420 unix:hat_unload_callback+824 (ffffffffffffe000, 73010000,
0, fffffffffffffff8, 0, 3000b361280)
Jan 29 15:39:23 msuu067 genunix: [ID 179002 kern.notice]   %l0-3: 0000000000000001 000002a107171528 000002a1071717e8 000000000000000
0

东方蜘蛛

做过POST吗？检查下CPU是否有问题～

laiqifei

CPU是没有问题的
回复 1# laiqifei


   

wait空白

压力测试跑跑，看看是不是有泄漏。

znnnz

是不是也有内存利用率的参数设置来防止内存耗尽啊

hjp1973

Pls see "SPARC T4 systems seeing "send_mondo" panics followed by a system hang [ID 1482273.1]" in handbook

东方蜘蛛

Solution
This is NOT a hardware issue and so NO hardware of any kind should be replaced.

If you see this type of panic please open a Service Request (SR) to the VSP SPARC T4 Hardware Group in My Oracle Support (MOS) and either upload or attach the corefile, explorer, and ILOM snapshot for analysis.

ericyan_cu

内存的插法是否正确？

estar

升级ilom到最新版本