主机宕机，dump的分析问题
主机上周无故宕机了，跟系统资源无关。以下是kdb分析dump的结果，麻烦各位大神看看是什么原因导致宕机的，谢谢！(12)> stat
SYSTEM_CONFIGURATION:
CHRP_SMP_PCI POWER_PC POWER_5 machine with 32 available CPU(s)(64-bit registers)
SYSTEM STATUS:
sysname... AIX
nodename.. XXXXXXX
release... 1
version... 6
build date Apr6 2011
build time 12:40:25
label..... 1114A_61N
machine... 00CE9A1B4C00
nid....... CE9A1B4C
time of crash: Sat Nov9 13:46:45 2013
age of system: 85 day, 12 hr., 45 min., 11 sec.
xmalloc debug: enabled
FRRs active... 0
FRRs started.. 0
CRASH INFORMATION:
CPU 12 CSA F00000003004D600 at time of crash, error code for LEDs: 30000000
pvthread+094200 STACK:
fpget+000204 (F10001001839EB80, 0000000002BD0180,
F10001001839EB80 [??])
fpalloc+000054 (??, ??, ??, ??, ??)
socksetup+000134 (??, ??)
socketpair+00058C (??, ??, ??, ??)
ovlya_addr_sc_flih_main+000130 ()
no real storage @ 11121A3A0
0900000000B19160 ()
no real storage @ FFFFFFFFFFF6650
(12)> status
CPU TIDTSLOT PIDPSLOTPROC_NAME
0 20005 2 20004 2wait
1 160039 22 C0024 12wait
2 17003B 23 D0026 13wait
3 18003D 24 E0028 14wait
4 1A0041 26 F002A 15wait
5 1B0043 2710002C 16wait
6 1C0045 2811002E 17wait
7 1D0047 29120030 18wait
8 1E0049 30130032 19wait
9 1F004B 31140034 20wait
10 20004D 32150036 21wait
11 21004F 33160038 22wait
12 1420135 23702D013C 1069cssdmonitor
13 230053 3518003C 24wait
14 240055 3619003E 25wait
15 250057 371A0040 26wait
16 20159 2050 20142 1026wait
17 3015B 2051 30144 1027wait
18 4015D 2052 40146 1028wait
19 5015F 2053 50148 1029wait
20 60161 2054 6014A 1030wait
21 70163 2055 7014C 1031wait
22 80165 2056 8014E 1032wait
23 90167 2057 90150 1033wait
24 A0169 2058 A0152 1034wait
25 B016B 2059 B0154 1035wait
26 C016D 2060 C0156 1036wait
27 D016F 2061 D0158 1037wait
28 E0171 2062 E015A 1038wait
29 F0173 2063 F015C 1039wait
30 100175 206410015E 1040wait
31 110177 2065110160 1041wait
32-127 Disabled
(12)> cpu 12
current cpu
(12)> proc
SLOT NAME STATE PID PPID ADSPACECL #THS
pvproc+10B400 1069*cssdmoni ACTIVE 02D013C 0000001 0000000B00DB0400 0 0012
NAME....... cssdmonitor
STATE...... stat:07.... xstat :0000
FLAGS...... flag:00200001 LOAD EXECED
........... flag2 :02000001 64BIT INHERITED
........... flag3 :00000102 NOSWAP FIXPRI
........... atomic :00040000 ORPHANPGRP
........... secflag:0001 ROOT
LINKS...... child :0000000000000000
........... siblings :F1000F0A00012C00 <pvproc+012C00>
........... uidinfo :00000000022AAB60
........... ganchor :0000000000000000
THREAD..... threadlist :F1000F0A10095200 <pvthread+095200>
DISPATCH... synch :FFFFFFFFFFFFFFFF
AACCT...... projid :00000000........... sprojid :00000000
........... subproj :0000000000000000
........... file id :0000000000000000 0000000000000000 00000000
........... kcid :00000000
........... flags :0000
WLM........ class/wlm:00/0000
........... time of SIGTERM:00000000
........... wlm_nvpages :00000000000000000
........... totalcputime :000001EB8D5715C0
........... totalscputime :0000007A9C848531
........... totaldiskio :0000000000000000
IDENTIFIER. uid :00000000........... suid :00000000
........... pid :002D013C........... ppid :00000001
........... sid :00540040........... pgrp :00540040
MISC...... lock @ F1000F0A0010B4F0 0000000000000000
.......... lock_d @ F1000F0A0010B5A8 0000000000000000
..... parent_lock @ F1000F0A0010B5A0 0000000000000000
..... session_lock @ F1000F0A0010B598 0000000000000000
........... pgrpl :0000000000000000
........... pgrpb :0000000000000000... ttyl :0000000000000000
........... ipc :0000000000000000... sigs_queued:0
........... dblist :0000000000000000... dbnext :0000000000000000
........... eyec :7076707250524F43(pvprPROC)
STATISTICS. nframes :0000000000004DB4... npsblks :0000000000000000
........... nvpages :0000000000004DB4... auditmask:00000000
........... ncpages :0000000000000000
SCHEDULER.. sched_next :0000000000000000... sched_back :0000000000000000
......... usched_lock @ F1000F0A0010B510 0000000000000000
........... uschedp :0000000000000000
........... asyncio :0000000000000000
CHECKPOINT. crid :00000000........... crid_token :FFFFFFFF
........... cridnext :0000000000000000... chksynch :FFFFFFFF
........... vpid :00000000........... vppid :00000000
........... vsid :00000000........... vpgrp :00000000
PROCFS..... procfsvn :0000000000000000
NUMA....... rset :0000000000000000
EWLM....... ewlmproc :0000000000000000
PROC....... procp :F1000130085F9C00... size :00000328
....... pri :00................. policy :02
BOP........ bop_flags:0000.............. monitor_count :0000
FLAGS...... flag:00000000
........... flag2 :00040801 64BIT RT_GRQ
........... int :00000000
........... atomic:00000000
THREAD..... threadcount:00000012........... active :00000010
........... suspended:00000000........... terminating:00000000
........... local :00000000........... wlm :00000010
........... wlmoc :00000000
SCHEDULE... nice : 0........... sched_pri: 255
DISPATCH... pevent :0000000000000000
IDENTIFIER. pid :002D013C
MISC....... adspace :0000000B00DB0400
........... adtable :0000000864091001... adspace_ldr:0000000AD0DAD400
........... eyec :70726F6350524F43(procPROC)
........... uprobe :0000000000000000 ... forktime :000DE1BA6BF48F77
SIGNAL..... infoq :F1000128089D4400
........... pending : 0000000000000000
........................ 0000000000000000
........................ 0000000000000000
........................ 0000000000000000
........... sigignore: 0000000000000000
........................ 0000000000000000
........................ 0000000000000000
........................ 0601000038409002
........................ INT PIPE URG IO WINCH PWR
........................ USR1 RECOVERY RECONFIG CPUFAIL
........... sigcatch : 0000000000000000
........................ 0000000000000000
........................ 0000000000000000
........................ 0000000000004628
........................ ILL ABRT BUS SEGV TERM
........... siginfo : 0000000000000000
........................ 0000000000000000
........................ 0000000000000000
........................ 0000000000000000
STATISTICS. page size:0000000000004C03... minflt :0000000000005215
........... majflt :0000000000000001... pctcpu :0000004D
....... inputdiskio :0000000000001000
....... inputio ops :0000000000000001
....... outputdiskio :000000000045AC00
....... outputio ops :0000000000000429
....... logdiskio :0000000000000000
....... logio ops :0000000000000000
SCHEDULER.. repage :0000000000000000... sched_count:00000000
........... cpticks :0000............... msgcnt :0000
........... majfltsec:00000001
........... rs_attinfo :0000000000000000........... sradassign :FFFFFFFF
........... rs_rss :0000000000004DD3........... boundcount :0000
. no. of threads w/rset: 0........... w/srad : 0
CHECKPOINT. chkblock :00000000........... chkfile :0000000000000000
POSIX RT TIMERS :Data not present in dump.
CPU-time... clock ticks:0013F085
........... active :0000000000000000
PROCFS..... prtrcset :0000000000000000
PVPROC..... pvprocp :F1000F0A0010B400... size :00000400
(12)> errpt
ERRORS NOT READ BY ERRDEMON (ORDERED CHRONOLOGICALLY):
Error Record:
erec_flags .............. 1
erec_len ................ 58
erec_timestamp .......... 527DCC45
erec_rec_len ............ 34
erec_cid ................ 0
erec_dupcount ........... 0
erec_duptime1 ........... 0
erec_duptime2 ........... 0
erec_rec.error_id ....... 9D035E4D
erec_rec.resource_name .. SYSVMM
00000000 00200000 00007FFF FFFFD000..... ..........
00000000 80000017 00000000 0000000E................
(12)> dr iar
iar : 00000000005A6844
.fpget+000204 ld r4,18(r20) r4=0000000002BD0180,18(r20)=0000000080000017
(12)> vmlog
Most recent VMM errorlog entry
Error id =DSI_PROC
Exception DSISR/ISISR=0000000000200000
Exception srval =00007FFFFFFFD000
Exception virt addr =0000000080000017
Exception value =0000000EERRNO_EFAULT
(12)> th
SLOT NAME STATE TID PRI RQ CPUIDCLWCHAN
pvthread+094200 2370*cssdmoni RUN 1420135 03C512 0
NAME................ cssdmonitor
.................tid :0000000001420135......tsleep :FFFFFFFFFFFFFFFF
...............flags :00000000..............flags2 :00000000
...........pmcontext :00000000
DATA.........pvprocp :F1000F0A0010B400 <pvproc+10B400>
LINKS.....prevthread :F1000F0A10016700 <pvthread+016700>
..........nextthread :F1000F0A10096800 <pvthread+096800>
DISPATCH.......synch :FFFFFFFFFFFFFFFF
SCHEDULER...affinity :0000000C.................pri :0000003C
.............boosted :00000000...............wchan :0000000000000000
...............state :00000002...............wtype :00000000
......boost_cnt_lock :0000 .....boost_cnt_other :0000
MISC ..tv_eyec :7076746850524F43 (pvthPROC)
CHECKPOINT......vtid :00000000.............chkfile :0000000000000000
LOCK........ lock_d @ F1000F0A10094230 0000000000000000
PROCFS......procfsvn :0000000000000000
NUMA............rset :0000000000000000
PROFILING.....prbase :0000000000000000....prpinned :0000000000000000
.............prflags :00000000............prbufcount :00000000
WLM........class/wlm :00/0000
.............wlm_tag :
THREAD.......threadp :F100013008614400........size :00000100
FLAGS............... CDEFER REHOMED
.................tid :0000000001420135......stackp :000000011121A390
.................scp :0000000000000000.......ulock :0000000000000000
...............uchan :0000000000000000....userdata :0000000111224800
..................cv :0000000000000000.......flags :0000000000400000
..............atomic :0000000000000000......flags2 :0000000000002000
DATA...........procp :F1000130085F9C00 <F1000130085F9C00>
...........pvthreadp :F1000F0A10094200 <pvthread+094200>
...............userp :F00000002FF48000 <__ublock+000A00>
............uthreadp :F00000003004D600 <F00000003004D600>
SLEEP/LOCK......usid :0000000000000000......wchan1 :0000000000000000
..............wchan2 :0000000000000000......swchan :0000000000000000
...........eventlist :0000000000000000......result :00000002
.............polevel :00000000..............pevent :0000000000000000
..............wevent :0000000000000000.......slist :0000000000000000
...........wchan1sid :00000007FFFFFFFFwchan1offset :00000000
...........lockcount :00000001..........adsp_flags :0000
DISPATCH.......ticks :00000003...............prior :0000000000000000
................next :0000000000000000......dispct :0000000000E2FCB5
...............fpuct :0000000000E2411C...homecount :00000000
............pri_band :00 .............cputime :00000073DFCCB42D
.........near_dispct :0000000000000000..far_dispct :0000000000000000
........allowed_cpus :0-511
.......prefunnel_cpu :00000000.......dispatch_hist :00
......threadcontrolp :0000000000000000
MISC........graphics :0000000000000000.ulock_listp :0000000000000000
...........lockowner :0000000000000000..kthreadseg :0000000A20E22400
..........time_start :00036FD12E0B031B.......credp :0000000000000000
....spurr_time_start :00036FD0B4004A97
..........wlm_charge :0..........wlm_evtcnt :00000000
............ipc_data :0000000000000000
..............t_eyec :7468726450524F43
............t_waitTm :0000000000000000 (thrdPROC)
...............iopri :00000000
......t_smt_priority :4 NORMAL
VMM...........t_delw :0000000000000000
SIGNAL........sigproc:00000000..............cursig :00000000
......(pending) sig: 0000000000000000
...................... 0000000000000000
...................... 0000000000000000
...................... 0000000000000000
............sigmask: 0000000000000000
...................... 0000000000000000
...................... 0000000000000000
...................... 0000000020000000
...................... USR1
SCHEDULER......cpuid :FFFFFFFF..............scpuid :FFFFFFFF
.........affinity_ts :00000000..............policy :00000000
.................cpu :00000000.............lockpri :00000000
.............wakepri :000000FF...........rehome_tb :00000073DFCCB42D
.............ceiling :000000FF................time :000000FF
.............sav_pri :0000003C..............t_nice :0000003C
...........run_queue :F1000100182E8280......cpu_tb :001054AF
.............home_rq :F1000100182E8280....ldispcpu :0015
.........home_sradid :FFFFFFFF
......... rs_attinfo :0000000000000000
.............suspend :00000001.............fsflags :00000000
..........norun_secs :00000000.......reaffin_count :0005
CHECKPOINT..chkerror :0000 ............chkblock :00000000
TIMERS...clock ticks :0002B9E0
PROCFS.......whystop :00000000............whatstop :00000000
PVTHREAD...pvthreadp :F1000F0A10094200........size :00000100
页:
[1]