aix 忽然当机，请教—十万火急

wwapple

System Model: IBM,7040-681
aix 5200-02
装有CICS、MQ Broker、DB2（Db2是为MQ Broker服务的，不是单独用来做数据库用的）

忽然间出现当机。
# errpt|more
IDENTIFIER TIMESTAMP  T C RESOURCE_NAME  DESCRIPTION
B6048838   1203021405 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
B6048838   1202214405 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
813FE820   1202180905 U S MQSeries       SOFTWARE PROGRAM ERROR
813FE820   1202180905 U S MQSeries       SOFTWARE PROGRAM ERROR
AFA89905   1202180805 I O grpsvcs        Group Services daemon started
97419D60   1202180805 I O topsvcs        Topology Services daemon started
A6DF45AA   1202170805 I O RMCdaemon      The daemon is started.
B38E3397   1202170605 U S SYSDUMP        Previous system dump information
E85C5C4C   1202170605 P S   CFGLFT       SOFTWARE PROGRAM ERROR
C0AA5338   1202170605 U S SYSDUMP        SYSTEM DUMP
9D035E4D   1202165805 P S SYSVMM         DATA STORAGE INTERRUPT, PROCESSOR
9DBCFDEE   1202170605 T O errdemon       ERROR LOGGING TURNED ON
B6048838   1202105905 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
B6048838   1202105905 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
B6048838   1202105905 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
B6048838   1202105905 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
B6048838   1202105905 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
B6048838   1202105905 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
B6048838   1202105905 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
B6048838   1202105905 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
B6048838   1202105905 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
B6048838   1202105905 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
# errpt -aj E85C5C4C
---------------------------------------------------------------------------
LABEL:          LFTERR
IDENTIFIER:     E85C5C4C

Date/Time:       Fri Dec  2 17:06:41 BEIS
Sequence Number: 56236
Machine Id:      0030CBFB4C00
Node Id:         localhost
Class:           S
Type:            PERM
Resource Name:     CFGLFT

Description
SOFTWARE PROGRAM ERROR

Probable Causes
SOFTWARE PROGRAM

Failure Causes
SOFTWARE PROGRAM

        Recommended Actions
        IF PROBLEM CONTINUES TO OCCUR REPEATEDLY THEN DO THE FOLLOWING
        CONTACT APPROPRIATE SERVICE REPRESENTATIVE
        REPORT DETAILED DATA

Detail Data
DETECTED   FAILED    RC   ERROR LOCATION
  cfglft              -1  1122  14
# errpt -aj 9D035E4D
---------------------------------------------------------------------------
LABEL:          DSI_PROC
IDENTIFIER:     9D035E4D

Date/Time:       Fri Dec  2 16:58:12 BEIS
Sequence Number: 56234
Machine Id:      0030CBFB4C00
Node Id:         localhost
Class:           S
Type:            PERM
Resource Name:   SYSVMM

Description
DATA STORAGE INTERRUPT, PROCESSOR

Probable Causes
SOFTWARE PROGRAM

Failure Causes
SOFTWARE PROGRAM

        Recommended Actions
        IF PROBLEM PERSISTS THEN DO THE FOLLOWING
        CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
DATA STORAGE INTERRUPT STATUS REGISTER
0000 0000 0000 0000
SEGMENT REGISTER, SEGREG
4200 0000 0000 0001
DATA STORAGE INTERRUPT ADDRESS REGISTER
1089 1400 F100 0089
EXVAL
420F 17C8 0000 0000
# errpt -aj A6DF45AA B38E3397
---------------------------------------------------------------------------
LABEL:          RMCD_INFO_0_ST
IDENTIFIER:     A6DF45AA

Date/Time:       Fri Dec  2 17:08:06 BEIS
Sequence Number: 56239
Machine Id:      0030CBFB4C00
Node Id:         sdtccm
Class:           O
Type:            INFO
Resource Name:   RMCdaemon

Description
The daemon is started.

Probable Causes
The Resource Monitoring and Control daemon has been started.

User Causes
The startsrc -s ctrmc command has been executed or
the rmcctrl -s command has been executed.

        Recommended Actions
        Confirm that the daemon should be started.

Detail Data
DETECTING MODULE
RSCT,rmcd.c,1.37,202
ERROR ID
6eKora0qv.Y1/0tJ.05.D8....................
REFERENCE CODE

老农

该详细看的不看。。。。。
B6048838

peapeacat

好象是软件引起的, 机器面板显示什么信息?

wwapple

# errpt -aj B6048838
--------------------------------------------------------------------------------------------------------
LABEL:          CORE_DUMP
IDENTIFIER:     B6048838

Date/Time:       Sat Dec  3 02:14:32 BEIS
Sequence Number: 56245
Machine Id:      0030CBFB4C00
Node Id:         sdtccm
Class:           S
Type:            PERM
Resource Name:   SYSPROC

Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED

Probable Causes
SOFTWARE PROGRAM

User Causes
USER GENERATED SIGNAL

        Recommended Actions
        CORRECT THEN RETRY


Failure Causes
SOFTWARE PROGRAM

        Recommended Actions
        RERUN THE APPLICATION PROGRAM
        IF PROBLEM PERSISTS THEN DO THE FOLLOWING
        CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
SIGNAL NUMBER
           5
USER'S PROCESS ID:
               1282166
FILE SYSTEM SERIAL NUMBER
          17
INODE NUMBER
        4191
PROCESSOR ID
           2
CORE FILE NAME
/var/cics_regions/CICSSDTC/dumps/dir1/core
PROGRAM NAME
cicsas
ADDITIONAL INFORMATION
_internal 7C
??
_global_l 208
_rec_mute E4
localtime 98
bde_local D8
SupER_IFl 5C8
SupER_Flu 4C
TasTA_Ter 9D8
TasTA_Run 1C78
main BD0
__start 8C

Symptom Data
REPORTABLE
1
INTERNAL ERROR
0
SYMPTOM CODE
PCSS/SPI2 FLDS/cicsas SIG/5 FLDS/_internal VALU/7c

---------------------------------------------------------------------------
LABEL:          CORE_DUMP
IDENTIFIER:     B6048838

Date/Time:       Fri Dec  2 10:59:20 BEIS
Sequence Number: 56040
Machine Id:      0030CBFB4C00
Node Id:         sdtccm
Class:           S
Type:            PERM
Resource Name:   SYSPROC

Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED

Probable Causes
SOFTWARE PROGRAM

User Causes
USER GENERATED SIGNAL

        Recommended Actions
        CORRECT THEN RETRY

Failure Causes
SOFTWARE PROGRAM

        Recommended Actions
        RERUN THE APPLICATION PROGRAM
        IF PROBLEM PERSISTS THEN DO THE FOLLOWING
        CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
SIGNAL NUMBER
           6
USER'S PROCESS ID:
               1253526
FILE SYSTEM SERIAL NUMBER
          27
INODE NUMBER
           2
PROCESSOR ID
          -1
CORE FILE NAME
/home/db2inst1/core
PROGRAM NAME
bipbroker
ADDITIONAL INFORMATION
pthread_k AC
??
_p_raise 58
raise 34
abort B8
terminate 16C
logSystem 11C
4#,-03- 6B8
54
??
__start 8C
Unable to generate symptom string.
Bad return from kprobe().
---------------------------------------------------------------------------
LABEL:          CORE_DUMP
IDENTIFIER:     B6048838

Date/Time:       Fri Dec  2 10:28:29 BEIS
Sequence Number: 55857
Machine Id:      0030CBFB4C00
Node Id:         sdtccm
Class:           S
Type:            PERM
Resource Name:   SYSPROC

Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED

Probable Causes
SOFTWARE PROGRAM

User Causes
USER GENERATED SIGNAL

        Recommended Actions
        CORRECT THEN RETRY

Failure Causes
SOFTWARE PROGRAM

        Recommended Actions
        RERUN THE APPLICATION PROGRAM
        IF PROBLEM PERSISTS THEN DO THE FOLLOWING
        CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
SIGNAL NUMBER
           5
USER'S PROCESS ID:
                778370
FILE SYSTEM SERIAL NUMBER
          28
INODE NUMBER
        4191
PROCESSOR ID
           2
CORE FILE NAME
/var/cics_regions/CICSSDTC/dumps/dir1/core
PROGRAM NAME
cicsas
ADDITIONAL INFORMATION
_internal 7C
??
_global_l 208
_rec_mute E4
localtime 98
bde_local D8
SupER_IFl 5C8
SupER_Flu 4C
TasTA_Ter 9D8
TasTA_Run 1C78
main BD0
__start 8C

Symptom Data
REPORTABLE
1
INTERNAL ERROR
0
SYMPTOM CODE
PCSS/SPI2 FLDS/cicsas SIG/5 FLDS/_internal VALU/7c
# errpt -aj 813FE820
---------------------------------------------------------------------------
LABEL:          AMQFFST1
IDENTIFIER:     813FE820

Date/Time:       Fri Dec  2 18:09:38 BEIS
Sequence Number: 56243
Machine Id:      0030CBFB4C00
Node Id:         sdtccm
Class:           S
Type:            UNKN
Resource Name:   MQSeries        NONE

Description
SOFTWARE PROGRAM ERROR

Probable Causes
UNDETERMINED

Failure Causes
SOFTWARE PROGRAM

        Recommended Actions
        REVIEW DETAILED DATA
        CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
DETECTING MODULE
lpiObtainQMDetails
SOFTWARE ERROR CODE
0000 0000
FILE NAME
/var/mqm/errors/AMQ42368.0.FDC
---------------------------------------------------------------------------
LABEL:          AMQFFST1
IDENTIFIER:     813FE820

Date/Time:       Fri Dec  2 18:09:38 BEIS
Sequence Number: 56242
Machine Id:      0030CBFB4C00
Node Id:         sdtccm
Class:           S
Type:            UNKN
Resource Name:   MQSeries        NONE

Description
SOFTWARE PROGRAM ERROR

Probable Causes
UNDETERMINED

Failure Causes
SOFTWARE PROGRAM

        Recommended Actions
        REVIEW DETAILED DATA
        CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
DETECTING MODULE
zutPreReadMachineIniFile
SOFTWARE ERROR CODE
0000 0001
FILE NAME
/var/mqm/errors/AMQ42368.0.FDC
我把# errpt -aj B6048838贴出来，在B6048838中，大部分是DB2 core的内容和几个CICS core的内容。
供楼上参考。

不过这个error的时间和当机的时间相差5个小时，真的是他么？

[ 本帖最后由 wwapple 于 2005-12-3 21:45 编辑 ]

wwapple

回复楼上，由于是突然当机，事后才发现的，所以机器面板上显示什么信息，现在也无从查起了。

老农

那也就是说宕机的时候没产生errorlog。。。。
要仔细查看你的系统了，呵呵

wwapple

我在网上也看到了一些类似的问题，也都是各说各的，虽然观点不少，但是好像对问题都帮助不大。
请教各位，能不能根据经验判断问题可能出在哪？
软件还是硬件？系统软件还是应用软件？

lpcome2004

CICS+MQ+DB2,恩,环境不错...属于有钱的公司...建议把那几个core dump传给IBM让他们分析去把!