weblogic11g定时重启时，偶尔会拉不起服务，报VM错误，求大神分析原因

wlwpz

1. DOMAIN_HOME: /webapp/user_projects/domains/domain_8004
   
2. .
   
3. .
   
4. Source JAVA Memory arguments: -Xms512m -Xmx512m -XX:MaxPermSize=256m
   
5. .
   
6. WLS Start Mode=Production
   
7. .
   
8. CLASSPATH=/usr/local/bea/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/opt/java6/lib/tools.jar:/usr/local/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/usr/local/bea/wlserver_10.3/server/lib/weblogic.jar:/usr/local/bea/modules/features/weblogic.server.modules_10.3.6.0.jar:/usr/local/bea/wlserver_10.3/server/lib/webservices.jar:/usr/local/bea/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/usr/local/bea/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:/usr/local/bea/wlserver_10.3/common/derby/lib/derbyclient.jar:/usr/local/bea/wlserver_10.3/server/lib/xqrl.jar:/usr/local/bea/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/opt/java6/lib/tools.jar:/usr/local/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/usr/local/bea/wlserver_10.3/server/lib/weblogic.jar:/usr/local/bea/modules/features/weblogic.server.modules_10.3.6.0.jar:/usr/local/bea/wlserver_10.3/server/lib/webservices.jar:/usr/local/bea/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/usr/local/bea/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:/usr/local/bea/wlserver_10.3/common/derby/lib/derbyclient.jar:/usr/local/bea/wlserver_10.3/server/lib/xqrl.jar:/opt/java6/lib/dt.jar:/opt/java6/lib/tools.jar
   
9. .
   
10. PATH=/usr/local/bea/wlserver_10.3/server/bin:/usr/local/bea/modules/org.apache.ant_1.7.1/bin:/opt/java6/jre/bin:/opt/java6/bin:/usr/local/bea/wlserver_10.3/server/bin:/usr/local/bea/modules/org.apache.ant_1.7.1/bin:/opt/java6/jre/bin:/opt/java6/bin:/usr/sbin:/usr/bin:/usr/ccs/bin:/usr/contrib/bin:/usr/contrib/Q4/bin:/opt/perl/bin:/opt/ipf/bin:/opt/gvsd/bin:/opt/nettladm/bin:/opt/fcms/bin:/opt/wbem/bin:/opt/wbem/sbin:/opt/sas/bin:/opt/graphics/common/bin:/opt/atok/bin:/usr/bin/X11:/usr/contrib/bin/X11:/opt/sec_mgmt/bastille/bin:/opt/caliper/bin:/opt/drd/bin:/opt/dsau/bin:/opt/dsau/sbin:/opt/resmon/bin:/opt/firefox:/opt/gnome/bin:/usr/contrib/kwdb/bin:/opt/mozilla:/opt/perl_32/bin:/opt/perl_64/bin:/opt/prm/bin:/opt/sfm/bin:/opt/swm/bin:/opt/sec_mgmt/spc/bin:/opt/ssh/bin:/opt/sentinel/bin:/opt/swa/bin:/opt/hpsmh/bin:/opt/thunderbird:/opt/langtools/bin:/opt/gwlm/bin:/opt/ignite/bin:/sbin:/opt/java6/bin/IA64W:/home/weblogic
    
11. .
    
12. ***************************************************
    
13. *  To start WebLogic Server, use a username and   *
    
14. *  password assigned to an admin-level user.  For *
    
15. *  server administration, use the WebLogic Server *
    
16. *  console at http://hostname:port/console        *
    
17. ***************************************************
    
18. starting weblogic with Java version:
    
19. Starting WLS with line:
    
20. /opt/java6/bin/IA64W/java -server -d64   -Dport=8004 -Xms1792m -Xmx1792m -Xmn800m -XX:PermSize=512m -XX:MaxPermSize=512m   -Dweblogic.ProductionModeEnabled=true  -Xmn800m -XX:+UseParNewGC  -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC  -XX:+UseCompressedOops -XX:+ForceMmapReserved -XX:MaxTenuringThreshold=10 -XX:SurvivorRatio=12 -XX:+PrintHeapAtGC -Xloggc:./log/gc.log   -da -Dplatform.home=/usr/local/bea/wlserver_10.3 -Dwls.home=/usr/local/bea/wlserver_10.3/server -Dweblogic.home=/usr/local/bea/wlserver_10.3/server   -Dweblogic.management.discover=true  -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole=false -Dweblogic.ext.dirs=/usr/local/bea/patch_wls1036/profiles/default/sysext_manifest_classpath  -Dweblogic.Name=myServer -Djava.security.policy=/usr/local/bea/wlserver_10.3/server/lib/weblogic.policy  weblogic.Server
    
21. 
    
22. Java HotSpot(TM) 64-Bit Server VM warning: UseCompressedOops only supported with an MPAS Java launcher
    
23. <2014-6-13 上午04时51分33秒 GMT+08:00> <Info> <Security> <BEA-090905> <Disabling CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true>
    
24. <2014-6-13 上午04时51分33秒 GMT+08:00> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG to FIPS186PRNG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true>
    
25. <2014-6-13 上午04时51分33秒 GMT+08:00> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with Java HotSpot(TM) 64-Bit Server VM Version 20.6-b01-jre1.6.0.14-rc1b2 from Hewlett-Packard Company>
    
26. <2014-6-13 上午04时51分34秒 GMT+08:00> <Info> <Management> <BEA-141107> <Version: WebLogic Server 10.3.6.0  Tue Nov 15 08:52:36 PST 2011 1441050 >
    
27. <2014-6-13 上午04时51分37秒 GMT+08:00> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
    
28. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool>
    
29. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <WebLogicServer> <BEA-000214> <WebLogic Server "myServer" version:
    
30. WebLogic Server 10.3.6.0  Tue Nov 15 08:52:36 PST 2011 1441050  Copyright (c) 1995, 2011, Oracle and/or its affiliates. All rights reserved.>
    
31. <2014-6-13 上午04时51分37秒 GMT+08:00> <Notice> <Log Management> <BEA-170019> <The server log file /webapp/user_projects/domains/domain_8004/servers/myServer/logs/myServer.log is opened. All server side log events will be written to this file.>
    
32. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Log Management> <BEA-170023> <The Server Logging is initialized with Java Logging API implementation.>
    
33. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Diagnostics> <BEA-320001> <The ServerDebug service initialized successfully.>
    
34. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "t3" is now configured.>
    
35. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "t3s" is now configured.>
    
36. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "http" is now configured.>
    
37. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "https" is now configured.>
    
38. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "iiop" is now configured.>
    
39. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "iiops" is now configured.>
    
40. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "ldap" is now configured.>
    
41. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "ldaps" is now configured.>
    
42. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "cluster" is now configured.>
    
43. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "clusters" is now configured.>
    
44. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "snmp" is now configured.>
    
45. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "admin" is now configured.>
    
46. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <Server> <BEA-002624> <The administration protocol is "t3s" and is now configured.>
    
47. <2014-6-13 上午04时51分37秒 GMT+08:00> <Info> <RJVM> <BEA-000570> <Network Configuration for Channel "myServer"
    
48. Listen Address                 :8004
    
49. Public Address                 N/A
    
50. Http Enabled                 true
    
51. Tunneling Enabled         false
    
52. Outbound Enabled         false
    
53. Admin Traffic Enabled         true>
    
54. <2014-6-13 上午04时51分37秒 GMT+08:00> <Debug> <RJVM> <BEA-000571> <Network Configuration Detail for Channel "myServer"
    
55. Channel Weight                 50
    
56. Accept Backlog                 300
    
57. Login Timeout                 5000ms
    
58. Max Message Size         10000000
    
59. Message Timeout         60s
    
60. Idle Timeout                 65s
    
61. Tunneling Timeout         40s
    
62. Tunneling Ping                 45s>
    
63. <2014-6-13 上午04时51分38秒 GMT+08:00> <Info> <Server> <BEA-002609> <Channel Service initialized.>
    
64. <2014-6-13 上午04时51分38秒 GMT+08:00> <Info> <Socket> <BEA-000416> <Using effective file descriptor limit of: 4,096 open sockets/files.>
    
65. <2014-6-13 上午04时51分38秒 GMT+08:00> <Info> <Socket> <BEA-000406> <DevPollSocketMuxer was built on May 30 2007 11:19:43>
    
66. <2014-6-13 上午04时51分38秒 GMT+08:00> <Info> <Socket> <BEA-000436> <Allocating 4 reader threads.>
    
67. <2014-6-13 上午04时51分38秒 GMT+08:00> <Info> <Socket> <BEA-000446> <Native IO Enabled.>
    
68. <2014-6-13 上午04时51分38秒 GMT+08:00> <Info> <IIOP> <BEA-002014> <IIOP subsystem enabled.>
    
69. <2014-6-13 上午04时51分40秒 GMT+08:00> <Info> <Security> <BEA-000000> <Starting OpenJPA 1.1.1-SNAPSHOT>
    
70. <2014-6-13 上午04时51分41秒 GMT+08:00> <Info> <Security> <BEA-000000> <StoreServiceImpl.initJDO - StoreService is initialized with Id = ldap_DlE/pSiIXjvOGI4X1kmXeaz6418=>
    
71. <2014-6-13 上午04时51分41秒 GMT+08:00> <Info> <Security> <BEA-090074> <Initializing Authenticator provider using LDIF template file /webapp/user_projects/domains/domain_8004/security/DefaultAuthenticatorInit.ldift.>
    
72. <2014-6-13 上午04时51分41秒 GMT+08:00> <Info> <Security> <BEA-090075> <The Authenticator provider has had its LDIF information loaded from: /webapp/user_projects/domains/domain_8004/security/DefaultAuthenticatorInit.ldift>
    
73. <2014-6-13 上午04时51分41秒 GMT+08:00> <Info> <Security> <BEA-000000> <BootStrapServiceImpl.loadLDIFTemplate - Did not find /webapp/user_projects/domains/domain_8004/servers/myServer/data/ldap/XACMLAuthorizermyrealmInit.initialized, will load full LDIFT.>
    
74. <2014-6-13 上午04时51分41秒 GMT+08:00> <Info> <Security> <BEA-090074> <Initializing Authorizer provider using LDIF template file /usr/local/bea/wlserver_10.3/server/lib/XACMLAuthorizerInit.ldift.>
    
75. <2014-6-13 上午04时51分42秒 GMT+08:00> <Info> <Security> <BEA-090075> <The Authorizer provider has had its LDIF information loaded from: /usr/local/bea/wlserver_10.3/server/lib/XACMLAuthorizerInit.ldift>
    
76. <2014-6-13 上午04时51分42秒 GMT+08:00> <Info> <Security> <BEA-000000> <BootStrapServiceImpl.loadLDIFTemplate - Did not find /webapp/user_projects/domains/domain_8004/servers/myServer/data/ldap/DefaultCredentialMappermyrealmInit.initialized, will load full LDIFT.>
    
77. <2014-6-13 上午04时51分42秒 GMT+08:00> <Info> <Security> <BEA-090827> <LDIF template file /usr/local/bea/wlserver_10.3/server/lib/DefaultCredentialMapperInit.ldift was empty. The WebLogic provider CredentialMapper has been bootstrapped but has not been initialized with any LDIF data.>
    
78. <2014-6-13 上午04时51分42秒 GMT+08:00> <Info> <Security> <BEA-000000> <BootStrapServiceImpl.loadLDIFTemplate - Did not find /webapp/user_projects/domains/domain_8004/servers/myServer/data/ldap/XACMLRoleMappermyrealmInit.initialized, will load full LDIFT.>
    
79. <2014-6-13 上午04时51分42秒 GMT+08:00> <Info> <Security> <BEA-090074> <Initializing RoleMapper provider using LDIF template file /webapp/user_projects/domains/domain_8004/security/XACMLRoleMapperInit.ldift.>
    
80. <2014-6-13 上午04时51分42秒 GMT+08:00> <Info> <Security> <BEA-090075> <The RoleMapper provider has had its LDIF information loaded from: /webapp/user_projects/domains/domain_8004/security/XACMLRoleMapperInit.ldift>
    
81. <2014-6-13 上午04时51分43秒 GMT+08:00> <Info> <Security> <BEA-090093> <No pre-WLS 8.1 Keystore providers are configured for server myServer for security realm myrealm.>
    
82. <2014-6-13 上午04时51分43秒 GMT+08:00> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
    
83. <2014-6-13 上午04时51分43秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "[snmp, https, t3, cluster-broadcast-secure, ldaps, cluster-broadcast, ldap, http, iiop, admin, t3s, iiops]" is now configured.>
    
84. <2014-6-13 上午04时51分43秒 GMT+08:00> <Info> <XML> <BEA-130036> <Initializing XMLRegistry.>
    
85. <2014-6-13 上午04时51分43秒 GMT+08:00> <Info> <messaging.interception> <BEA-400000> <Initializing message interception service>
    
86. <2014-6-13 上午04时51分43秒 GMT+08:00> <Info> <Store> <BEA-280008> <Opening the persistent file store "_WLS_myServer" for recovery: directory=/webapp/user_projects/domains/domain_8004/servers/myServer/data/store/default requestedWritePolicy="Direct-Write" fileLockingEnabled=true driver="wlfileio3".>
    
87. <2014-6-13 上午04时51分44秒 GMT+08:00> <Info> <Store> <BEA-280101> <The persistent file store "_WLS_myServer" is forced to use buffered I/O because the OS/hardware environment does not support unbuffered I/O. The store is still operating in a transactionally safe synchronous mode. See store open log messages for the final write policy.>
    
88. <2014-6-13 上午04时51分44秒 GMT+08:00> <Info> <Store> <BEA-280009> <The persistent file store "_WLS_myServer" (82c9b20b-cff6-4c9b-8105-6c8144ba8800) has been opened: blockSize=512 actualWritePolicy="Direct-Write(single-handle-buffered)" explicitIOEnforced=false records=0.>
    
89. <2014-6-13 上午04时51分44秒 GMT+08:00> <Info> <JDBC> <BEA-001135> <Initializing the JDBC service.>
    
90. <2014-6-13 上午04时51分44秒 GMT+08:00> <Info> <JDBC> <BEA-001137> <Initialization complete.>
    
91. <2014-6-13 上午04时51分44秒 GMT+08:00> <Info> <JDBC> <BEA-001138> <Resuming the JDBC service.>
    
92. <2014-6-13 上午04时51分44秒 GMT+08:00> <Info> <JDBC> <BEA-001140> <Resume complete.>
    
93. <2014-6-13 上午04时51分44秒 GMT+08:00> <Info> <Connector> <BEA-190000> <Initializing J2EE Connector Service......>
    
94. <2014-6-13 上午04时51分44秒 GMT+08:00> <Info> <Connector> <BEA-190001> <J2EE Connector Service initialized successfully>
    
95. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040305> <JMS service is initialized and in standby mode.>
    
96. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040090> <Deployed 8 default connection factories.>
    
97. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040407> <Default connection factory "DefaultXAConnectionFactory2" with its JNDI name "weblogic.jms.XAConnectionFactory2" is started.>
    
98. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040407> <Default connection factory "MessageDrivenBeanConnectionFactory" with its JNDI name "weblogic.jms.MessageDrivenBeanConnectionFactory" is started.>
    
99. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040407> <Default connection factory "DefaultConnectionFactory" with its JNDI name "weblogic.jms.ConnectionFactory" is started.>
    
100. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040407> <Default connection factory "TopicConnectionFactory" with its JNDI name "javax.jms.TopicConnectionFactory" is started.>
     
101. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040407> <Default connection factory "DefaultXAConnectionFactory" with its JNDI name "weblogic.jms.XAConnectionFactory" is started.>
     
102. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040407> <Default connection factory "DefaultXAConnectionFactory0" with its JNDI name "weblogic.jms.XAConnectionFactory0" is started.>
     
103. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040407> <Default connection factory "QueueConnectionFactory" with its JNDI name "javax.jms.QueueConnectionFactory" is started.>
     
104. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040407> <Default connection factory "DefaultXAConnectionFactory1" with its JNDI name "weblogic.jms.XAConnectionFactory1" is started.>
     
105. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <JMS> <BEA-040306> <JMS service is active now.>
     
106. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <HTTP> <BEA-101128> <Initializing HTTP services.>
     
107. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <HTTP> <BEA-101135> <myServer is the default Web server.>
     
108. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <HTTP> <BEA-101052> <[HttpServer (defaultWebserver) name: myServer] Initialized>
     
109. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <HTTP> <BEA-101129> <Initializing the Web application container.>
     
110. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <WTC> <BEA-180046> <INFO: Logging service enabled.>
     
111. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <WTC> <BEA-180046> <INFO: TC Configuration Helper instantiated!>
     
112. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <WTC> <BEA-180046> <INFO: TC Task Manager instantiated!>
     
113. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <WTC> <BEA-180046> <INFO: TC security service instantiated!>
     
114. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <WTC> <BEA-180046> <INFO: TC Outbound routing service instantiated!>
     
115. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <WTC> <BEA-180046> <INFO: TC Transaction service instantiated!>
     
116. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <WebService> <BEA-220031> <The server does not support reliable SOAP messaging.>
     
117. <2014-6-13 上午04时51分45秒 GMT+08:00> <Info> <WebService> <BEA-220027> <Web Service reliable agents are started on the server.>
     
118. (0) 0xc00000000a8f8fc0        VMError::report_and_die{_ZN7VMError14report_and_dieEv} + 0x680 at /wsp/jinteg/SVN/jinteg_h6.0.14.rc1b2/hotspot/src/share/vm/utilities/vmError.cpp:859 [/opt/java6/jre/lib/IA64W/server/libjvm.so]
     
119. (1) 0xc00000000a404250        os::Hpux::JVM_handle_hpux_signal{_ZN2os4Hpux22JVM_handle_hpux_signalEiP9__siginfoPvi} + 0x6f0 at /wsp/jinteg/SVN/jinteg_h6.0.14.rc1b2/hotspot/src/os_cpu/hp-ux_ia64/vm/os_hp-ux_ia64.cpp:1081 [/opt/java6/jre/lib/IA64W/server/libjvm.so]
     
120. (2) 0xc00000000a3d7100        os::Hpux::signalHandler{_ZN2os4Hpux13signalHandlerEiP9__siginfoPv} + 0x80 at /wsp/jinteg/SVN/jinteg_h6.0.14.rc1b2/hotspot/src/os/hp-ux/vm/os_hp-ux.cpp:4383 [/opt/java6/jre/lib/IA64W/server/libjvm.so]
     
121. (3) 0xe000000120757440  ---- Signal 4 (SIGILL) delivered ----
     
122. (4) 0xc00000000a353410        nmethod::handler_for_exception_and_pc{_ZN7nmethod28handler_for_exception_and_pcE6HandlePh} + 0xf0 at /wsp/jinteg/SVN/jinteg_h6.0.14.rc1b2/hotspot/src/share/vm/code/nmethod.cpp:410 [/opt/java6/jre/lib/IA64W/server/libjvm.so]
     
123. (5) 0xc00000000a562c40        OptoRuntime::handle_exception_C{_ZN11OptoRuntime18handle_exception_CEP10JavaThread} + 0x340 at /wsp/jinteg/SVN/jinteg_h6.0.14.rc1b2/hotspot/src/share/vm/opto/runtime.cpp:983 [/opt/java6/jre/lib/IA64W/server/libjvm.so]
     
124. (6) 0x9ffffffecf84f140
     
125. *** Internal error (-3) while unwinding stack [/wsp/jinteg/SVN/jinteg_h6.0.14.rc1b2/hotspot/src/os_cpu/hp-ux_ia64/vm/thread_hp-ux_ia64.cpp:145] ***
     
126. #
     
127. # A fatal error has been detected by the Java Runtime Environment:
     
128. #
     
129. #  SIGILL (4) at pc=c00000000a353410, pid=9071, tid=36
     
130. #
     
131. # JRE version: 6.0
     
132. # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.6-b01-jre1.6.0.14-rc1b2 mixed mode hp-ux-ia64 )
     
133. # Problematic frame:
     
134. # V  [libjvm.so+0x4ab6c0f8]  JVM_StartCompiler+0x4aba4118
     
135. #
     
136. # An error report file with more information is saved as:
     
137. # /webapp/user_projects/domains/domain_8004/hs_err_pid9071.log
     
138. #
     
139. #  Please report this error to HP customer support.
     
140. #
     
141. @(#)Unexpected Exception regs: see 0x9fffffffbf69f8e0
     
142. 
     
143. Current pc:-
     
144.         pc: 0xc00000000a353410
     
145. 
     
146. Context Branch Registers:-
     
147.         br0: 0xc00000000a562c40
     
148.         br1: 0x9fffffffaf025fc0
     
149.         br2: 0x0
     
150.         br3: 0x0
     
151.         br4: 0x0
     
152.         br5: 0x0
     
153.         br6: 0xc00000000a562900
     
154.         br7: 0xc00000000a3491c0
     
155. 
     
156. Context bspstore        = 0x9fffffffa98010f0
     
157. Context bsp             = 0x9fffffffa98010f0
     
158. number of stacked grs   = 23
     
159. bsp of signalling frame = 0x9fffffffa9801038
     
160. predicate registers     = 0x3243
     
161. 
     
162. General Register contents at error:-
     
163.         nat:0        gr  1: 0x9fffffffbf5af980
     
164.         nat:0        gr  2: 0x1
     
165.         nat:0        gr  3: 0x9fffffff3fb178a8
     
166.         nat:0        gr  4: 0x6000000004244800
     
167.         nat:0        gr  5: 0x9fffffffa9800fb0
     
168.         nat:0        gr  6: 0x9fffffffa98a9000
     
169.         nat:0        gr  7: 0x9fffffffa987a000
     
170.         nat:0        gr  8: 0x9fffffffaf1dfef0
     
171.         nat:0        gr  9: 0xffffffffffffffff
     
172.         nat:0        gr 10: 0x60000000040e6678
     
173.         nat:0        gr 11: 0x0
     
174.         nat:0        gr 12: 0x9fffffffa98fc600
     
175.         nat:0        gr 13: 0x600000000426c8a0
     
176.         nat:0        gr 14: 0x60000000040e6600
     
177.         nat:0        gr 15: 0x60000000040e66f8
     
178.         nat:0        gr 16: 0x60000000040e6700
     
179.         nat:0        gr 17: 0x1
     
180.         nat:0        gr 18: 0x9fffffffbf428270
     
181.         nat:0        gr 19: 0x0
     
182.         nat:0        gr 20: 0x9fffffffbf4fc620
     
183.         nat:0        gr 21: 0x0
     
184.         nat:0        gr 22: 0x9fffffff3fb17c90
     
185.         nat:0        gr 23: 0x3c
     
186.         nat:0        gr 24: 0x0
     
187.         nat:0        gr 25: 0x9fffffffaf005550
     
188.         nat:0        gr 26: 0x0
     
189.         nat:0        gr 27: 0x0
     
190.         nat:0        gr 28: 0x1
     
191.         nat:0        gr 29: 0x6
     
192.         nat:0        gr 30: 0x9fffffffa98fd869
     
193.         nat:0        gr 31: 0xc00000000000030a
     
194. 
     
195. Stacked General Registers contents at error:-
     
196.         gr 32 = 0x6000000004244800
     
197.         gr 33 = 0x9fffffffbf5af980
     
198.         gr 34 = 0xc000000000000811
     
199.         gr 35 = 0x9fffffffaf08b8a0
     
200.         gr 36 = 0x32c3
     
201.         gr 37 = 0x1
     
202.         gr 38 = 0x9fffffffa98fc618
     
203.         gr 39 = 0x0
     
204.         gr 40 = 0x9fffffffa98fc640
     
205.         gr 41 = 0x60000000042449b8
     
206.         gr 42 = 0x9fffffffaf1dfc00
     
207.         gr 43 = 0x1
     
208.         gr 44 = 0x1
     
209.         gr 45 = 0x0
     
210.         gr 46 = 0x0
     
211.         gr 47 = 0x9fffffffaf1dfdf0
     
212.         gr 48 = 0x9fffffffaf1dfb50
     
213.         gr 49 = 0x9fffffffbf4fd987
     
214.         gr 50 = 0x9fffffffaf1dfb50
     
215.         gr 51 = 0x6000000001b027b8
     
216.         gr 52 = 0x9fffffffaf1dfdf0
     
217.         gr 53 = 0xc00000000a562ac0
     
218.         gr 54 = 0x9fffffffbf4fe090
     
219. 
     
220. Unflushed Caller Register Summary at error:-
     
221. number of unflushed caller registers= 0
     
222. 
     
223. DOMAIN_HOME: /webapp/user_projects/domains/domain_8004
     
224. .
     
225. .
     
226. Source JAVA Memory arguments: -Xms512m -Xmx512m -XX:MaxPermSize=256m
     
227. .
     
228. WLS Start Mode=Production
     
229. .
     
230. CLASSPATH=/usr/local/bea/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/opt/java6/lib/tools.jar:/usr/local/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/usr/local/bea/wlserver_10.3/server/lib/weblogic.jar:/usr/local/bea/modules/features/weblogic.server.modules_10.3.6.0.jar:/usr/local/bea/wlserver_10.3/server/lib/webservices.jar:/usr/local/bea/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/usr/local/bea/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:/usr/local/bea/wlserver_10.3/common/derby/lib/derbyclient.jar:/usr/local/bea/wlserver_10.3/server/lib/xqrl.jar:/opt/java6/lib/dt.jar:/opt/java6/lib/tools.jar
     
231. .
     
232. PATH=/usr/local/bea/wlserver_10.3/server/bin:/usr/local/bea/modules/org.apache.ant_1.7.1/bin:/opt/java6/jre/bin:/opt/java6/bin:/usr/sbin:/usr/bin:/usr/ccs/bin:/usr/contrib/bin:/usr/contrib/Q4/bin:/opt/perl/bin:/opt/ipf/bin:/opt/gvsd/bin:/opt/nettladm/bin:/opt/fcms/bin:/opt/wbem/bin:/opt/wbem/sbin:/opt/sas/bin:/opt/graphics/common/bin:/opt/atok/bin:/usr/bin/X11:/usr/contrib/bin/X11:/opt/sec_mgmt/bastille/bin:/opt/caliper/bin:/opt/drd/bin:/opt/dsau/bin:/opt/dsau/sbin:/opt/resmon/bin:/opt/firefox:/opt/gnome/bin:/usr/contrib/kwdb/bin:/opt/mozilla:/opt/perl_32/bin:/opt/perl_64/bin:/opt/prm/bin:/opt/sfm/bin:/opt/swm/bin:/opt/sec_mgmt/spc/bin:/opt/ssh/bin:/opt/sentinel/bin:/opt/swa/bin:/opt/hpsmh/bin:/opt/thunderbird:/opt/langtools/bin:/opt/gwlm/bin:/opt/ignite/bin:/sbin:/opt/java6/jre/bin/IA64W:/home/weblogic
     
233. .
     
234. ***************************************************
     
235. *  To start WebLogic Server, use a username and   *
     
236. *  password assigned to an admin-level user.  For *
     
237. *  server administration, use the WebLogic Server *
     
238. *  console at http://hostname:port/console        *
     
239. ***************************************************
     
240. starting weblogic with Java version:
     
241. java version "1.6.0.14"
     
242. Java(TM) SE Runtime Environment (build 1.6.0.14-jinteg_07_mar_2012_00_45-b00)
     
243. Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01-jre1.6.0.14-rc1b2, mixed mode)
     
244. Starting WLS with line:
     
245. /opt/java6/bin/IA64W/java -server -d64   -Dport=8004 -Xms1792m -Xmx1792m -Xmn800m -XX:PermSize=512m -XX:MaxPermSize=512m   -Dweblogic.ProductionModeEnabled=true  -Xmn800m -XX:+UseParNewGC  -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC  -XX:+UseCompressedOops -XX:+ForceMmapReserved -XX:MaxTenuringThreshold=10 -XX:SurvivorRatio=12 -XX:+PrintHeapAtGC -Xloggc:./log/gc.log   -da -Dplatform.home=/usr/local/bea/wlserver_10.3 -Dwls.home=/usr/local/bea/wlserver_10.3/server -Dweblogic.home=/usr/local/bea/wlserver_10.3/server   -Dweblogic.management.discover=true  -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole=false -Dweblogic.ext.dirs=/usr/local/bea/patch_wls1036/profiles/default/sysext_manifest_classpath  -Dweblogic.Name=myServer -Djava.security.policy=/usr/local/bea/wlserver_10.3/server/lib/weblogic.policy  weblogic.Server
     
246. 
     
247. Java HotSpot(TM) 64-Bit Server VM warning: UseCompressedOops only supported with an MPAS Java launcher
     
248. <2014-6-13 上午09时33分47秒 GMT+08:00> <Info> <Security> <BEA-090905> <Disabling CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true>
     
249. <2014-6-13 上午09时33分47秒 GMT+08:00> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG to FIPS186PRNG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true>
     
250. <2014-6-13 上午09时33分47秒 GMT+08:00> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with Java HotSpot(TM) 64-Bit Server VM Version 20.6-b01-jre1.6.0.14-rc1b2 from Hewlett-Packard Company>
     
251. <2014-6-13 上午09时33分49秒 GMT+08:00> <Info> <Management> <BEA-141107> <Version: WebLogic Server 10.3.6.0  Tue Nov 15 08:52:36 PST 2011 1441050 >
     
252. <2014-6-13 上午09时33分51秒 GMT+08:00> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
     
253. <2014-6-13 上午09时33分51秒 GMT+08:00> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool>
     
254. <2014-6-13 上午09时33分51秒 GMT+08:00> <Info> <WebLogicServer> <BEA-000214> <WebLogic Server "myServer" version:
     
255. WebLogic Server 10.3.6.0  Tue Nov 15 08:52:36 PST 2011 1441050  Copyright (c) 1995, 2011, Oracle and/or its affiliates. All rights reserved.>
     
256. <2014-6-13 上午09时33分51秒 GMT+08:00> <Notice> <Log Management> <BEA-170019> <The server log file /webapp/user_projects/domains/domain_8004/servers/myServer/logs/myServer.log is opened. All server side log events will be written to this file.>
     
257. <2014-6-13 上午09时33分51秒 GMT+08:00> <Info> <Log Management> <BEA-170023> <The Server Logging is initialized with Java Logging API implementation.>
     
258. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Diagnostics> <BEA-320001> <The ServerDebug service initialized successfully.>
     
259. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "t3" is now configured.>
     
260. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "t3s" is now configured.>
     
261. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "http" is now configured.>
     
262. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "https" is now configured.>
     
263. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "iiop" is now configured.>
     
264. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "iiops" is now configured.>
     
265. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "ldap" is now configured.>
     
266. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "ldaps" is now configured.>
     
267. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "cluster" is now configured.>
     
268. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "clusters" is now configured.>
     
269. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "snmp" is now configured.>
     
270. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "admin" is now configured.>
     
271. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002624> <The administration protocol is "t3s" and is now configured.>
     
272. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <RJVM> <BEA-000570> <Network Configuration for Channel "myServer"
     
273. Listen Address                 :8004
     
274. Public Address                 N/A
     
275. Http Enabled                 true
     
276. Tunneling Enabled         false
     
277. Outbound Enabled         false
     
278. Admin Traffic Enabled         true>
     
279. <2014-6-13 上午09时33分52秒 GMT+08:00> <Debug> <RJVM> <BEA-000571> <Network Configuration Detail for Channel "myServer"
     
280. Channel Weight                 50
     
281. Accept Backlog                 300
     
282. Login Timeout                 5000ms
     
283. Max Message Size         10000000
     
284. Message Timeout         60s
     
285. Idle Timeout                 65s
     
286. Tunneling Timeout         40s
     
287. Tunneling Ping                 45s>
     
288. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Server> <BEA-002609> <Channel Service initialized.>
     
289. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Socket> <BEA-000416> <Using effective file descriptor limit of: 4,096 open sockets/files.>
     
290. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Socket> <BEA-000406> <DevPollSocketMuxer was built on May 30 2007 11:19:43>
     
291. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Socket> <BEA-000436> <Allocating 4 reader threads.>
     
292. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <Socket> <BEA-000446> <Native IO Enabled.>
     
293. <2014-6-13 上午09时33分52秒 GMT+08:00> <Info> <IIOP> <BEA-002014> <IIOP subsystem enabled.>
     
294. <2014-6-13 上午09时33分55秒 GMT+08:00> <Info> <Security> <BEA-000000> <Starting OpenJPA 1.1.1-SNAPSHOT>
     
295. <2014-6-13 上午09时33分55秒 GMT+08:00> <Info> <Security> <BEA-000000> <StoreServiceImpl.initJDO - StoreService is initialized with Id = ldap_DlE/pSiIXjvOGI4X1kmXeaz6418=>
     
296. <2014-6-13 上午09时33分55秒 GMT+08:00> <Info> <Security> <BEA-090516> <The Authenticator provider has preexisting LDAP data.>
     
297. <2014-6-13 上午09时33分55秒 GMT+08:00> <Info> <Security> <BEA-090516> <The Authorizer provider has preexisting LDAP data.>
     
298. <2014-6-13 上午09时33分56秒 GMT+08:00> <Info> <Security> <BEA-090516> <The CredentialMapper provider has preexisting LDAP data.>
     
299. <2014-6-13 上午09时33分56秒 GMT+08:00> <Info> <Security> <BEA-090516> <The RoleMapper provider has preexisting LDAP data.>
     
300. <2014-6-13 上午09时33分56秒 GMT+08:00> <Info> <Security> <BEA-090093> <No pre-WLS 8.1 Keystore providers are configured for server myServer for security realm myrealm.>
     
301. <2014-6-13 上午09时33分56秒 GMT+08:00> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
     
302. <2014-6-13 上午09时33分57秒 GMT+08:00> <Info> <Server> <BEA-002622> <The protocol "[snmp, https, t3, cluster-broadcast-secure, ldaps, cluster-broadcast, ldap, http, iiop, admin, t3s, iiops]" is now configured.>
     
303. <2014-6-13 上午09时33分57秒 GMT+08:00> <Info> <XML> <BEA-130036> <Initializing XMLRegistry.>
     
304. <2014-6-13 上午09时33分57秒 GMT+08:00> <Info> <messaging.interception> <BEA-400000> <Initializing message interception service>
     
305. <2014-6-13 上午09时33分57秒 GMT+08:00> <Info> <Store> <BEA-280008> <Opening the persistent file store "_WLS_myServer" for recovery: directory=/webapp/user_projects/domains/domain_8004/servers/myServer/data/store/default requestedWritePolicy="Direct-Write" fileLockingEnabled=true driver="wlfileio3".>
     
306. <2014-6-13 上午09时33分58秒 GMT+08:00> <Info> <Store> <BEA-280101> <The persistent file store "_WLS_myServer" is forced to use buffered I/O because the OS/hardware environment does not support unbuffered I/O. The store is still operating in a transactionally safe synchronous mode. See store open log messages for the final write policy.>
     
307. <2014-6-13 上午09时33分58秒 GMT+08:00> <Emergency> <Store> <BEA-280060> <The persistent store "_WLS_myServer" encountered a fatal error, and it must be shut down: weblogic.store.PersistentStoreFatalException: [Store:280073]The file store "_WLS_myServer" could not be opened because it contained a file with the invalid version 1. A file of version 2 was expected.
     
308. weblogic.store.PersistentStoreFatalException: [Store:280073]The file store "_WLS_myServer" could not be opened because it contained a file with the invalid version 1. A file of version 2 was expected.
     
309.         at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:121)
     
310.         at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:435)
     
311.         at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:423)
     
312.         at weblogic.store.admin.AdminHandler.activate(AdminHandler.java:126)
     
313.         at weblogic.store.admin.FileAdminHandler.activate(FileAdminHandler.java:191)
     
314.         at weblogic.store.admin.DefaultStoreService.start(DefaultStoreService.java:60)
     
315.         at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
     
316.         at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
     
317.         at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
     
318. >
     
319. No matching store mbean found for =_WLS_myServer, must be a default store.
     
320. <2014-6-13 上午09时33分58秒 GMT+08:00> <Error> <Store> <BEA-280061> <The persistent store "_WLS_myServer" could not be deployed: weblogic.store.PersistentStoreFatalException: [Store:280073]The file store "_WLS_myServer" could not be opened because it contained a file with the invalid version 1. A file of version 2 was expected.
     
321. weblogic.store.PersistentStoreFatalException: [Store:280073]The file store "_WLS_myServer" could not be opened because it contained a file with the invalid version 1. A file of version 2 was expected.
     
322.         at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:121)
     
323.         at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:435)
     
324.         at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:423)
     
325.         at weblogic.store.admin.AdminHandler.activate(AdminHandler.java:126)
     
326.         at weblogic.store.admin.FileAdminHandler.activate(FileAdminHandler.java:191)
     
327.         at weblogic.store.admin.DefaultStoreService.start(DefaultStoreService.java:60)
     
328.         at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
     
329.         at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
     
330.         at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
     
331. >
     
332. <2014-6-13 上午09时33分58秒 GMT+08:00> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:
     
333. 
     
334. There are 1 nested errors:
     
335. 
     
336. weblogic.management.DeploymentException:
     
337.         at weblogic.store.admin.AdminHandler.activate(AdminHandler.java:129)
     
338.         at weblogic.store.admin.FileAdminHandler.activate(FileAdminHandler.java:191)
     
339.         at weblogic.store.admin.DefaultStoreService.start(DefaultStoreService.java:60)
     
340.         at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
     
341.         at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
     
342.         at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
     
343. Caused by: weblogic.store.PersistentStoreFatalException: [Store:280073]The file store "_WLS_myServer" could not be opened because it contained a file with the invalid version 1. A file of version 2 was expected.
     
344.         at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:121)
     
345.         at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:435)
     
346.         at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:423)
     
347.         at weblogic.store.admin.AdminHandler.activate(AdminHandler.java:126)
     
348.         ... 5 more
     
349. 
     
350. >
     
351. <2014-6-13 上午09时33分58秒 GMT+08:00> <Critical> <Health> <BEA-310006> <Critical Subsystem PersistentStore._WLS_myServer has failed. Setting server state to FAILED.
     
352. Reason: weblogic.store.PersistentStoreFatalException: [Store:280073]The file store "_WLS_myServer" could not be opened because it contained a file with the invalid version 1. A file of version 2 was expected.>
     
353. <2014-6-13 上午09时33分58秒 GMT+08:00> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
     
354. <2014-6-13 上午09时33分58秒 GMT+08:00> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
     
355. <2014-6-13 上午09时33分58秒 GMT+08:00> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
     
356. <2014-6-13 上午09时33分58秒 GMT+08:00> <Info> <WebLogicServer> <BEA-000236> <Stopping execute threads.>
     
357. <2014-6-13 上午09时33分58秒 GMT+08:00> <Critical> <WebLogicServer> <BEA-000385> <Server health failed. Reason: health of critical service 'PersistentStore._WLS_myServer' failed>
     
358. <2014-6-13 上午09时33分58秒 GMT+08:00> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
     
359. <2014-6-13 上午09时33分58秒 GMT+08:00> <Error> <> <BEA-000000> <***** This VM does not support thread dumps *****
     
360. >

复制代码

现象只是偶然出现，本机上有5个domain，定时重启时 偶然会有部分domain拉不起来，求大神，帮忙分析~~

wlwpz

1. #
   
2. # A fatal error has been detected by the Java Runtime Environment:
   
3. #
   
4. #  SIGILL (4) at pc=c00000000a353410, pid=9071, tid=36
   
5. #
   
6. # JRE version: 6.0
   
7. # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.6-b01-jre1.6.0.14-rc1b2 mixed mode hp-ux-ia64 )
   
8. # Problematic frame:
   
9. # V  [libjvm.so+0x4ab6c0f8]  JVM_StartCompiler+0x4aba4118
   
10. #
    
11. #  Please report this error to HP customer support.
    
12. #
    
13. 
    
14. ---------------  T H R E A D  ---------------
    
15. 
    
16. Current thread (6000000004244800):  JavaThread "Thread-10" [_thread_in_vm, id=36, lwp_id=30464, stack(9fffffffa9881000,9fffffffa9901000)]
    
17. 
    
18. siginfo:si_signo=SIGILL: si_errno=0, si_code=1 (ILL_ILLOPC), si_addr=c00000000a353410
    
19. 
    
20. Top of Stack: (sp=9fffffffa98fc600)
    
21. 9fffffffa98fc600:   6dc079982f41aabf 4d1588bbd01f4b8f
    
22. 9fffffffa98fc610:   8000000484d6d318 6000000001b027b8
    
23. 9fffffffa98fc620:   000000008ef7d171 bc1f76fc04cde747
    
24. 9fffffffa98fc630:   6901ff98eaf900b5 f3e314e68374363f
    
25. 9fffffffa98fc640:   8380f8149dcf4f23 28fe8c6ee1171110
    
26. 9fffffffa98fc650:   9f47fa7981faf865 ba9f5fa15dfc737a
    
27. 9fffffffa98fc660:   885fa5c7f9b58cf4 f25b942996de2783
    
28. 9fffffffa98fc670:   000000003f0dc1a5 ab65406f12ac89df
    
29. 9fffffffa98fc680:   00456fe2e4f7f407 fbce4cfb4e51c5e3
    
30. 9fffffffa98fc690:   b4a997bc158fd126 98a398e46d4358cd
    
31. 9fffffffa98fc6a0:   ef31fe1175d11158 f17b18df19ca8a63
    
32. 9fffffffa98fc6b0:   83adf8af58bd9b82 60000000000019d8
    
33. 9fffffffa98fc6c0:   6000000000001190 e3715adb4b467e1c
    
34. 9fffffffa98fc6d0:   612b3f31f8dea758 fd19359c65f1b5c8
    
35. 9fffffffa98fc6e0:   0000000030a7e2e2 f1db0230f6ee5d12
    
36. 9fffffffa98fc6f0:   ed36aff6a11bda72 40e11d06b32fc870
    
37. 9fffffffa98fc700:   80cf11aefe4b23f8 8b942a3663fe4b0a
    
38. 9fffffffa98fc710:   00000000639e2798 0000000000000005
    
39. 9fffffffa98fc720:   0000000000000000 12a29d07aa25f9bb
    
40. 9fffffffa98fc730:   6000000001d4a940 6000000001d4a938
    
41. 9fffffffa98fc740:   9fffffffbf400a30 90a13969bce6a262
    
42. 9fffffffa98fc750:   000000080000002f 0000000c0000000f
    
43. 9fffffffa98fc760:   000000086176612f 696f2f46696c653b
    
44. 9fffffffa98fc770:   0000000000000001 2f48656170244865
    
45. 9fffffffa98fc780:   617046696c654865 616465723b4c6a61
    
46. 9fffffffa98fc790:   00000008616e672f 0000000000000001
    
47. 9fffffffa98fc7a0:   56006f2f66696c65 0000000865637442
    
48. 9fffffffa98fc7b0:   7566666572506f6f 6c3b295600747269
    
49. 9fffffffa98fc7c0:   6e673b294c6a6176 612f6c616e672f53
    
50. 9fffffffa98fc7d0:   7472696e673b0061 c000000000285b20
    
51. 9fffffffa98fc7e0:   0101016900003b29 00000008612f6c61
    
52. 9fffffffa98fc7f0:   6000000001d4b0b8 6000000001d4b0c0
    
53. 9fffffffa98fc800:   6000000001b20ee0 6000000001d4b0c8
    
54. 9fffffffa98fc810:   0000000000000000 6000000001d4af08
    
55. 9fffffffa98fc820:   000000006771e8cd c00000000039f0a0
    
56. 9fffffffa98fc830:   600000000519c2f0 6000000001b20ed0
    
57. 9fffffffa98fc840:   6000000001b20ee0 6000000001b212b8
    
58. 9fffffffa98fc850:   0000000040c037f0 c00000000039f0a0
    
59. 9fffffffa98fc860:   47c8cc053c49ed2d 0000000000000001
    
60. 9fffffffa98fc870:   0000000000000000 0000000000000000
    
61. 9fffffffa98fc880:   0000000000000000 0000000000000000
    
62. 9fffffffa98fc890:   0000000000000000 0000000000000000
    
63. 9fffffffa98fc8a0:   6000000004244800 0000000000000001
    
64. 9fffffffa98fc8b0:   6000000001d49f30 6000000001d4a7d8
    
65. 9fffffffa98fc8c0:   6000000001d51f18 9fffffff441674f0
    
66. 9fffffffa98fc8d0:   9ffffffecf859ab0 9fffffff441674f0
    
67. 9fffffffa98fc8e0:   9ffffffecf859978 600000000413b780
    
68. 9fffffffa98fc8f0:   0000000000000000 0000000000000000
    
69. 9fffffffa98fc900:   0000000000000000 0000000000000000
    
70. 9fffffffa98fc910:   0000000000000000 0000000000000000
    
71. 9fffffffa98fc920:   0000000000000000 0000000000000000
    
72. 9fffffffa98fc930:   0000000000000000 0000000000000000
    
73. 9fffffffa98fc940:   0000000000000000 0000000000000000
    
74. 9fffffffa98fc950:   0000000000000000 0000000000000000
    
75. 9fffffffa98fc960:   0000000000000000 0000000000000000
    
76. 9fffffffa98fc970:   0000000000000000 0000000000000000
    
77. 9fffffffa98fc980:   0000000000000000 0000000000000000
    
78. 9fffffffa98fc990:   0000000000000000 0000000000000000
    
79. 9fffffffa98fc9a0:   0000000000000000 0000000000000000
    
80. 9fffffffa98fc9b0:   0000000000000000 0000000000000000
    
81. 9fffffffa98fc9c0:   0000000000000000 0000000000000000
    
82. 9fffffffa98fc9d0:   0000000000000000 0000000000000000
    
83. 9fffffffa98fc9e0:   0000000000000000 0000000000000000
    
84. 9fffffffa98fc9f0:   0000000000000000 0000000000000000
    
85. 9fffffffa98fca00:   0000000000000000 0000000000000000
    
86. 9fffffffa98fca10:   9fffffffa98ffc70 0000000000000000
    
87. 9fffffffa98fca20:   9fffffffa98ffc80 0000000000000000
    
88. 9fffffffa98fca30:   9fffffffa98ffc90 0000000000000000
    
89. 9fffffffa98fca40:   9fffffffa98ffca0 0000000000000000
    
90. 9fffffffa98fca50:   9fffffffa98ffcb0 fa9b4aef4c17f125
    
91. 9fffffffa98fca60:   9fffffffa98ffcc0 0000000000000000
    
92. 9fffffffa98fca70:   9fffffffa98ffcd0 0000000300000001
    
93. 9fffffffa98fca80:   9fffffffa98ffce0 600000000413b780
    
94. 9fffffffa98fca90:   6000000001d49f30 6000000001d4a6b0
    
95. 9fffffffa98fcaa0:   6000000001d51f18 9fffffffa98fd600
    
96. 9fffffffa98fcab0:   6000000001d51f18 9fffffffa98fd600
    
97. 9fffffffa98fcac0:   000000000008a6a8 00000002ffa4d1c8
    
98. 9fffffffa98fcad0:   0000000000043b40 600000000003c970
    
99. 9fffffffa98fcae0:   600000000003c9d0 0000000000000000
    
100. 9fffffffa98fcaf0:   60000000040e6b14 60000000040e6b30
     
101. 9fffffffa98fcb00:   9fffffffa98fcb30 0000000000077480
     
102. 9fffffffa98fcb10:   0000000000000000 9fffffff3f801ec0
     
103. 9fffffffa98fcb20:   0000000000000000 9fffffffaf0138d0
     
104. 9fffffffa98fcb30:   6000000001d4a680 00000001cf859978
     
105. 9fffffffa98fcb40:   6000000004244800 600000000413b780
     
106. 9fffffffa98fcb50:   6000000001d49f30 6000000001d4a6c0
     
107. 9fffffffa98fcb60:   6000000001d51f18 9fffffffa98fd600
     
108. 9fffffffa98fcb70:   6000000000043b40 6000000004244800
     
109. 9fffffffa98fcb80:   0000000e01b028f0 cf859ab001b027d0
     
110. 9fffffffa98fcb90:   0000000000000000 9ffffffecf859978
     
111. 9fffffffa98fcba0:   9fffffff441674f0 0000000000000000
     
112. 9fffffffa98fcbb0:   0000000000000000 0000000000000000
     
113. 9fffffffa98fcbc0:   0000000000000000 0000000000000000
     
114. 9fffffffa98fcbd0:   0000000000000000 0001010000000000
     
115. 9fffffffa98fcbe0:   00000000ffffffff 9fffffffa98fcb98
     
116. 9fffffffa98fcbf0:   9fffffffa98fcbd9 0000000200000008
     
117. 9fffffffa98fcc00:   0001003100000000 6000000001b02918
     
118. 9fffffffa98fcc10:   0000000000000001 6000000004244800
     
119. 9fffffffa98fcc20:   0000000001d4a678 6000000001b027d0
     
120. 9fffffffa98fcc30:   6000000001d4a680 0000000000000000
     
121. 9fffffffa98fcc40:   6000000004244800 6000000001b027d0
     
122. 9fffffffa98fcc50:   0000000000000000 01002a0039000000
     
123. 9fffffffa98fcc60:   0101003100008c58 6000000001d4a678
     
124. 9fffffffa98fcc70:   0000000000000000 0000000000000000
     
125. 9fffffffa98fcc80:   00000000ffffffff ff004a0039c9a142
     
126. 9fffffffa98fcc90:   00393a00019200d3 d32371393949ff48
     
127. 9fffffffa98fcca0:   06719259221cff06 071152ff42040049
     
128. 9fffffffa98fccb0:   ff98040000077a10 00000002ff8836d8
     
129. 9fffffffa98fccc0:   00000000015e6810 600000000003ccd0
     
130. 9fffffffa98fccd0:   000000000003cd30 600000000003cc70
     
131. 9fffffffa98fcce0:   60000000040e6b14 60000000040e6b30
     
132. 9fffffffa98fccf0:   0000000500000000 60000000000011b8
     
133. 9fffffffa98fcd00:   00000000e47278bf 0000000000540775
     
134. 9fffffffa98fcd10:   0000000000000000 0000000000000000
     
135. 9fffffffa98fcd20:   600000000166f1f0 60000000016722b1
     
136. 9fffffffa98fcd30:   60000000016722b1 9fffffffa98fce70
     
137. 9fffffffa98fcd40:   0100000004244800 c00000000039f0a0
     
138. 9fffffffa98fcd50:   000000000008c3a8 00000002ffa4cc38
     
139. 9fffffffa98fcd60:   006c6c656374696f 600000000003ccd0
     
140. 9fffffffa98fcd70:   600000000003cd30 600000000003cc70
     
141. 9fffffffa98fcd80:   60000000040e6b14 60000000040e6b30
     
142. 9fffffffa98fcd90:   0000000500000000 60000000000017d0
     
143. 9fffffffa98fcda0:   60000000000019d8 0000000000000000
     
144. 9fffffffa98fcdb0:   6000000004244800 0000000000000000
     
145. 9fffffffa98fcdc0:   6000000004244800 0000000000000000
     
146. 9fffffffa98fcdd0:   c00000000039f0a0 0000000000000000
     
147. 9fffffffa98fcde0:   6000000004244800 c00000000039f0a0
     
148. 9fffffffa98fcdf0:   7765626c6f676963 2f73746f72652f69
     
149. 
     
150. Instructions: (pc=c00000000a353410)
     
151. c00000000a3533f0:   19 88 00 00 00 24 00 81 54 04 42 03 48 00 00 43
     
152. c00000000a353400:   19 a8 00 20 38 10 00 04 20 0c 70 83 08 00 84 03
     
153. c00000000a353410:   19 58 54 00 40 04 00 04 54 0c 70 03 58 ff ff 4a
     
154. c00000000a353420:   19 40 00 00 00 24 00 00 00 02 00 80 08 00 84 00
     
155. 
     
156. General Register to memory mapping:
     
157. GR1  =
     
158. [error occurred during error reporting (printing register info), id 0xb]
     
159. 
     
160. Stack: [9fffffffa9881000,9fffffffa9901000],  sp=9fffffffa98fc600,  free space=493k
     
161. Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
     
162. V  [libjvm.so+0x4ab6c0f8]  JVM_StartCompiler+0x4aba4118
     
163. 
     
164. Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
     
165. v  ~ExceptionBlob
     
166. J  java.net.URLClassLoader.findClass(Ljava/lang/String;)Ljava/lang/Class;
     
167. j  sun.misc.Launcher$ExtClassLoader.findClass(Ljava/lang/String;)Ljava/lang/Class;+7
     
168. J  java.lang.ClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class;
     
169. j  sun.misc.Launcher$AppClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class;+41
     
170. j  java.lang.ClassLoader.loadClass(Ljava/lang/String;)Ljava/lang/Class;+3
     
171. v  ~StubRoutines::call_stub
     
172. j  weblogic.store.io.file.Heap.reserveSpace(I)V+398
     
173. j  weblogic.store.io.file.Heap.multiWrite(Ljava/util/List;)[J+180
     
174. j  weblogic.store.io.file.FileStoreIO.flush()V+377
     
175. j  weblogic.store.internal.PersistentStoreImpl.run()V+133
     
176. j  java.lang.Thread.run()V+11
     
177. v  ~StubRoutines::call_stub
     
178. 
     
179. ---------------  P R O C E S S  ---------------
     
180. 
     
181. Java Threads: ( => current thread )
     
182. =>6000000004244800 JavaThread "Thread-10" [_thread_in_vm, id=36, lwp_id=30464, stack(9fffffffa9881000,9fffffffa9901000)]
     
183.   60000000016ef800 JavaThread "DoSManager" daemon [_thread_blocked, id=35, lwp_id=30426, stack(9ffffffecd981000,9ffffffecda01000)]
     
184.   6000000004198400 JavaThread "VDE Transaction Processor Thread" daemon [_thread_blocked, id=33, lwp_id=30424, stack(9ffffffecdd81000,9ffffffecde01000)]
     
185.   6000000004195800 JavaThread "ExecuteThread: '3' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=32, lwp_id=30415, stack(9ffffffecdf81000,9ffffffece001000)]
     
186.   6000000004102000 JavaThread "ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=31, lwp_id=30414, stack(9ffffffece181000,9ffffffece201000)]
     
187.   6000000004058c00 JavaThread "ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=30, lwp_id=30413, stack(9ffffffece381000,9ffffffece401000)]
     
188.   600000000021f800 JavaThread "ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" daemon [_thread_in_native, id=29, lwp_id=30412, stack(9fffffffa9a81000,9fffffffa9b01000)]
     
189.   60000000035e0c00 JavaThread "[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=28, lwp_id=30411, stack(9ffffffece581000,9ffffffece601000)]
     
190.   60000000035c3000 JavaThread "weblogic.timers.TimerThread" daemon [_thread_blocked, id=27, lwp_id=30410, stack(9ffffffece781000,9ffffffece801000)]
     
191.   60000000035a9c00 JavaThread "weblogic.time.TimeEventGenerator" daemon [_thread_blocked, id=26, lwp_id=30409, stack(9fffffffa9c81000,9fffffffa9d01000)]
     
192.   600000000341f000 JavaThread "[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=25, lwp_id=30408, stack(9ffffffece981000,9ffffffecea01000)]
     
193.   60000000029fdc00 JavaThread "Timer-1" daemon [_thread_blocked, id=24, lwp_id=30407, stack(9fffffffa9e81000,9fffffffa9f01000)]
     
194.   6000000001f2d800 JavaThread "Timer-0" daemon [_thread_blocked, id=23, lwp_id=30371, stack(9fffffffaa081000,9fffffffaa101000)]
     
195.   600000000167c400 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=22, lwp_id=30368, stack(9ffffffeceb81000,9ffffffecec01000)]
     
196.   600000000166bc00 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=20, lwp_id=30366, stack(9ffffffeced81000,9ffffffecee01000)]
     
197.   6000000001667400 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=19, lwp_id=30365, stack(9ffffffecef81000,9ffffffecf001000)]
     
198.   6000000001664800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=18, lwp_id=30364, stack(9ffffffecf181000,9ffffffecf201000)]
     
199.   6000000001643400 JavaThread "Surrogate Locker Thread (Concurrent GC)" daemon [_thread_blocked, id=17, lwp_id=30363, stack(9fffffffaa281000,9fffffffaa301000)]
     
200.   6000000001619000 JavaThread "Finalizer" daemon [_thread_blocked, id=16, lwp_id=30362, stack(9ffffffecf481000,9ffffffecf501000)]
     
201.   6000000001616400 JavaThread "Reference Handler" daemon [_thread_blocked, id=15, lwp_id=30361, stack(9ffffffecf681000,9ffffffecf701000)]
     
202.   6000000000070400 JavaThread "main" [_thread_blocked, id=1, lwp_id=-1, stack(9fffffffffefd000,9fffffffffffe000)]
     
203. 
     
204. Other Threads:
     
205.   6000000001609c00 VMThread [stack: 9ffffffecf783000,9ffffffecf7c5000] [id=14, lwp_id=30360]
     
206.   600000000167a000 WatcherThread [stack: 9ffffffecf283000,9ffffffecf2c5000] [id=21, lwp_id=30367]
     
207. 
     
208. VM state: not at safepoint (normal execution)
     
209. 
     
210. VM Mutex/Monitor currently owned by a thread: None
     
211. 
     
212. Heap
     
213. par new generation   total 760704K, used 276437K [9ffffffecf800000, 9fffffff01800000, 9fffffff01800000)
     
214.   eden space 702208K,  39% used [9ffffffecf800000, 9ffffffee05f5438, 9ffffffefa5c0000)
     
215.   from space 58496K,   0% used [9ffffffefa5c0000, 9ffffffefa5c0000, 9ffffffefdee0000)
     
216.   to   space 58496K,   0% used [9ffffffefdee0000, 9ffffffefdee0000, 9fffffff01800000)
     
217. concurrent mark-sweep generation total 1015808K, used 0K [9fffffff01800000, 9fffffff3f800000, 9fffffff3f800000)
     
218. concurrent-mark-sweep perm gen total 524288K, used 75169K [9fffffff3f800000, 9fffffff5f800000, 9fffffff5f800000)
     
219. 
     
220. Code Cache  [9fffffffaf000000, 9fffffffaf400000, 9fffffffbf000000)
     
221. total_blobs=879 nmethods=497 adapters=336 free_code_cache=265863616 largest_free_block=9024
     
222. 
     
223. Dynamic libraries:
     
224. /opt/java6/bin/IA64W/java
     
225.         text:0x4000000000000000-0x400000000001b450 data:0x6000000000000000-0x6000000000000500
     
226. /usr/lib/hpux64/libpthread.so.1
     
227.         text:0xc0000000000bc000-0xc0000000001d1030 data:0x9fffffffbf7ec000-0x9fffffffbf7efcd0
     
228. /opt/java6/bin/IA64W/../../jre/lib/IA64W/jli//libjli.so
     
229.         text:0xc0000000082b4000-0xc0000000082d0ab0 data:0x9fffffffbf7ea000-0x9fffffffbf7eb4a0
     
230. /usr/lib/hpux64/libdl.so.1
     
231.         text:0xc000000000490000-0xc000000000494570 data:0x9fffffffbf7e9000-0x9fffffffbf7e9160
     
232. /usr/lib/hpux64/libc.so.1
     
233.         text:0xc0000000001d4000-0xc00000000048d7e0 data:0x9fffffffbf7cf000-0x9fffffffbf7e52b8
     
234. /opt/java6/jre/lib/IA64W/server/libjvm.so
     
235.         text:0xc0000000082f8000-0xc00000000aa66c60 data:0x9fffffffbf3a1000-0x9fffffffbf6cd478
     
236. /usr/lib/hpux64/libm.so.1
     
237.         text:0xc000000000700000-0xc000000000beaf20 data:0x9fffffffbf7c0000-0x9fffffffbf7c20d8
     
238. /usr/lib/hpux64/librt.so.1
     
239.         text:0xc0000000012c4000-0xc0000000012cb580 data:0x9fffffffbf7e8000-0x9fffffffbf7e80e0
     
240. /usr/lib/hpux64/libuca.so.1
     
241.         text:0xc000000000c84000-0xc000000000c8c020 data:0x9fffffffbf7ce000-0x9fffffffbf7ce150
     
242. /usr/lib/hpux64/libelf.so.1
     
243.         text:0xc00000000aa68000-0xc00000000aa946f0 data:0x9fffffffbf7c3000-0x9fffffffbf7c39a0
     
244. /usr/lib/hpux64/libunwind.so.1
     
245.         text:0xc000000000bec000-0xc000000000c53410 data:0x9fffffffbf7b7000-0x9fffffffbf7b7d68
     
246. /usr/lib/hpux64/libCsup.so.1
     
247.         text:0xc0000000006a0000-0xc0000000006fef80 data:0x9fffffffbf7b0000-0x9fffffffbf7b63d8
     
248. /opt/java6/jre/lib/IA64W/libverify.so
     
249.         text:0xc00000000aa98000-0xc00000000aab9a80 data:0x9fffffffbf7aa000-0x9fffffffbf7ab210
     
250. /opt/java6/jre/lib/IA64W/libjava.so
     
251.         text:0xc00000000aabc000-0xc00000000ab259c0 data:0x9fffffffbf7a4000-0x9fffffffbf7a7648
     
252. /usr/lib/hpux64/libhg.so
     
253.         text:0xc0000000082d2000-0xc0000000082d4f00 data:0x9fffffffbf7a0000-0x9fffffffbf7a00b8
     
254. /opt/java6/jre/lib/IA64W/libzip.so
     
255.         text:0xc00000000ab28000-0xc00000000ab5bf20 data:0x9fffffffbf79e000-0x9fffffffbf79f870
     
256. /usr/lib/nls/loc/hpux64/locales.3/zh_CN.gb18030
     
257.         text:0xc0000000082d6000-0xc0000000082d86e0 data:0x9ffffffecf59c000-0x9ffffffecf5fcb80
     
258. /usr/lib/nls/loc/hpux32/../hpux64/methods.1/libgb18030.so
     
259.         text:0xc00000000ab5c000-0xc00000000acdfba0 data:0x9ffffffecf2f0000-0x9ffffffecf3ffd90
     
260. /opt/java6/jre/lib/IA64W/libnet.so
     
261.         text:0xc00000000ace0000-0xc00000000ad0f3b0 data:0x9fffffffbf79b000-0x9fffffffbf79bbd0
     
262. /usr/lib/hpux64/libnss_dns.so.1
     
263.         text:0xc000000008060000-0xc00000000806c290 data:0x9fffffffbf79a000-0x9fffffffbf79a348
     
264. /usr/lib/hpux64/libnss_nis.so.1
     
265.         text:0xc000000008050000-0xc00000000805f250 data:0x9fffffffbf797000-0x9fffffffbf797ad8
     
266. /usr/lib/hpux64/libnsl.so.1
     
267.         text:0xc000000000fc0000-0xc0000000010b48a0 data:0x9fffffffbf786000-0x9fffffffbf791c78
     
268. /usr/lib/hpux64/libxti.so.1
     
269.         text:0xc000000000c54000-0xc000000000c83510 data:0x9fffffffbf782000-0x9fffffffbf7849a0
     
270. /usr/lib/hpux64/libnss_files.so.1
     
271.         text:0xc000000000c90000-0xc000000000ca4f70 data:0x9fffffffbf796000-0x9fffffffbf796c80
     
272. /opt/java6/jre/lib/IA64W/libnio.so
     
273.         text:0xc00000000ad10000-0xc00000000ad1dd10 data:0x9fffffffbf785000-0x9fffffffbf785598
     
274. /opt/java6/jre/lib/IA64W/libmanagement.so
     
275.         text:0xc00000000ad20000-0xc00000000ad2c770 data:0x9fffffffbf781000-0x9fffffffbf781360
     
276. /usr/local/bea/wlserver_10.3/server/native/hpux11/IPF64/libmuxer.so
     
277.         text:0xc00000000ad30000-0xc00000000ad39890 data:0x9fffffffbf73b000-0x9fffffffbf73b8b0
     
278. /usr/local/bea/wlserver_10.3/server/native/hpux11/IPF64/libwlfileio3.so
     
279.         text:0xc00000000ad3c000-0xc00000000ad44390 data:0x9fffffffbf73a000-0x9fffffffbf73a2e8
     
280. 
     
281. VM Arguments:
     
282. jvm_args: -Dport=8004 -Xms1792m -Xmx1792m -Xmn800m -XX:PermSize=512m -XX:MaxPermSize=512m -Dweblogic.ProductionModeEnabled=true -Xmn800m -XX:+UseParNewGC -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:+UseCompressedOops -XX:+ForceMmapReserved -XX:MaxTenuringThreshold=10 -XX:SurvivorRatio=12 -XX:+PrintHeapAtGC -Xloggc:./log/gc.log -da -Dplatform.home=/usr/local/bea/wlserver_10.3 -Dwls.home=/usr/local/bea/wlserver_10.3/server -Dweblogic.home=/usr/local/bea/wlserver_10.3/server -Dweblogic.management.discover=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole=false -Dweblogic.ext.dirs=/usr/local/bea/patch_wls1036/profiles/default/sysext_manifest_classpath -Dweblogic.Name=myServer -Djava.security.policy=/usr/local/bea/wlserver_10.3/server/lib/weblogic.policy
     
283. java_command: weblogic.Server
     
284. Launcher Type: SUN_STANDARD
     
285. 
     
286. Environment Variables:
     
287. JAVA_HOME=/opt/java6
     
288. CLASSPATH=/usr/local/bea/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/opt/java6/lib/tools.jar:/usr/local/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/usr/local/bea/wlserver_10.3/server/lib/weblogic.jar:/usr/local/bea/modules/features/weblogic.server.modules_10.3.6.0.jar:/usr/local/bea/wlserver_10.3/server/lib/webservices.jar:/usr/local/bea/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/usr/local/bea/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:/usr/local/bea/wlserver_10.3/common/derby/lib/derbyclient.jar:/usr/local/bea/wlserver_10.3/server/lib/xqrl.jar:/usr/local/bea/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/opt/java6/lib/tools.jar:/usr/local/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/usr/local/bea/wlserver_10.3/server/lib/weblogic.jar:/usr/local/bea/modules/features/weblogic.server.modules_10.3.6.0.jar:/usr/local/bea/wlserver_10.3/server/lib/webservices.jar:/usr/local/bea/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/usr/local/bea/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:/usr/local/bea/wlserver_10.3/common/derby/lib/derbyclient.jar:/usr/local/bea/wlserver_10.3/server/lib/xqrl.jar:/opt/java6/lib/dt.jar:/opt/java6/lib/tools.jar
     
289. PATH=/usr/local/bea/wlserver_10.3/server/bin:/usr/local/bea/modules/org.apache.ant_1.7.1/bin:/opt/java6/jre/bin:/opt/java6/bin:/usr/local/bea/wlserver_10.3/server/bin:/usr/local/bea/modules/org.apache.ant_1.7.1/bin:/opt/java6/jre/bin:/opt/java6/bin:/usr/sbin:/usr/bin:/usr/ccs/bin:/usr/contrib/bin:/usr/contrib/Q4/bin:/opt/perl/bin:/opt/ipf/bin:/opt/gvsd/bin:/opt/nettladm/bin:/opt/fcms/bin:/opt/wbem/bin:/opt/wbem/sbin:/opt/sas/bin:/opt/graphics/common/bin:/opt/atok/bin:/usr/bin/X11:/usr/contrib/bin/X11:/opt/sec_mgmt/bastille/bin:/opt/caliper/bin:/opt/drd/bin:/opt/dsau/bin:/opt/dsau/sbin:/opt/resmon/bin:/opt/firefox:/opt/gnome/bin:/usr/contrib/kwdb/bin:/opt/mozilla:/opt/perl_32/bin:/opt/perl_64/bin:/opt/prm/bin:/opt/sfm/bin:/opt/swm/bin:/opt/sec_mgmt/spc/bin:/opt/ssh/bin:/opt/sentinel/bin:/opt/swa/bin:/opt/hpsmh/bin:/opt/thunderbird:/opt/langtools/bin:/opt/gwlm/bin:/opt/ignite/bin:/sbin:/opt/java6/bin/IA64W:/home/weblogic
     
290. SHELL=/usr/bin/sh
     
291. SHLIB_PATH=/usr/local/bea/patch_wls1036/profiles/default/native:/usr/local/bea/wlserver_10.3/server/native/hpux11/IPF64:/usr/local/bea/wlserver_10.3/server/native/hpux11/IPF64/oci920_8
     
292. 
     
293. Signal Handlers:
     
294. SIGSEGV: [libjvm.so+0xffd03288], sa_mask[0]=0xbf7ffeff, sa_flags=0x00000051
     
295. SIGBUS : [libjvm.so+0xffd03288], sa_mask[0]=0xbf7ffeff, sa_flags=0x00000051
     
296. SIGFPE : [libjvm.so+0xffd03288], sa_mask[0]=0xbf7ffeff, sa_flags=0x00000050
     
297. SIGPIPE: [libjvm.so+0xffd03288], sa_mask[0]=0xbf7ffeff, sa_flags=0x00000050
     
298. SIGXFSZ: [libjvm.so+0xffd03288], sa_mask[0]=0xbf7ffeff, sa_flags=0x00000050
     
299. SIGILL : [libjvm.so+0xffd03288], sa_mask[0]=0xbf7ffeff, sa_flags=0x00000051
     
300. SIGQUIT: [libjvm.so+0xffcfea38], sa_mask[0]=0xbf7ffeff, sa_flags=0x00000040
     
301. SIGHUP : SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000004
     
302. SIGINT : SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000004
     
303. SIGTERM: [libjvm.so+0xffcfea38], sa_mask[0]=0xbf7ffeff, sa_flags=0x00000040
     
304. SIGRTMIN: [libjvm.so+0xffd01c78], sa_mask[0]=0x00000000, sa_flags=0x00000010
     
305. SIGRTMIN+1: [libjvm.so+0xffd03288], sa_mask[0]=0xbf7ffeff, sa_flags=0x00000050
     
306. SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000004
     
307. SIGUSR2: [libjvm.so+0xffd01a88], sa_mask[0]=0x00000000, sa_flags=0x00000000
     
308. SIGPROF: [libjvm.so+0xffcfea38], sa_mask[0]=0xbf7ffeff, sa_flags=0x00000040
     
309. 
     
310. 
     
311. ---------------  S Y S T E M  ---------------
     
312. 
     
313. OS:HP-UX
     
314. uname:HP-UX B.11.31 U ia64
     
315. rlimit: STACK 1048576k, CORE 2097151k, NOFILE 4096, AS infinity
     
316. load average:1.52 1.05 0.54
     
317. 
     
318. CPU:total 8
     
319. 
     
320. Memory: 4k page, physical 33523420k(14791772k free)
     
321. 
     
322. vm_info: Java HotSpot(TM) 64-Bit Server VM (20.6-b01-jre1.6.0.14-rc1b2) for hp-ux-ia64 JRE (1.6.0), built on Mar  7 2012 04:57:39 by "jinteg" with aCC
     
323. 
     
324. time: Fri Jun 13 04:51:45 2014
     
325. elapsed time: 13 seconds
     

复制代码

wlwpz

根据HP官网给出的信息，也不能具体的确定问题~~~郁闷

1. HP Consumer Support
   
2. "***Internal error (-3) while unwinding stack"
   
3. 
   
4. Print
   
5.   Share
   
6. Feedback
   
7. Issue
   
8. 
   
9. A java program aborted with the following error message:
   
10. (0) 0xc00000000a3bada0      VMError::report_and_die{_ZN7VMError14report_and_dieEv} + 0x6b0 at /CLO/Components/JAVA_HOTSPOT/Src/src/share/vm/utilities/vmError.cpp:790 [/opt/java6/jre/lib/IA64W/server/libjvm.so](1) 0xc000000009ee3c90       os::Hpux::JVM_handle_hpux_signal{_ZN2os4Hpux22JVM_handle_hpux_signalEiP9__siginfoPvi} + 0x6f0 at /CLO/Components/JAVA_HOTSPOT/Src/src/os_cpu/hp-ux_ia64/vm/os_hp-ux_ia64.cpp:1051 [/opt/java6/jre/lib/IA64W/server/libjvm.so](2) 0xc000000009ebed00      os::Hpux::signalHandler{_ZN2os4Hpux13signalHandlerEiP9__siginfoPv} + 0x80 at /CLO/Components/JAVA_HOTSPOT/Src/src/os/hp-ux/vm/os_hp-ux.cpp:4281 [/opt/java6/jre/lib/IA64W/server/libjvm.so](3) 0xe0000001080028e0  ---- Signal 10 (SIGBUS) delivered ----(4) 0x9fffffffece34660*** Internal error (-3) while unwinding stack[/CLO/Components/JAVA_HOTSPOT/Src/src/os_cpu/hp-ux_ia64/vm/thread_hp-ux_ia64.cpp:142] ***## A fatal error has been detected by the Java Runtime Environment:##  SIGBUS (10) at pc=9fffffffece34660, pid=4391, tid=43## JRE version: 6.0# Java VM: Java HotSpot(TM) 64-Bit Server VM (17.1-b03-jre1.6.0.09-rc1 mixed mode hp-ux-ia64 )# Problematic frame:# J  com.itextpdf.text.pdf.PRTokeniser.nextToken()Z## An error report file with more information is saved as:# /apps/tomcat/instance1/logs/hs_err_pid4391.log##  Please report this error to HP customer support.#@(#)Unexpected Exception regs: see 0x9fffffffef798eb0
    
11. Why did java get the error *** Internal error (-3) while unwinding stack? Does this error message indicate a problem with java and is there a solution in a later version of java?
    
12. Solution
    
13. 
    
14. This type of internal error when unwinding the stack simply means that the stack was corrupted when java tried to dump a stack trace while aborting the application. This error while unwinding the stack may be the cause of the abort, but usually java isn't the root cause of the problem that caused the stack to be trashed.
    
15. A corrupted stack problem is not typically an indicator of a problem with the JVM itself, but instead usually means that there is a problem with the application java is running. It usually indicates there is a user programming defect that causes the program to corrupt its own stack.
    
16. The couple of typical examples of how the stack can get corrupted are:
    
17. Buffer overruns, such as moving 1000 bytes into an array that only has room for 100 bytes. All 1000 bytes will be moved and the last 900 will corrupt the adjacent virtual memory for that process.
    
18. Corrupted, uninitialized or improperly initialized array subscripts, such as trying to write bytes out to a virtual address of hex zero. That would be an invalid address to any application. Many times, a call may fail (and returned zero) and that failure was not detected and the value "returned" was not checked for validity before being used.
    
19. To find out why the application is aborting, it would be very helpful to open a call with the HP Response Center and request a java core analysis. HP does not typically do core analysis on any user written or third party application, except in the case of java. HP will do a full core analysis on java cores upon request.

复制代码

rdcwayx

关于这个错误BEA-280060， 你试着每次清理 tmp/cache 目录，再重启。

wlwpz

感谢版主的建议，自启动脚本里的有清理相关文件行，总感觉是JVM出错了，具体原因没法定位呀

rm -rf ./servers/myServer/tmp/* ;
rm -rf ./servers/myServer/data/*;
rm -rf ./log/*.log ; rm -rf ./log/*.txt
nohup ./startWebLogic.sh  >> ./log/8003.out.log 2>&1  &

回复 4# rdcwayx


   

Shell_HAT

每台服务器上不要同时启动多个实例
一个实例启动完毕之后再启动下一个实例
试试