免费注册 查看新帖 |

Chinaunix

  平台 论坛 博客 文库
最近访问板块 发新帖
查看: 7987 | 回复: 1

服务器挂了。给看下具体那出问题了。 [复制链接]

论坛徽章:
0
发表于 2013-09-29 18:41 |显示全部楼层
2机 message日志:

1机 message日志:
Sep 26 03:10:10 XZ-XJ-AAA01 syslogd: line 31: unknown priority name "err                                           @loghost"Sep 26 03:10:10 XZ-XJ-AAA01 syslogd: line 32: unknown priority name "err        @59.43.50.100"Sep 26 03:10:10 XZ-XJ-AAA01 syslogd: line 33: unknown priority name "err        @59.43.50.102"Sep 26 03:10:10 XZ-XJ-AAA01 syslogd: line 34: unknown priority name "info       @59.43.50.100"Sep 26 03:10:10 XZ-XJ-AAA01 syslogd: line 35: unknown priority name "info       @59.43.50.102"Sep 26 08:37:56 XZ-XJ-AAA01 su: [ID 810491 auth.crit] 'su root' failed for xznoc on /dev/pts/32Sep 27 10:06:28 XZ-XJ-AAA01 su: [ID 810491 auth.crit] 'su root' failed for xznoc on /dev/pts/24Sep 27 10:09:04 XZ-XJ-AAA01 last message repeated 3 timesSep 27 10:38:28 XZ-XJ-AAA01 e1000g: [ID 801725 kern.info] NOTICE: pciex8086,105e - e1000g[1] : link downSep 27 10:38:28 XZ-XJ-AAA01 e1000g: [ID 801725 kern.info] NOTICE: pciex8086,105e - e1000g[2] : link downSep 27 10:38:30 XZ-XJ-AAA01 llt: [ID 140958 kern.notice] LLT INFO V-14-1-10205 link 0 (e1000g1) node 1 in troubleSep 27 10:38:30 XZ-XJ-AAA01 llt: [ID 140958 kern.notice] LLT INFO V-14-1-10205 link 1 (e1000g2) node 1 in troubleSep 27 10:38:36 XZ-XJ-AAA01 llt: [ID 487101 kern.notice] LLT INFO V-14-1-10032 link 0 (e1000g1) node 1 inactive 8 sec (11736191Sep 27 10:38:36 XZ-XJ-AAA01 llt: [ID 487101 kern.notice] LLT INFO V-14-1-10032 link 1 (e1000g2) node 1 inactive 8 sec (116911477)Sep 27 10:38:37 XZ-XJ-AAA01 llt: [ID 487101 kern.notice] LLT INFO V-14-1-10032 link 0 (e1000g1) node 1 inactive 9 sec (11736191

2主机vcs日志:

AAA01主机vcs 日志27日故障前没有日志,26日日志如下:
2013/09/26 17:50:52 VCS ERROR V-16-2-13067 (XZ-XJ-AAA02) Agent is calling clean for resource(solid_app) because the resource became OFFLINE unexpectedly, on its own.
2013/09/26 17:50:52 VCS INFO V-16-2-13068 (XZ-XJ-AAA02) Resource(solid_app) - clean completed successfully.
2013/09/26 17:50:52 VCS INFO V-16-1-10307 Resource solid_app (Owner: Unspecified, Group: solid) is offline on XZ-XJ-AAA02 (Not initiated by VCS)
2013/09/26 17:50:52 VCS NOTICE V-16-1-10300 Initiating Offline of Resource solid_ip (Owner: Unspecified, Group: solid) on System XZ-XJ-AAA02
2013/09/26 17:50:52 VCS NOTICE V-16-1-10300 Initiating Offline of Resource solid_mount (Owner: Unspecified, Group: solid) on System XZ-XJ-AAA02
2013/09/26 17:51:58 VCS ERROR V-16-2-13027 (XZ-XJ-AAA02) Resource(nms_net) - monitor procedure did not complete within the expected time.
2013/09/26 19:20:19 VCS INFO V-16-1-53504 VCS Engine Alive message!!
2013/09/26 23:20:20 VCS INFO V-16-1-53504 VCS Engine Alive message!!

27日故障前2主机上数据库日志有报错:
从8点59分开始有creation failed报错,数据库开始出现问题:
Fri Sep 27 08:59:52 2013
ksvcreate: Process(q001) creation failed
Fri Sep 27 09:00:04 2013
Process startup failed, error stack:
Fri Sep 27 09:00:04 2013
Errors in file /export/home/oracle/ora10g/admin/ora10g/bdump/ora10g_psp0_6501.trc:
ORA-27300: OS system dependent operation:pipe failed with status: 24
ORA-27301: OS failure message: Too many open files
ORA-27302: failure occurred at: skgpspawn2
Fri Sep 27 09:00:05 2013
Process q001 died, see its trace file
Fri Sep 27 09:00:05 2013
ksvcreate: Process(q001) creation failed
Fri Sep 27 09:00:17 2013
Process startup failed, error stack:
Fri Sep 27 09:00:17 2013
Errors in file /export/home/oracle/ora10g/admin/ora10g/bdump/ora10g_psp0_6501.trc:
ORA-27300: OS system dependent operation:pipe failed with status: 24
ORA-27301: OS failure message: Too many open files
ORA-27302: failure occurred at: skgpspawn2
Fri Sep 27 09:00:18 2013
Process q001 died, see its trace file
Fri Sep 27 09:00:18 2013
ksvcreate: Process(q001) creation failed
Fri Sep 27 09:00:30 2013
Process startup failed, error stack:
Fri Sep 27 09:00:30 2013
Errors in file /export/home/oracle/ora10g/admin/ora10g/bdump/ora10g_psp0_6501.trc:
ORA-27300: OS system dependent operation:pipe failed with status: 24
ORA-27301: OS failure message: Too many open files
ORA-27302: failure occurred at: skgpspawn2
Fri Sep 27 09:00:31 2013
Process q001 died, see its trace file
Fri Sep 27 09:00:31 2013
ksvcreate: Process(q001) creation failed
Fri Sep 27 09:00:34 2013
Process startup failed, error stack:



附带explorer

论坛徽章:
2
双鱼座
日期:2014-02-23 12:10:03操作系统版块每日发帖之星
日期:2015-12-17 06:20:00
发表于 2013-09-30 11:02 |显示全部楼层
2013/09/26 17:48:52 VCS WARNING V-16-1-10023 Agent NIC not sending alive messages since Thu Sep 26 17:46:36 2013
2013/09/26 17:48:52 VCS WARNING V-16-1-53025 Agent NIC has faulted; ipm connection was lost; restarting the agent
2013/09/26 17:49:12 VCS ERROR V-16-1-10015 Cannot start /opt/VRTSvcs/bin/NIC/NICAgent please check file
2013/09/26 17:49:12 VCS WARNING V-16-1-10485 Excessive delay between successive calls to GAB heartbeat (20 seconds)
2013/09/26 17:50:52 VCS ERROR V-16-2-13067 (XZ-XJ-AAA02) Agent is calling clean for resource(solid_app) because the resource became OFFLINE unexpectedly, on its own.
检查下主机的网络连接情况、交换机情况。。。。。VCS不熟。。。。
您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

  

北京盛拓优讯信息技术有限公司. 版权所有 京ICP备16024965号-6 北京市公安局海淀分局网监中心备案编号:11010802020122 niuxiaotong@pcpop.com 17352615567
未成年举报专区
中国互联网协会会员  联系我们:huangweiwei@itpub.net
感谢所有关心和支持过ChinaUnix的朋友们 转载本站内容请注明原作者名及出处

清除 Cookies - ChinaUnix - Archiver - WAP - TOP