linux oracle 10.2 rac 故障

zhangmq

各位好，小弟在rhel5.1下安装linux oracle_10.2 rac（创建库时有警告，当时没留意）。重启发现节点一老是有问题，有些服务起不来
[oracle@node1 bin]$ ./crs_stat -t
Name           Type           Target    State     Host        
------------------------------------------------------------
ora....B1.inst application    ONLINE    OFFLINE               
ora....B2.inst application    ONLINE    ONLINE    node2      
ora.RACDB.db   application    ONLINE    ONLINE    node2      
ora....SM1.asm application    ONLINE    OFFLINE               
ora....E1.lsnr application    ONLINE    OFFLINE               
ora.node1.gsd  application    ONLINE    ONLINE    node1      
ora.node1.ons  application    ONLINE    ONLINE    node1      
ora.node1.vip  application    ONLINE    ONLINE    node2      
ora....SM2.asm application    ONLINE    ONLINE    node2      
ora....E2.lsnr application    ONLINE    ONLINE    node2      
ora.node2.gsd  application    ONLINE    ONLINE    node2      
ora.node2.ons  application    ONLINE    ONLINE    node2      
ora.node2.vip  application    ONLINE    ONLINE    node2      

SQL> conn / as sysdba
Connected to an idle instance.
SQL> startup
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+RAC_DISK/RACDB/spfileRACDB.ora'
ORA-17503: ksfdopn:2 Failed to open file +RAC_DISK/RACDB/spfileRACDB.ora
ORA-15077: could not locate ASM instance serving a required diskgroup
SQL> quit

zhangmq

监听也不正常
[oracle@node1 bin]$ lsnrctl status

LSNRCTL for Linux: Version 10.2.0.1.0 - Production on 01-APR-2011 19:58:39

Copyright (c) 1991, 2005, Oracle.  All rights reserved.

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 10.2.0.1.0 - Production
Start Date                01-APR-2011 19:58:03
Uptime                    0 days 0 hr. 0 min. 36 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /opt/ora10g/product/10.2.0/db_1/network/admin/listener.ora
Listener Log File         /opt/ora10g/product/10.2.0/db_1/network/log/listener.log
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=node1)(PORT=1521)))
The listener supports no services
The command completed successfully
[oracle@node1 bin]$

节点2倒是正常的
oracle@node2 admin]$ lsnrctl status

LSNRCTL for Linux: Version 10.2.0.1.0 - Production on 01-APR-2011 19:59:31

Copyright (c) 1991, 2005, Oracle.  All rights reserved.

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_NODE2
Version                   TNSLSNR for Linux: Version 10.2.0.1.0 - Production
Start Date                01-APR-2011 18:30:25
Uptime                    0 days 1 hr. 29 min. 6 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /opt/ora10g/product/10.2.0/db_1/network/admin/listener.ora
Listener Log File         /opt/ora10g/product/10.2.0/db_1/network/log/listener_node2.log
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.10.222)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.10.212)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM2", status BLOCKED, has 1 handler(s) for this service...
Service "+ASM_XPT" has 1 instance(s).
  Instance "+ASM2", status BLOCKED, has 1 handler(s) for this service...
Service "RACDB" has 1 instance(s).
  Instance "RACDB2", status READY, has 1 handler(s) for this service...
Service "RACDBXDB" has 1 instance(s).
  Instance "RACDB2", status READY, has 1 handler(s) for this service...
Service "RACDB_XPT" has 1 instance(s).
  Instance "RACDB2", status READY, has 1 handler(s) for this service...
The command completed successfully
[oracle@node2 admin]$

renxiao2003

ORA-01565: error in identifying file '+RAC_DISK/RACDB/spfileRACDB.ora'
ORA-17503: ksfdopn:2 Failed to open file +RAC_DISK/RACDB/spfileRACDB.ora

没做过RAC，但从错误看，你这个节点应该没有访问到spfile文件。

duolanshizhe

恩  我来谈几点：

1、实例一可能asm实例有问题，请查看。或者贴出相关信息

2、10.2的rac，没有打上patchset3  那是相当恐怖的一件事情

3、实例一的asm的实例没有起来导致db实例不能起来，从而监听也没有注册的服务，这个很正常！

zhangmq

[oracle@node1 bin]$ srvctl status database -d RACDB
Instance RACDB1 is not running on node node1
Instance RACDB2 is running on node node2

zhangmq

[oracle@node1 bin]$ srvctl status database -d RACDB
Instance RACDB1 is not running on node node1
Instance RACDB2 is running on node node2
[oracle@node1 bin]$ srvctl status instance -d RACDB -i RACDB1,RACDB2
Instance RACDB1 is not running on node node1
Instance RACDB2 is running on node node2
[oracle@node1 bin]$ srvctl status asm -n node1
ASM instance +ASM1 is not running on node node1.
[oracle@node1 bin]$ srvctl status asm -n node2
ASM instance +ASM2 is running on node node2.
[oracle@node1 bin]$ srvctl status nodeapps -n node1
VIP is running on node: node2
GSD is running on node: node1
Listener is not running on node: node1
ONS daemon is running on node: node1
[oracle@node1 bin]$ srvctl status nodeapps -n node2
VIP is running on node: node2
GSD is running on node: node2
Listener is running on node: node2
ONS daemon is running on node: node2
[oracle@node1 bin]$

zhangmq

对啦，跟执行/etc/init.d/oracleasm listdisks没看到东西有没直接关系，昨天安装时是能找到vol1/vol2的

zhangmq

顶！

zhangmq

[root@node1 bin]# srvctl start instance -d RACDB -i RACDB1
PRKP-1001 : Error starting instance RACDB1 on node node1
node1ra.node1.vip:checkIf: Default gateway is not defined (host=node1)
node1ra.node1.vip:Interface eth0 checked failed (host=node1)
node1ra.node1.vip:Invalid parameters, or failed to bring up VIP (host=node1)
node1ra.node1.vip:checkIf: Default gateway is not defined (host=node1)
node1ra.node1.vip:Interface eth0 checked failed (host=node1)
node1ra.node1.vip:Invalid parameters, or failed to bring up VIP (host=node1)
CRS-0215: Could not start resource 'ora.RACDB.RACDB1.inst'.

zhangmq

发现node1网卡1的网关没配置，重新配置好后执行
[root@node1 oracle]# srvctl start instance -d RACDB -i RACDB1
PRKP-1001 : Error starting instance RACDB1 on node node1
node1ra.node1.vip:IP:192.168.10.221 is already up in the network (host=node1)
node1ra.node1.vip:IP:192.168.10.221 is already up in the network (host=node1)
CRS-0215: Could not start resource 'ora.RACDB.RACDB1.inst'.