oracle变态行为

hyong2011

安装oracle客户端oracle-instantclient11.2-basic，oracle-instantclient11.2-devel后，在用户目录下产生大量的trace日志文件，都找不到关闭的方法？
安装oracle服务器版本后，core_dump等trace日志也不能关，每天产生大量的日志文件，这些难道不影响服务器性能么？？

duolanshizhe

首先告诉你存在就是合理的！
正常情况下，不应该出现大量日志的，首先从自身找问题，日志是诊断问题的利器！
再者说，在现有硬件情况下，写日志导致的性能下降基本上可以忽略不计，不过可能会占用一些空间，这也不是问题！

hyong2011

1、问题是装个客户端连接ORACLE每天都产生十几万个文件，每个文件就几百字节，删起来要花好多时间，一般PC服务器对小文件的处理好像性能不怎么的哦，
2、服务器的日志全是TNS  WAIT ，虽然配置了DG，这应该是正常的吧

duolanshizhe

这两类日志文件貌似不正常，数量多要么是bug，要么是配置存在问题！

请贴出相关日志内容，以便进一步分析与定位！

hyong2011

ALTER日志：

Fatal NI connect error 12170.

  VERSION INFORMATION:
TNS for Linux: Version 11.2.0.1.0 - Production
Oracle Bequeath NT Protocol Adapter for Linux: Version 11.2.0.1.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 11.2.0.1.0 - Production
  Time: 15-OCT-2013 16:20:26
  Tracing not turned on.
  Tns error struct:
    ns main err code: 12535

TNS-12535: TNSperation timed out
    ns secondary err code: 12560
    nt main err code: 505

TNS-00505: Operation timed out
    nt secondary err code: 110
    nt OS err code: 0
  Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=121.8.248.106)(PORT=5604))
2013-10-15 16:20:48.373000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20598 (LGWR switch)
  Current log# 3 seq# 20598 mem# 0: /data/oralog/redo03.log
Archived Log entry 4732 added for thread 1 sequence 20597 ID 0x4fe35bd1 dest 1:
2013-10-15 16:23:37.066000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20599 (LGWR switch)
  Current log# 1 seq# 20599 mem# 0: /data/oralog/redo01.log
Archived Log entry 4734 added for thread 1 sequence 20598 ID 0x4fe35bd1 dest 1:
2013-10-15 16:25:54.929000 +08:00
Thread 1 cannot allocate new log, sequence 20600
Checkpoint not complete
  Current log# 1 seq# 20599 mem# 0: /data/oralog/redo01.log
2013-10-15 16:25:59.118000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20600 (LGWR switch)
  Current log# 2 seq# 20600 mem# 0: /data/oralog/redo02.log
Archived Log entry 4736 added for thread 1 sequence 20599 ID 0x4fe35bd1 dest 1:
2013-10-15 16:28:34.478000 +08:00
Thread 1 cannot allocate new log, sequence 20601
Checkpoint not complete
  Current log# 2 seq# 20600 mem# 0: /data/oralog/redo02.log
2013-10-15 16:28:36.527000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20601 (LGWR switch)
  Current log# 3 seq# 20601 mem# 0: /data/oralog/redo03.log
Archived Log entry 4738 added for thread 1 sequence 20600 ID 0x4fe35bd1 dest 1:
2013-10-15 16:31:50.828000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20602 (LGWR switch)
  Current log# 1 seq# 20602 mem# 0: /data/oralog/redo01.log
Archived Log entry 4740 added for thread 1 sequence 20601 ID 0x4fe35bd1 dest 1:
2013-10-15 16:35:16.066000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20603 (LGWR switch)
  Current log# 2 seq# 20603 mem# 0: /data/oralog/redo02.log
Archived Log entry 4742 added for thread 1 sequence 20602 ID 0x4fe35bd1 dest 1:
2013-10-15 16:37:46.271000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20604 (LGWR switch)
  Current log# 3 seq# 20604 mem# 0: /data/oralog/redo03.log
Archived Log entry 4744 added for thread 1 sequence 20603 ID 0x4fe35bd1 dest 1:
2013-10-15 16:40:49.318000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20605 (LGWR switch)
  Current log# 1 seq# 20605 mem# 0: /data/oralog/redo01.log
Archived Log entry 4746 added for thread 1 sequence 20604 ID 0x4fe35bd1 dest 1:
2013-10-15 16:43:24.932000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20606 (LGWR switch)
  Current log# 2 seq# 20606 mem# 0: /data/oralog/redo02.log
Archived Log entry 4748 added for thread 1 sequence 20605 ID 0x4fe35bd1 dest 1:
2013-10-15 16:45:49.564000 +08:00
Thread 1 cannot allocate new log, sequence 20607
Checkpoint not complete
  Current log# 2 seq# 20606 mem# 0: /data/oralog/redo02.log
2013-10-15 16:45:52.350000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20607 (LGWR switch)
  Current log# 3 seq# 20607 mem# 0: /data/oralog/redo03.log
Archived Log entry 4750 added for thread 1 sequence 20606 ID 0x4fe35bd1 dest 1:
2013-10-15 16:47:32.250000 +08:00
Thread 1 cannot allocate new log, sequence 20608
Checkpoint not complete
  Current log# 3 seq# 20607 mem# 0: /data/oralog/redo03.log
2013-10-15 16:47:34.391000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20608 (LGWR switch)
Current log# 1 seq# 20608 mem# 0: /data/oralog/redo01.log
Archived Log entry 4752 added for thread 1 sequence 20607 ID 0x4fe35bd1 dest 1:
2013-10-15 16:48:48.571000 +08:00
Thread 1 cannot allocate new log, sequence 20609
Checkpoint not complete
  Current log# 1 seq# 20608 mem# 0: /data/oralog/redo01.log
2013-10-15 16:48:52.730000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20609 (LGWR switch)
  Current log# 2 seq# 20609 mem# 0: /data/oralog/redo02.log
Archived Log entry 4754 added for thread 1 sequence 20608 ID 0x4fe35bd1 dest 1:
2013-10-15 16:50:15.013000 +08:00
Thread 1 cannot allocate new log, sequence 20610
Checkpoint not complete
  Current log# 2 seq# 20609 mem# 0: /data/oralog/redo02.log
2013-10-15 16:50:17.479000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20610 (LGWR switch)
  Current log# 3 seq# 20610 mem# 0: /data/oralog/redo03.log
Archived Log entry 4756 added for thread 1 sequence 20609 ID 0x4fe35bd1 dest 1:
2013-10-15 16:50:59.124000 +08:00
AUD: Audit Commit Delay exceeded, written a copy to OS Audit Trail
2013-10-15 16:52:12.586000 +08:00
Thread 1 cannot allocate new log, sequence 20611
Checkpoint not complete
  Current log# 3 seq# 20610 mem# 0: /data/oralog/redo03.log
2013-10-15 16:52:13.967000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20611 (LGWR switch)
  Current log# 1 seq# 20611 mem# 0: /data/oralog/redo01.log
Archived Log entry 4758 added for thread 1 sequence 20610 ID 0x4fe35bd1 dest 1:
2013-10-15 16:54:37.067000 +08:00
Thread 1 cannot allocate new log, sequence 20612
Checkpoint not complete
  Current log# 1 seq# 20611 mem# 0: /data/oralog/redo01.log
2013-10-15 16:54:39.500000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20612 (LGWR switch)
  Current log# 2 seq# 20612 mem# 0: /data/oralog/redo02.log
Archived Log entry 4760 added for thread 1 sequence 20611 ID 0x4fe35bd1 dest 1:
2013-10-15 16:56:49.739000 +08:00
Thread 1 cannot allocate new log, sequence 20613
Checkpoint not complete
  Current log# 2 seq# 20612 mem# 0: /data/oralog/redo02.log
2013-10-15 16:56:55.981000 +08:00
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
Destination LOG_ARCHIVE_DEST_2 no longer supports SYNCHRONIZATION
Thread 1 advanced to log sequence 20613 (LGWR switch)
  Current log# 3 seq# 20613 mem# 0: /data/oralog/redo03.log
Archived Log entry 4762 added for thread 1 sequence 20612 ID 0x4fe35bd1 dest 1:

hyong2011

trace文件
*** 2013-10-15 17:02:48.465
WAIT #0: nam='log file parallel write' ela= 5643 files=1 blocks=3 requests=1 obj#=-1 tim=1381827768465456
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=3 requests=1 obj#=-1 tim=1381827768465480
WAIT #0: nam='LGWR wait on LNS' ela= 3 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768465492
WAIT #0: nam='log file parallel write' ela= 11891 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768477422
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768477439
WAIT #0: nam='LGWR wait on LNS' ela= 3 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768477450
WAIT #0: nam='log file parallel write' ela= 11932 files=1 blocks=3 requests=1 obj#=-1 tim=1381827768489413
WAIT #0: nam='log file parallel write' ela= 0 files=1 blocks=3 requests=1 obj#=-1 tim=1381827768489430
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768489441
WAIT #0: nam='log file parallel write' ela= 7138 files=1 blocks=2 requests=1 obj#=-1 tim=1381827768496612
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=2 requests=1 obj#=-1 tim=1381827768496628
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768496638
WAIT #0: nam='log file parallel write' ela= 16676 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768513346
WAIT #0: nam='log file parallel write' ela= 0 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768513366
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768513377
WAIT #0: nam='log file parallel write' ela= 7178 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768520583
WAIT #0: nam='log file parallel write' ela= 0 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768520599
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768520610
WAIT #0: nam='LGWR real time apply sync' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768520620
WAIT #0: nam='LGWR-LNS wait on channel' ela= 272 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768520915
WAIT #0: nam='LGWR wait on LNS' ela= 15 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768520930
WAIT #0: nam='rdbms ipc message' ela= 588 timeout=185 p2=0 p3=0 obj#=-1 tim=1381827768521529
WAIT #0: nam='log file parallel write' ela= 16931 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768538493
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768538510
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768538520
WAIT #0: nam='log file parallel write' ela= 10740 files=1 blocks=2 requests=1 obj#=-1 tim=1381827768549289
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=2 requests=1 obj#=-1 tim=1381827768549306
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768549316
WAIT #0: nam='LGWR real time apply sync' ela= 1 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768549326
WAIT #0: nam='LGWR-LNS wait on channel' ela= 257 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768549606
WAIT #0: nam='LGWR wait on LNS' ela= 17 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768549622
WAIT #0: nam='rdbms ipc message' ela= 2720 timeout=182 p2=0 p3=0 obj#=-1 tim=1381827768552352
WAIT #0: nam='log file parallel write' ela= 10101 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768562494
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768562514
WAIT #0: nam='LGWR wait on LNS' ela= 3 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768562538
WAIT #0: nam='log file parallel write' ela= 5859 files=1 blocks=3 requests=1 obj#=-1 tim=1381827768568433
WAIT #0: nam='log file parallel write' ela= 0 files=1 blocks=3 requests=1 obj#=-1 tim=1381827768568450
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768568461
WAIT #0: nam='log file parallel write' ela= 17995 files=1 blocks=26 requests=1 obj#=-1 tim=1381827768586488
WAIT #0: nam='log file parallel write' ela= 0 files=1 blocks=26 requests=1 obj#=-1 tim=1381827768586505
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768586516
WAIT #0: nam='log file parallel write' ela= 5928 files=1 blocks=2 requests=1 obj#=-1 tim=1381827768592472
WAIT #0: nam='log file parallel write' ela= 0 files=1 blocks=2 requests=1 obj#=-1 tim=1381827768592488
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768592499
WAIT #0: nam='LGWR real time apply sync' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768592509
WAIT #0: nam='LGWR-LNS wait on channel' ela= 268 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768592800
WAIT #0: nam='LGWR wait on LNS' ela= 16 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768592816
WAIT #0: nam='rdbms ipc message' ela= 5830 timeout=178 p2=0 p3=0 obj#=-1 tim=1381827768598657
WAIT #0: nam='log file parallel write' ela= 11732 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768610423
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768610441
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768610451
WAIT #0: nam='LGWR real time apply sync' ela= 1 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768610461
WAIT #0: nam='LGWR-LNS wait on channel' ela= 277 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768610762
WAIT #0: nam='LGWR wait on LNS' ela= 16 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768610777
WAIT #0: nam='rdbms ipc message' ela= 2634 timeout=176 p2=0 p3=0 obj#=-1 tim=1381827768613423
WAIT #0: nam='log file parallel write' ela= 13716 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768627173
WAIT #0: nam='log file parallel write' ela= 0 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768627190
WAIT #0: nam='LGWR wait on LNS' ela= 2 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768627201
WAIT #0: nam='LGWR real time apply sync' ela= 1 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768627211
WAIT #0: nam='LGWR-LNS wait on channel' ela= 275 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768627510
WAIT #0: nam='LGWR wait on LNS' ela= 15 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768627525
WAIT #0: nam='rdbms ipc message' ela= 998 timeout=174 p2=0 p3=0 obj#=-1 tim=1381827768628533
WAIT #0: nam='log file parallel write' ela= 17760 files=1 blocks=3 requests=1 obj#=-1 tim=1381827768646323
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=3 requests=1 obj#=-1 tim=1381827768646352
WAIT #0: nam='LGWR wait on LNS' ela= 3 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768646372
WAIT #0: nam='log file parallel write' ela= 12196 files=1 blocks=76 requests=1 obj#=-1 tim=1381827768658623
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=76 requests=1 obj#=-1 tim=1381827768658661
WAIT #0: nam='LGWR wait on LNS' ela= 6 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768658679
WAIT #0: nam='log file parallel write' ela= 11902 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768670610
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768670653
WAIT #0: nam='LGWR-LNS wait on channel' ela= 5 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768670715
WAIT #0: nam='LGWR-LNS wait on channel' ela= 338 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768671069
WAIT #0: nam='LGWR wait on LNS' ela= 36 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768671087
WAIT #0: nam='log file parallel write' ela= 11491 files=1 blocks=18 requests=1 obj#=-1 tim=1381827768682616
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=18 requests=1 obj#=-1 tim=1381827768682672
WAIT #0: nam='LGWR wait on LNS' ela= 8 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768682699
WAIT #0: nam='log file parallel write' ela= 6140 files=1 blocks=97 requests=1 obj#=-1 tim=1381827768688928
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=97 requests=1 obj#=-1 tim=1381827768688989
WAIT #0: nam='LGWR wait on LNS' ela= 6 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768689009
WAIT #0: nam='log file parallel write' ela= 19285 files=1 blocks=67 requests=2 obj#=-1 tim=1381827768708366
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=67 requests=2 obj#=-1 tim=1381827768708422
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768708452
WAIT #0: nam='log file parallel write' ela= 17898 files=1 blocks=17 requests=2 obj#=-1 tim=1381827768726449
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=17 requests=2 obj#=-1 tim=1381827768726509
WAIT #0: nam='LGWR wait on LNS' ela= 10 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768726541
WAIT #0: nam='log file parallel write' ela= 11933 files=1 blocks=40 requests=2 obj#=-1 tim=1381827768738573
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=40 requests=2 obj#=-1 tim=1381827768738612
WAIT #0: nam='LGWR wait on LNS' ela= 6 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768738632
WAIT #0: nam='log file parallel write' ela= 23885 files=1 blocks=21 requests=2 obj#=-1 tim=1381827768762587
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=21 requests=2 obj#=-1 tim=1381827768762641
WAIT #0: nam='LGWR wait on LNS' ela= 10 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768762667
WAIT #0: nam='log file parallel write' ela= 11824 files=1 blocks=20 requests=2 obj#=-1 tim=1381827768774575
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=20 requests=2 obj#=-1 tim=1381827768774634
WAIT #0: nam='LGWR wait on LNS' ela= 11 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768774661
WAIT #0: nam='LGWR wait for redo copy' ela= 4 copy latch #=11 p2=0 p3=0 obj#=-1 tim=1381827768774695
WAIT #0: nam='log file parallel write' ela= 6050 files=1 blocks=65 requests=1 obj#=-1 tim=1381827768780809
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=65 requests=1 obj#=-1 tim=1381827768780857
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768780882
WAIT #0: nam='log file parallel write' ela= 5956 files=1 blocks=30 requests=1 obj#=-1 tim=1381827768786916
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=30 requests=1 obj#=-1 tim=1381827768786971
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768786995
WAIT #0: nam='log file parallel write' ela= 5897 files=1 blocks=25 requests=1 obj#=-1 tim=1381827768792968
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=25 requests=1 obj#=-1 tim=1381827768793012
WAIT #0: nam='LGWR wait on LNS' ela= 5 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768793029
WAIT #0: nam='LGWR real time apply sync' ela= 5 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768793048
WAIT #0: nam='LGWR-LNS wait on channel' ela= 318 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768793410
WAIT #0: nam='LGWR wait on LNS' ela= 21 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768793430
WAIT #0: nam='rdbms ipc message' ela= 465 timeout=157 p2=0 p3=0 obj#=-1 tim=1381827768793906
WAIT #0: nam='log file parallel write' ela= 11017 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768804972
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=1 requests=1 obj#=-1 tim=1381827768805037
WAIT #0: nam='LGWR wait on LNS' ela= 10 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768805069
WAIT #0: nam='log file parallel write' ela= 8004 files=1 blocks=43 requests=1 obj#=-1 tim=1381827768813171
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=43 requests=1 obj#=-1 tim=1381827768813223
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768813246
WAIT #0: nam='log file parallel write' ela= 6840 files=1 blocks=63 requests=1 obj#=-1 tim=1381827768820160
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=63 requests=1 obj#=-1 tim=1381827768820221
WAIT #0: nam='LGWR wait on LNS' ela= 10 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768820253
WAIT #0: nam='log file parallel write' ela= 5844 files=1 blocks=29 requests=1 obj#=-1 tim=1381827768826195
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=29 requests=1 obj#=-1 tim=1381827768826260
WAIT #0: nam='LGWR wait on LNS' ela= 10 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768826322
WAIT #0: nam='log file parallel write' ela= 5810 files=1 blocks=18 requests=1 obj#=-1 tim=1381827768832231
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=18 requests=1 obj#=-1 tim=1381827768832283
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768832315
WAIT #0: nam='log file parallel write' ela= 6006 files=1 blocks=57 requests=1 obj#=-1 tim=1381827768838419
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=57 requests=1 obj#=-1 tim=1381827768838479
WAIT #0: nam='LGWR wait on LNS' ela= 8 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768838509
WAIT #0: nam='log file parallel write' ela= 5848 files=1 blocks=17 requests=1 obj#=-1 tim=1381827768844448
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=17 requests=1 obj#=-1 tim=1381827768844507
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768844538
WAIT #0: nam='log file parallel write' ela= 11351 files=1 blocks=93 requests=1 obj#=-1 tim=1381827768855988
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=93 requests=1 obj#=-1 tim=1381827768856044
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768856068
WAIT #0: nam='log file parallel write' ela= 6720 files=1 blocks=45 requests=1 obj#=-1 tim=1381827768862864
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=45 requests=1 obj#=-1 tim=1381827768862926
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768862958
WAIT #0: nam='log file parallel write' ela= 12127 files=1 blocks=101 requests=2 obj#=-1 tim=1381827768875202
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=101 requests=2 obj#=-1 tim=1381827768875266
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768875298
WAIT #0: nam='log file parallel write' ela= 10444 files=1 blocks=111 requests=2 obj#=-1 tim=1381827768885848
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=111 requests=2 obj#=-1 tim=1381827768885908
WAIT #0: nam='LGWR wait on LNS' ela= 10 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768885939
WAIT #0: nam='log file parallel write' ela= 17597 files=1 blocks=49 requests=1 obj#=-1 tim=1381827768903650
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=49 requests=1 obj#=-1 tim=1381827768903716
WAIT #0: nam='LGWR wait on LNS' ela= 10 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768903748
WAIT #0: nam='log file parallel write' ela= 6519 files=1 blocks=129 requests=1 obj#=-1 tim=1381827768910383
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=129 requests=1 obj#=-1 tim=1381827768910442
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768910473
WAIT #0: nam='log file parallel write' ela= 11789 files=1 blocks=30 requests=1 obj#=-1 tim=1381827768922381
WAIT #0: nam='log file parallel write' ela= 1 files=1 blocks=30 requests=1 obj#=-1 tim=1381827768922445
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768922477
WAIT #0: nam='log file parallel write' ela= 11733 files=1 blocks=186 requests=1 obj#=-1 tim=1381827768934316
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=186 requests=1 obj#=-1 tim=1381827768934385
WAIT #0: nam='LGWR wait on LNS' ela= 10 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768934417
WAIT #0: nam='log file parallel write' ela= 9124 files=1 blocks=104 requests=1 obj#=-1 tim=1381827768943641
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=104 requests=1 obj#=-1 tim=1381827768943703
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768943735
WAIT #0: nam='log file parallel write' ela= 8856 files=1 blocks=43 requests=1 obj#=-1 tim=1381827768952683
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=43 requests=1 obj#=-1 tim=1381827768952745
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768952776
WAIT #0: nam='log file parallel write' ela= 9142 files=1 blocks=48 requests=1 obj#=-1 tim=1381827768962015
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=48 requests=1 obj#=-1 tim=1381827768962099
WAIT #0: nam='LGWR wait on LNS' ela= 9 p1=0 p2=0 p3=0 obj#=-1 tim=1381827768962124
WAIT #0: nam='control file sequential read' ela= 9 file#=0 block#=1 blocks=1 obj#=-1 tim=1381827768962213
WAIT #0: nam='control file sequential read' ela= 10 file#=1 block#=1 blocks=1 obj#=-1 tim=1381827768962241
WAIT #0: nam='control file sequential read' ela= 4 file#=0 block#=16 blocks=1 obj#=-1 tim=1381827768962256
WAIT #0: nam='control file sequential read' ela= 4 file#=0 block#=18 blocks=1 obj#=-1 tim=1381827768962269
WAIT #0: nam='control file sequential read' ela= 5 file#=0 block#=19 blocks=1 obj#=-1 tim=1381827768962287
WAIT #0: nam='control file sequential read' ela= 6 file#=0 block#=281 blocks=1 obj#=-1 tim=1381827768962304
WAIT #0: nam='control file sequential read' ela= 4 file#=0 block#=21 blocks=1 obj#=-1 tim=1381827768962318
WAIT #0: nam='control file parallel write' ela= 44566 files=2 block#=22 requests=2 obj#=-1 tim=1381827769006900
WAIT #0: nam='control file parallel write' ela= 42019 files=2 block#=17 requests=2 obj#=-1 tim=1381827769048970
WAIT #0: nam='control file parallel write' ela= 45106 files=2 block#=15 requests=2 obj#=-1 tim=1381827769094123
WAIT #0: nam='control file parallel write' ela= 12595 files=2 block#=1 requests=2 obj#=-1 tim=1381827769106779
WAIT #0: nam='control file sequential read' ela= 10 file#=0 block#=1 blocks=1 obj#=-1 tim=1381827769106827
WAIT #0: nam='log file parallel write' ela= 19406 files=1 blocks=8 requests=1 obj#=-1 tim=1381827769126312
WAIT #0: nam='log file parallel write' ela= 2 files=1 blocks=8 requests=1 obj#=-1 tim=1381827769126361
WAIT #0: nam='LGWR wait on LNS' ela= 8 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769126384
WAIT #0: nam='log file sequential read' ela= 6 log#=0 block#=1 blocks=1 obj#=-1 tim=1381827769126458
WAIT #0: nam='control file sequential read' ela= 7 file#=0 block#=31 blocks=1 obj#=-1 tim=1381827769126485
WAIT #0: nam='log file single write' ela= 22317 log#=0 block#=1 blocks=1 obj#=-1 tim=1381827769148852
WAIT #0: nam='control file parallel write' ela= 31378 files=2 block#=21 requests=2 obj#=-1 tim=1381827769180307
WAIT #0: nam='control file parallel write' ela= 29567 files=2 block#=18 requests=2 obj#=-1 tim=1381827769209928
WAIT #0: nam='control file parallel write' ela= 27836 files=2 block#=16 requests=2 obj#=-1 tim=1381827769237804
WAIT #0: nam='control file parallel write' ela= 18501 files=2 block#=1 requests=2 obj#=-1 tim=1381827769256361
WAIT #0: nam='control file sequential read' ela= 7 file#=0 block#=1 blocks=1 obj#=-1 tim=1381827769256409
*** 2013-10-15 17:02:49.256 6875 krsu.c
Making upinblc request to NSS2 (ocis 0x2b14102ead10). Begin time is <10/15/2013 17:02:49> and NET_TIMEOUT is <30> seconds
NetServer pid:4547
WAIT #0: nam='LGWR-LNS wait on channel' ela= 4 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769256487
WAIT #0: nam='LGWR-LNS wait on channel' ela= 85 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769256586
WAIT #0: nam='LGWR wait on LNS' ela= 31 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769256600
Receiving response from NSS2
WAIT #0: nam='LGWR-LNS wait on channel' ela= 3234 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769259868
WAIT #0: nam='LGWR wait on LNS' ela= 19 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769259886
WAIT #0: nam='control file sequential read' ela= 6 file#=0 block#=721 blocks=1 obj#=-1 tim=1381827769259909
WAIT #0: nam='control file sequential read' ela= 4 file#=0 block#=739 blocks=1 obj#=-1 tim=1381827769259923
WAIT #0: nam='control file sequential read' ela= 5 file#=0 block#=811 blocks=1 obj#=-1 tim=1381827769259936
WAIT #0: nam='control file sequential read' ela= 4 file#=0 block#=1059 blocks=1 obj#=-1 tim=1381827769259948
WAIT #0: nam='control file sequential read' ela= 5 file#=0 block#=21 blocks=1 obj#=-1 tim=1381827769259968
WAIT #0: nam='control file parallel write' ela= 59464 files=2 block#=1060 requests=2 obj#=-1 tim=1381827769319464
WAIT #0: nam='control file sequential read' ela= 8 file#=0 block#=327 blocks=1 obj#=-1 tim=1381827769319519
WAIT #0: nam='control file parallel write' ela= 34180 files=2 block#=22 requests=2 obj#=-1 tim=1381827769353871
WAIT #0: nam='control file sequential read' ela= 6 file#=0 block#=19 blocks=1 obj#=-1 tim=1381827769353935
Receiving response from NSS2
WAIT #0: nam='LGWR-LNS wait on channel' ela= 3882 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769357866
WAIT #0: nam='LGWR wait on LNS' ela= 20 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769357884
*** 2013-10-15 17:02:49.357 7020 krsu.c
Making upinbls request to NSS2 (ocis 0x2b14102ead10). Begin time is <10/15/2013 17:02:49> and NET_TIMEOUT is <30> seconds
NetServer pid:4547
WAIT #0: nam='LGWR-LNS wait on channel' ela= 83 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769358005
WAIT #0: nam='LGWR wait on LNS' ela= 15 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769358019
Un-promoting destination LOG_ARCHIVE_DEST_2
WAIT #0: nam='LGWR-LNS wait on channel' ela= 447 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769358613
WAIT #0: nam='LGWR wait on LNS' ela= 18 p1=0 p2=0 p3=0 obj#=-1 tim=1381827769358630
WAIT #0: nam='control file sequential read' ela= 5 file#=0 block#=22 blocks=1 obj#=-1 tim=1381827769358650
WAIT #0: nam='log file sequential read' ela= 4 log#=0 block#=1 blocks=1 obj#=-1 tim=1381827769358668
WAIT #0: nam='log file single write' ela= 24624 log#=0 block#=1 blocks=1 obj#=-1 tim=1381827769383305
WAIT #0: nam='control file sequential read' ela= 7 file#=0 block#=281 blocks=1 obj#=-1 tim=1381827769383341
WAIT #0: nam='control file parallel write' ela= 24350 files=2 block#=22 requests=2 obj#=-1 tim=1381827769407712
WAIT #0: nam='control file sequential read' ela= 6 file#=0 block#=595 blocks=1 obj#=-1 tim=1381827769407740
WAIT #0: nam='control file sequential read' ela= 4 file#=0 block#=22 blocks=1 obj#=-1 tim=1381827769407762
WAIT #0: nam='control file parallel write' ela= 47756 files=2 block#=596 requests=2 obj#=-1 tim=1381827769455530
WAIT #0: nam='control file sequential read' ela= 6 file#=0 block#=19 blocks=1 obj#=-1 tim=1381827769455561

*** 2013-10-15 17:02:49.479
WAIT #0: nam='control file parallel write' ela= 23767 files=2 block#=20 requests=2 obj#=-1 tim=1381827769479344
WAIT #0: nam='control file parallel write' ela= 24408 files=2 block#=17 requests=2 obj#=-1 tim=1381827769503780
WAIT #0: nam='control file parallel write' ela= 33076 files=2 block#=15 requests=2 obj#=-1 tim=1381827769536877
WAIT #0: nam='control file parallel write' ela= 26021 files=2 block#=1 requests=2 obj#=-1 tim=1381827769562937
WAIT #0: nam='control file sequential read' ela= 9 file#=0 block#=1 blocks=1 obj#=-1 tim=1381827769562992
WAIT #0: nam='control file sequential read' ela= 11 file#=0 block#=31 blocks=1 obj#=-1 tim=1381827769567701