Category Archives: Oracle备份恢复

存储故障后oracle报—ORA-01122/ORA-01207故障处理

Posted on by

联系：手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题：存储故障后oracle报—ORA-01122/ORA-01207故障处理

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

客户存储异常,通过硬件恢复解决存储故障之后,oracle数据库无法正常启动(存储cache丢失),尝试recover数据库报ORA-00283 ORA-01122 ORA-01110 ORA-01207错误
以前处理过比较类似的存储故障case:
又一起存储故障导致ORA-00333 ORA-00312恢复
存储故障,强制拉库报ORA-600 kcbzib_kcrsds_1处理

SQL> recover database until cancel;
ORA-00283: 恢复会话因错误而取消
ORA-01122: 数据库文件 536 验证失败
ORA-01110: 数据文件 536: '+DATA/orcl/dt_img_dat511.ora'
ORA-01207: 文件比控制文件更新 - 旧的控制文件

Sun May 05 00:09:03 2024
ALTER DATABASE RECOVER  database until cancel  
Media Recovery Start
 started logmerger process
Sun May 05 00:09:10 2024
SUCCESS: diskgroup FRA was mounted
Sun May 05 00:09:10 2024
NOTE: dependency between database orcl and diskgroup resource ora.FRA.dg is established
Sun May 05 00:09:14 2024
WARNING! Recovering data file 1 from a fuzzy backup. It might be an online
backup taken without entering the begin backup command.
Media Recovery failed with error 1122
Slave exiting with ORA-283 exception
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_pr00_8048.trc:
ORA-00283: 恢复会话因错误而取消
ORA-01122: 数据库文件 536 验证失败
ORA-01110: 数据文件 536: '+DATA/orcl/dt_img_dat511.ora'
ORA-01207: 文件比控制文件更新 - 旧的控制文件
Sun May 05 00:09:16 2024
Recovery Slave PR00 previously exited with exception 283
ORA-283 signalled during: ALTER DATABASE RECOVER  database until cancel  ...

using backup controlfile进行恢复

SQL> recover database using backup controlfile until cancel;
ORA-00279: 更改 18646239951 (在 04/25/2024 17:14:50 生成) 对于线程 1 是必需的
ORA-00289: 建议:
+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003886.199435.1167240505
ORA-00280: 更改 18646239951 (用于线程 1) 在序列 #1003886 中


指定日志: {<RET>=suggested | filename | AUTO | CANCEL}
auto
ORA-00279: 更改 18646239951 (在 04/25/2024 17:11:40 生成) 对于线程 2 是必需的
ORA-00289: 建议:
+FRA/orcl/archivelog/2024_04_25/thread_2_seq_677876.199531.1167239807
ORA-00280: 更改 18646239951 (用于线程 2) 在序列 #677876 中


ORA-00279: 更改 18646255791 (在 04/25/2024 17:16:46 生成) 对于线程 2 是必需的
ORA-00289: 建议:
+FRA/orcl/archivelog/2024_04_25/thread_2_seq_677877.199472.1167240099
ORA-00280: 更改 18646255791 (用于线程 2) 在序列 #677877 中
ORA-00278: 此恢复不再需要日志文件
'+FRA/orcl/archivelog/2024_04_25/thread_2_seq_677876.199531.1167239807'


ORA-00279: 更改 18646295647 (在 04/25/2024 17:21:38 生成) 对于线程 2 是必需的
ORA-00289: 建议:
+FRA/orcl/archivelog/2024_04_25/thread_2_seq_677878.199379.1167240623
ORA-00280: 更改 18646295647 (用于线程 2) 在序列 #677878 中
ORA-00278: 此恢复不再需要日志文件
'+FRA/orcl/archivelog/2024_04_25/thread_2_seq_677877.199472.1167240099'


ORA-00279: 更改 18646331784 (在 04/25/2024 17:28:25 生成) 对于线程 1 是必需的
ORA-00289: 建议:
+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507
ORA-00280: 更改 18646331784 (用于线程 1) 在序列 #1003887 中
ORA-00278: 此恢复不再需要日志文件
'+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003886.199435.1167240505'


ORA-00308: 无法打开归档日志
'+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507'
ORA-17503: ksfdopn: 2 未能打开文件
+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507
ORA-15012: ASM file
'+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507' does not exist


ORA-10879: error signaled in parallel recovery slave
ORA-01547: 警告: RECOVER 成功但 OPEN RESETLOGS 将出现如下错误
ORA-01194: 文件 1 需要更多的恢复来保持一致性
ORA-01110: 数据文件 1: '+DATA/orcl/system01.dbf'

通过分析,确认由于cache丢失导致thread_1_seq_1003887这个日志丢失（而且redo已经被覆盖）
20240506-2

20240506-1
数据库无法通过正常recover的思路解决.通过屏蔽一致性,强制打开数据库,alert日志报ORA-600 2662错误

Sat May 04 17:23:00 2024
Redo thread 2 internally disabled at seq 1 (CKPT)
ARC1: Archiving disabled thread 2 sequence 1
Archived Log entry 2 added for thread 2 sequence 1 ID 0x0 dest 1:
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_ora_3684.trc  (incident=47066):
ORA-00600: ??????, ??: [2662], [4], [1466533588], [4], [1466584862], [12583040], [], [], [], [], [], []
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_ora_3684.trc:
ORA-00600: ??????, ??: [2662], [4], [1466533588], [4], [1466584862], [12583040], [], [], [], [], [], []
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_ora_3684.trc:
ORA-00600: ??????, ??: [2662], [4], [1466533588], [4], [1466584862], [12583040], [], [], [], [], [], []
Error 600 happened during db open, shutting down database
USER (ospid: 3684): terminating the instance due to error 600
Instance terminated by USER, pid = 3684
ORA-1092 signalled during: alter database open resetlogs...

通过修改数据库scn,open数据库报ORA-600 4137

Sun May 05 00:12:41 2024
replication_dependency_tracking turned off (no async multimaster replication found)
LOGSTDBY: Validating controlfile with logical metadata
LOGSTDBY: Validation complete
Completed: alter database open resetlogs
Sun May 05 00:12:56 2024
Trace dumping is performing id=[cdmp_20240505001256]
Sun May 05 00:12:56 2024
ORACLE Instance orcl1 (pid = 22) - Error 600 encountered while recovering transaction (28, 21).
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_smon_5896.trc:
ORA-00600: ??????, ??: [4137], [28.21.42965783], [0], [0], [], [], [], [], [], [], [], []

这个错误比较明显,由于28号回滚段异常导致,对异常回滚段进行处理,重建undo,数据库恢复主要工作完成

Oracle 23ai rm redo*.log恢复

Posted on by

联系：手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题：Oracle 23ai rm redo*.log恢复

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

在oracle 23ai的pdb中创建用户和表,并且插入数据(不提交),在另外一个会话中abort库,并从os层面rm删除掉redo文件,模拟数据库当前redo丢失,数据库恢复
创建用户和表并插入数据

[oracle@xifenfei ~]$ sqlplus sys/oracle@free as sysdba

SQL*Plus: Release 23.0.0.0.0 - Production on Fri May 3 15:40:55 2024
Version 23.4.0.24.05

Copyright (c) 1982, 2024, Oracle.  All rights reserved.


Connected to:
Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free
Version 23.4.0.24.05

SQL> 
SQL> 
SQL> show pdbs;

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 FREEPDB1                       READ WRITE NO

SQL> alter session set container=FREEPDB1;

Session altered.

SQL> create user xff identified by oracle;

User created.

SQL> grant dba to xff ;

Grant succeeded.

SQL> conn xff/oracle@FREEPDB1
Connected.
SQL> create table t1 as select * from dba_objects;

Table created.


SQL> insert into t1 select *from t1;

75877 rows created.

SQL> /

151754 rows created.

另外一个会话中abort库

[oracle@xifenfei ~]$ sqlplus sys/oracle@free as sysdba

SQL*Plus: Release 23.0.0.0.0 - Production on Fri May 3 15:43:30 2024
Version 23.4.0.24.05

Copyright (c) 1982, 2024, Oracle.  All rights reserved.


Connected to:
Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free
Version 23.4.0.24.05

SQL> shutdown abort;
ORACLE instance shut down.
SQL> 
SQL> 
SQL> 
SQL> exit
Disconnected from Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free
Version 23.4.0.24.05

操作系统层面rm -rf 删除redo

[oracle@xifenfei ~]$ cd $ORACLE_BASE/oradata
[oracle@xifenfei oradata]$ ls
FREE
[oracle@xifenfei oradata]$ cd FREE/
[oracle@192 FREE]$ ls
control01.ctl  FREEPDB1  redo01.log  redo03.log    system01.dbf  undotbs2.dbf
control02.ctl  pdbseed   redo02.log  sysaux01.dbf  temp01.dbf    users01.dbf
[oracle@xifenfei FREE]$ ls -ltr
total 2441036
drwxr-x---. 2 oracle     1000         85 May  1 16:49 pdbseed
-rw-r-----. 1 oracle oinstall   20979712 May  1 16:51 temp01.dbf
drwxr-x---. 2 oracle     1000        104 May  1 16:55 FREEPDB1
-rw-r-----. 1 oracle oinstall  209715712 May  3 15:23 redo01.log
-rw-r-----. 1 oracle oinstall  209715712 May  3 15:23 redo02.log
-rw-r-----. 1 oracle oinstall    7348224 May  3 15:23 users01.dbf
-rw-r-----. 1 oracle oinstall 1080041472 May  3 15:43 system01.dbf
-rw-r-----. 1 oracle oinstall  692068352 May  3 15:43 sysaux01.dbf
-rw-rw----. 1 oracle oinstall   52436992 May  3 15:43 undotbs2.dbf
-rw-r-----. 1 oracle oinstall  209715712 May  3 15:43 redo03.log
-rw-r-----. 1 oracle oinstall   18759680 May  3 15:43 control01.ctl
-rw-r-----. 1 oracle oinstall   18759680 May  3 15:43 control02.ctl
[oracle@xifenfei FREE]$ rm -rf redo0*
[oracle@192 FREE]$ ls -l redo*
ls: cannot access 'redo*': No such file or directory
[oracle@xifenfei FREE]$

尝试启动数据库，报ora-00313，ora-00312，ora-27037等错误

[oracle@xifenfei ~]$ sqlplus sys/oracle@free as sysdba

SQL*Plus: Release 23.0.0.0.0 - Production on Fri May 3 15:44:17 2024
Version 23.4.0.24.05

Copyright (c) 1982, 2024, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup mount;
ORACLE instance started.

Total System Global Area 1603726344 bytes
Fixed Size                  5360648 bytes
Variable Size             671088640 bytes
Database Buffers          922746880 bytes
Redo Buffers                4530176 bytes
Database mounted.
SQL> recover database;
ORA-00283: recovery session canceled due to errors
ORA-00313: open failed for members of log group 3 of thread 1
ORA-00312: online log 3 thread 1: '/opt/oracle/oradata/FREE/redo03.log'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 7

由于所有redo均被删除(包含当前redo),因此只能强制resetlogs方式打开库,尝试打开数据库

SQL> shutdown immediate;
ORA-01109: database not open

Database dismounted.
ORACLE instance shut down.
SQL> startup mount pfile='/tmp/pfile'
ORACLE instance started.

Total System Global Area 1603726344 bytes
Fixed Size                  5360648 bytes
Variable Size             671088640 bytes
Database Buffers          922746880 bytes
Redo Buffers                4530176 bytes
Database mounted.
SQL> recover database until cancel;
ORA-00279: change 4244588 generated at 05/03/2024 15:23:22 needed for thread 1
ORA-00289: suggestion :
/opt/oracle/product/23ai/dbhomeFree/dbs/arch1_6_1167842962.dbf
ORA-00280: change 4244588 for thread 1 is in sequence #6


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
cancel
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1: '/opt/oracle/oradata/FREE/system01.dbf'


ORA-01112: media recovery not started

SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-00603: ORACLE server session terminated by irrecoverable error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [],
[], [], [], [], [], [], []
Process ID: 5596
Session ID: 29 Serial number: 63204

数据库遇到常见的ORA-600 kcbzib_kcrsds_1错误,这类错误类似oracle在12c之前版本中的ORA-600 2662错误
ORA-600 kcbzib_kcrsds_1报错
存储故障,强制拉库报ORA-600 kcbzib_kcrsds_1处理
redo异常强制拉库报ORA-600 kcbzib_kcrsds_1修复
ORA-00603 ORA-01092 ORA-600 kcbzib_kcrsds_1
这种一般就是scn问题,通过修改数据库scn,数据库启动报ORA-16433错误

[oracle@xifenfei ~]$ sqlplus sys/oracle@free as sysdba

SQL*Plus: Release 23.0.0.0.0 - Production on Fri May 3 15:49:00 2024
Version 23.4.0.24.05

Copyright (c) 1982, 2024, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup nomount pfile='/tmp/pfile';
ORACLE instance started.

Total System Global Area 1603726344 bytes
Fixed Size                  5360648 bytes
Variable Size             671088640 bytes
Database Buffers          922746880 bytes
Redo Buffers                4530176 bytes
SQL> alter database mount;

Database altered.

SQL> recover database;
ORA-00283: recovery session canceled due to errors
ORA-16433: The database or pluggable database must be opened in read/write mode.


SQL> shutdown abort;
ORACLE instance shut down.

处理ctl,open数据库成功

SQL> startup nomount pfile='/tmp/pfile'
ORACLE instance started.

Total System Global Area 1603726344 bytes
Fixed Size                  5360648 bytes
Variable Size             671088640 bytes
Database Buffers          922746880 bytes
Redo Buffers                4530176 bytes
SQL> CREATE CONTROLFILE REUSE DATABASE "FREE" NORESETLOGS  NOARCHIVELOG
  2      MAXLOGFILES 16
  3      MAXLOGMEMBERS 3
  4      MAXDATAFILES 1024
  5      MAXINSTANCES 8
  6      MAXLOGHISTORY 292
  7  LOGFILE
  8    GROUP 1 '/opt/oracle/oradata/FREE/redo01.log'  SIZE 200M BLOCKSIZE 512,
  9    GROUP 2 '/opt/oracle/oradata/FREE/redo02.log'  SIZE 200M BLOCKSIZE 512,
 10    GROUP 3 '/opt/oracle/oradata/FREE/redo03.log'  SIZE 200M BLOCKSIZE 512
 11  -- STANDBY LOGFILE
 12  DATAFILE
 13    '/opt/oracle/oradata/FREE/system01.dbf',
 14    '/opt/oracle/oradata/FREE/pdbseed/system01.dbf',
 15    '/opt/oracle/oradata/FREE/sysaux01.dbf',
 16    '/opt/oracle/oradata/FREE/pdbseed/sysaux01.dbf',
 17    '/opt/oracle/oradata/FREE/users01.dbf',
 18    '/opt/oracle/oradata/FREE/pdbseed/undotbs01.dbf',
 19    '/opt/oracle/oradata/FREE/FREEPDB1/system01.dbf',
 20    '/opt/oracle/oradata/FREE/FREEPDB1/sysaux01.dbf',
 21    '/opt/oracle/oradata/FREE/FREEPDB1/undotbs01.dbf',
 22    '/opt/oracle/oradata/FREE/FREEPDB1/users01.dbf',
 23    '/opt/oracle/oradata/FREE/undotbs2.dbf'
 24  CHARACTER SET AL32UTF8
 25  ;

Control file created.

SQL> recover database;
Media recovery complete.
SQL> alter database open;

Database altered.

SQL> show pdbs;

          CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------------- ------------------------------ ---------- ----------
               2 PDB$SEED                       READ ONLY  NO
               3 FREEPDB1                       READ WRITE NO

至此当前数据库redo故障模拟和恢复基本完成

ORA-00742 ORA-00312 恢复

Posted on by

联系：手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题：ORA-00742 ORA-00312 恢复

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有客户反馈,断电之后数据库启动报ORA-00742和ORA-00312,无法正常open
ORA-742-ORA-312

我们远程上去尝试open库结果也报同样错误

[oracle@oldhis oradata]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Wed Apr 10 09:40:03 2024

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> recover database;
Media recovery complete.

SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-00742: Log read detects lost write in thread %d sequence %d block %d
ORA-00312: online log 3 thread 1: '/oradata/shrdh/redo03.log'


SQL> select group#,status from v$log;

    GROUP# STATUS
---------- ----------------
         1 INACTIVE
         3 CURRENT
         2 INACTIVE

因为recover已经成功,但是依旧报ORA-742错误,尝试查询scn相关信息

SQL> set pages 10000
set numw 16
SELECT status,
checkpoint_change#,
checkpoint_time,last_change#,
count(*) ROW_NUM
FROM v$datafile
GROUP BY status, checkpoint_change#, checkpoint_time,last_change#
ORDER BY status, checkpoint_change#, checkpoint_time;


set numw 16
col CHECKPOINT_TIME for a40
set lines 150
set pages 1000
SELECT status,
to_char(checkpoint_time,'yyyy-mm-dd hh24:mi:ss') checkpoint_time,FUZZY,checkpoint_change#,
count(*) ROW_NUM
FROM v$datafile_header
GROUP BY status, checkpoint_change#, to_char(checkpoint_time,'yyyy-mm-dd hh24:mi:ss'),fuzzy
ORDER BY status, checkpoint_change#, checkpoint_time;

SQL> SQL>   2    3    4    5    6    7  
STATUS  CHECKPOINT_CHANGE# CHECKPOIN     LAST_CHANGE#          ROW_NUM
------- ------------------ --------- ---------------- ----------------
ONLINE          1279351848 26-MAR-24       1279351848               19
SYSTEM          1279351848 26-MAR-24       1279351848                1

SQL> SQL> SQL> SQL> SQL> SQL> SQL>   2    3    4    5    6  
STATUS  CHECKPOINT_TIME                          FUZ CHECKPOINT_CHANGE#          ROW_NUM
------- ---------------------------------------- --- ------------------ ----------------
ONLINE  2024-03-26 00:05:45                      NO          1279351848               20

基于这样的情况,我们判断数据库直接open成功

SQL> recover database using backup controlfile;
ORA-00279: change 1279351848 generated at 03/26/2024 00:05:45 needed for thread 1
ORA-00289: suggestion : /oradata/arch/shrdh/shrdh_1_12984_974767526.arc
ORA-00280: change 1279351848 for thread 1 is in sequence #12984


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
/oradata/shrdh/redo03.log
Log applied.
Media recovery complete.
SQL> alter database open resetlogs;

Database altered.

后面比较不幸，数据库报ORA-600 4194错误导致数据库异常

Wed Apr 10 09:43:08 2024
ALTER DATABASE RECOVER  database using backup controlfile  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 4 slaves
ORA-279 signalled during: ALTER DATABASE RECOVER  database using backup controlfile  ...
Wed Apr 10 09:43:24 2024
ALTER DATABASE RECOVER    LOGFILE '/oradata/shrdh/redo03.log'  
Media Recovery Log /oradata/shrdh/redo03.log
Media Recovery Complete (shrdh)
Completed: ALTER DATABASE RECOVER    LOGFILE '/oradata/shrdh/redo03.log'  
alter database open resetlogs
RESETLOGS after complete recovery through change 1279351849
Clearing online redo logfile 1 /oradata/shrdh/redo01.log
Clearing online log 1 of thread 1 sequence number 12982
Clearing online redo logfile 1 complete
Clearing online redo logfile 2 /oradata/shrdh/redo02.log
Clearing online log 2 of thread 1 sequence number 12983
Clearing online redo logfile 2 complete
Clearing online redo logfile 3 /oradata/shrdh/redo03.log
Clearing online log 3 of thread 1 sequence number 12984
Clearing online redo logfile 3 complete
Resetting resetlogs activation ID 1820377766 (0x6c80c2a6)
Online log /oradata/shrdh/redo01.log: Thread 1 Group 1 was previously cleared
Online log /oradata/shrdh/redo02.log: Thread 1 Group 2 was previously cleared
Online log /oradata/shrdh/redo03.log: Thread 1 Group 3 was previously cleared
Wed Apr 10 09:43:34 2024
Setting recovery target incarnation to 2
Wed Apr 10 09:43:34 2024
Assigning activation ID 2011515185 (0x77e54931)
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: /oradata/shrdh/redo01.log
Successful open of redo thread 1
Wed Apr 10 09:43:34 2024
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Apr 10 09:43:34 2024
SMON: enabling cache recovery
[25089] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:1273646224 end:1273646494 diff:270 (2 seconds)
Dictionary check beginning
Dictionary check complete
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is ZHS16GBK
No Resource Manager plan active
replication_dependency_tracking turned off (no async multimaster replication found)
Errors in file /u01/app/oracle/diag/rdbms/shrdh/shrdh/trace/shrdh_smon_21704.trc  (incident=84296):
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/shrdh/shrdh/incident/incdir_84296/shrdh_smon_21704_i84296.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Starting background process QMNC
Wed Apr 10 09:43:35 2024
QMNC started with pid=24, OS id=25340 
LOGSTDBY: Validating controlfile with logical metadata
LOGSTDBY: Validation complete
Block recovery from logseq 1, block 61 to scn 1279351933
Recovery of Online Redo Log: Thread 1 Group 1 Seq 1 Reading mem 0
  Mem# 0: /oradata/shrdh/redo01.log
Block recovery stopped at EOT rba 1.99.16
Block recovery completed at rba 1.99.16, scn 0.1279351933
Block recovery from logseq 1, block 61 to scn 1279351919
Recovery of Online Redo Log: Thread 1 Group 1 Seq 1 Reading mem 0
  Mem# 0: /oradata/shrdh/redo01.log
Block recovery completed at rba 1.87.16, scn 0.1279351922
Errors in file /u01/app/oracle/diag/rdbms/shrdh/shrdh/trace/shrdh_smon_21704.trc:
ORA-01595: error freeing extent (2) of rollback segment (7))
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [], [], [], [], []
Completed: alter database open resetlogs
Wed Apr 10 09:43:37 2024
Errors in file /u01/app/oracle/diag/rdbms/shrdh/shrdh/trace/shrdh_m000_25343.trc  (incident=84392):
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/shrdh/shrdh/incident/incdir_84392/shrdh_m000_25343_i84392.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Wed Apr 10 09:43:37 2024
Starting background process CJQ0
Wed Apr 10 09:43:37 2024
CJQ0 started with pid=29, OS id=25357 
Starting background process SMCO
Wed Apr 10 09:43:37 2024
SMCO started with pid=30, OS id=25360 
Wed Apr 10 09:43:38 2024
Flush retried for xcb 0x115b42d28, pmd 0x1148dea70
Block recovery from logseq 1, block 61 to scn 1279351933
Recovery of Online Redo Log: Thread 1 Group 1 Seq 1 Reading mem 0
  Mem# 0: /oradata/shrdh/redo01.log
Block recovery completed at rba 1.99.16, scn 0.1279351934
Errors in file /u01/app/oracle/diag/rdbms/shrdh/shrdh/trace/shrdh_pmon_21679.trc  (incident=84208):
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/shrdh/shrdh/incident/incdir_84208/shrdh_pmon_21679_i84208.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /u01/app/oracle/diag/rdbms/shrdh/shrdh/trace/shrdh_pmon_21679.trc:
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [], [], [], [], []
PMON (ospid: 21679): terminating the instance due to error 472
Wed Apr 10 09:43:47 2024
Instance terminated by PMON, pid = 21679

报错比较明显,对undo进行处理即可.

数据库open成功后报ORA-00353 ORA-00354错误引起的一系列问题(本质ntfs文件系统异常)

Posted on by

联系：手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题：数据库open成功后报ORA-00353 ORA-00354错误引起的一系列问题(本质ntfs文件系统异常)

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

服务器异常断电之后，开机启动数据库启动成功，但是报ORA-00353 ORA-00354以及ORA-600 kdsgrp1错误

Sun Mar 31 01:19:51 2024
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Sun Mar 31 01:19:51 2024
SMON: enabling cache recovery
[4528] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:64250 end:64859 diff:609 (6 seconds)
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is ZHS16GBK
No Resource Manager plan active
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
Sun Mar 31 01:19:57 2024
QMNC started with pid=40, OS id=4912 
Sun Mar 31 01:20:03 2024
Completed: alter database open
Sun Mar 31 01:20:04 2024
Starting background process CJQ0
Sun Mar 31 01:20:04 2024
CJQ0 started with pid=20, OS id=5104 
Setting Resource Manager plan SCHEDULER[0x32DE]:DEFAULT_MAINTENANCE_PLAN via scheduler window
Setting Resource Manager plan DEFAULT_MAINTENANCE_PLAN via parameter
Sun Mar 31 01:20:07 2024
Starting background process VKRM
Sun Mar 31 01:20:07 2024
VKRM started with pid=48, OS id=4160 
Sun Mar 31 01:20:32 2024
Incomplete read from log member 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'. Trying next member.
Incomplete read from log member 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'. Trying next member.
Errors in file E:\APP\ADMINISTRATOR\diag\rdbms\xifenfei\xifenfei\trace\xifenfei_j000_4180.trc(incident=132393):
ORA-00353: 日志损坏接近块 13792 更改 33686946 时间 03/31/2024 00:17:54
ORA-00334: 归档日志: 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'
Incomplete read from log member 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'. Trying next member.
Incomplete read from log member 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'. Trying next member.
Errors in file E:\APP\ADMINISTRATOR\diag\rdbms\xifenfei\xifenfei\trace\xifenfei_j000_4180.trc(incident=132394):
ORA-00353: 日志损坏接近块 13792 更改 33686946 时间 03/31/2024 00:17:54
ORA-00334: 归档日志: 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'
ORA-00353: 日志损坏接近块 13792 更改 33686946 时间 03/31/2024 00:17:54
ORA-00334: 归档日志: 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'
Errors in file E:\APP\ADMINISTRATOR\diag\rdbms\xifenfei\xifenfei\incident\incdir_132393\xifenfei_j000_4180_i132393.trc:
ORA-00354: 损坏重做日志块标头
ORA-00353: 日志损坏接近块 13792 更改 33686946 时间 03/31/2024 00:17:54
ORA-00334: 归档日志: 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'
ORA-00353: 日志损坏接近块 13792 更改 33686946 时间 03/31/2024 00:17:54
ORA-00334: 归档日志: 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'
Sun Mar 31 01:20:35 2024
Sweep [inc][132394]: completed
Errors in file E:\APP\ADMINISTRATOR\diag\rdbms\xifenfei\xifenfei\trace\xifenfei_j000_4180.trc:
ORA-00354: 损坏重做日志块标头
ORA-00353: 日志损坏接近块 13792 更改 33686946 时间 03/31/2024 00:17:54
ORA-00334: 归档日志: 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'
Errors in file E:\APP\ADMINISTRATOR\diag\rdbms\xifenfei\xifenfei\trace\xifenfei_j000_4180.trc(incident=132395):
ORA-00600: 内部错误代码, 参数: [kdsgrp1], [], [], [], [], [], [], [], [], [], [], []
Incomplete read from log member 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'. Trying next member.
Incomplete read from log member 'E:\APP\ARCHIVELOG\ARC0000005281_1157968160.0001'. Trying next member.
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.

再次重启数据库报ORA-600 2131错误,数据库无法正常mount

Sun Mar 31 08:59:20 2024
alter database mount exclusive
Errors in file E:\APP\ADMINISTRATOR\diag\rdbms\xifenfei\xifenfei\trace\xifenfei_ora_3232.trc  (incident=144175):
ORA-00600: ??????, ??: [2131], [9], [8], [], [], [], [], [], [], [], [], []
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
ORA-600 signalled during: alter database mount exclusive...

因为这个库有ctl备份,通过rman还原ctl备份,然后尝试recover库,结果报ORA-00310 ORA-00334（由于需要的redo无法正常应用导致）
20240403223225

对于这类情况,只能通过屏蔽一致性强制打开库
20240403223432
数据报ORA-600 2662错误,此类错误比较简单,使用patch scn工具一键搞定(ORA-600 2662快速恢复之Patch scn工具),数据库open成功,导出数据完整恢复
对于正常open的库,出现此类问题属于反常现象,通过分析系统事件确定是由于ntfs文件系统本身有问题导致
20240403223855

ORA-600 ktsiseginfo1故障

Posted on by

联系：手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题：ORA-600 ktsiseginfo1故障

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

oracle 9i的库在运行途中突然报ORA-600 kcbnew_3错误

Sun Mar 31 14:25:11 2024
Undo Segment 69 Onlined
Sun Mar 31 14:25:11 2024
Created Undo Segment _SYSSMU69$
Sun Mar 31 14:25:11 2024
Created Undo Segment _SYSSMU70$
Undo Segment 70 Onlined
Sun Mar 31 14:28:41 2024
Starting control autobackup
Control autobackup written to SBT_TAPE device
	comment 'API Version 2.0,MMS Version 1.2.0.0',
	media 'A90063L4'
	handle 'c-1757743655-20240331-01'
Sun Mar 31 14:31:07 2024
Starting control autobackup
Control autobackup written to SBT_TAPE device
	comment 'API Version 2.0,MMS Version 1.2.0.0',
	media 'A90063L4'
	handle 'c-1757743655-20240331-02'
Sun Mar 31 14:34:31 2024
SMON offlining US=70
Sun Mar 31 14:34:31 2024
Created Undo Segment _SYSSMU73$
Undo Segment 73 Onlined
Sun Mar 31 14:34:31 2024
Created Undo Segment _SYSSMU74$
Undo Segment 74 Onlined
Sun Mar 31 14:34:31 2024
Created Undo Segment _SYSSMU81$
Undo Segment 81 Onlined
Sun Mar 31 14:37:36 2024
Errors in file /u2/oradb/admin/xifenfei/udump/xifenfei_ora_3741.trc:
ORA-00600: 内部错误代码，参数: [kcbnew_3], [156], [], [], [], [], [], []
Sun Mar 31 14:39:22 2024
Errors in file /u2/oradb/admin/xifenfei/bdump/xifenfei_smon_6890.trc:
ORA-01595: error freeing extent (6) of rollback segment (70))
ORA-00600: internal error code, arguments: [kcbnew_3], [156], [], [], [], [], [], []
Sun Mar 31 14:45:05 2024
Errors in file /u2/oradb/admin/xifenfei/udump/xifenfei_ora_4046.trc:
ORA-00600: 内部错误代码，参数: [kcbnew_3], [156], [], [], [], [], [], []
Sun Mar 31 14:47:47 2024
Failure to extend rollback segment 73 because of 3113 condition
FULL status of rollback segment 73 set.
Sun Mar 31 14:47:47 2024
FULL status of rollback segment 73 cleared.
Sun Mar 31 14:48:03 2024
Errors in file /u2/oradb/admin/xifenfei/udump/xifenfei_ora_4074.trc:
ORA-00600: 内部错误代码，参数: [kcbnew_3], [156], [], [], [], [], [], []
Sun Mar 31 14:48:35 2024
Errors in file /u2/oradb/admin/xifenfei/bdump/xifenfei_qmn0_4044.trc:
ORA-00600: internal error code, arguments: [kcbnew_3], [156], [], [], [], [], [], []
Sun Mar 31 14:50:01 2024
Failure to extend rollback segment 73 because of 3113 condition
FULL status of rollback segment 73 set.
Sun Mar 31 14:50:02 2024
FULL status of rollback segment 73 cleared.
Sun Mar 31 14:50:41 2024
Restarting dead background process QMN0
QMN0 started with pid=53
Sun Mar 31 14:51:09 2024
Failure to extend rollback segment 73 because of 3113 condition
FULL status of rollback segment 73 set.
Sun Mar 31 14:51:09 2024
FULL status of rollback segment 73 cleared.
Sun Mar 31 14:54:36 2024
Failure to extend rollback segment 73 because of 3113 condition
FULL status of rollback segment 73 set.
Sun Mar 31 14:54:36 2024
FULL status of rollback segment 73 cleared.
Sun Mar 31 14:54:56 2024
Errors in file /u2/oradb/admin/xifenfei/bdump/xifenfei_smon_6890.trc:
ORA-01595: error freeing extent (6) of rollback segment (70))
ORA-00600: internal error code, arguments: [kcbnew_3], [156], [], [], [], [], [], []

从上述信息看,由于ora-600 kcbnew_3错误导致70/73号回滚段异常,smon无法正常对其进行扩展,关于ora-600 kcbnew_3报错描述
20240331201941

客户尝试关闭数据库重启,数据库报ora-600 ktsiseginfo1错误，无法正常启动

Sun Mar 31 15:13:03 2024
Shutting down instance: further logons disabled
Sun Mar 31 15:15:59 2024
Shutting down instance (immediate)
Sun Mar 31 15:15:59 2024
Failure to extend rollback segment 73 because of 1089 condition
Sun Mar 31 15:15:59 2024
FULL status of rollback segment 73 set.
Sun Mar 31 15:15:59 2024
Failure to extend rollback segment 73 because of 1089 condition
FULL status of rollback segment 73 set.
Sun Mar 31 15:15:59 2024
Failure to extend rollback segment 73 because of 1089 condition
Sun Mar 31 15:15:59 2024
Failure to extend rollback segment 73 because of 1089 condition
Sun Mar 31 15:15:59 2024
Failure to extend rollback segment 73 because of 1089 condition
Sun Mar 31 15:15:59 2024
Failure to extend rollback segment 73 because of 1089 condition
Sun Mar 31 15:15:59 2024
FULL status of rollback segment 73 cleared.
Sun Mar 31 15:15:59 2024
Failure to extend rollback segment 73 because of 1089 condition
Sun Mar 31 15:15:59 2024
FULL status of rollback segment 73 set.
Sun Mar 31 15:15:59 2024
Failure to extend rollback segment 73 because of 1089 condition
Sun Mar 31 15:15:59 2024
FULL status of rollback segment 73 cleared.
Sun Mar 31 15:15:59 2024
FULL status of rollback segment 73 cleared.
Sun Mar 31 15:15:59 2024
License high water mark = 161
Sun Mar 31 15:21:04 2024
Active call for process 2878 user 'oracle' program 'oracle@ttperp (TNS V1-V3)'
SHUTDOWN: waiting for active calls to complete.
Sun Mar 31 15:21:07 2024
Failure to extend rollback segment 73 because of 1089 condition
FULL status of rollback segment 73 set.
Sun Mar 31 15:21:07 2024
SMON: FULL status of rollback segment 73 cleared.
Sun Mar 31 15:21:08 2024
ALTER DATABASE CLOSE NORMAL
SMON: disabling tx recovery
SMON: disabling cache recovery
Sun Mar 31 15:21:16 2024
Shutting down archive processes
Archiving is disabled
Sun Mar 31 15:21:16 2024
ARCH shutting down
Sun Mar 31 15:21:16 2024
ARCH shutting down
Sun Mar 31 15:21:16 2024
ARC1: Archival stopped
Sun Mar 31 15:21:16 2024
ARC0: Archival stopped
Sun Mar 31 15:21:17 2024
Thread 1 closed at log sequence 313828
Successful close of redo thread 1.
Sun Mar 31 15:21:18 2024
Completed: ALTER DATABASE CLOSE NORMAL
Sun Mar 31 15:21:18 2024
ALTER DATABASE DISMOUNT
Completed: ALTER DATABASE DISMOUNT
ARCH: Archiving is disabled
Shutting down archive processes
Archiving is disabled
Archive process shutdown avoided: 0 active
ARCH: Archiving is disabled
Shutting down archive processes
Archiving is disabled
Archive process shutdown avoided: 0 active
Sun Mar 31 15:23:10 2024
Starting ORACLE instance (normal)
Sun Mar 31 15:23:10 2024
Running with 1 strand for Non-Enterprise Edition
Sun Mar 31 15:23:10 2024
WARNING: EINVAL creating segment of size 0x0000000065400000
fix shm parameters in /etc/system or equivalent
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
SCN scheme 2
Running with 1 strand for Non-Enterprise Edition
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up ORACLE RDBMS Version: 9.2.0.4.0.
System parameters with non-default values:
  processes                = 1000
  timed_statistics         = TRUE
  shared_pool_size         = 218103808
  sga_max_size             = 1679366944
  large_pool_size          = 67108864
  java_pool_size           = 33554432
  control_files            = /u2/oradb/oradata/xifenfei/control01.ctl
  db_block_size            = 8192
  db_cache_size            = 838860800
  compatible               = 9.2.0.0.0
  log_archive_start        = TRUE
  log_archive_dest         = /u6/archive_u6
  log_archive_format       = %s_%t.dbf
  db_file_multiblock_read_count= 16
  fast_start_mttr_target   = 0
  undo_management          = AUTO
  undo_tablespace          = UNDOTBS7
  undo_retention           = 10800
  remote_login_passwordfile= EXCLUSIVE
  db_domain                = 
  instance_name            = xifenfei
  utl_file_dir             = /u1/usr/oracle/utl
  job_queue_processes      = 10
  hash_join_enabled        = TRUE
  background_dump_dest     = /u2/oradb/admin/xifenfei/bdump
  user_dump_dest           = /u2/oradb/admin/xifenfei/udump
  core_dump_dest           = /u2/oradb/admin/xifenfei/cdump
  sort_area_size           = 104857600
  db_name                  = xifenfei
  open_cursors             = 300
  star_transformation_enabled= FALSE
  query_rewrite_enabled    = FALSE
  pga_aggregate_target     = 209715200
  aq_tm_processes          = 1
PMON started with pid=2
DBW0 started with pid=3
LGWR started with pid=4
CKPT started with pid=5
SMON started with pid=6
RECO started with pid=7
CJQ0 started with pid=8
QMN0 started with pid=9
Sun Mar 31 15:23:11 2024
ARCH: STARTING ARCH PROCESSES
ARC0 started with pid=10
ARC0: Archival started
ARC1 started with pid=11
Sun Mar 31 15:23:11 2024
ARCH: STARTING ARCH PROCESSES COMPLETE
Oracle Data Guard is not available in this edition of Oracle.
Sun Mar 31 15:23:11 2024
ARC1: Archival started
Sun Mar 31 15:23:11 2024
ARC1: Thread not mounted
Sun Mar 31 15:23:11 2024
ARC0: Thread not mounted
Sun Mar 31 15:23:11 2024
ALTER DATABASE   MOUNT
Sun Mar 31 15:23:15 2024
Successful mount of redo thread 1, with mount id 2392685535.
Sun Mar 31 15:23:15 2024
Database mounted in Exclusive Mode.
Completed: ALTER DATABASE   MOUNT
Sun Mar 31 15:23:15 2024
ALTER DATABASE OPEN
Sun Mar 31 15:23:15 2024
LGWR: Primary database is in CLUSTER CONSISTENT mode
Thread 1 opened at log sequence 313828
  Current log# 5 seq# 313828 mem# 0: /u3/oracle/xifenfei/redolog/redo05a.log
  Current log# 5 seq# 313828 mem# 1: /u4/oracle/xifenfei/redolog/redo05b.log
Successful open of redo thread 1.
Sun Mar 31 15:23:17 2024
SMON: enabling cache recovery
Sun Mar 31 15:23:17 2024
Undo Segment 69 Onlined
Undo Segment 70 Onlined
Undo Segment 71 Onlined
Undo Segment 72 Onlined
Sun Mar 31 15:23:21 2024
Errors in file /u2/oradb/admin/xifenfei/udump/xifenfei_ora_4384.trc:
ORA-00600: 内部错误代码，参数: [ktsiseginfo1], [20], [65], [73], [], [], [], []
Sun Mar 31 15:23:22 2024
Errors in file /u2/oradb/admin/xifenfei/udump/xifenfei_ora_4384.trc:
ORA-00600: 内部错误代码，参数: [ktsiseginfo1], [20], [65], [73], [], [], [], []
Error 600 happened during db open, shutting down database
USER: terminating instance due to error 600
Instance terminated by USER, pid = 4384
ORA-1092 signalled during: ALTER DATABASE OPEN...

启动的时候由于回滚段异常导致该问题,临时解决问题可以通过规避掉异常回滚段open库,但是这个问题的本质是由于seg$和undo$中记录不匹配导致,建议进一步通过hcheck进行检查分析,并进行后续处理,参考mos相关描述:
20240331201653