服务器断电一起的一例ORA-01207故障处理

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:服务器断电一起的一例ORA-01207故障处理

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有一个客户反馈服务器异常断电,导致数据库无法正常启动(启动报ORA-01207: file is more recent than control file – old control file 错误)

SQL> startup mount
ORACLE 例程已经启动。

Total System Global Area 5.4784E+10 bytes
Fixed Size                  2188768 bytes
Variable Size            4.2010E+10 bytes
Database Buffers         1.2751E+10 bytes
Redo Buffers               20566016 bytes
数据库装载完毕。
SQL> recover database
ORA-00283: recovery session canceled due to errors
ORA-01122: database file 6 failed verification check
ORA-01110: data file 6: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\HSHIS1'
ORA-01207: file is more recent than control file - old control file

尝试拷贝alert日志到本地进行分析,发现无法正常拷贝
alert-error


怀疑文件系统或者底层磁盘有损坏,进一步检查系统日志,确认发现大量ntfs层面异常提示(而且多个盘都有)
ntfs-err

对于这样的情况,先对数据文件进行了备份,运气还不错,文件全部备份成功,并通过Oracle Database Recovery Check 脚本进行检查当前故障库状态,确认很多文件的checkpoint点相差很远
11

对于这样的一个非归档数据库,肯定无法正常打开,只能通过屏蔽一致性尝试强制打开库,结果在打开过程中出现ORA-600 4194错误

SQL> alter database open ;
alter database open 
*
第 1 行出现错误:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [],
[], [], [], []
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [],
[], [], [], []
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [],
[], [], [], []
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [],
[], [], [], []
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [],
[], [], [], []
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [],
[], [], [], []
进程 ID: 7820
会话 ID: 177 序列号: 147

解决异常undo,数据库正常open成功,但是在数据库导出数据过程中报ORA-600 2662和ORA-600 2663错误(这种情况一般比较少见,大部分ORA-600 2662/2663都是在open过程中,这种库已经open成功,后续查询报该错误的比较少见,但是原理上有可能[启动的时候没有读到这些block,后面放到这些表的block scn大于文件头scn,所以报该错误])

Fri Sep 05 01:04:12 2025
QMNC started with pid=54, OS id=7708 
Completed: alter database open
Fri Sep 05 01:04:13 2025
Errors in file d:\app\administrator\diag\rdbms\his\his\trace\his_m001_8624.trc  (incident=10058):
ORA-00600: internal error code, arguments: [2662], [2], [1384717872], [2], [1385084764], [41943042],
Incident details in: d:\app\administrator\diag\rdbms\his\his\incident\incdir_10058\his_m001_8624_i10058.trc
Fri Sep 05 01:04:13 2025
Starting background process CJQ0
Fri Sep 05 01:04:13 2025
CJQ0 started with pid=62, OS id=6852 
Fri Sep 05 01:04:13 2025
db_recovery_file_dest_size of 3912 MB is 0.00% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Errors in file d:\app\administrator\diag\rdbms\his\his\trace\his_ora_9220.trc  (incident=10084):
ORA-00600: 内部错误代码, 参数: [2662], [2], [1384717989], [2], [1385446402], [37802453], [], [], [],
Fri Sep 05 01:04:18 2025
Errors in file d:\app\administrator\diag\rdbms\his\his\trace\his_j017_11248.trc  (incident=10059):
ORA-00600: internal error code, arguments: [2662], [2], [1384717990], [2], [1385446402], [37802453],
Errors in file d:\app\administrator\diag\rdbms\his\his\trace\his_j017_11248.trc:
ORA-00600: internal error code, arguments: [2662], [2], [1384717990], [2], [1385446402], [37802453],
ORA-06512: at "HOSPITAL.RPT_CASE", line 5684
ORA-06512: at line 3
Errors in file d:\app\administrator\diag\rdbms\his\his\trace\his_ora_9220.trc  (incident=10085):
ORA-00600: 内部错误代码, 参数: [2662], [2], [1384717990], [2], [1385446402], [37802453], [], [], [], 

这个问题相对比较简单,通过Patch SCN小工具修改正确的Oracle SCN即可
Patch_SCN09


RMAN SBT_TAPE备份通过小程序修改实现直接DISK通道还原

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:RMAN SBT_TAPE备份通过小程序修改实现直接DISK通道还原

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有朋友把oracle rman备份从cv带库里拷贝到文件系统,然后希望把他还原到数据库中.通过尝试发现直接带库的文件无法直接在文件系统中被rman调用还原

C:\Users\XFF>sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on 星期三 9月 3 18:07:58 2025

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

已连接到空闲例程。

SQL> startup nomount pfile='d:/pfile.txt'
ORACLE 例程已经启动。

Total System Global Area 4275781632 bytes
Fixed Size                  2288080 bytes
Variable Size             939525680 bytes
Database Buffers         3321888768 bytes
Redo Buffers               12079104 bytes
SQL> exit
从 Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options 断开

C:\Users\XFF>
C:\Users\XFF>
C:\Users\XFF>
C:\Users\XFF>rman target /

恢复管理器: Release 11.2.0.4.0 - Production on 星期三 9月 3 18:08:38 2025

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

已连接到目标数据库: ORCL (未装载)

RMAN> restore controlfile from 'H:\TEMP\Oracle_Restore\405\c-1737595250-20250822-00';

启动 restore 于 03-9月 -25
使用目标数据库控制文件替代恢复目录
分配的通道: ORA_DISK_1
通道 ORA_DISK_1: SID=28 设备类型=DISK

通道 ORA_DISK_1: 正在还原控制文件
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: restore 命令 (在 09/03/2025 18:08:54 上) 失败
ORA-19870: 还原备份片段 H:\TEMP\ORACLE_RESTORE\405\C-1737595250-20250822-00 时出错
ORA-19505: 无法识别文件"H:\TEMP\ORACLE_RESTORE\405\C-1737595250-20250822-00"
ORA-27046: 文件大小不是逻辑块大小的倍数
OSD-04000: 逻辑块大小不匹配 (OS 512)

以前也写过相关的文章,由于tape的备份和disk通道备份的文件头本身格式都不一样,具体参考:RMAN SBT_TAPE备份无法被DISK通道识别
尝试使用dbms_backup_restore还原依旧失败

C:\Users\XFF>sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on 星期三 9月 3 18:19:23 2025

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


连接到:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> DECLARE
  2   devtype varchar2(256);
  3   done boolean;
  4   BEGIN
  5   devtype:=sys.dbms_backup_restore.deviceAllocate (type=>'',ident=>'t1');
  6   sys.dbms_backup_restore.restoreSetDatafile;
  7   sys.dbms_backup_restore.restoreControlfileTo(cfname=>'H:\CONTROL01.CTL');
  8   sys.dbms_backup_restore.restoreBackupPiece(done=>done,
  9     handle=>'H:\TEMP\Oracle_Restore\405\c-1737595250-20250822-00', params=>null);
 10  sys.dbms_backup_restore.deviceDeallocate;
 11  END;
 12  /
DECLARE
*
第 1 行出现错误:
ORA-19624: 操作失败, 如果可能请重试
ORA-19870: 还原备份片段 H:\TEMP\ORACLE_RESTORE\405\C-1737595250-20250822-00
时出错
ORA-19505: 无法识别文件"H:\TEMP\ORACLE_RESTORE\405\C-1737595250-20250822-00"
ORA-27046: 文件大小不是逻辑块大小的倍数
OSD-04000: 逻辑块大小不匹配 (OS 512)
ORA-06512: 在 "SYS.X$DBMS_BACKUP_RESTORE", line 5940
ORA-06512: 在 line 8

对于这种情况,我尝试自己开发一个小程序,来模拟disk通道备份集,欺骗数据库来实现还原操作.
QQ20250903-182124


通过程序模拟出来的disk通道的文件,每个后缀加上了disk
QQ20250903-182136

再次通过尝试通过disk通道还原操作

C:\Users\XFF>rman target /

恢复管理器: Release 11.2.0.4.0 - Production on 星期三 9月 3 18:23:14 2025

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

已连接到目标数据库: ORCL (未装载)

RMAN> restore controlfile from 'H:\TEMP\Oracle_Restore\405\c-1737595250-20250822-00.disk';

启动 restore 于 03-9月 -25
使用目标数据库控制文件替代恢复目录
分配的通道: ORA_DISK_1
通道 ORA_DISK_1: SID=28 设备类型=DISK

通道 ORA_DISK_1: 正在还原控制文件
通道 ORA_DISK_1: 还原完成, 用时: 00:00:01
输出文件名=H:\CONTROL01.CTL
完成 restore 于 03-9月 -25

RMAN> alter database mount;

数据库已装载
释放的通道: ORA_DISK_1

RMAN> catalog start with 'H:\TEMP\Oracle_Restore\405\*.disk';

搜索与样式 H:\TEMP\Oracle_Restore\405\*.disk 匹配的所有文件

数据库未知文件的列表
=====================================
文件名: H:\TEMP\ORACLE_RESTORE\405\322_ORCL_0141onds_1_1.disk
文件名: H:\TEMP\ORACLE_RESTORE\405\322_ORCL_0241onec_1_1.disk
文件名: H:\TEMP\ORACLE_RESTORE\405\322_ORCL_0441onev_1_1.disk
文件名: H:\TEMP\ORACLE_RESTORE\405\c-1737595250-20250822-00.disk
文件名: H:\TEMP\ORACLE_RESTORE\405\c-1737595250-20250822-01.disk

是否确实要将上述文件列入目录 (输入 YES 或 NO)? yes
正在编制文件目录...
目录编制完毕

已列入目录的文件的列表
=======================
文件名: H:\TEMP\ORACLE_RESTORE\405\322_ORCL_0141onds_1_1.disk
文件名: H:\TEMP\ORACLE_RESTORE\405\322_ORCL_0241onec_1_1.disk
文件名: H:\TEMP\ORACLE_RESTORE\405\322_ORCL_0441onev_1_1.disk
文件名: H:\TEMP\ORACLE_RESTORE\405\c-1737595250-20250822-00.disk
文件名: H:\TEMP\ORACLE_RESTORE\405\c-1737595250-20250822-01.disk

RMAN>
RMAN> report schema;

RMAN-06139: 警告: 控制文件对于 REPORT SCHEMA 不是最新
db_unique_name 为 ORCL 的数据库的数据库方案报表

永久数据文件列表
===========================
文件大小 (MB) 表空间           回退段数据文件名称
---- -------- -------------------- ------- ------------------------
1    0        SYSTEM               ***     /data/oracle/oradata/orcl/system01.dbf
2    0        SYSAUX               ***     /data/oracle/oradata/orcl/sysaux01.dbf
3    0        UNDOTBS1             ***     /data/oracle/oradata/orcl/undotbs01.dbf
4    0        USERS                ***     /data/oracle/oradata/orcl/users01.dbf
5    0        EXAMPLE              ***     /data/oracle/oradata/orcl/example01.dbf

临时文件列表
=======================
文件大小 (MB) 表空间           最大大小 (MB) 临时文件名称
---- -------- -------------------- ----------- --------------------
1    20       TEMP                 32767       /data/oracle/oradata/orcl/temp01.dbf

RMAN> run
2> {
3> set newname for datafile 1 to 'h:\system01.dbf';
4> set newname for datafile 2 to 'h:\sysaux01.dbf';
5> set newname for datafile 3 to 'h:\undotbs01.dbf';
6> set newname for datafile 4 to 'h:\users01.dbf';
7> set newname for datafile 5 to 'h:\example01.dbf';
8> restore database;
9> switch datafile all;
10> }

正在执行命令: SET NEWNAME

正在执行命令: SET NEWNAME

正在执行命令: SET NEWNAME

正在执行命令: SET NEWNAME

正在执行命令: SET NEWNAME

启动 restore 于 03-9月 -25
分配的通道: ORA_DISK_1
通道 ORA_DISK_1: SID=28 设备类型=DISK

通道 ORA_DISK_1: 正在开始还原数据文件备份集
通道 ORA_DISK_1: 正在指定从备份集还原的数据文件
通道 ORA_DISK_1: 将数据文件 00001 还原到 h:\system01.dbf
通道 ORA_DISK_1: 将数据文件 00002 还原到 h:\sysaux01.dbf
通道 ORA_DISK_1: 将数据文件 00003 还原到 h:\undotbs01.dbf
通道 ORA_DISK_1: 将数据文件 00004 还原到 h:\users01.dbf
通道 ORA_DISK_1: 将数据文件 00005 还原到 h:\example01.dbf
通道 ORA_DISK_1: 正在读取备份片段 H:\TEMP\ORACLE_RESTORE\405\322_ORCL_0141ONDS_1_1.DISK
通道 ORA_DISK_1: 段句柄 = H:\TEMP\ORACLE_RESTORE\405\322_ORCL_0141ONDS_1_1.DISK 标记 = TAG20250822T124236
通道 ORA_DISK_1: 已还原备份片段 1
通道 ORA_DISK_1: 还原完成, 用时: 00:00:01
完成 restore 于 03-9月 -25

数据文件 1 已转换成数据文件副本
输入数据文件副本 RECID=7 STAMP=1210875892 文件名=H:\SYSTEM01.DBF
数据文件 2 已转换成数据文件副本
输入数据文件副本 RECID=8 STAMP=1210875892 文件名=H:\SYSAUX01.DBF
数据文件 3 已转换成数据文件副本
输入数据文件副本 RECID=9 STAMP=1210875892 文件名=H:\UNDOTBS01.DBF
数据文件 4 已转换成数据文件副本
输入数据文件副本 RECID=10 STAMP=1210875892 文件名=H:\USERS01.DBF
数据文件 5 已转换成数据文件副本
输入数据文件副本 RECID=11 STAMP=1210875892 文件名=H:\EXAMPLE01.DBF

RMAN>
RMAN> run
2> {
3> set archivelog destination to 'h:/';
4> restore archivelog all;
5> }

正在执行命令: SET ARCHIVELOG DESTINATION

启动 restore 于 03-9月 -25
使用通道 ORA_DISK_1

通道 ORA_DISK_1: 正在开始将归档日志还原到用户指定的目标
归档日志目标=h:/
通道 ORA_DISK_1: 正在还原归档日志
归档日志线程=1 序列=6
通道 ORA_DISK_1: 正在还原归档日志
归档日志线程=1 序列=7
通道 ORA_DISK_1: 正在读取备份片段 H:\TEMP\ORACLE_RESTORE\405\322_ORCL_0441ONEV_1_1.DISK
通道 ORA_DISK_1: 段句柄 = H:\TEMP\ORACLE_RESTORE\405\322_ORCL_0441ONEV_1_1.DISK 标记 = TAG20250822T124311
通道 ORA_DISK_1: 已还原备份片段 1
通道 ORA_DISK_1: 还原完成, 用时: 00:00:01
完成 restore 于 03-9月 -25

通过上述操作,证明写的小程序oracle rman 备份集从tape通道备份方式修改为直接走disk 通道还原完整没有问题.

删除数据库文件并部分覆盖情况下Oracle恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:删除数据库文件并部分覆盖情况下Oracle恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有客户由于磁盘空间满了,人工删除了数据库30多个数据文件,导致数据库无法正常工作,然后又被人offline这些文件启动数据库,并运行了一段时间,导致写入了大量的trace和部分数据库归档日志,导致被删除的数据文件发生了覆盖,对于这样情况,通过底层反删除工具对磁盘进行扫描,发现了部分被删除文件,但是大小基本上显示0kb
dbf-0kb


对于这种情况,os层面反删除恢复,肯定无法恢复出来合适的数据文件,只能做底层数据块扫描恢复,参考以前类似case:
rm -rf误删Oracle数据库恢复
win系统删除oracle数据文件恢复
Oracle 数据文件大小为0kb或者文件丢失恢复
解决一次硬件恢复之后数据文件0kb的故障恢复case
这个客户的情况相对复杂一些:
1. 该磁盘分区中有历史库(也就是说单纯的软件直接按照rdba方式无法直接区分出来合适的数据块,儿实现数据重组)
2. 删除的文件比较多(33个数据文件),分区较大(5T+)
3. 删除文件之后,分区还写入了不少数据,会引起一些覆盖和导致碎片数量增加,导致工作量增加和恢复效果变差
通过对客户alert日志分析,发现一个好消息,客户数据每个数据文件是固定大小(没有设置自增长),这种情况,一般来说数据比较连续,碎片相对比较容易区分出来.
add_datafile

通过工具扫描识别出来oracle block,并把结果记录到数据库中,然后通过人工在数据库中对其进行挑选识别,然后生成dd语句恢复出来数据文件,比如这个只是被覆盖了文件头的22号文件,就比较容易恢复
dd-ok

对于一些碎片严重的文件,就需要人工生成大量dd语句来恢复
dd

对于所有恢复出来的文件,使用工具检查坏块情况
check

然后把这些数据文件中的数据恢复到新库中,完成本次数据恢复工作,最大限度抢救客户数据.

存储掉电强制拉库引起ORA-01555和ORA-01189/ORA-01190故障处理

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:存储掉电强制拉库引起ORA-01555和ORA-01189/ORA-01190故障处理

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

机房存储突然掉电导致Oracle数据库访问存储异常,数据库报出大量的ORA-27072: File I/O error,Linux-x86_64 Error: 5: Input/output error,ORA-15081: failed to submit an I/O operation to a disk等错误,实例直接crash

Wed Aug 27 07:11:53 2025
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_m000_17596.trc:
ORA-27072: File I/O error
Linux-x86_64 Error: 5: Input/output error
Additional information: 4
Additional information: 6297632
Additional information: -1
WARNING: Read Failed. group:1 disk:0 AU:3075 offset:16384 size:16384
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ckpt_6165.trc:
ORA-00202: control file: '+DG/xff/controlfile/current.284.918834897'
ORA-15081: failed to submit an I/O operation to a disk
WARNING: failed to read mirror side 1 of virtual extent 0 logical extent 0 of 
  file 284 in group [1.2747812198] from disk DG_0000  allocation unit 3075 reason error; 
  if possible, will try another mirror side
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_m000_17596.trc:
ORA-00202: control file: '+DG/xff/controlfile/current.284.918834897'
ORA-15081: failed to submit an I/O operation to a disk
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ckpt_6165.trc:
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 5: Input/output error
Additional information: -1
Additional information: 16384
WARNING: Write Failed. group:1 disk:0 AU:3080 offset:49152 size:16384
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ckpt_6165.trc:
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 5: Input/output error
Additional information: -1
Additional information: 16384
WARNING: Write Failed. group:1 disk:0 AU:3075 offset:49152 size:16384
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ckpt_6165.trc:
ORA-15080: synchronous I/O operation to a disk failed
WARNING: failed to write mirror side 1 of virtual extent 0 logical 
 extent 0 of file 284 in group 1 on disk 0 allocation unit 3075 
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ckpt_6165.trc:
ORA-15080: synchronous I/O operation to a disk failed
WARNING: failed to write mirror side 1 of virtual extent 0 logical extent 0
  of file 283 in group 1 on disk 0 allocation unit 3080 
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ckpt_6165.trc:
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: '+DG/xff/controlfile/current.283.918834897'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: '+DG/xff/controlfile/current.284.918834897'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ckpt_6165.trc:
ORA-00221: error on write to control file
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: '+DG/xff/controlfile/current.283.918834897'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: '+DG/xff/controlfile/current.284.918834897'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
CKPT (ospid: 6165): terminating the instance due to error 221
Wed Aug 27 07:11:53 2025
ORA-1092 : opitsk aborting process

存储恢复之后,尝试open数据库报ORA-00333错误(该错误一般是由于redo写丢失导致)

Wed Aug 27 16:36:32 2025
ALTER DATABASE OPEN
This instance was first to open
Beginning crash recovery of 2 threads
 parallel recovery started with 31 processes
Started redo scan
Incomplete read from log member '+DG/xff/onlinelog/group_2.287.918834905'. Trying next member.
Incomplete read from log member '+DG/xff/onlinelog/group_2.288.918834911'. Trying next member.
Incomplete read from log member '+DG/xff/onlinelog/group_2.287.918834905'. Trying next member.
Abort recovery for domain 0
Aborting crash recovery due to error 333
Errors in file /u01/app/oracle/diag/rdbms/xff/xff2/trace/xff2_ora_10257.trc:
ORA-00333: redo log read error block 1275904 count 5721
Abort recovery for domain 0
Errors in file /u01/app/oracle/diag/rdbms/xff/xff2/trace/xff2_ora_10257.trc:
ORA-00333: redo log read error block 1275904 count 5721
ORA-333 signalled during: ALTER DATABASE OPEN...
1
现场人员使用隐含参数,尝试直接拉库操作报ORA-00704 ORA-01555错误,导致拉库失败
1
Wed Aug 27 16:47:11 2025
ALTER DATABASE RECOVER  database until cancel  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 32 slaves
ORA-279 signalled during: ALTER DATABASE RECOVER  database until cancel  ...
Wed Aug 27 16:47:56 2025
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Media Recovery Log +DG
Wed Aug 27 16:47:56 2025
Errors with log +DG
Errors in file /u01/app/oracle/diag/rdbms/xff/xff2/trace/xff2_pr00_24154.trc:
ORA-00308: cannot open archived log '+DG'
ORA-17503: ksfdopn:2 Failed to open file +DG
ORA-15045: ASM file name '+DG' is not in reference form
ORA-308 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
ALTER DATABASE RECOVER CANCEL 
Errors in file /u01/app/oracle/diag/rdbms/xff/xff2/trace/xff2_pr00_24154.trc:
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1: '+DG/xff/datafile/system.279.918834827'
Slave exiting with ORA-1547 exception
Errors in file /u01/app/oracle/diag/rdbms/xff/xff2/trace/xff2_pr00_24154.trc:
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1: '+DG/xff/datafile/system.279.918834827'
ORA-1547 signalled during: ALTER DATABASE RECOVER CANCEL ...
Wed Aug 27 16:48:09 2025
alter database open resetlogs
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 25330611827626
Resetting resetlogs activation ID 3307041102 (0xc51d714e)
Deleted Oracle managed file +DG/xff/onlinelog/group_1.285.918834899
Deleted Oracle managed file +DG/xff/onlinelog/group_1.286.918834901
Deleted Oracle managed file +DG/xff/onlinelog/group_2.287.918834905
Deleted Oracle managed file +DG/xff/onlinelog/group_2.288.918834911
Wed Aug 27 16:48:28 2025
Deleted Oracle managed file +DG/xff/onlinelog/group_3.289.918834917
Deleted Oracle managed file +DG/xff/onlinelog/group_3.290.918834923
Deleted Oracle managed file +DG/xff/onlinelog/group_4.293.918835035
Deleted Oracle managed file +DG/xff/onlinelog/group_4.294.918835037
Wed Aug 27 16:48:48 2025
Deleted Oracle managed file +DG/xff/onlinelog/group_5.295.918835041
Deleted Oracle managed file +DG/xff/onlinelog/group_5.296.918835047
Deleted Oracle managed file +DG/xff/onlinelog/group_6.297.918835055
Wed Aug 27 16:48:58 2025
Deleted Oracle managed file +DG/xff/onlinelog/group_6.298.918835061
Wed Aug 27 16:49:10 2025
Setting recovery target incarnation to 3
Wed Aug 27 16:49:10 2025
This instance was first to open
Picked broadcast on commit scheme to generate SCNs
Wed Aug 27 16:49:10 2025
Assigning activation ID 3598492411 (0xd67ca2fb)
Thread 2 opened at log sequence 1
  Current log# 4 seq# 1 mem# 0: +DG/xff/onlinelog/group_4.294.1210265317
  Current log# 4 seq# 1 mem# 1: +DG/xff/onlinelog/group_4.293.1210265323
Successful open of redo thread 2
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Aug 27 16:49:10 2025
SMON: enabling cache recovery
Instance recovery: looking for dead threads
Instance recovery: lock domain invalid but no dead threads
ORA-01555 caused by SQL statement below (SQL ID: 4krwuz0ctqxdt, SCN: 0x1709.be1eb3b1):
select ctime, mtime, stime from obj$ where obj# = :1
Errors in file /u01/app/oracle/diag/rdbms/xff/xff2/trace/xff2_ora_23787.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 1
ORA-01555: snapshot too old: rollback segment number 20 with name "_SYSSMU20_1295954159$" too small
Errors in file /u01/app/oracle/diag/rdbms/xff/xff2/trace/xff2_ora_23787.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 1
ORA-01555: snapshot too old: rollback segment number 20 with name "_SYSSMU20_1295954159$" too small
Error 704 happened during db open, shutting down database
USER (ospid: 23787): terminating the instance due to error 704
Instance terminated by USER, pid = 23787
ORA-1092 signalled during: alter database open resetlogs...
opiodr aborting process unknown ospid (23787) as a result of ORA-1092

现场进行了一系列尝试操作,最后我接手数据库之时报错为:ORA-01190 ORA-01110,无法recover,也无法重建controlfile,陷入了死局

Completed: ALTER DATABASE   MOUNT
Sat Aug 30 10:03:20 2025
ALTER DATABASE OPEN
This instance was first to open
Abort recovery for domain 0
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ora_6435.trc:
ORA-01190: control file or data file 1 is from before the last RESETLOGS
ORA-01110: data file 1: '+DG/xff/datafile/system0829.dbf'
ORA-1190 signalled during: ALTER DATABASE OPEN...
Sat Aug 30 00:56:32 2025
NOTE: Loaded library: System 
SUCCESS: diskgroup DG was mounted
Errors in file /u01/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ora_17302.trc:
ORA-01189: file is from a different RESETLOGS than previous files
ORA-01110: data file 2: '+DG/xff/datafile/sysaux.280.918834827'
ORA-1503 signalled during: create controlfile reuse database xff noarchivelog noresetlogs

对于这种情况,通过Oracle recovery check脚本可以直接发现异常(WRONG RESETLOGS)
wrong-resetlogs


使用Oracle Recovery Tools小工具实现快速恢复
orarecovery

再尝试重建ctl成功
rectl

然后修改数据库scn信息,顺利open数据库
open

后续建议客户逻辑迁移该库

ORA-600 kcratr_nab_less_than_odr和ORA-600 2662故障处理

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:ORA-600 kcratr_nab_less_than_odr和ORA-600 2662故障处理

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

异常断电之后,oracle启动报ORA-600 kcratr_nab_less_than_odr错误

Sun Aug 17 11:06:09 2025
ALTER DATABASE OPEN
Beginning crash recovery of 1 threads
 parallel recovery started with 11 processes
Started redo scan
Completed redo scan
 read 0 KB redo, 0 data blocks need recovery
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [40785], [33267], [40630]
Sun Aug 17 11:06:20 2025
Aborting crash recovery due to error 600
Sun Aug 17 11:06:20 2025
Trace dumping is performing id=[cdmp_20250817110620]
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_2920.trc:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [40785], [33267], [40630]
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_2920.trc:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [40785], [33267], [40630]
ORA-600 signalled during: ALTER DATABASE OPEN...

这个错误处理有多次处理经验
12c启动报kcratr_nab_less_than_odr
又一例ORA-600 kcratr_nab_less_than_odr
ORA-600 kcratr_nab_less_than_odr故障解决
差点被误操作的ORA-600 kcratr_nab_less_than_odr故障
11.2.0.4库中遇到ORA-600 kcratr_nab_less_than_odr报错
ORA-600 kcratr_nab_less_than_odr和ORA-600 4194故障处理
一般重建ctl或者using backup ctl方式恢复即可实现0丢失打开库,但是这个库尝试打开报ORA-600 2662错误

SQL> startup nomount;
ORACLE 例程已经启动。

Total System Global Area 6847938560 bytes
Fixed Size                  2188768 bytes
Variable Size            4680845856 bytes
Database Buffers         2147483648 bytes
Redo Buffers               17420288 bytes
SQL> @rectl.sql

控制文件已创建。

SQL>
SQL>
SQL> recover database;
完成介质恢复。
SQL> alter database open;
alter database open
*
第 1 行出现错误:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [2662], [4413], [1200914792],
[4413], [1201104184], [12583040], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [2662], [4413], [1200914791],
[4413], [1201104184], [12583040], [], [], [], [], [], []
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [2662], [4413], [1200914789],
[4413], [1201104184], [12583040], [], [], [], [], [], []
进程 ID: 3132
会话 ID: 3018 序列号: 1

对于这样的ORA-600 2662错误是比较常见的问题,直接通过Patch_SCN工具修改scn即可正常打开库
patch_scn-ora-600-2662

Mon Aug 18 03:30:41 2025
ALTER DATABASE RECOVER  database  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 12 slaves
Mon Aug 18 03:30:41 2025
Recovery of Online Redo Log: Thread 1 Group 2 Seq 40787 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO02.LOG
Completed: ALTER DATABASE RECOVER  database  
alter database open 
Beginning crash recovery of 1 threads
 parallel recovery started with 11 processes
Started redo scan
Completed redo scan
 read 1 KB redo, 0 data blocks need recovery
Started redo application at
 Thread 1: logseq 40787, block 2, scn 18954891612040
Recovery of Online Redo Log: Thread 1 Group 2 Seq 40787 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO02.LOG
Completed redo application of 0.00MB
Completed crash recovery at
 Thread 1: logseq 40787, block 4, scn 18954891632047
 0 data blocks read, 0 data blocks written, 1 redo k-bytes read
Mon Aug 18 03:30:47 2025
Thread 1 advanced to log sequence 40788 (thread open)
Thread 1 opened at log sequence 40788
  Current log# 3 seq# 40788 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO03.LOG
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Aug 18 03:30:47 2025
SMON: enabling cache recovery
Dictionary check beginning
Tablespace 'TEMP' #3 found in data dictionary,
but not in the controlfile. Adding to controlfile.
Dictionary check complete
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
*********************************************************************
WARNING: The following temporary tablespaces contain no files.
         This condition can occur when a backup controlfile has
         been restored.  It may be necessary to add files to these
         tablespaces.  That can be done using the SQL statement:
 
         ALTER TABLESPACE <tablespace_name> ADD TEMPFILE
 
         Alternatively, if these temporary tablespaces are no longer
         needed, then they can be dropped.
           Empty temporary tablespace: TEMP
*********************************************************************
Database Characterset is ZHS16GBK
replication_dependency_tracking turned off (no async multimaster replication found)
LOGSTDBY: Validating controlfile with logical metadata
LOGSTDBY: Validation complete
Completed: alter database open

后续增加tempfile,导出数据完成本次恢复任务