Category Archives: Oracle

记录一次asm disk加入到vg通过恢复直接open库的案例

Posted on by

联系：手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题：记录一次asm disk加入到vg通过恢复直接open库的案例

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

客户在不清楚磁盘被asm disk使用的情况下,直接分区做pv,加入到vg中并且分配给了lv,导致数据库异常
QQ20250504
144809

通过操作系统层面分析,确认客户把data磁盘组的一个磁盘给处理掉了,导致数据库报错

WARNING: ASMB force dismounting group 2 (DATA) due to failover
SUCCESS: diskgroup DATA was dismounted
2025-05-04T07:03:19.910082+08:00
KCF: read, write or open error, block=0x201544 online=1
        file=102 '+DATA/ORCL/F7D939D6DBE06C71E053C30114AC1F10/DATAFILE/xifenfei_61.dbf'
        error=15078 txt: ''
2025-05-04T07:03:19.918972+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_dbwc_18507.trc:
2025-05-04T07:03:19.952045+08:00
KCF: read, write or open error, block=0x2013e7 online=1
        file=102 '+DATA/ORCL/F7D939D6DBE06C71E053C30114AC1F10/DATAFILE/xifenfei_61.dbf'
        error=15078 txt: ''
2025-05-04T07:03:19.964538+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_dbw7_18486.trc:
2025-05-04T07:03:19.967133+08:00
KCF: read, write or open error, block=0x230e71 online=1
        file=105 '+DATA/ORCL/F7D939D6DBE06C71E053C30114AC1F10/DATAFILE/xifenfei_64.dbf'
        error=15078 txt: ''
2025-05-04T07:03:19.973289+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_dbw2_18466.trc:
2025-05-04T07:03:19.978514+08:00
KCF: read, write or open error, block=0x1f6e91 online=1
        file=86 '+DATA/ORCL/F7D939D6DBE06C71E053C30114AC1F10/DATAFILE/xifenfei_52.dbf'
        error=15078 txt: ''
2025-05-04T07:03:19.991060+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_dbwd_18511.trc:
2025-05-04T07:03:19.995762+08:00
KCF: read, write or open error, block=0x7f8 online=1
        file=15 '+DATA/ORCL/F7D939D6DBE06C71E053C30114AC1F10/DATAFILE/undotbs01.dbf'
        error=15078 txt: ''
2025-05-04T07:03:20.006862+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_dbwa_18498.trc:
2025-05-04T07:03:20.020739+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_imr0_18937.trc:

这个客户比较幸运,处理该磁盘之后,没有往对应的lv中写入太多数据,导致覆盖部分很少

[root@rac01 rules.d]# df -h
文件系统               容量  已用  可用 已用% 挂载点
/dev/mapper/nlas-root  800G  272G  528G   34% /
devtmpfs               284G     0  284G    0% /dev
tmpfs                  284G  637M  283G    1% /dev/shm
tmpfs                  284G  4.0G  280G    2% /run
tmpfs                  284G     0  284G    0% /sys/fs/cgroup
/dev/mapper/nlas-home  200G   64M  200G    1% /home
/dev/sda1              197M  158M   40M   80% /boot
tmpfs                   57G   40K   57G    1% /run/user/0
tmpfs                   57G   48K   57G    1% /run/user/1000
[root@rac01 rules.d]# pvs
  PV         VG   Fmt  Attr PSize   PFree
  /dev/sda2  nlas lvm2 a--  564.00g    0 
  /dev/sdb1  nlas lvm2 a--   <2.00t 1.51t
[root@rac01 rules.d]# vgs
  VG   #PV #LV #SN Attr   VSize VFree
  nlas   2   3   0 wz--n- 2.55t 1.51t
[root@rac01 rules.d]# lvs
  LV   VG   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  home nlas -wi-ao---- 200.00g                                                    
  root nlas -wi-ao---- 800.00g                                                    
  swap nlas -wi-ao----  64.00g

通过底层对磁盘进行分析,发现备份的磁盘头均以损坏,通过深入分析确认f1b1在sdb磁盘的第10个au上,通过相关信息,使用dul工具加载磁盘组,并分析元数据信息,发现恢复数据需要的元数据都可以正常加载
asm-dul

直接使用dul抽取数据到文件系统,然后open数据库成功
open-asm
然后通过rman 检测坏块(3T多的库只有不到5000个坏块,相对来说效果非常好),对于坏块对象进行处理,完美完成本次恢复工作.对于这次能够有这样好的恢复效果有几个因素:
1)asm disk 加入到vg,并分配给lv之后,立刻停止写入操作,避免了因为写入数据而覆盖asm 磁盘的带来的风险
2)由于是19c库,默认au为4M,使得数据库文件数据相对比较靠后,覆盖几率小了一点
3)由于文件系统是xfs,相对覆盖比ext4会少很多
4)是云环境的ssd磁盘,没有触发trim功能
以前类似asm disk异常恢复的相关case汇总:
asm磁盘加入vg恢复
asm磁盘dd破坏恢复
asm磁盘分区丢失恢复
pvid=yes导致asm无法mount
win asm disk header 异常恢复
又一例asm disk 加入vg故障
pvcreate asm disk导致asm磁盘组异常恢复
asm disk被加入到另外一个磁盘组故障恢复
再一例asm disk被误加入vg并且扩容lv恢复
再一起asm disk被格式化成ext3文件系统故障恢复
一次完美的asm disk被格式化ntfs恢复
asm disk误设置pvid导致asm diskgroup无法mount恢复
asm disk被分区,格式化为ext4恢复
oracle asm disk格式化恢复—格式化为ext4文件系统
分享oracleasm createdisk重新创建asm disk后数据0丢失恢复案例

Oracle Recovery Tools修复ORA-00742、ORA-600 ktbair2: illegal inheritance故障

Posted on by

联系：手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题：Oracle Recovery Tools修复ORA-00742、ORA-600 ktbair2: illegal inheritance故障

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

接到客户反馈,一套运行在虚拟化平台中的Oracle数据库,由于机房断电,导致数据库无法启动,最初启动报错

2025-04-22T16:59:48.922227+08:00
Completed: alter database mount exclusive
alter database open
2025-04-22T16:59:52.609726+08:00
Ping without log force is disabled:
  instance mounted in exclusive mode.
2025-04-22T16:59:52.937852+08:00
Beginning crash recovery of 1 threads
 parallel recovery started with 15 processes
 Thread 1: Recovery starting at checkpoint rba (logseq 11031 block 139863), scn 0
2025-04-22T16:59:53.094081+08:00
Started redo scan
2025-04-22T16:59:53.203464+08:00
Completed redo scan
 read 15533 KB redo, 1140 data blocks need recovery
2025-04-22T16:59:53.515972+08:00
Hex dump of (file 3,block 82642) in trace file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p001_6612.trc

Corrupt block relative dba: 0x00c142d2 (file 3,block 82642)
Fractured block found during crash/instance recovery
Data in bad block:
 type: 6 format: 2 rdba: 0x00c142d2
 last change scn: 0x0000.0001.7a0219e2 seq: 0x1 flg: 0x06
 spare3: 0x0
 consistency value in tail: 0x0fcb0601
 check value in block header: 0x82eb
 computed block checksum: 0x172a

2025-04-22T16:59:53.563004+08:00
Reading datafile 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF' for corrupt data at rdba: 0x00c142d2 (file 3,block 82642)
Reread (file 3,block 82642) found same corrupt data (no logical check)
2025-04-22T16:59:54.578467+08:00
Hex dump of (file 3,block 207498) in trace file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p000_6064.trc
2025-04-22T16:59:54.578467+08:00

Corrupt block relative dba: 0x00c32a8a (file 3,block 207498)
Fractured block found during crash/instance recovery
Data in bad block:
 type: 6 format: 2 rdba: 0x00c32a8a
 last change scn: 0x0000.0001.731ec74f seq: 0x2 flg: 0x06
 spare3: 0x0
 consistency value in tail: 0x10520601
 check value in block header: 0x61c3
 computed block checksum: 0xe27a

Reading datafile 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF' for corrupt data at rdba: 0x00c32a8a (file 3,block 207498)
Reread (file 3,block 207498) found same corrupt data (no logical check)
Hex dump of (file 4, block 3959) in trace file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p000_6064.trc

Corrupt block relative dba: 0x01000f77 (file 4, block 3959)
Fractured block found during crash/instance recovery
Data in bad block:
 type: 2 format: 2 rdba: 0x01000f77
 last change scn: 0x0000.0001.79fa3621 seq: 0x4 flg: 0x04
 spare3: 0x0
 consistency value in tail: 0xcf6c0201
 check value in block header: 0x99d2
 computed block checksum: 0x79e0

Reading datafile 'D:\APP\ADMINISTRATOR\ORADATA\HIS\UNDOTBS01.DBF' for corrupt data at rdba: 0x01000f77 (file 4,block 3959)
Reread (file 4, block 3959) found same corrupt data (no logical check)
2025-04-22T16:59:54.640991+08:00
Started redo application at
 Thread 1: logseq 11031, block 139863, offset 0
2025-04-22T16:59:54.672246+08:00
Recovery of Online Redo Log: Thread 1 Group 3 Seq 11031 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO03.LOG
2025-04-22T16:59:54.703494+08:00
Hex dump of (file 3,block 89738) in trace file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc

Corrupt block relative dba: 0x00c15e8a (file 3,block 89738)
Fractured block found during crash/instance recovery
Data in bad block:
 type: 6 format: 2 rdba: 0x00c15e8a
 last change scn: 0x0000.0001.7a09108a seq: 0x2 flg: 0x06
 spare3: 0x0
 consistency value in tail: 0x278e0602
 check value in block header: 0xd721
 computed block checksum: 0x2705

Reading datafile 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF' for corrupt data at rdba: 0x00c15e8a (file 3,block 89738)
Reread (file 3,block 89738) found same corrupt data (no logical check)
2025-04-22T16:59:54.734756+08:00
Hex dump of (file 3,block 9504) in trace file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p005_3660.trc

Reading datafile 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF' for corrupt data at rdba: 0x00c02520 (file 3,block 9504)
Reread (file 3,block 9504) found same corrupt data (logically corrupt)
*****************************************************************
An internal routine has requested a dump of selected redo.
This usually happens following a specific internal error, when
analysis of the redo logs will help Oracle Support with the
diagnosis.
It is recommended that you retain all the redo logs generated (by
all the instances) during the past 12 hours, in case additional
redo dumps are required to help with the diagnosis.
*****************************************************************
2025-04-22T16:59:54.750364+08:00
Exception [type: ACCESS_VIOLATION, UNABLE_TO_READ] [ADDR:0xC] [PC:0x7FF64221F2F8, kdxlin()+4824]
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p008_3964.trc  (incident=521460):
ORA-07445: 出现异常错误: 核心转储 [kdxlin()+4824][ACCESS_VIOLATION][ADDR:0xC][PC:0x7FF64221F2F8][UNABLE_TO_READ][]
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_521460\his_p008_3964_i521460.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-04-22T16:59:55.156605+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p005_3660.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00312: 联机日志 1 线程 1: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
2025-04-22T16:59:55.828516+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p00a_3428.trc  (incident=521476):
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_521476\his_p00a_3428_i521476.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-04-22T16:59:56.844113+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc  (incident=521444):
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_521444\his_p006_5236_i521444.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-04-22T16:59:59.062944+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p005_3660.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00334: 归档日志: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
2025-04-22T16:59:59.156645+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p00a_3428.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00312: 联机日志 1 线程 1: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_521477\his_p00a_3428_i521477.trc
2025-04-22T17:00:04.250414+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00334: 归档日志: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
2025-04-22T17:00:04.344200+08:00
Slave encountered ORA-10388 exception during crash recovery
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc  (incident=521445):
ORA-01578: ORACLE 数据块损坏 (文件号 3, 块号 299732)
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10564: tablespace SYSAUX
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 11351
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_521445\his_p006_5236_i521445.trc
2025-04-22T17:00:05.312967+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p00a_3428.trc:
ORA-01578: ORACLE 数据块损坏 (文件号 3, 块号 216445)
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10564: tablespace SYSAUX
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED DATA BLOCK', data object# 11171
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
2025-04-22T17:00:05.359805+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-04-22T17:00:06.172286+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc:
ORA-01578: ORACLE 数据块损坏 (文件号 3, 块号 299732)
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10564: tablespace SYSAUX
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 11351
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
2025-04-22T17:00:06.172286+08:00
Slave encountered ORA-1578 exception during crash recovery
Slave exiting with ORA-1578 exception
2025-04-22T17:00:06.172286+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc:
ORA-01578: ORACLE 数据块损坏 (文件号 3, 块号 299732)
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10564: tablespace SYSAUX
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 11351
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
ORA-1578 signalled during: alter database open...

这里面主要报错有几大类:
1.在需要实例恢复的block中有坏块,而且主要集中在3号文件(sysaux)上
2. ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
3. ORA-600 ktbair2: illegal inheritance错误
对于这些问题的本质都是数据库无法正常实例恢复,尝试按照数据文件级别进行恢复操作,结果只有datafile 3 不成功

2025-04-22T20:11:36.724880+08:00
ALTER DATABASE RECOVER  datafile 3  
2025-04-22T20:11:36.740523+08:00
Media Recovery Start
2025-04-22T20:11:36.771771+08:00
Serial Media Recovery started
2025-04-22T20:11:36.896781+08:00
Recovery of Online Redo Log: Thread 1 Group 3 Seq 11031 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO03.LOG
Exception [type: ACCESS_VIOLATION, UNABLE_TO_READ] [ADDR:0xC] [PC:0x7FF64221F2F8, kdxlin()+4824]
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_ora_2512.trc  (incident=721216):
ORA-07445: ??????: ???? [kdxlin()+4824] [ACCESS_VIOLATION] [ADDR:0xC] [PC:0x7FF64221F2F8] [UNABLE_TO_READ] []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_721216\his_ora_2512_i721216.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-04-22T20:12:56.585813+08:00
ALTER DATABASE RECOVER  datafile 3  
2025-04-22T20:12:56.601434+08:00
Media Recovery Start
2025-04-22T20:12:56.601434+08:00
Serial Media Recovery started
2025-04-22T20:12:57.351435+08:00
Recovery of Online Redo Log: Thread 1 Group 3 Seq 11031 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO03.LOG
2025-04-22T20:12:58.492071+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_ora_5332.trc  (incident=721217):
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_721217\his_ora_5332_i721217.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-04-22T20:13:00.710878+08:00
*****************************************************************
An internal routine has requested a dump of selected redo.
This usually happens following a specific internal error, when
analysis of the redo logs will help Oracle Support with the
diagnosis.
It is recommended that you retain all the redo logs generated (by
all the instances) during the past 12 hours, in case additional
redo dumps are required to help with the diagnosis.
*****************************************************************
2025-04-22T20:13:01.398399+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_ora_5332.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00312: 联机日志 1 线程 1: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
2025-04-22T20:13:03.820304+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_mz00_2200.trc:
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
2025-04-22T20:13:06.070340+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_ora_5332.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00334: 归档日志: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
2025-04-22T20:13:06.085990+08:00
Media Recovery failed with error 10562
ORA-283 signalled during: ALTER DATABASE RECOVER  datafile 3  ...

对于这样的故障,前段时间通过Oracle Recovery Tools工具处理过类似异常:Oracle Recovery Tools修复ORA-600 6101/kdxlin:psno out of range故障
QQ20250422-201804

然后尝试recover datafile 3,并打开数据库成功
QQ20250422-201916
至此数据库open成功,由于该库有不少坏块(包括system的一些字典),后续使用逻辑方式迁移数据到新库中(其中由于C_OBJ#簇中有坏块,导致迁移过程中大量index丢失,通过一些方法使用obj$,inde$等基表人工生成index.sql对于丢失的index进行弥补),最终客户业务正常运行

11.2.0.4库中遇到ORA-600 kcratr_nab_less_than_odr报错

Posted on by

联系：手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题：11.2.0.4库中遇到ORA-600 kcratr_nab_less_than_odr报错

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

今天有一个客户11.2.0.4的库由于断电报ORA-600 kcratr_nab_less_than_odr的故障

C:\Users\Administrator>sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on 星期二 4月 22 10:56:36 2025

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


连接到:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select * from v$version;

BANNER
--------------------------------------------------------------------------------

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
PL/SQL Release 11.2.0.4.0 - Production
CORE    11.2.0.4.0      Production
TNS for 64-bit Windows: Version 11.2.0.4.0 - Production
NLSRTL Version 11.2.0.4.0 - Production

SQL> startup mount
ORACLE 例程已经启动。

Total System Global Area 1.9174E+10 bytes
Fixed Size                  2289320 bytes
Variable Size            3355443544 bytes
Database Buffers         1.5771E+10 bytes
Redo Buffers               45932544 bytes
数据库装载完毕。
SQL> alter database open;
alter database open
*
第 1 行出现错误:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [4653],
[50575], [50922], [], [], [], [], [], [], []

alert日志报错信息

Tue Apr 22 05:08:12 2025
ALTER DATABASE   MOUNT
Successful mount of redo thread 1, with mount id 3880792636
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE   MOUNT
Tue Apr 22 05:08:17 2025
ALTER DATABASE OPEN
Beginning crash recovery of 1 threads
 parallel recovery started with 15 processes
Started redo scan
Completed redo scan
 read 2451 KB redo, 1442 data blocks need recovery
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\ydhl\ydhl\trace\ydhl_ora_2164.trc  (incident=240429):
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [4653], [50575], [50922],[],[],[],[],[],[],[]
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\ydhl\ydhl\incident\incdir_240429\ydhl_ora_2164_i240429.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Aborting crash recovery due to error 600
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\ydhl\ydhl\trace\ydhl_ora_2164.trc:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [4653], [50575], [50922],[],[],[],[],[],[],[]
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\ydhl\ydhl\trace\ydhl_ora_2164.trc:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [4653], [50575], [50922],[],[],[],[],[],[],[]
ORA-600 signalled during: ALTER DATABASE OPEN...

这个故障在一般在11.2.0.1版本中最为常见,处理起来比较简单,一般数据也会0丢失
12c启动报kcratr_nab_less_than_odr
又一例ORA-600 kcratr_nab_less_than_odr
ORA-600 kcratr_nab_less_than_odr故障解决
差点被误操作的ORA-600 kcratr_nab_less_than_odr故障
这里处理比较简单,重建控制文件就完成了恢复工作
102328

Oracle 19c 202504补丁（RUs+OJVM）-19.27

Posted on by

联系：手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题：Oracle 19c 202504补丁（RUs+OJVM）-19.27

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

 

19.0.0.0
 Description  Database Update  GI Update  Windows Bundle Patch
 APR2025 (19.27.0.0.0) 37642901  37641958  37532350
 JAN2025 (19.26.0.0.0) 37260974  37257886  37486199
 OCT2024 (19.25.0.0.0) 36912597  36916690  36878821
 JUL2024 (19.24.0.0.0) 36582781  36582629  36521936
 APR2024 (19.23.0.0.0) 36233263  36233126  36219938
 JAN2024 (19.22.0.0.0) 35943157  35940989  35962832
 OCT2023 (19.21.0.0.0) 35643107  35642822  35681552
 JUL2023 (19.20.0.0.0) 35320081  35319490  35348034
 APR2023 (19.19.0.0.0) 35042068  35037840  35046439
 JAN2023 (19.18.0.0.0) 34765931  34762026  34750795
 Oct2022 (19.17.0.0.0) 34419443  34416665  34468114
 JUL2022 (19.16.0.0.0) 34133642  34130714  34110685
 APR2022 (19.15.0.0.0) 33806152  33803476  33829175
 JAN2022 (19.14.0.0.0) 33515361  33509923  33575656
 OCT2021(19.13.0.0.0) 33192793  33182768  33155330
 JUL2021 (19.12.0.0.0) 32904851  32895426  32832237
 APR2021 (19.11.0.0.0) 32545013  32545008  32409154
 JAN2021 (19.10.0.0.0) 32218454  32226239  32062765
 OCT2020 (19.9.0.0.0) 31771877  31750108  31719903
 JUL2020  (19.8.0.0.0) 31281355  31305339  31247621
 APR2020 (19.7.0.0.0) 30869156  30899722  30901317
 JAN2020 (19.6.0.0.0) 30557433  30501910  30445947
 OCT2019 (19.5.0.0.0) 30125133  30116789  30151705
 JUL2019 (19.4.0.0.0) 29834717  29708769   NA
 APR2019 (19.3.0.0.0) 29517242  29517302   NA

 

 

 

19.0.0.0
 Description  OJVM Update  OJVM + DB Update  OJVM + GI Update
 APR2025 (19.27.0.0.250415)  37499406  37591483  37591516
 JAN2025 (19.26.0.0.250121)  37102264  37262172  37262208
 OCT2024 (19.25.0.0.241015)  36878697  36866623  36866740
 JUL2024 (19.24.0.0.240716)  36414915  36522340  36522439
 APR2024 (19.23.0.0.240416)  36199232  36209492  36209493
 JAN2024 (19.22.0.0.240116)  35926646  36031426  36031453
 OCT2023 (19.21.0.0.231017)  35648110  35742413  35742441
 JUL2023 (19.20.0.0.230718)  35354406  35370174  35370167
 APR2023 (19.19.0.0.230418)  35050341  35058163  35058172
 JAN2023 (19.18.0.0.230117)  34786990  34773489  34773504
 OCT2022 (19.17.0.0.221018)  34411846  34449114  34449117
 JUL2022 (19.16.0.0.220719)  34086870  34160831  34160854
 APR2022 (19.15.0.0.220419)  33808367  33859194  33859214
 JAN2022 (19.14.0.0.220118)  33561310  33567270  33567274
 OCT2021 (19.13.0.0.211019)  33192694  33248420  33248471
 JUL2021 (19.12.0.0.210720)  32876380  32900021  32900083
 APR2021 (19.11.0.0.210420)  32399816  32578972  32578973
 JAN2021 (19.10.0.0.210119)  32067171  32126828  32126842
 OCT2020 (19.9.0.0.201020)  31668882  31720396  31720429
 JUL2020 (19.8.0.0.200714)  31219897  31326362  31326369
 APR2020 (19.7.0.0.200414)  30805684  30783543  30783556
 JAN2020 (19.6.0.0.200114)  30484981  30463595  30463609
 OCT2019 (19.5.0.0.191015)  30128191  30133124  30133178
 JUL2019 (19.4.0.0.190716)  29774421  29699079  29699097
 APR2019 (19.3.0.0.190416)  29548437  29621253  29621299

参考:Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases (Doc ID 2118136.2)

Oracle Recovery Tools修复ORA-600 6101/kdxlin:psno out of range故障

Posted on by

联系：手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题：Oracle Recovery Tools修复ORA-600 6101/kdxlin:psno out of range故障

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

数据库异常断电,然后启动异常,我接手该库,尝试recover恢复

SQL> recover database;
ORA-10562: Error occurred while applying redo to data block (file# 2, block#
63710)
ORA-10564: tablespace SYSAUX
ORA-01110: ???????? 2: 'H:\TEMP\GDLISNET\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 179030
ORA-00600: ????????????, ????: [6101], [0], [700], [32], [], [], [], [], [],
[], [], []

SQL> recover datafile 1;
完成介质恢复。
SQL> recover datafile 3;
完成介质恢复。
SQL> recover datafile 4;
完成介质恢复。
SQL> recover datafile 5;
完成介质恢复。
SQL> recover datafile 6;
完成介质恢复。
SQL> recover datafile 2;
ORA-00283: ??????????
ORA-10562: Error occurred while applying redo to data block (file# 2, block#
63744)
ORA-10564: tablespace SYSAUX
ORA-01110: ???? 2: 'H:\TEMP\GDLISNET\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 67482
ORA-00600: ??????, ??: [kdxlin:psno out of range], [], [], [], [], [], [], [],
[], [], [], []

alert日志信息

Mon Apr 14 12:31:55 2025
ALTER DATABASE RECOVER  database  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 20 slaves
Mon Apr 14 12:31:56 2025
Recovery of Online Redo Log: Thread 1 Group 2 Seq 6845 Reading mem 0
  Mem# 0: H:\TEMP\GDLISNET\REDO02.LOG
Mon Apr 14 12:31:56 2025
Errors in file c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\trace\gdlisnet_pr0e_9840.trc  (incident=1489):
ORA-00600: 内部错误代码, 参数: [6101], [0], [700], [32], [], [], [], [], [], [], [], []
Incident details in: c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\incident\incdir_1489\gdlisnet_pr0e_9840_i1489.trc
Slave exiting with ORA-10562 exception
Errors in file c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\trace\gdlisnet_pr0e_9840.trc:
ORA-10562: Error occurred while applying redo to data block (file# 2, block# 63710)
ORA-10564: tablespace SYSAUX
ORA-01110: 数据文件 2: 'H:\TEMP\GDLISNET\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 179030
ORA-00600: 内部错误代码, 参数: [6101], [0], [700], [32], [], [], [], [], [], [], [], []
Mon Apr 14 12:31:57 2025
Checker run found 6 new persistent data failures
Recovery Slave PR0E previously exited with exception 10562
Media Recovery failed with error 448
Errors in file c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\trace\gdlisnet_pr00_14232.trc:
ORA-00283: 恢复会话因错误而取消
ORA-00448: 后台进程正常结束
Slave exiting with ORA-283 exception
Errors in file c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\trace\gdlisnet_pr00_14232.trc:
ORA-00283: 恢复会话因错误而取消
ORA-00448: 后台进程正常结束
ORA-10562 signalled during: ALTER DATABASE RECOVER  database  ...

ALTER DATABASE RECOVER  datafile 2  
Media Recovery Start
Serial Media Recovery started
Recovery of Online Redo Log: Thread 1 Group 2 Seq 6845 Reading mem 0
  Mem# 0: H:\TEMP\GDLISNET\REDO02.LOG
Errors in file c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\trace\gdlisnet_ora_22400.trc  (incident=1369):
ORA-00600: ??????, ??: [kdxlin:psno out of range], [], [], [], [], [], [], [], [], [], [], []
Incident details in: c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\incident\incdir_1369\gdlisnet_ora_22400_i1369.trc
Media Recovery failed with error 10562
ORA-283 signalled during: ALTER DATABASE RECOVER  datafile 2  ...

发现2号数据文件异常,报ORA-600 6101和ORA-600 kdxlin:psno out of range错误,出现该错误的原因主要是由于数据文件的bock和redo中信息不匹配导致,对于这种情况,我这边选择使用Oracle Recovery Tools工具进行快速恢复
QQ20250414-123421
再次recover并且直接打开库

SQL> set numw 16
SQL> col CHECKPOINT_TIME for a40
SQL> set lines 150
SQL> set pages 1000
SQL> SELECT status,
  2  to_char(checkpoint_time,'yyyy-mm-dd hh24:mi:ss') checkpoint_time,FUZZY,checkpoint_change#,
  3  count(*) ROW_NUM
  4  FROM v$datafile_header
  5  GROUP BY status, checkpoint_change#, to_char(checkpoint_time,'yyyy-mm-dd hh24:mi:ss'),fuzzy
  6  ORDER BY status, checkpoint_change#, checkpoint_time;

STATUS  CHECKPOINT_TIME                          FUZ CHECKPOINT_CHANGE#          ROW_NUM
------- ---------------------------------------- --- ------------------ ----------------
ONLINE  2025-03-28 22:00:09                      NO           118575875                6

SQL>
SQL>
SQL> alter database open;
alter database open
*
第 1 行出现错误:
ORA-01113: 文件 2 需要介质恢复
ORA-01110: 数据文件 2: 'H:\TEMP\GDLISNET\SYSAUX01.DBF'


SQL> recover datafile 2;
完成介质恢复。
SQL> alter database open;

数据库已更改。

SQL>

逻辑方法导出该库,完成该库的恢复工作