一个运行在hp平台的10g rac突然异常,之后运行一段时间就自动重启,客户让对其进行分析和解决
Thu May 8 06:23:21 2025
ALTER DATABASE OPEN
Picked broadcast on commit scheme to generate SCNs
Thu May 8 06:23:21 2025
Thread 1 opened at log sequence 74302
Current log# 1 seq# 74302 mem# 0: /dev/vgdata/rrac_redo01
Successful open of redo thread 1
Thu May 8 06:23:21 2025
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Thu May 8 06:23:21 2025
SMON: enabling cache recovery
Thu May 8 06:23:22 2025
Successfully onlined Undo Tablespace 1.
Thu May 8 06:23:22 2025
SMON: enabling tx recovery
Thu May 8 06:23:22 2025
Database Characterset is ZHS16CGB231280
Opening with internal Resource Manager plan
where NUMA PG = 1, CPUs = 4
replication_dependency_tracking turned off (no async multimaster replication found)
Thu May 8 06:23:23 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Starting background process QMNC
QMNC started with pid=22, OS id=15792
Thu May 8 06:23:25 2025
ORACLE Instance orcl1 (pid = 13) - Error 607 encountered while recovering transaction (9, 33) on object 775794.
Thu May 8 06:23:25 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Thu May 8 06:23:26 2025
Completed: ALTER DATABASE OPEN
Thu May 8 06:23:26 2025
Doing block recovery for file 118 block 333578
Block recovery from logseq 74302, block 22 to scn 46740761996
Thu May 8 06:23:26 2025
Recovery of Online Redo Log: Thread 1 Group 1 Seq 74302 Reading mem 0
Mem# 0: /dev/vgdata/rrac_redo01
Block recovery stopped at EOT rba 74302.33.16
Block recovery completed at rba 74302.33.16, scn 10.3791089036
Thu May 8 06:23:33 2025
Trace dumping is performing id=[cdmp_20250508062324]
Thu May 8 06:25:55 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Thu May 8 06:25:58 2025
ORACLE Instance orcl1 (pid = 13) - Error 607 encountered while recovering transaction (9, 33) on object 775794.
Thu May 8 06:27:32 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
Doing block recovery for file 118 block 333578
Block recovery from logseq 74302, block 372 to scn 46740952565
Thu May 8 06:27:41 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
ORACLE Instance orcl1 (pid = 13) - Error 607 encountered while recovering transaction (9, 33) on object 775794.
Thu May 8 06:27:43 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Doing block recovery for file 118 block 333578
Block recovery from logseq 74302, block 372 to scn 46740952565
Thu May 8 06:27:45 2025
Recovery of Online Redo Log: Thread 1 Group 1 Seq 74302 Reading mem 0
Mem# 0: /dev/vgdata/rrac_redo01
Block recovery completed at rba 74302.394.16, scn 10.3791279606
Thu May 8 06:27:47 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Thu May 8 06:28:07 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_pmon_15690.trc:
ORA-00474: SMON process terminated with error
Thu May 8 06:28:07 2025
PMON: terminating instance due to error 474
这个数据库重启是由于smon进程异常导致数据库关闭,然后rac自动拉起数据库,从而出现了循环重启启动,smon异常的主要原因是由于“ORACLE Instance orcl1 (pid = 13) – Error 607 encountered while recovering transaction (9, 33) on object 775794.”这个表示有实物要回滚,但是遇到了ORA-600 kddummy_blkchk异常,无法完成回滚从而使得smon进程异常进而使得数据实例crash.对于这个问题处理相对比较简单
1. 通过ORA-600 kddummy_blkchk找出来报错对象
ORA-00600: internal error code, arguments: [kddummy_blkchk], [a], [b], 1
ARGUMENTS:
Arg [a] Absolute file number
Arg [b] Block number
Arg 1 Internal error code returned from kcbchk() which indicates the problem encountered.
2. 屏蔽实物回滚,打开数据库,然后对通过dba_extents 查询出来的对象/或者根据事务报错的object信息查询出来对象进行重建,完成本次恢复任务
关于ORA-00600: internal error code, arguments: [kddummy_blkchk], [a], [b], 相关目前已知bug
| Bug |
Fixed |
Description |
| 12349316 |
11.2.0.4, 12.1.0.2, 12.2.0.1 |
DBMS_SPACE_ADMIN.TABLESPACE_FIX_BITMAPS fails with ORA-600 [kddummy_blkchk] / ORA-600 [kdBlkCheckError] / ORA-607 |
| 17325413 |
11.2.0.3.BP23, 11.2.0.4.2, 11.2.0.4.BP04, 12.1.0.1.3, 12.1.0.2, 12.2.0.1 |
Drop column with DEFAULT value and NOT NULL definition ends up with Dropped Column Data still on Disk leading to Corruption |
| 13715932 |
11.2.0.4, 12.1.0.1 |
ORA-600 [kddummy_blkchk] [18038] while adding extents to a large datafile |
| 12417369 |
11.2.0.2.5, 11.2.0.2.BP13, 11.2.0.2.GIPSU05, 11.2.0.3, 12.1.0.1 |
Block corruption from rollback on compressed table |
| 10324526 |
10.2.0.5.4, 11.1.0.7.8, 11.2.0.2.3, 11.2.0.2.BP06, 11.2.0.3, 12.1.0.1 |
ORA-600 [kddummy_blkchk] [6106] / corruption on COMPRESS table in TTS |
| 10113224 |
11.2.0.3, 12.1.0.1 |
Index coalesce may generate invalid redo if blocks in the buffer cache are invalid/corrupted |
| 9726702 |
11.2.0.3, 12.1.0.1 |
DBMS_SPACE_ADMIN.assm_segment_verify reports HWM related inconsistencies |
| 9724970 |
11.2.0.1.BP08, 11.2.0.2.2, 11.2.0.2.BP02, 11.2.0.3, 12.1.0.1 |
Block Corruption with PDML UPDATE. ORA_600 [4511] OERI[kdblkcheckerror] by block check |
| 9711859 |
10.2.0.5.1, 11.1.0.7.6, 11.2.0.2, 12.1.0.1 |
ORA-600 [ktsptrn_fix-extmap] / ORA-600 [kdblkcheckerror] during extent allocation caused by bug 8198906 |
| 9581240 |
11.1.0.7.9, 11.2.0.2, 12.1.0.1 |
Corruption / ORA-600 [kddummy_blkchk] [6101] / ORA-600 [7999] after RENAME operation during ROLLBACK |
| 9350204 |
11.2.0.3, 12.1.0.1 |
Spurious ORA-600 [kddummy_blkchk] .. [6145] during CR operations on tables with ROWDEPENDENCIES |
| 9231605 |
11.1.0.7.4, 11.2.0.1.3, 11.2.0.1.BP02, 11.2.0.2, 12.1.0.1 |
Block corruption with missing row on a compressed table after DELETE |
| 9119771 |
11.2.0.2, 12.1.0.1 |
OERI [kddummy_blkchk]…[6108] from ‘SHRINK SPACE CASCADE’ |
| 9019113 |
11.2.0.1.BP02, 11.2.0.2, 12.1.0.1 |
ORA-600 [17182] ORA-7445 [memcpy] ORA-600 [kdBlkCheckError] for OLTP COMPRESS table in OLTP Compression REDO during RECOVERY |
| 8951812 |
11.2.0.2, 12.1.0.1 |
Corrupt index by rebuild online. Possible OERI [kddummy_blkchk] by SMON |
| 8720802 |
10.2.0.5, 11.2.0.1.BP07, 11.2.0.2, 12.1.0.1 |
Add check for row piece pointing to itself (db_block_checking,dbv,rman,analyze) |
| 8331063 |
11.2.0.3, 12.1.0.1 |
Corrupt Undo. ORA-600 [2015] in Undo Block During Rollback |
| 6523037 |
11.2.0.1.BP07, 11.2.0.2.2, 11.2.0.2.BP01, 11.2.0.3, 12.1.0.1 |
Corruption / ORA-600 [kddummy_blkchk] [6110] on update |
| 8277580 |
11.1.0.7.2, 11.2.0.1, 11.2.0.2, 12.1.0.1 |
Corruption on compressed tables during Recovery and Quick Multi Delete (QMD). |
| 9964102 |
11.2.0.1 |
OERI:2015 / OERI:kddummy_blkchk / undo corruption from supplementat logging with compressed tables |
| 8613137 |
11.1.0.7.2, 11.2.0.1 |
ORA-600 updating table with DEFERRED constraints |
| 8360192 |
11.1.0.7.6, 11.2.0.1 |
ORA-600 [kdBlkCheckError] [6110] / corruption from insert |
| 8239658 |
10.2.0.5, 11.2.0.1 |
Dump / corruption writing row to compressed table |
| 8198906 |
10.2.0.5, 11.2.0.1 |
OERI [kddummy_blkchk] / OERI [5467] for an aborted transaction of allocating extents |
| 7715244 |
11.1.0.7.2, 11.2.0.1 |
Corruption on compressed tables. Error codes 6103 / 6110 |
| 7662491 |
10.2.0.4.2, 10.2.0.5, 11.1.0.7.4, 11.2.0.1 |
Array Update can corrupt a row. Errors OERI[kghstack_free1] or OERI[kddummy_blkchk][6110] |
| 7411865 |
10.2.0.4.2, 10.2.0.5, 11.1.0.7.1, 11.2.0.1 |
OERI:13030 / ORA-1407 / block corruption from UPDATE .. RETURNING DML with trigger |
| 7331181 |
11.2.0.1 |
ORA-1555 or OERI [kddummy_blkchk] [file#] [block#] [6126] during CR Rollback in query |
| 7293156 |
11.1.0.7, 11.2.0.1 |
ORA-600 [2023] by Parallel Transaction Rollback when applying Multi-block undo Head-piece / Tail-piece |
| 7041254 |
11.1.0.7.5, 11.2.0.1 |
ORA-19661 during RMAN restore check logical of compressed backup / IOT dummy key |
| 6760697 |
10.2.0.4.3, 10.2.0.5, 11.1.0.7, 11.2.0.1 |
DBMS_SPACE_ADMIN.ASSM_SEGMENT_VERIFY does not detect certain segment header block corruption |
| 6647480 |
10.2.0.4.4, 10.2.0.5, 11.1.0.7.3, 11.2.0.1 |
Corruption / OERI [kddummy_blkchk] .. [18021] with ASSM |
| 6134368 |
10.2.0.5, 11.2.0.1 |
ORA-1407 / block corruption from UPDATE .. RETURNING DML with trigger – SUPERCEDED |
| 6057203 |
10.2.0.4, 11.1.0.7, 11.2.0.1 |
Corruption with zero length column (ZLC) / OERI [kcbchg1_6] from Parallel update |
| 6653934 |
10.2.0.4.2, 10.2.0.5, 11.1.0.7 |
Dump / block corruption from ONLINE segment shrink with ROWDEPENDENCIES |
| 6674196 |
10.2.0.4, 10.2.0.5, 11.1.0.6 |
OERI / buffer cache corruption using ASM, OCFS or any ksfd client like ODM |
| 5599596 |
10.2.0.4, 11.1.0.6 |
Block corruption / OERI [kddummy_blkchk] on clustered or compressed tables |
| 5496041 |
10.2.0.4, 11.1.0.6 |
OERI[6006] / index corruption on compressed index |
| 5386204 |
10.2.0.4.1, 10.2.0.5, 11.1.0.6 |
Block corruption / OERI[kddummy_blkchk] after direct load of ASSM segment |
| 5363584 |
10.2.0.4, 11.1.0.6 |
Array insert into table can corrupt redo |
| 4602031 |
10.2.0.2, 11.1.0.6 |
Block corruption from UPDATE or MERGE into compressed table |
| 4493447 |
11.1.0.6 |
Spurious ORA-600 [kddummy_blkchk] [file#] [block#] [6145] on rollback of array update |
| 4329302 |
11.1.0.6 |
OERI [kddummy_blkchk] [file#] [block#] [6145] on rollback of update with logminer |
| 6075487 |
10.2.0.4 |
OERI[kddummy_blkchk]..[18020/18026] for DDL on plugged ASSM tablespace with FLASHBACK |
| 4054640 |
10.1.0.5, 10.2.0.1 |
Block corruption / OERI [kddummy_blkchk] at physical standby |
| 4000840 |
10.1.0.4, 10.2.0.1, 9.2.0.7 |
Update of a row with more than 255 columns can cause block corruption |
| 3772033 |
9.2.0.7, 10.1.0.4, 10.2.0.1 |
OERI[ktspfmb_create1] creating a LOB in ASSM using 2k blocksize |
