断电引起的ORA-08102: 未找到索引关键字, 对象号 39故障处理

Posted on 2024 年 12 月 21 日 by 惜分飞

最近有客户在虚拟化平台运行oracle,由于机房掉电,导致oracle数据库无法正常启动,通过第三方恢复,oracle被强制拉起,但是无法进行ddl操作,比如创建表报ORA-08102: 未找到索引关键字, 对象号 39, 文件 1, 块 122448 (2) 错误

通过obj#确认具体对象为i_obj#(也就是obj$对象上的一个index)

由于这类对象属于数据库的底层核心对象,无法直接rebulid他们,根据以往经验,可以通过bbed对其进行修复,或者参考类似文章进行重建:
分享I_OBJ4 ORA-8102故障恢复案例
使用bbed 修复I_OBJ4 index 报ORA-8102错误
通过bbed修改obj$中dataobj$重现I_OBJ4索引报ORA-08102错误
bootstrap$核心index(I_OBJ1,I_USER1,I_FILE#_BLOCK#,I_IND1,I_TS#,I_CDEF1等)异常恢复—ORA-00701错误解决
这个问题解决之后,该客户还有另外一个问题需要解决（不然数据库运行一段时间之后就会crash）

Wed Dec 18 09:13:03 2024
Errors in file E:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_smon_2536.trc  (incident=1105672):
ORA-00600: 内部错误代码, 参数: [13013], [5001], [268], [8459081], [1], [8459081], [3], [], [], [], [], []
Incident details in: E:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\incident\incdir_1105672\orcl_smon_2536_i1105672.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Non-fatal internal error happenned while SMON was doing logging scn->time mapping.
SMON encountered 1 out of maximum 100 non-fatal internal errors.

这个问题本质就是SMON_SCN_TIME表的异常导致(一般13013是由于表和index不一致导致),对于这类问题处理，参考:
关于SMON_SCN_TIME若干问题说明
处理完上述两个明显故障之后,然后使用expdp不落地方式把客户数据迁移到新库,完成本次恢复任务

ORA-00227: corrupt block detected in control file

Posted on 2024 年 12 月 18 日 by 惜分飞

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-00227: corrupt block detected in control file

由于服务器断电，导致oracle数据库无法正常启动，recover报ORA-00353 ORA-00312错

SQL> recover datafile 1;
ORA-00283: recovery session canceled due to errors
ORA-00368: checksum error in redo log block
ORA-00353: log corruption near block 8 change 237896529 time 12/03/2024 22:03:32
ORA-00312: online log 1 thread 1: 'E:\ORADATA\xifenfei\REDO01.LOG'

报错提示比较明显,是由于oracle恢复需要的redo损坏,导致无法进行正常恢复,这种情况下,只能尝试强制打开库

SQL> recover database until cancel;
ORA-00279: change 237857808 generated at 12/03/2024 07:06:31 needed for thread 1
ORA-00289: suggestion : C:\ORACLE\PRODUCT\10.2.0.3\DB_1\RDBMS\ARC20892_0929553713.001
ORA-00280: change 237857808 for thread 1 is in sequence #20892


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
cancel
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1: 'E:\ORADATA\xifenfei\SYSTEM01.DBF'


ORA-01112: media recovery not started


SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-01092: ORACLE instance terminated. Disconnection forced

对于这种ORA-01092错误,需要查看alert日志确认具体报错原因

Tue Dec 17 22:03:59 2024
alter database open resetlogs
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 237857808
Resetting resetlogs activation ID 2015910641 (0x78285af1)
Tue Dec 17 22:04:00 2024
Setting recovery target incarnation to 2
Tue Dec 17 22:04:00 2024
Advancing SCN to 16106127360 according to _minimum_giga_scn
Tue Dec 17 22:04:00 2024
Assigning activation ID 2274407914 (0x8790b5ea)
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: E:\ORADATA\xifenfei\REDO01.LOG
Successful open of redo thread 1
Tue Dec 17 22:04:01 2024
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Tue Dec 17 22:04:01 2024
SMON: enabling cache recovery
Tue Dec 17 22:04:01 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\udump\xifenfei_ora_816.trc:
ORA-00704: bootstrap process failure
ORA-39700: database must be opened with UPGRADE option

Tue Dec 17 22:04:01 2024
Error 704 happened during db open, shutting down database
USER: terminating instance due to error 704
Tue Dec 17 22:04:01 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_pmon_2472.trc:
ORA-00704: bootstrap process failure

Tue Dec 17 22:04:01 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_reco_2576.trc:
ORA-00704: bootstrap process failure

Tue Dec 17 22:04:01 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_smon_2584.trc:
ORA-00704: bootstrap process failure

Tue Dec 17 22:04:02 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_ckpt_2216.trc:
ORA-00704: bootstrap process failure

Tue Dec 17 22:04:02 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_lgwr_1556.trc:
ORA-00704: bootstrap process failure

Tue Dec 17 22:04:02 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_dbw0_1528.trc:
ORA-00704: bootstrap process failure

Tue Dec 17 22:04:02 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_psp0_2896.trc:
ORA-00704: bootstrap process failure

Tue Dec 17 22:04:02 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_mman_904.trc:
ORA-00704: bootstrap process failure

Tue Dec 17 22:04:02 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_dbw1_1732.trc:
ORA-00704: bootstrap process failure

Instance terminated by USER, pid = 816
ORA-1092 signalled during: alter database open resetlogs...

由于对oracle粗心对于oracle版本判断失误,导致打开数据库失败,使用正确版本打开数据库发现ctl有报错,导致打开依旧失败(这种错误一般比较少见，大部分ctl异常都是在oracle mount状态无法成功)

Tue Dec 17 22:10:48 2024
Recovery of Online Redo Log: Thread 1 Group 1 Seq 1 Reading mem 0
  Mem# 0: E:\ORADATA\xifenfei\REDO01.LOG
Tue Dec 17 22:10:48 2024
Completed redo application
Tue Dec 17 22:10:48 2024
Completed crash recovery at
 Thread 1: logseq 1, block 3, scn 16106147363
 0 data blocks read, 0 data blocks written, 1 redo blocks read
Tue Dec 17 22:10:48 2024
Read from controlfile member 'E:\ORADATA\xifenfei\CONTROL01.CTL' has found a corrupted block (blk# 432, seq# 132194)
Hex dump of (file 0, block 432) in trace file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_lgwr_2728.trc
Corrupt block relative dba: 0x000001b0 (file 0, block 432)
Fractured block found during control file block read
Data in bad block:
 type: 0 format: 0 rdba: 0x00000000
 last change scn: 0x0000.00000000 seq: 0x0 flg: 0x00
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x03e71501
 check value in block header: 0x0
 block checksum disabled
Hex dump of (file 0, block 432) in trace file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_lgwr_2728.trc
Corrupt block relative dba: 0x000001b0 (file 0, block 432)
Fractured block found during control file block read
Data in bad block:
 type: 0 format: 0 rdba: 0x00000000
 last change scn: 0x0000.00000000 seq: 0x0 flg: 0x00
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x03e71501
 check value in block header: 0x0
 block checksum disabled
Tue Dec 17 22:10:48 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_lgwr_2728.trc:
ORA-00202: control file: 'E:\ORADATA\xifenfei\CONTROL01.CTL'

Tue Dec 17 22:10:48 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_lgwr_2728.trc:
ORA-00227: corrupt block detected in control file: (block 432, # blocks 1)
ORA-00202: control file: 'E:\ORADATA\xifenfei\CONTROL01.CTL'

LGWR: terminating instance due to error 227
Tue Dec 17 22:10:48 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_pmon_2056.trc:
ORA-00227: corrupt block detected in control file: (block , # blocks )

Tue Dec 17 22:10:49 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_dbw0_1636.trc:
ORA-00227: corrupt block detected in control file: (block , # blocks )

Tue Dec 17 22:10:49 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_reco_2468.trc:
ORA-00227: corrupt block detected in control file: (block , # blocks )

Tue Dec 17 22:10:49 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_smon_2996.trc:
ORA-00227: corrupt block detected in control file: (block , # blocks )

Tue Dec 17 22:10:49 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_ckpt_2292.trc:
ORA-00227: corrupt block detected in control file: (block , # blocks )

Tue Dec 17 22:10:49 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_dbw1_508.trc:
ORA-00227: corrupt block detected in control file: (block , # blocks )

Tue Dec 17 22:10:49 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_mman_1728.trc:
ORA-00227: corrupt block detected in control file: (block , # blocks )

Tue Dec 17 22:10:49 2024
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_psp0_1724.trc:
ORA-00227: corrupt block detected in control file: (block , # blocks )

Instance terminated by LGWR, pid = 2728

重建ctl,打开数据库成功,导出数据,完成本次恢复任务

手工删除19c rac

Posted on 2024 年 12 月 15 日 by 惜分飞

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：手工删除19c rac

在某些时候,需要删除掉手工删除19c RAC,重新安装,以下是比较简便的操作(root用户操作)

source /home/grid/.bash_profile
crsctl stop crs
systemctl disable oracle-ohasd
systemctl stop oracle-ohasd
systemctl disable oracle-tfa.service
systemctl stop oracle-tfa

rm -rf /etc/oracle
rm -rf /etc/ora*
rm -rf /u01
rm -rf /tmp/CVU*
rm -rf /tmp/.oracle
rm -rf /var/tmp/.oracle
rm -f /etc/init.d/init.ohasd
rm -f /etc/systemd/system/oracle-ohasd.service
rm -f /etc/systemd/system/oracle-tfa.service

dd if=/dev/zero of=/dev/asm_ocr1 bs=1024 count=1
dd if=/dev/zero of=/dev/asm_ocr2 bs=1024 count=1
dd if=/dev/zero of=/dev/asm_ocr3 bs=1024 count=1

解决oracle数据文件路径有回车故障

Posted on 2024 年 12 月 14 日 by 惜分飞

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：解决oracle数据文件路径有回车故障

最近遇到一个硬件恢复朋友的请求,oracle数据库文件恢复出来了,但是在linux上面启动的时候,有两个文件无法检测到,dbv检测正常.

通过分析是由于文件无法找到原因导致

进一步检查发现原库这两个文件结尾带有回车,但是恢复出来的文件不带回车

对于这个故障,我在测试环境进行了重现并且给予解决
1. 创建带回车键数据文件

SQL> create tablespace xifenfei datafile '/u01/app/oracle/oradata/xifenfei/xff01.dbf
  2  ' size 128m;

Tablespace created.

SQL> alter tablespace xifenfei add datafile '/u01/app/oracle/oradata/xifenfei/xff02.dbf' size 128M;

Tablespace altered.

SQL> select name from v$datafile;

NAME
--------------------------------------------------------------------------------
/u01/app/oracle/oradata/xifenfei/system01.dbf
/u01/app/oracle/oradata/xifenfei/sysaux01.dbf
/u01/app/oracle/oradata/xifenfei/undotbs01.dbf
/u01/app/oracle/oradata/xifenfei/users01.dbf
/u01/app/oracle/oradata/xifenfei/xff01.dbf
/u01/app/oracle/oradata/xifenfei/xff02.dbf

6 rows selected.

2.操作系统层面查看文件（在我的ssh工具中,可以看到带回车键文件和不带回车文件不一样,使用的是crt工具,其他工具是否显示不确定）

[oracle@xifenfei ~]$ cd /u01/app/oracle/oradata/xifenfei/
[oracle@xifenfei xifenfei]$ ls -l xff*
-rw-r----- 1 oracle oinstall 134225920 Dec 14 08:05 xff01.dbf?
-rw-r----- 1 oracle oinstall 134225920 Dec 14 08:05 xff02.dbf

3. 操作系统层面重命名数据文件

[oracle@xifenfei xifenfei]$ mv xff01.dbf* xff01.dbf
[oracle@xifenfei xifenfei]$ ls -l xff*
-rw-r----- 1 oracle oinstall 134225920 Dec 14 08:05 xff01.dbf
-rw-r----- 1 oracle oinstall 134225920 Dec 14 08:05 xff02.dbf

3. 数据库层面重启看文件情况,发现文件不能被正常发现(当然不能,文件被os层面mv了)

SQL> shutdown abort;
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.

Total System Global Area  551165952 bytes
Fixed Size                  2255112 bytes
Variable Size             369100536 bytes
Database Buffers          171966464 bytes
Redo Buffers                7843840 bytes
Database mounted.
SQL> select file#, CHECKPOINT_CHANGE# from v$datafile_header;

     FILE# CHECKPOINT_CHANGE#
---------- ------------------
         1          306775013
         2          306775013
         3          306775013
         4          306775013
         5                  0
         6          306779423

6 rows selected.

RMAN> report schema;

Report of database schema for database with db_unique_name XIFENFEI

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    770      SYSTEM               ***     /u01/app/oracle/oradata/xifenfei/system01.dbf
2    1950     SYSAUX               ***     /u01/app/oracle/oradata/xifenfei/sysaux01.dbf
3    70       UNDOTBS1             ***     /u01/app/oracle/oradata/xifenfei/undotbs01.dbf
4    12       USERS                ***     /u01/app/oracle/oradata/xifenfei/users01.dbf
5    0        XIFENFEI             ***     /u01/app/oracle/oradata/xifenfei/xff01.dbf

6    128      XIFENFEI             ***     /u01/app/oracle/oradata/xifenfei/xff02.dbf

4. 解决控制文件和数据文件实际名称不一致问题

RMAN> catalog datafilecopy '/u01/app/oracle/oradata/xifenfei/xff01.dbf';

using target database control file instead of recovery catalog
cataloged datafile copy
datafile copy file name=/u01/app/oracle/oradata/xifenfei/xff01.dbf RECID=1 STAMP=1187684217

RMAN> switch datafile 5 to copy;

datafile 5 switched to datafile copy "/u01/app/oracle/oradata/xifenfei/xff01.dbf"

RMAN> report schema;

Report of database schema for database with db_unique_name XIFENFEI

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    770      SYSTEM               ***     /u01/app/oracle/oradata/xifenfei/system01.dbf
2    1950     SYSAUX               ***     /u01/app/oracle/oradata/xifenfei/sysaux01.dbf
3    70       UNDOTBS1             ***     /u01/app/oracle/oradata/xifenfei/undotbs01.dbf
4    12       USERS                ***     /u01/app/oracle/oradata/xifenfei/users01.dbf
5    128      XIFENFEI             ***     /u01/app/oracle/oradata/xifenfei/xff01.dbf
6    128      XIFENFEI             ***     /u01/app/oracle/oradata/xifenfei/xff02.dbf

List of Temporary Files
=======================
File Size(MB) Tablespace           Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1    123      TEMP                 32767       /u01/app/oracle/oradata/xifenfei/temp01.dbf


RMAN> alter database open;

database opened

Oracle Recovery Tools工具一键解决ORA-00376 ORA-01110故障（文件offline）

Posted on 2024 年 12 月 07 日 by 惜分飞

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：Oracle Recovery Tools工具一键解决ORA-00376 ORA-01110故障（文件offline）

客户在win上面迁移数据文件,由于原库非归档,结果导致有两个文件scn不一致,无法打开库,结果他们选择offline文件,然后打开数据库

Wed Dec 04 14:06:04 2024
alter database open
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_6056.trc:
ORA-01113: 文件 10 需要介质恢复
ORA-01110: 数据文件 10: 'C:\PROGRAM FILES\ORACLE\XFF1.DBF'
ORA-1113 signalled during: alter database open...
Wed Dec 04 14:08:18 2024
alter database datafile 'c:\program files\oracle\XFF1.dbf' offline drop
Completed: alter database datafile 'c:\program files\oracle\XFF1.dbf' offline drop
Wed Dec 04 14:08:31 2024
alter database open
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_6056.trc:
ORA-01113: 文件 26 需要介质恢复
ORA-01110: 数据文件 26: 'C:\PROGRAM FILES\ORACLE\XFF2.DBF'
ORA-1113 signalled during: alter database open...
Wed Dec 04 14:08:31 2024
Checker run found 1 new persistent data failures
Wed Dec 04 14:08:51 2024
alter database datafile 'c:\program files\oracle\XFF2.dbf' offline drop
Completed: alter database datafile 'c:\program files\oracle\XFF2.dbf' offline drop
alter database open
Wed Dec 04 14:08:57 2024
Thread 1 opened at log sequence 136210
  Current log# 1 seq# 136210 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO01.LOG
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Dec 04 14:08:57 2024
SMON: enabling cache recovery
Successfully onlined Undo Tablespace 2.
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is AL32UTF8
No Resource Manager plan active
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
Wed Dec 04 14:08:59 2024
QMNC started with pid=20, OS id=4264 
Completed: alter database open

后面自行尝试recover 数据文件没有成功

Wed Dec 04 14:42:50 2024
ALTER DATABASE RECOVER  datafile 26  
Media Recovery Start
Serial Media Recovery started
ORA-279 signalled during: ALTER DATABASE RECOVER  datafile 26  ...
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Media Recovery Log D:\APP\ADMINISTRATOR\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2024_12_04\O1_MF_1_135983_%U_.ARC
Errors with log D:\APP\ADMINISTRATOR\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2024_12_04\O1_MF_1_135983_%U_.ARC
ORA-308 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
ALTER DATABASE RECOVER CANCEL 
Media Recovery Canceled
Completed: ALTER DATABASE RECOVER CANCEL

由于这两个文件处于offline状态导致客户很多操作报ORA-00376 ORA-01110之类错

ORA-00376: file 10 cannot be read at this time
ORA-01110: data file 10: 'C:\PROGRAM FILES\ORACLE\XFF1.DBF'

对于这类故障使用Oracle Recovery Tools工具,一键恢复

然后直接recover 数据文件成功

对于这类缺少归档数据文件offline的故障Oracle Recovery Tools可以快速傻瓜式恢复
软件下载:OraRecovery下载
使用说明:使用说明

13429648788

qq咨询

专业Oracle数据库恢复技术支持

专业Oracle数据库恢复，或许是您恢复数据的最后机会@phone:17813235971

Category Archives: Oracle

断电引起的ORA-08102: 未找到索引关键字, 对象号 39故障处理

ORA-00227: corrupt block detected in control file

手工删除19c rac

解决oracle数据文件路径有回车故障

Oracle Recovery Tools工具一键解决ORA-00376 ORA-01110故障（文件offline）

2025 年七月
S	M	T	W	T	F	S
« Jun
		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31