Oracle误删除数据文件恢复

Posted on 2024 年 02 月 20 日 by 惜分飞

有客户通过sftp误删除oracle数据文件,咨询我们是否可以恢复,通过远程上去检查,发现运气不错,数据库还没有crash,通过句柄找到被删除文件

oracle@cwgstestdb[testwctdb]/proc/20611/fd$ls -ltr
total 0
lr-x------ 1 oracle oinstall 64 Feb 20 14:03 9 -> /oracle/db19c/rdbms/mesg/oraus.msb
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 8 -> /oracle/db19c/dbs/lkTESTWCTDB
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 7 -> /oracle/db19c/dbs/hc_testwctdb.dat
lr-x------ 1 oracle oinstall 64 Feb 20 14:03 6 -> /var/lib/sss/mc/passwd
lr-x------ 1 oracle oinstall 64 Feb 20 14:03 5 -> /proc/20611/fd
lr-x------ 1 oracle oinstall 64 Feb 20 14:03 4 -> /oracle/db19c/rdbms/mesg/oraus.msb
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 305 -> /oradata/ftms_zx_test01_data8.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 304 -> /oradata/ftms_zx_test01_data7.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 303 -> /oradata/ftms_zx_test01_data6.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 302 -> '/oradata/ftms_zx_test01_data5.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 301 -> '/oradata/ftms_zx_test01_data4.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 300 -> '/oradata/ftms_zx_test01_data3.dbf (deleted)'
lr-x------ 1 oracle oinstall 64 Feb 20 14:03 3 -> /dev/null
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 299 -> '/oradata/ftms_zx_test01_data2.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 298 -> '/oradata/ftms_zx_test01_data1.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 297 -> '/oradata/ftms_zx_test01_data.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 296 -> /oradata/ftms_zx_test_data.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 295 -> '/oradata/TESTWCTDB/sd.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 294 -> /oradata/TESTWCTDB/ftms_cs3_jiamiceshi
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 293 -> /langchao/dumpdata/FTMS_CS_TDE.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 292 -> /oradata/ftms_zx_test01.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 291 -> /langchao/dumpdata/FTMS_CS_DATA4.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 290 -> '/oradata/ftms_zx_data5.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 289 -> /langchao/dumpdata/FTMS_CS_DATA3.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 288 -> /langchao/dumpdata/FTMS_CS_DATA2.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 287 -> /langchao/dumpdata/FTMS_JD_DATA2.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 286 -> '/oradata/LCBIPECDS _TEMP_DAT.DBF'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 285 -> '/oradata/rTB_MBFE_TEMP (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 284 -> '/oradata/TESTWCTDB/temp01.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 283 -> '/oradata/ftms_credit_data5.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 282 -> /oradata/ftmshtdata.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 281 -> '/oradata/dump_data/FTMS_CSBF_DATA.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 280 -> /langchao/dumpdata/FTMS_NEWBL2_DATA.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 279 -> /langchao/dumpdata/FTMS_CS_DATA.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 278 -> /oradata/LCBIPECDS_DAT.DBF
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 277 -> /oradata/rTB_MBFE
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 276 -> /oradata/udpcount_02.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 275 -> /oradata/udpcount_01.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 274 -> '/oradata/ftms_credit_data_6.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 273 -> /langchao/dumpdata/FTMS_JD_DATA.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 272 -> '/oradata/ftms_old.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 271 -> '/oradata/ftms_credit_data2.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 270 -> /langchao/dumpdata/PJDIP_DATA.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 269 -> '/oradata/ftms_credit_data.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 268 -> /langchao/dumpdata/FTMS_NEWBL_DATA.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 267 -> '/oradata/ftms_zx_data4.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 266 -> /langchao/dumpdata/QIANZHANG_DATA.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 265 -> '/oradata/ftms_zx_data3.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 264 -> '/oradata/ftms_zx_data2.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 263 -> '/oradata/ftms_zx_data.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 262 -> /langchao/dumpdata/FTMSDIP_DATA.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 261 -> /oradata/TESTWCTDB/users01.dbf
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 260 -> '/oradata/TESTWCTDB/undotbs01.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 259 -> '/oradata/TESTWCTDB/sysaux01.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 258 -> '/oradata/TESTWCTDB/system01.dbf (deleted)'
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 257 -> /oradata/TESTWCTDB/control02.ctl
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 256 -> /oradata/TESTWCTDB/control01.ctl
l-wx------ 1 oracle oinstall 64 Feb 20 14:03 2 -> /dev/null
lrwx------ 1 oracle oinstall 64 Feb 20 14:03 10 -> 'socket:[823411]'
l-wx------ 1 oracle oinstall 64 Feb 20 14:03 1 -> /dev/null
lr-x------ 1 oracle oinstall 64 Feb 20 14:03 0 -> /dev/null

查询数据文件大小（被删除的文件文件大小通过v$datafile查询为0）

SQL> select name,bytes/1024/1024/1024 from v$datafile;

NAME                                                                             BYTES/1024/1024/1024
-------------------------------------------------------------------------------- --------------------
/oradata/TESTWCTDB/system01.dbf                                                                     0
/oradata/TESTWCTDB/sysaux01.dbf                                                                     0
/oradata/TESTWCTDB/undotbs01.dbf                                                                    0
/oradata/TESTWCTDB/users01.dbf                                                             .004882813
/langchao/dumpdata/FTMSDIP_DATA.dbf                                                                 3
/oradata/ftms_zx_data.dbf                                                                           0
/oradata/ftms_zx_data2.dbf                                                                          0
/oradata/ftms_zx_data3.dbf                                                                          0
/langchao/dumpdata/QIANZHANG_DATA.dbf                                                               5
/oradata/ftms_zx_data4.dbf                                                                          0
/langchao/dumpdata/FTMS_NEWBL_DATA.dbf                                                             30
/oradata/ftms_credit_data.dbf                                                                       0
/langchao/dumpdata/PJDIP_DATA.dbf                                                                  20
/oradata/ftms_credit_data2.dbf                                                                      0
/oradata/ftms_old.dbf                                                                               0
/langchao/dumpdata/FTMS_JD_DATA.dbf                                                                15
/oradata/ftms_credit_data_6.dbf                                                                     0
/oradata/udpcount_01.dbf                                                                            5
/oradata/udpcount_02.dbf                                                                            5
/oradata/rTB_MBFE                                                                              .03125
/oradata/LCBIPECDS_DAT.DBF                                                                         .5
/langchao/dumpdata/FTMS_CS_DATA.dbf                                                                30
/langchao/dumpdata/FTMS_NEWBL2_DATA.dbf                                                            30
/oradata/dump_data/FTMS_CSBF_DATA.dbf                                                               0
/oradata/ftmshtdata.dbf                                                                    .087890625
/oradata/ftms_credit_data5.dbf                                                                      0
/langchao/dumpdata/FTMS_JD_DATA2.dbf                                                                3
/langchao/dumpdata/FTMS_CS_DATA2.dbf                                                       31.9999847
/langchao/dumpdata/FTMS_CS_DATA3.dbf                                                               10
/oradata/ftms_zx_data5.dbf                                                                          0
/langchao/dumpdata/FTMS_CS_DATA4.dbf                                                        12.109375
/oradata/ftms_zx_test01.dbf                                                                19.0527344
/langchao/dumpdata/FTMS_CS_TDE.dbf                                                                  1
/oradata/TESTWCTDB/ftms_cs3_jiamiceshi                                                     .029296875
/oradata/TESTWCTDB/sd.dbf                                                                           0
/oradata/ftms_zx_test_data.dbf                                                             .009765625
/oradata/ftms_zx_test01_data.dbf                                                                    0
/oradata/ftms_zx_test01_data1.dbf                                                                   0
/oradata/ftms_zx_test01_data2.dbf                                                                   0
/oradata/ftms_zx_test01_data3.dbf                                                                   0
/oradata/ftms_zx_test01_data4.dbf                                                                   0
/oradata/ftms_zx_test01_data5.dbf                                                                   0
/oradata/ftms_zx_test01_data6.dbf                                                          12.5976563
/oradata/ftms_zx_test01_data7.dbf                                                          9.08203125
/oradata/ftms_zx_test01_data8.dbf                                                                6.25

45 rows selected.

把数据文件拷贝回来

cp /proc/20611/fd/302   /langchao/orabak/
cp /proc/20611/fd/301   /langchao/orabak/
cp /proc/20611/fd/300   /langchao/orabak/
cp /proc/20611/fd/299   /langchao/orabak/
cp /proc/20611/fd/298   /langchao/orabak/
cp /proc/20611/fd/297   /langchao/orabak/
cp /proc/20611/fd/295   /langchao/orabak/
cp /proc/20611/fd/290   /langchao/orabak/
cp /proc/20611/fd/285   /langchao/orabak/
cp /proc/20611/fd/284   /langchao/orabak/
cp /proc/20611/fd/283   /langchao/orabak/
cp /proc/20611/fd/281   /langchao/orabak/
cp /proc/20611/fd/274   /langchao/orabak/
cp /proc/20611/fd/272   /langchao/orabak/
cp /proc/20611/fd/271   /langchao/orabak/
cp /proc/20611/fd/269   /langchao/orabak/
cp /proc/20611/fd/267   /langchao/orabak/
cp /proc/20611/fd/265   /langchao/orabak/
cp /proc/20611/fd/264   /langchao/orabak/
cp /proc/20611/fd/263   /langchao/orabak/
cp /proc/20611/fd/260   /langchao/orabak/
cp /proc/20611/fd/259   /langchao/orabak/
cp /proc/20611/fd/258   /langchao/orabak/

由于涉及system表空间数据文件被删除,无法在open情况下直接操作,直接关闭数据库,启动到mount状态,重命名数据文件路径,recover数据文件,open库,恢复完成
参考以前类似恢复:
Solaris rm datafile recovery—利用句柄误删除数据文件恢复
如果数据库已经关闭,需要考虑以下类似恢复方式:
dbca删除库和rm删库恢复
 记录一次rm -rf 删除数据文件异常恢复

ORA-01595/ORA-600 4194处理

Posted on 2024 年 02 月 05 日 by 惜分飞

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-01595/ORA-600 4194处理

又一个客户由于服务器断电，导致oracle无法正常启动报ORA-600 4194错误

Sat Jan 20 13:45:22 2024
ALTER DATABASE OPEN
Beginning crash recovery of 1 threads
 parallel recovery started with 32 processes
Started redo scan
Completed redo scan
 read 31 KB redo, 5 data blocks need recovery
Started redo application at
 Thread 1: logseq 9, block 3
Recovery of Online Redo Log: Thread 1 Group 3 Seq 9 Reading mem 0
  Mem# 0: D:\ORACLE\ORADATA\xff\REDO03.LOG
Completed redo application of 0.02MB
Completed crash recovery at
 Thread 1: logseq 9, block 66, scn 81924188
 5 data blocks read, 5 data blocks written, 31 redo k-bytes read
Thread 1 advanced to log sequence 10 (thread open)
Thread 1 opened at log sequence 10
  Current log# 1 seq# 10 mem# 0: D:\ORACLE\ORADATA\xff\REDO01.LOG
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
SMON: enabling cache recovery
Successfully onlined Undo Tablespace 2.
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is AL32UTF8
No Resource Manager plan active
Errors in file d:\oracle\diag\rdbms\xff\xff\trace\xff_smon_5164.trc  (incident=384222):
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
Incident details in: d:\oracle\diag\rdbms\xff\xff\incident\incdir_384222\xff_smon_5164_i384222.trc
Errors in file d:\oracle\diag\rdbms\xff\xff\trace\xff_ora_6832.trc  (incident=384270):
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
Incident details in: d:\oracle\diag\rdbms\xff\xff\incident\incdir_384270\xff_ora_6832_i384270.trc
Doing block recovery for file 3 block 207
Resuming block recovery (PMON) for file 3 block 207
Block recovery from logseq 10, block 64 to scn 81924393
Recovery of Online Redo Log: Thread 1 Group 1 Seq 10 Reading mem 0
  Mem# 0: D:\ORACLE\ORADATA\xff\REDO01.LOG
Block recovery stopped at EOT rba 10.66.16
Block recovery completed at rba 10.66.16, scn 0.81924391
Doing block recovery for file 3 block 160
Resuming block recovery (PMON) for file 3 block 160
Block recovery from logseq 10, block 64 to scn 81924389
Recovery of Online Redo Log: Thread 1 Group 1 Seq 10 Reading mem 0
  Mem# 0: D:\ORACLE\ORADATA\xff\REDO01.LOG
Block recovery completed at rba 10.64.16, scn 0.81924390
Errors in file d:\oracle\diag\rdbms\xff\xff\trace\xff_smon_5164.trc:
ORA-01595: 释放区 (2) 回退段 (3) 时出错
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
Trace dumping is performing id=[cdmp_20240120134525]
Doing block recovery for file 3 block 3857
Resuming block recovery (PMON) for file 3 block 3857
Block recovery from logseq 10, block 64 to scn 81924396
Recovery of Online Redo Log: Thread 1 Group 1 Seq 10 Reading mem 0
  Mem# 0: D:\ORACLE\ORADATA\xff\REDO01.LOG
Block recovery stopped at EOT rba 10.66.16
Block recovery completed at rba 10.66.16, scn 0.81924391
Doing block recovery for file 3 block 272
Resuming block recovery (PMON) for file 3 block 272
Block recovery from logseq 10, block 64 to scn 81924390
Recovery of Online Redo Log: Thread 1 Group 1 Seq 10 Reading mem 0
  Mem# 0: D:\ORACLE\ORADATA\xff\REDO01.LOG
Block recovery completed at rba 10.66.16, scn 0.81924391
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
Sat Jan 20 13:45:26 2024
QMNC started with pid=56, OS id=8052 
Exception [type: ACCESS_VIOLATION, UNABLE_TO_READ] [ADDR:0x66F8B134] [PC:0x93C7D88, kgegpa()+38]
Dump file d:\oracle\diag\rdbms\xff\xff\trace\alert_xff.log
Doing block recovery for file 3 block 3857
Resuming block recovery (PMON) for file 3 block 3857
Block recovery from logseq 10, block 64 to scn 81924396
Recovery of Online Redo Log: Thread 1 Group 1 Seq 10 Reading mem 0
  Mem# 0: D:\ORACLE\ORADATA\xff\REDO01.LOG
Block recovery completed at rba 10.66.16, scn 0.81924399
Errors in file d:\oracle\diag\rdbms\xff\xff\trace\xff_mmon_7876.trc  (incident=396089):
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
ORA-06512: 在 "SYS.DBMS_HA_ALERTS_PRVT", line 310
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], 
Doing block recovery for file 3 block 3857
Resuming block recovery (PMON) for file 3 block 3857
Block recovery from logseq 10, block 64 to scn 81924396
Recovery of Online Redo Log: Thread 1 Group 1 Seq 10 Reading mem 0
  Mem# 0: D:\ORACLE\ORADATA\xff\REDO01.LOG
Block recovery completed at rba 10.66.16, scn 0.81924399
Errors in file d:\oracle\diag\rdbms\xff\xff\trace\xff_pmon_8876.trc  (incident=384102):
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
Sat Jan 20 13:45:39 2024
Errors in file d:\oracle\diag\rdbms\xff\xff\trace\xff_pmon_8876.trc:
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
PMON (ospid: 8876): terminating the instance due to error 472
Instance terminated by PMON, pid = 8876

这个故障比较简单,由于undo回滚异常导致,对异常的undo进行处理,数据库正常open,重建undo表空间导出数据,完成本次恢复工作

从ORA-00283 ORA-16433报错开始恢复

Posted on 2024 年 02 月 03 日 by 惜分飞

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：从ORA-00283 ORA-16433报错开始恢复

接手一个客户无法正常启动的故障数据库，尝试recover 报ORA-00283 ORA-16433错误

[oracle@xff trace]$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.2.0 Production on Sat Jan 27 04:46:23 2024

Copyright (c) 1982, 2014, Oracle.  All rights reserved.


???:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> show pdbs;
SQL> select open_mode from v$database;

OPEN_MODE
--------------------
MOUNTED

SQL>
SQL> recover database;
ORA-00283: recovery session canceled due to errors
ORA-16433: The database must be opened in read/write mode

通过对控制文件进行处理,再次尝试recover库

SQL> recover database;
ORA-00399: corrupt change description in redo log
ORA-00353: log corruption near block 134877 change 3249721295 time 01/27/2024 00:21:05
ORA-00312: online log 1 thread 1:'/u01/app/oracle/oradata/xff/redo01.log'

由于redo和数据文件不匹配,无法正常recover库,尝试强制打开库报ORA-600 2662错误

SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [2662], [0], [3249721308], [0],[3249730440], [16777344],[],[],[],[],[],[]
ORA-00600: internal error code, arguments: [2662], [0], [3249721307], [0],[3249730440], [16777344],[],[],[],[],[],[]
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [2662], [0], [3249721303], [0],[3249730440], [16777344],[],[],[],[],[],[]
Process ID: 117336
Session ID: 1146 Serial number: 11764

基于这种错误,尝试oradebug修改scn

SQL> oradebug setmypid
oradebug DUMPvar SGA kcsgscn_
Statement processed.
SQL> kcslf kcsgscn_ [06001FBB0, 06001FBE0) = 00000000 00000000 00000000 00000000 00000000 
SQL> oradebug poke 0x06001FBB0 4 0x10000000
oradebug DUMPvar SGA kcsgscn_
ORA-32521: error parsing ORADEBUG command:

发现报ORA-32521错误,证明常规的oradebug方法无法修改scn,参考相关文章:
oradebug poke ORA-32521/ORA-32519故障解决
第一次通过其他方法处理,由于计算失误导致数据库启动报ORA-600 2252错误

SQL> ALTER DATABASE OPEN RESETLOGS;
ALTER DATABASE OPEN RESETLOGS
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [2252], [45264], [0], [11641],[3340959744], [],[],[],[],[],[]

该错误是相关文章参考:
记录一次ORA-00600[2252]故障解决
 ORA-00600: internal error code, arguments: [2252], [3987]
主机断电系统回到N年前数据库报ORA-600 kcm_headroom_warn_1错误
处理正确的scn值之后,数据库open成功,然后逻辑方式导出数据,恢复工作完成

SQL> alter database open ;

Database altered.

近期又遇到ORA-600 16703和ORA-702故障

Posted on 2024 年 01 月 30 日 by 惜分飞

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：近期又遇到ORA-600 16703和ORA-702故障

比较常见的由于软件注入恶意代码或者被人恶意破坏的常见故障ORA-600 16703和ORA-702故障

上面的故障是由于bootstrap$表被删除导致，以前处理过类似case:
恶意删除bootstrap$导致数据库无法正常启动
恶意删除bootstrap$导致数据库无法正常启动
2019恢复第一单—ORA-704-ORA-702恢复
ORA-00702: bootstrap verison ” inconsistent with version ’8.0.0.0.0′

该故障的原因是一般是由于在互联网上下载了带恶意脚本的oracle安装介质,300天之后数据库重启会导致tab$被删除,参见:
ORA-600 16703故障解析—tab$表被清空
警告：互联网中有oracle介质被注入恶意程序导致—ORA-600 16703
再次提醒oracle使用者:
1. 不要使用从第三方网站下载非官方的数据库访问软件（发生过被注入恶意脚本的plsql dev案例:plsql dev引起的数据库被黑勒索比特币实现原理分析和解决方案）
2. 从官方下载oracle 安装程序和补丁程序（防止被注入恶意脚本,参考:警告：互联网中有oracle介质被注入恶意程序导致—ORA-600 16703）

ORA-01033: ORACLE initialization or shutdown in progress 故障处理

Posted on 2024 年 01 月 22 日 by 惜分飞

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-01033: ORACLE initialization or shutdown in progress 故障处理

客户反馈数据库使用plsql dev登录报ORA-01033: ORACLE initialization or shutdown in progress的错误

出现该错误一般是由于数据库没有正常open成功,查看oracle 告警日志发现

Mon Jan 22 16:55:50 2024
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: alter database mount exclusive
alter database open
Beginning crash recovery of 1 threads
 parallel recovery started with 15 processes
Started redo scan
Completed redo scan
 read 139 KB redo, 70 data blocks need recovery
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_7792.trc  (incident=20565):
ORA-00600: ??????, ??: [kcratr_nab_less_than_odr], [1], [1916], [28210], [28222], [], [], [], [], [], [], []
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_20565\orcl_ora_7792_i20565.trc
Mon Jan 22 16:55:57 2024
Trace dumping is performing id=[cdmp_20240122165557]
Aborting crash recovery due to error 600
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_7792.trc:
ORA-00600: ??????, ??: [kcratr_nab_less_than_odr], [1], [1916], [28210], [28222], [], [], [], [], [], [], []
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_7792.trc:
ORA-00600: ??????, ??: [kcratr_nab_less_than_odr], [1], [1916], [28210], [28222], [], [], [], [], [], [], []
ORA-600 signalled during: alter database open...

这个错误比较常见,解决起来比较简单,参考:
kcratr_nab_less_than_odr
12c启动报kcratr_nab_less_than_odr
又一例ORA-600 kcratr_nab_less_than_odr
在恢复过程中中还遇到了ORA-00700 kcrf_split_brain_error错误,但是没有影响数据库open

Mon Jan 22 20:13:55 2024
alter database open
Beginning crash recovery of 1 threads
Started redo application at
 Thread 1: logseq 1916, block 27931
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1916 Reading mem 0
  Mem# 0: D:\TEMP\ORCL\REDO02.LOG
Completed redo application of 0.00MB
Completed crash recovery at
 Thread 1: logseq 1916, block 28210, scn 43957072
 0 data blocks read, 0 data blocks written, 139 redo k-bytes read
Errors in file d:\app\xifenfei\diag\rdbms\orcl\orcl\trace\orcl_ora_6104.trc  (incident=15729):
ORA-00700: 软内部错误, 参数: [kcrf_split_brain_error], [1], [1916], [28222], [28209], [4], [], [], [], [], [], []
Incident details in: d:\app\xifenfei\diag\rdbms\orcl\orcl\incident\incdir_15729\orcl_ora_6104_i15729.trc
Mon Jan 22 20:13:56 2024
Trace dumping is performing id=[cdmp_20240122201356]
Mon Jan 22 20:13:56 2024
Thread 1 advanced to log sequence 1917 (thread open)
Thread 1 opened at log sequence 1917
  Current log# 3 seq# 1917 mem# 0: D:\TEMP\ORCL\REDO03.LOG
Successful open of redo thread 1
Mon Jan 22 20:13:56 2024
SMON: enabling cache recovery
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is AL32UTF8
replication_dependency_tracking turned off (no async multimaster replication found)
WARNING: AQ_TM_PROCESSES is set to 0. System operation                     might be adversely affected.
Completed: alter database open

至此数据库open成功但是dbv检测system有很多坏块需要分析处理

C:\Users\XIFENFEI>dbv file=d:/temp/orcl/system01.dbf

DBVERIFY: Release 11.2.0.1.0 - Production on 星期一 1月 22 21:07:18 2024

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - 开始验证: FILE = D:\TEMP\ORCL\SYSTEM01.DBF
页 106278 流入 - 很可能是介质损坏
Corrupt block relative dba: 0x00419f26 (file 1, block 106278)
Fractured block found during dbv:
Data in bad block:
 type: 6 format: 2 rdba: 0x00419f26
 last change scn: 0x0000.01410f78 seq: 0x2 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x00000000
 check value in block header: 0xbf11
 computed block checksum: 0xaf18

页 106279 标记为损坏
Corrupt block relative dba: 0x00419f27 (file 1, block 106279)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 106280 标记为损坏
Corrupt block relative dba: 0x00419f28 (file 1, block 106280)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 106281 标记为损坏
Corrupt block relative dba: 0x00419f29 (file 1, block 106281)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 106282 标记为损坏
Corrupt block relative dba: 0x00419f2a (file 1, block 106282)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 106283 标记为损坏
Corrupt block relative dba: 0x00419f2b (file 1, block 106283)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 106284 标记为损坏
Corrupt block relative dba: 0x00419f2c (file 1, block 106284)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 106285 标记为损坏
Corrupt block relative dba: 0x00419f2d (file 1, block 106285)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 106286 标记为损坏
Corrupt block relative dba: 0x00419f2e (file 1, block 106286)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x2c310602
 check value in block header: 0xbbb2
 block checksum disabled

页 143094 流入 - 很可能是介质损坏
Corrupt block relative dba: 0x00422ef6 (file 1, block 143094)
Fractured block found during dbv:
Data in bad block:
 type: 6 format: 2 rdba: 0x00422ef6
 last change scn: 0x0000.028f863b seq: 0x2 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x00000000
 check value in block header: 0xda23
 computed block checksum: 0x4210

页 143095 标记为损坏
Corrupt block relative dba: 0x00422ef7 (file 1, block 143095)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 143096 标记为损坏
Corrupt block relative dba: 0x00422ef8 (file 1, block 143096)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 143097 标记为损坏
Corrupt block relative dba: 0x00422ef9 (file 1, block 143097)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 143098 标记为损坏
Corrupt block relative dba: 0x00422efa (file 1, block 143098)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 143099 标记为损坏
Corrupt block relative dba: 0x00422efb (file 1, block 143099)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 143100 标记为损坏
Corrupt block relative dba: 0x00422efc (file 1, block 143100)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 143101 标记为损坏
Corrupt block relative dba: 0x00422efd (file 1, block 143101)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x00000000
 check value in block header: 0xbbb2
 block checksum disabled

页 143102 标记为损坏
Corrupt block relative dba: 0x00422efe (file 1, block 143102)
Bad header found during dbv:
Data in bad block:
 type: 178 format: 3 rdba: 0xc8c9c1b6
 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00
 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf
 consistency value in tail: 0x8a780602
 check value in block header: 0xbbb2
 block checksum disabled



DBVERIFY - 验证完成

检查的页总数: 152320
处理的页总数 (数据): 115189
失败的页总数 (数据): 0
处理的页总数 (索引): 13086
失败的页总数 (索引): 0
处理的页总数 (其他): 9741
处理的总页数 (段)  : 1
失败的总页数 (段)  : 0
空的页总数: 14286
标记为损坏的总页数: 18
流入的页总数: 2
加密的总页数        : 0
最高块 SCN            : 44036082 (0.44036082)

通过分析aud$的extent,确认这些坏块全部属于该对象

SQL> select block_id,blocks from dba_extents where segment_name='AUD$';

  BLOCK_ID     BLOCKS
---------- ----------
…………
    102016       1024
    103040       1024
    104064       1024
    105088       1024
    106112       1024
…………
    141056       1024
    142080       1024
    143104       1024

已选择124行。

处理方法比较简单,直接truncate aud$表即可