忘记执行end backup命令数据库恢复

遇到两次begin backup忘记end backup导致的悲剧.虽然不是自己亲身经历,但是感触很深,这里做了一个小实验,说明在begin backup后忘记end backup,而又丢失了备份归档日志,且数据库异常重启的事故恢复(这里为了加大实验难道,并且使用begin backup命令后的热备文件恢复)
模拟begin end

SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            /u01/oracle/oradata/xifenfei/archive
Oldest online log sequence     37
Next log sequence to archive   39
Current log sequence           39
SQL> alter tablespace bbed begin backup;

Tablespace altered.

SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            /u01/oracle/oradata/xifenfei/archive
Oldest online log sequence     37
Next log sequence to archive   39
Current log sequence           39
SQL> drop table chf.t_xff;

Table dropped.

SQL> create table chf.t_xff
  2    as
  3  select * from dba_objects;

Table created.

SQL> alter system switch logfile;

System altered.

SQL>   delete from chf.t_XFF;

30811 rows deleted.

SQL>   commit;

Commit complete.

SQL>   alter system  switch logfile;

System altered.

SQL> /

System altered.

SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            /u01/oracle/oradata/xifenfei/archive
Oldest online log sequence     40
Next log sequence to archive   42
Current log sequence           42

cp备份文件

[oracle@xifenfei xifenfei]$ cp bbed01.dbf bbed01.dbf_05
[oracle@xifenfei xifenfei]$ cp bbed02.dbf bbed02.dbf_05

继续操作数据库

SQL> alter system switch logfile;

System altered.

SQL>   insert into chf.t_xff
  2    select * from dba_objects;

30811 rows created.

SQL> commit;

Commit complete.

SQL>  archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            /u01/oracle/oradata/xifenfei/archive
Oldest online log sequence     41
Next log sequence to archive   43
Current log sequence           43
SQL> alter system switch logfile;

System altered.

模拟异常关闭数据库

SQL> shutdown immediate;
ORA-01149: cannot shutdown - file 11 has online backup set
ORA-01110: data file 11: '/u01/oracle/oradata/xifenfei/bbed01.dbf'
SQL> shutdown abort;
ORACLE instance shut down.

删除部分归档日志(模拟归档日志丢失)

[oracle@xifenfei archive]$ mv 1_39.dbf 1_39.dbf_bak
[oracle@xifenfei archive]$ mv 1_40.dbf 1_40.dbf_bak

启动数据库

[oracle@xifenfei xifenfei]$ sqlplus "/ as sysdba"

SQL*Plus: Release 9.2.0.4.0 - Production on Tue Jun 5 03:02:56 2012

Copyright (c) 1982, 2002, Oracle Corporation.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area  353441008 bytes
Fixed Size                   451824 bytes
Variable Size             184549376 bytes
Database Buffers          167772160 bytes
Redo Buffers                 667648 bytes
Database mounted.
ORA-01113: file 11 needs media recovery
ORA-01110: data file 11: '/u01/oracle/oradata/xifenfei/bbed01.dbf'

分析相关SCN

SQL> select file#,online_status "STATUS",to_char(change#,'9999999999999999') "SCN",
  2  To_char(time,'yyyy-mm-dd hh24:mi:ss')"TIME" from v$recover_file;

     FILE# STATUS  SCN               TIME
---------- ------- ----------------- -------------------
        11 ONLINE     12286828683164 2012-06-05 02:55:43
        12 ONLINE     12286828683164 2012-06-05 02:55:43

SQL> select file#,to_char(checkpoint_change#,'999999999999999') "SCN",
  2  to_char(last_change#,'999999999999999')"STOP_SCN" from v$datafile;

     FILE# SCN              STOP_SCN
---------- ---------------- ----------------
         1   12286828684636
         2   12286828684636
         3   12286828684636
         4   12286828684636
         5   12286828684636
         6   12286828684636
         7   12286828684636
         8   12286828684636
         9   12286828684636
        10   12286828684636
        11   12286828683164
        12   12286828683164

12 rows selected.

SQL> select file#,to_char(checkpoint_change#,'9999999999999999') "SCN",
  2  to_char(RESETLOGS_CHANGE#,'9999999999999999') "RESETLOGS SCN"
  3  from v$datafile_header;

     FILE# SCN               RESETLOGS SCN
---------- ----------------- -----------------
         1    12286828684636            174968
         2    12286828684636            174968
         3    12286828684636            174968
         4    12286828684636            174968
         5    12286828684636            174968
         6    12286828684636            174968
         7    12286828684636            174968
         8    12286828684636            174968
         9    12286828684636            174968
        10    12286828684636            174968
        11    12286828683164            174968
        12    12286828683164            174968

12 rows selected.

SQL> select file#,to_char(CHANGE#,'9999999999999999') "SCN",
  2  to_char(TIME,'yyyy-mm-dd hh24:mi:ss') "TIME" from v$backup;

     FILE# SCN               TIME
---------- ----------------- -------------------
         1                 0
         2                 0
         3                 0
         4                 0
         5                 0
         6                 0
         7                 0
         8                 0
         9                 0 
        10                 0
        11    12286828683164 2012-06-05 02:55:43
        12    12286828683164 2012-06-05 02:55:43

12 rows selected.

发现数据库未end backup

Tue Jun  5 02:55:43 2012
alter tablespace bbed begin backup
Tue Jun  5 02:55:43 2012
Completed: alter tablespace bbed begin backup

尝试end backup
出现这个错误是正常的,因为我替换回来的bbed表空间数据文件的版本信息可能和控制文件的不一致,解决方法是重建控制文件

SQL> alter tablespace bbed end backup;
alter tablespace bbed end backup
*
ERROR at line 1:
ORA-01235: END BACKUP failed for 2 file(s) and succeeded for 0
ORA-01122: database file 12 failed verification check
ORA-01110: data file 12: '/u01/oracle/oradata/xifenfei/bbed02.dbf'
ORA-01208: data file is an old version - not accessing current version
ORA-01122: database file 11 failed verification check
ORA-01110: data file 11: '/u01/oracle/oradata/xifenfei/bbed01.dbf'
ORA-01208: data file is an old version - not accessing current version

重建控制文件

SQL> shutdown abort;
ORACLE instance shut down.
SQL> STARTUP NOMOUNT
Total System Global Area  353441008 bytes
Fixed Size                   451824 bytes
Variable Size             184549376 bytes
Database Buffers          167772160 bytes
Redo Buffers                 667648 bytes

SQL>@ctl.sql
Control file created.

尝试恢复数据库

SQL> recover database;
ORA-00279: change 12286828683164 generated at 06/05/2012 02:55:43 needed for
thread 1
ORA-00289: suggestion : /u01/oracle/oradata/xifenfei/archive/1_39.dbf
ORA-00280: change 12286828683164 for thread 1 is in sequence #39


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
auto
ORA-00308: cannot open archived log
'/u01/oracle/oradata/xifenfei/archive/1_39.dbf'
ORA-27037: unable to obtain file status
Linux Error: 2: No such file or directory
Additional information: 3


ORA-00308: cannot open archived log
'/u01/oracle/oradata/xifenfei/archive/1_39.dbf'
ORA-27037: unable to obtain file status
Linux Error: 2: No such file or directory
Additional information: 3

执行end backup

SQL> alter tablespace bbed end backup;

Tablespace altered.

再次查看相关SCN
可以发现end backup之后,datafile header 的scn发生了改变,说明begin backup主要是冻住了datafile header scn

SQL> select file#,online_status "STATUS",to_char(change#,'9999999999999999') "SCN",
  2  To_char(time,'yyyy-mm-dd hh24:mi:ss')"TIME" from v$recover_file;

     FILE# STATUS  SCN               TIME
---------- ------- ----------------- -------------------
         1 ONLINE     12286828684636 2012-06-05 03:00:46
         2 ONLINE     12286828684636 2012-06-05 03:00:46
         3 ONLINE     12286828684636 2012-06-05 03:00:46
         4 ONLINE     12286828684636 2012-06-05 03:00:46
         5 ONLINE     12286828684636 2012-06-05 03:00:46
         6 ONLINE     12286828684636 2012-06-05 03:00:46
         7 ONLINE     12286828684636 2012-06-05 03:00:46
         8 ONLINE     12286828684636 2012-06-05 03:00:46
         9 ONLINE     12286828684636 2012-06-05 03:00:46
        10 ONLINE     12286828684636 2012-06-05 03:00:46
        11 ONLINE     12286828683821 2012-06-05 02:56:26
        12 ONLINE     12286828683821 2012-06-05 02:56:26

12 rows selected.

SQL> select file#,to_char(checkpoint_change#,'999999999999999') "SCN",
  2  to_char(last_change#,'999999999999999')"STOP_SCN" from v$datafile;

     FILE# SCN              STOP_SCN
---------- ---------------- ----------------
         1   12286828684636
         2   12286828684636
         3   12286828684636
         4   12286828684636
         5   12286828684636
         6   12286828684636
         7   12286828684636
         8   12286828684636
         9   12286828684636
        10   12286828684636
        11   12286828684636
        12   12286828684636

12 rows selected.

SQL> select file#,to_char(checkpoint_change#,'9999999999999999') "SCN",
  2  to_char(RESETLOGS_CHANGE#,'9999999999999999') "RESETLOGS SCN"
  3  from v$datafile_header;

     FILE# SCN               RESETLOGS SCN
---------- ----------------- -----------------
         1    12286828684636            174968
         2    12286828684636            174968
         3    12286828684636            174968
         4    12286828684636            174968
         5    12286828684636            174968
         6    12286828684636            174968
         7    12286828684636            174968
         8    12286828684636            174968
         9    12286828684636            174968
        10    12286828684636            174968
        11    12286828683821            174968
        12    12286828683821            174968

12 rows selected.

再次尝试恢复数据库

SQL> recover database;
ORA-00279: change 12286828683821 generated at 06/05/2012 02:56:26 needed for
thread 1
ORA-00289: suggestion : /u01/oracle/oradata/xifenfei/archive/1_41.dbf
ORA-00280: change 12286828683821 for thread 1 is in sequence #41


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
auto
Log applied.
Media recovery complete.
SQL> alter database open;

Database altered.

总结说明
在数据库忘记end backup,而又被异常重启数据库时候,会提示你需要恢复.这个时候如果你有所有的归档日志,那没有任何问题,直接recover就可以了.如果因为begin backup命令执行比较久,部分归档日志丢失,这个时候不能直接recover,可以先尝试end backup,然后在recover.如果在这个时候还发现有部分日志不存在,那只能考虑bbed修改datafile header的scn.
温馨提醒:各位dba在执行begin backup之后一定要记得end backup

rman通过nfs备份

挂载nfs

[root@xifenfei tmp]# mount -t nfs 192.168.1.90:/tmp/nfs /nfs
[root@xifenfei tmp]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              18G   12G  5.2G  70% /
tmpfs                 737M     0  737M   0% /dev/shm
/dev/sdb1              20G  7.8G   11G  42% /u01/oracle/oradata
192.168.1.90:/tmp/nfs
                       18G   13G  3.9G  77% /nfs

rman备份

[oracle@xifenfei nfs]$ rman target /

Recovery Manager: Release 10.2.0.1.0 - Production on Wed May 30 16:31:40 2012

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

connected to target database: XFF (DBID=3426707456)

RMAN> backup datafile 1 format '/nfs/rman/system01_%U';

Starting backup at 3-JUN-12
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=146 devtype=DISK
channel ORA_DISK_1: starting full datafile backupset
channel ORA_DISK_1: specifying datafile(s) in backupset
RMAN-03009: failure of backup command on ORA_DISK_1 channel at 05/30/2012 16:32:17
ORA-19602: cannot backup or copy active file in NOARCHIVELOG mode
continuing other job steps, job failed will not be re-run
channel ORA_DISK_1: starting full datafile backupset
channel ORA_DISK_1: specifying datafile(s) in backupset
including current control file in backupset
including current SPFILE in backupset
channel ORA_DISK_1: starting piece 1 at 3-JUN-12
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on ORA_DISK_1 channel at 05/30/2012 16:32:20
ORA-19504: failed to create file "/nfs/rman/system01_02nc9rgh_1_1"
ORA-27054: NFS file system where the file is created or resides is not mounted with correct options
Additional information: 3

重新挂载nfs

[root@xifenfei ~]# umount /nfs
[root@xifenfei tmp]# mount -t nfs -o rw,bg,hard,rsize=32768,wsize=32768,
>vers=3,nointr,timeo=600,tcp 192.168.1.90:/tmp/nfs /nfs

rman重新备份

[oracle@xifenfei ~]$ rman target /

Recovery Manager: Release 10.2.0.1.0 - Production on Wed May 30 16:38:14 2012

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

connected to target database: XFF (DBID=3426707456)

RMAN> backup datafile 1 format '/nfs/rman/system01_%U';

Starting backup at 3-JUN-12
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=143 devtype=DISK
channel ORA_DISK_1: starting full datafile backupset
channel ORA_DISK_1: specifying datafile(s) in backupset
input datafile fno=00001 name=/u01/oracle/oradata/XFF/system01.dbf
channel ORA_DISK_1: starting piece 1 at 3-JUN-12
channel ORA_DISK_1: finished piece 1 at 3-JUN-12
piece handle=/nfs/rman/system01_07nc9rrv_1_1 tag=TAG20120530T163823 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:02:15
channel ORA_DISK_1: starting full datafile backupset
channel ORA_DISK_1: specifying datafile(s) in backupset
including current control file in backupset
including current SPFILE in backupset
channel ORA_DISK_1: starting piece 1 at 3-JUN-12
channel ORA_DISK_1: finished piece 1 at 3-JUN-12
piece handle=/nfs/rman/system01_08nc9s07_1_1 tag=TAG20120530T163823 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:02
Finished backup at 3-JUN-12

查看nfs源端文件

[root@xifenfei rman]# pwd
/tmp/nfs/rman
[root@xifenfei rman]# ls -l
total 378300
-rw-r-----  1 54321 54321 379846656 Jun  3 13:45 system01_07nc9rrv_1_1
-rw-r-----  1 54321 54321   7143424 Jun  3 13:45 system01_08nc9s07_1_1

要点说明
Mount Options for Oracle files when used with NFS on NAS devices [ID 359515.1]

ORA-39126: 在 KUPW$WORKER.PUT_DDLS [TABLE_STATISTICS] 中 Worker 发生意外致命错误

使用impdp导入数据报如下错误导致导入终止

处理对象类型 SCHEMA_EXPORT/TABLE/TRIGGER
处理对象类型 SCHEMA_EXPORT/TABLE/INDEX/FUNCTIONAL_AND_BITMAP/INDEX
处理对象类型 SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/FUNCTIONAL_AND_BITMAP/INDEX_STATISTICS
处理对象类型 SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
ORA-39126: 在 KUPW$WORKER.PUT_DDLS [TABLE_STATISTICS] 中 Worker 发生意外致命错误

ORA-06502: PL/SQL: 数字或值错误
LPX-00225: end-element tag "HIST_GRAM_LIST_ITEM" does not match start-element tag "EPVALUE"

ORA-06512: 在 "SYS.DBMS_SYS_ERROR", line 95
ORA-06512: 在 "SYS.KUPW$WORKER", line 9001

----- PL/SQL Call Stack -----
  object      line  object
  handle    number  name
26ABF4B0     20462  package body SYS.KUPW$WORKER
26ABF4B0      9028  package body SYS.KUPW$WORKER
26ABF4B0     16665  package body SYS.KUPW$WORKER
26ABF4B0      3956  package body SYS.KUPW$WORKER
26ABF4B0      9725  package body SYS.KUPW$WORKER
26ABF4B0      1775  package body SYS.KUPW$WORKER
290D454C         2  anonymous block

ORA-39097: 数据泵作业出现意外的错误 -1427
ORA-39065: DISPATCH 中出现意外的主进程异常错误
ORA-01427: 单行子查询返回多个行

作业 "EAS"."SYS_IMPORT_SCHEMA_01" 因致命错误于 15:21:20 停止

从这里可以看出是在执行TABLE_STATISTICS的时候因为EPVALUE列的数据类型和导入数据不匹配,问题发生上面错误,导致impdp job终止.

解决办法
参考文档:[ID 878626.1]
1.如果数据已经expdp导出,建议在导入的时候屏蔽掉统计信息导入EXCLUDE=STATISTICS,导入后使用DBMS_STATS 重新收集统计信息
2.如果数据尚未expdp导出,建议在导出的时候屏蔽掉统计信息导出EXCLUDE=STATISTICS导入后使用DBMS_STATS 重新收集统计信息

undo坏块导致数据库异常终止案例

在处理的众多undo问题的数据库中,这个是第一例遇到因为undo出现坏块导致数据库自动down案例(oracle 9.2.0.8 aix)
undo坏块导致数据库down

Fri Jun  1 00:45:13 2012
Successfully onlined Undo Tablespace 1.
Fri Jun  1 00:45:13 2012
SMON: enabling tx recovery
Fri Jun  1 00:45:13 2012
Database Characterset is ZHS16GBK
Fri Jun  1 00:45:13 2012
SMON: about to recover undo segment 52
SMON: about to recover undo segment 52
SMON: mark undo segment 52 as available
SMON: Restarting fast_start parallel rollback
SMON: about to recover undo segment 52
SMON: mark undo segment 52 as available
SMON: ignoring slave err,downgrading to serial rollback
SMON: about to recover undo segment 52
ORACLE Instance acc1 (pid = 9) - Error 1578 encountered while recovering transaction (52, 29).
Fri Jun  1 00:45:14 2012
Errors in file /oraacc/app/admin/acc/bdump/acc1_smon_734734.trc:
ORA-01578: ORACLE data block corrupted (file # 169, block # 55887)
ORA-01110: data file 169: '/dev/raccount07_01lv'
Fri Jun  1 00:45:15 2012
replication_dependency_tracking turned off (no async multimaster replication found)
Completed: alter database open
Fri Jun  1 00:45:16 2012
Errors in file /oraacc/app/admin/acc/bdump/acc1_pmon_766940.trc:
ORA-00474: SMON process terminated with error
Fri Jun  1 00:45:16 2012
PMON: terminating instance due to error 474
Instance terminated by PMON, pid = 766940

这里可以看出数据库因为回滚段52出现坏块导致回滚的时候smon终止,数据库down

检测坏块

$ dbv file='/dev/raccount07_01lv' blocksize=8192

DBVERIFY: Release 9.2.0.8.0 - Production on Fri Jun 1 01:25:10 2012

Copyright (c) 1982, 2002, Oracle Corporation.  All rights reserved.

DBVERIFY - Verification starting : FILE = /dev/raccount07_01lv

DBV-00200: Block, dba 708893135, already marked corrupted

DBV-00200: Block, dba 708893151, already marked corrupted

DBV-00200: Block, dba 708893263, already marked corrupted


DBVERIFY - Verification complete

Total Pages Examined         : 1048320
Total Pages Processed (Data) : 0
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 0
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 1048320
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 0
Total Pages Marked Corrupt   : 3
Total Pages Influx           : 0
Highest block SCN            : 12843497215383 (2990.1545000343)

检测对应文件号/数据块号

SQL> select DBMS_UTILITY.data_block_address_file(708893135) file#,
  2  DBMS_UTILITY.data_block_address_block(708893135) block#  from dual;

     FILE#     BLOCK#
---------- ----------
       169      55759

SQL> select DBMS_UTILITY.data_block_address_file(708893151) file#,
  2  DBMS_UTILITY.data_block_address_block(708893151) block#  from dual;

     FILE#     BLOCK#
---------- ----------
       169      55775

SQL> select DBMS_UTILITY.data_block_address_file(708893263) file#,
  2  DBMS_UTILITY.data_block_address_block(708893263) block#  from dual;

     FILE#     BLOCK#
---------- ----------
       169      55887

解决办法

--隐含参数
_corrupted_rollback_segments= _SYSSMU52$

--open库后
alter session set "_smu_debug_mode"=4;
drop rollback segment "_SYSSMU52$";

使用bbed修复损坏datafile header

相关信息和准备工作

SQL> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle9i Enterprise Edition Release 9.2.0.4.0 - Production
PL/SQL Release 9.2.0.4.0 - Production
CORE    9.2.0.3.0       Production
TNS for Linux: Version 9.2.0.4.0 - Production
NLSRTL Version 9.2.0.4.0 - Production

SQL> select to_char(sysdate,'yyyy-mm-dd hh24:mi:ss') "www.orasos.com" from dual;

www.orasos.com
-------------------
2012-05-29 19:39:48

启动数据块异常

SQL> startup
ORACLE instance started.

Total System Global Area  353441008 bytes
Fixed Size                   451824 bytes
Variable Size             184549376 bytes
Database Buffers          167772160 bytes
Redo Buffers                 667648 bytes
Database mounted.
ORA-01110: data file 11: '/u01/oracle/oradata/xifenfei/bbed01.dbf'
ORA-01115: IO error reading block from file 11 (block # 1)
ORA-27072: skgfdisp: I/O error

bbed检测datafile header

[oracle@xifenfei ~]$ bbed password=blockedit blocksize=8192  listfile=/home/oracle/bbed.file mode=edit

BBED: Release 2.0.0.0.0 - Limited Production on Sat May 26 05:29:37 2012

Copyright (c) 1982, 2002, Oracle Corporation.  All rights reserved.

************* !!! For Oracle Internal Use only !!! ***************

BBED> info
 File#  Name                                                        Size(blks)
 -----  ----                                                        ----------
     1  /home/oracle/users01.dbf                                             0
     2  /home/oracle/system01.dbf.head                                       0
     3  /home/oracle/data11.ora                                              0
     4  /u01/oracle/oradata/xifenfei/system01.dbf                            0
     5  /u01/oracle/oradata/xifenfei/users01.dbf                             0
     6  /home/oracle/data11.ora.10                                           0
    11  /u01/oracle/oradata/xifenfei/bbed01.dbf                              0
    12  /u01/oracle/oradata/xifenfei/bbed02.dbf                              0

BBED> set file 11
        FILE#           11

BBED> set block 1
        BLOCK#          1

BBED> map
 File: /u01/oracle/oradata/xifenfei/bbed01.dbf (11)
 Block: 1                                     Dba:0x02c00001
------------------------------------------------------------
BBED-00400: invalid blocktype (00)

BBED> d
 File: /u01/oracle/oradata/xifenfei/bbed01.dbf (11)
 Block: 1                Offsets:    0 to  511           Dba:0x02c00001
------------------------------------------------------------------------
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 

 <32 bytes per line>
--header 记录全部为0,证明数据文件header坏掉

拷贝数据块
为了方便,拷贝同一个表空间的数据块

BBED> set file 12
        FILE#           12

BBED> set block 1
        BLOCK#          1

BBED> d count 16
 File: /u01/oracle/oradata/xifenfei/bbed02.dbf (12)
 Block: 1                Offsets:    0 to   15           Dba:0x03000001
------------------------------------------------------------------------
 0b020000 01000003 00000000 00000104 

 <32 bytes per line>

BBED> copy dba 0x03000001 to dba 0x02c00001
Warning: contents of previous BIFILE will be lost. Proceed? (Y/N) y
 File: /u01/oracle/oradata/xifenfei/bbed01.dbf (11)
 Block: 1                Offsets:    0 to   15           Dba:0x02c00001
------------------------------------------------------------------------
 0b020000 01000003 00000000 00000104 

 <32 bytes per line>

BBED> show
        FILE#           11
        BLOCK#          1
        OFFSET          0
        DBA             0x02c00001 (46137345 11,1)
        FILENAME        /u01/oracle/oradata/xifenfei/bbed01.dbf
        BIFILE          bifile.bbd
        LISTFILE        /home/oracle/bbed.file
        BLOCKSIZE       8192
        MODE            Edit
        EDIT            Unrecoverable
        IBASE           Dec
        OBASE           Dec
        WIDTH           80
        COUNT           16
        LOGFILE         log.bbd
        SPOOL           No

BBED> map
 File: /u01/oracle/oradata/xifenfei/bbed01.dbf (11)
 Block: 1                                     Dba:0x02c00001
------------------------------------------------------------
 Data File Header

 struct kcvfh, 360 bytes                    @0       

 ub4 tailchk                                @8188    

修改数据块内容

BBED>  p kcvfh
struct kcvfh, 360 bytes                     @0       
   struct kcvfhbfh, 20 bytes                @0       
      ub1 type_kcbh                         @0        0x0b
      ub1 frmt_kcbh                         @1        0x02
      ub1 spare1_kcbh                       @2        0x00
      ub1 spare2_kcbh                       @3        0x00
      ub4 rdba_kcbh                         @4        0x03000001
      ub4 bas_kcbh                          @8        0x00000000
      ub2 wrp_kcbh                          @12       0x0000
      ub1 seq_kcbh                          @14       0x01
      ub1 flg_kcbh                          @15       0x04 (KCBHFCKV)
      ub2 chkval_kcbh                       @16       0xb10a
      ub2 spare3_kcbh                       @18       0x0000
   struct kcvfhhdr, 76 bytes                @20      
      ub4 kccfhswv                          @20       0x09200000
      ub4 kccfhcvn                          @24       0x08000000
      ub4 kccfhdbi                          @28       0x5314b4cd
      text kccfhdbn[0]                      @32      X
      text kccfhdbn[1]                      @33      I
      text kccfhdbn[2]                      @34      F
      text kccfhdbn[3]                      @35      E
      text kccfhdbn[4]                      @36      N
      text kccfhdbn[5]                      @37      F
      text kccfhdbn[6]                      @38      E
      text kccfhdbn[7]                      @39      I
      ub4 kccfhcsq                          @40       0x000001d8
      ub4 kccfhfsz                          @44       0x00001400
      s_blkz kccfhbsz                       @48       0x00
      ub2 kccfhfno                          @52       0x000c
      ub2 kccfhtyp                          @54       0x0003
      ub4 kccfhacid                         @56       0x00000000
      ub4 kccfhcks                          @60       0x00000000
      text kccfhtag[0]                      @64       
      text kccfhtag[1]                      @65       
      text kccfhtag[2]                      @66       
      text kccfhtag[3]                      @67       
      text kccfhtag[4]                      @68       
      text kccfhtag[5]                      @69       
      text kccfhtag[6]                      @70       
      text kccfhtag[7]                      @71       
      text kccfhtag[8]                      @72       
      text kccfhtag[9]                      @73       
      text kccfhtag[10]                     @74       
      text kccfhtag[11]                     @75       
      text kccfhtag[12]                     @76       
      text kccfhtag[13]                     @77       
      text kccfhtag[14]                     @78       
      text kccfhtag[15]                     @79       
      text kccfhtag[16]                     @80       
      text kccfhtag[17]                     @81       
      text kccfhtag[18]                     @82       
      text kccfhtag[19]                     @83       
      text kccfhtag[20]                     @84       
      text kccfhtag[21]                     @85       
      text kccfhtag[22]                     @86       
      text kccfhtag[23]                     @87       
      text kccfhtag[24]                     @88       
      text kccfhtag[25]                     @89       
      text kccfhtag[26]                     @90       
      text kccfhtag[27]                     @91       
      text kccfhtag[28]                     @92       
      text kccfhtag[29]                     @93       
      text kccfhtag[30]                     @94       
      text kccfhtag[31]                     @95       
   ub4 kcvfhrdb                             @96       0x00000000
   struct kcvfhcrs, 8 bytes                 @100     
      ub4 kscnbas                           @100      0xc00a3405
      ub2 kscnwrp                           @104      0x0b2c
   ub4 kcvfhcrt                             @108      0x2ebeb8c3
   ub4 kcvfhrlc                             @112      0x2e51408f
   struct kcvfhrls, 8 bytes                 @116     
      ub4 kscnbas                           @116      0x0002ab78
      ub2 kscnwrp                           @120      0x0000
   ub4 kcvfhbti                             @124      0x00000000
   struct kcvfhbsc, 8 bytes                 @128     
      ub4 kscnbas                           @128      0x00000000
      ub2 kscnwrp                           @132      0x0000
   ub2 kcvfhbth                             @136      0x0000
   ub2 kcvfhsta                             @138      0x0000 (NONE)
   struct kcvfhckp, 36 bytes                @140     
      struct kcvcpscn, 8 bytes              @140     
         ub4 kscnbas                        @140      0xc00b6467
         ub2 kscnwrp                        @144      0x0b2c
      ub4 kcvcptim                          @148      0x2ebf0c07
      ub2 kcvcpthr                          @152      0x0001
      union u, 12 bytes                     @156     
         struct kcvcprba, 12 bytes          @156     
            ub4 kcrbaseq                    @156      0x00000015
            ub4 kcrbabno                    @160      0x0000429a
            ub2 kcrbabof                    @164      0x0010
         struct kcvcptr, 12 bytes           @156     
            struct kcrtrscn, 8 bytes        @156     
               ub4 kscnbas                  @156      0x00000015
               ub2 kscnwrp                  @160      0x429a
            ub4 kcrtrtim                    @164      0x09110010
      ub1 kcvcpetb[0]                       @168      0x02
      ub1 kcvcpetb[1]                       @169      0x00
      ub1 kcvcpetb[2]                       @170      0x00
      ub1 kcvcpetb[3]                       @171      0x00
      ub1 kcvcpetb[4]                       @172      0x00
      ub1 kcvcpetb[5]                       @173      0x00
      ub1 kcvcpetb[6]                       @174      0x00
      ub1 kcvcpetb[7]                       @175      0x00
   ub4 kcvfhcpc                             @176      0x0000000d
   ub4 kcvfhrts                             @180      0x2ebeea4f
   ub4 kcvfhccc                             @184      0x0000000c
   struct kcvfhbcp, 36 bytes                @188     
      struct kcvcpscn, 8 bytes              @188     
         ub4 kscnbas                        @188      0x00000000
         ub2 kscnwrp                        @192      0x0000
      ub4 kcvcptim                          @196      0x00000000
      ub2 kcvcpthr                          @200      0x0000
      union u, 12 bytes                     @204     
         struct kcvcprba, 12 bytes          @204     
            ub4 kcrbaseq                    @204      0x00000000
            ub4 kcrbabno                    @208      0x00000000
            ub2 kcrbabof                    @212      0x0000
         struct kcvcptr, 12 bytes           @204     
            struct kcrtrscn, 8 bytes        @204     
               ub4 kscnbas                  @204      0x00000000
               ub2 kscnwrp                  @208      0x0000
            ub4 kcrtrtim                    @212      0x00000000
      ub1 kcvcpetb[0]                       @216      0x00
      ub1 kcvcpetb[1]                       @217      0x00
      ub1 kcvcpetb[2]                       @218      0x00
      ub1 kcvcpetb[3]                       @219      0x00
      ub1 kcvcpetb[4]                       @220      0x00
      ub1 kcvcpetb[5]                       @221      0x00
      ub1 kcvcpetb[6]                       @222      0x00
      ub1 kcvcpetb[7]                       @223      0x00
   ub4 kcvfhbhz                             @224      0x00000000
   struct kcvfhxcd, 16 bytes                @228     
      ub4 space_kcvmxcd[0]                  @228      0x00000000
      ub4 space_kcvmxcd[1]                  @232      0x00000000
      ub4 space_kcvmxcd[2]                  @236      0x00000000
      ub4 space_kcvmxcd[3]                  @240      0x00000000
   word kcvfhtsn                            @244      12
   ub2 kcvfhtln                             @248      0x0004
   text kcvfhtnm[0]                         @250     B
   text kcvfhtnm[1]                         @251     B
   text kcvfhtnm[2]                         @252     E
   text kcvfhtnm[3]                         @253     D
   text kcvfhtnm[4]                         @254      
   text kcvfhtnm[5]                         @255      
   text kcvfhtnm[6]                         @256      
   text kcvfhtnm[7]                         @257      
   text kcvfhtnm[8]                         @258      
   text kcvfhtnm[9]                         @259      
   text kcvfhtnm[10]                        @260      
   text kcvfhtnm[11]                        @261      
   text kcvfhtnm[12]                        @262      
   text kcvfhtnm[13]                        @263      
   text kcvfhtnm[14]                        @264      
   text kcvfhtnm[15]                        @265      
   text kcvfhtnm[16]                        @266      
   text kcvfhtnm[17]                        @267      
   text kcvfhtnm[18]                        @268      
   text kcvfhtnm[19]                        @269      
   text kcvfhtnm[20]                        @270      
   text kcvfhtnm[21]                        @271      
   text kcvfhtnm[22]                        @272      
   text kcvfhtnm[23]                        @273      
   text kcvfhtnm[24]                        @274      
   text kcvfhtnm[25]                        @275      
   text kcvfhtnm[26]                        @276      
   text kcvfhtnm[27]                        @277      
   text kcvfhtnm[28]                        @278      
   text kcvfhtnm[29]                        @279      
   ub4 kcvfhrfn                             @280      0x0000000c
   struct kcvfhrfs, 8 bytes                 @284     
      ub4 kscnbas                           @284      0x00000000
      ub2 kscnwrp                           @288      0x0000
   ub4 kcvfhrft                             @292      0x2ebee9f9
   struct kcvfhafs, 8 bytes                 @296     
      ub4 kscnbas                           @296      0x00000000
      ub2 kscnwrp                           @300      0x0000
   ub4 kcvfhbbc                             @304      0x00000000
   ub4 kcvfhncb                             @308      0x00000000
   ub4 kcvfhmcb                             @312      0x00000000
   ub4 kcvfhlcb                             @316      0x00000000
   ub4 kcvfhbcs                             @320      0x00000000
   ub2 kcvfhofb                             @324      0x0000
   ub2 kcvfhnfb                             @326      0x0000
   ub4 kcvfhprc                             @328      0x00000000
   struct kcvfhprs, 8 bytes                 @332     
      ub4 kscnbas                           @332      0x00000000
      ub2 kscnwrp                           @336      0x0000
   struct kcvfhprfs, 8 bytes                @340     
      ub4 kscnbas                           @340      0x00000000
      ub2 kscnwrp                           @344      0x0000
   ub4 kcvfhtrt                             @356      0x00000000

/*需要修改内容
ub4 rdba_kcbh                         @4        0x03000001
ub4 kccfhfsz                          @44       0x00001400
ub2 kccfhfno                          @52       0x000c
struct kcvfhcrs, 8 bytes                 @100     
      ub4 kscnbas                           @100      0xc00a3405
      ub2 kscnwrp                           @104      0x0b2c
ub4 kcvfhrfn                             @280      0x0000000c
/
/*修改值(通过错误提示结合file$表)
rdba_kcbh  02c00001
kccfhfsz  00000500
kccfhfno 000b
kscnbas c00a32b8
kcvfhrfn 0000000b
/

BBED> set offset 4
        OFFSET          4

BBED> m /x 0100c002
 File: /u01/oracle/oradata/xifenfei/bbed01.dbf (11)
 Block: 1                Offsets:    4 to   19           Dba:0x02c00001
------------------------------------------------------------------------
 0100c002 00000000 00000104 0ab10000 

 <32 bytes per line>

BBED> set offset 44
        OFFSET          44

BBED> m /x 00050000
 File: /u01/oracle/oradata/xifenfei/bbed01.dbf (11)
 Block: 1                Offsets:   44 to   59           Dba:0x02c00001
------------------------------------------------------------------------
 00050000 00200000 0c000300 00000000 

 <32 bytes per line>

BBED> set offset 52
        OFFSET          52

BBED> m /x 
BBED-00203: incomplete/malformed command


BBED> m /x 0b00                                                                                                        
 File: /u01/oracle/oradata/xifenfei/bbed01.dbf (11)
 Block: 1                Offsets:   52 to   67           Dba:0x02c00001
------------------------------------------------------------------------
 0b000300 00000000 00000000 00000000 

 <32 bytes per line>

BBED> set offset 100
        OFFSET          100

BBED> m /x b8320ac0
 File: /u01/oracle/oradata/xifenfei/bbed01.dbf (11)
 Block: 1                Offsets:  100 to  115           Dba:0x02c00001
------------------------------------------------------------------------
 b8320ac0 2c0b0000 c3b8be2e 8f40512e 

 <32 bytes per line>

BBED> set offset 280
        OFFSET          280

BBED> m /x 0b000000
 File: /u01/oracle/oradata/xifenfei/bbed01.dbf (11)
 Block: 1                Offsets:  280 to  295           Dba:0x02c00001
------------------------------------------------------------------------
 0b000000 00000000 00000000 f9e9be2e 

 <32 bytes per line>

BBED> sum apply
Check value for File 11, Block 1:
current = 0xa777, required = 0xa777

重建控制文件open数据库

SQL> alter database backup controlfile to trace as '/tmp/t_xifenfie.ctl';

Database altered.

SQL> shutdown immediate;
ORA-01109: database not open


Database dismounted.
ORACLE instance shut down.

SQL> startup nomount
ORACLE instance started.

Total System Global Area  353441008 bytes
Fixed Size                   451824 bytes
Variable Size             184549376 bytes
Database Buffers          167772160 bytes
Redo Buffers                 667648 bytes

SQL> CREATE CONTROLFILE REUSE DATABASE "XIFENFEI" NORESETLOGS  ARCHIVELOG
  2      MAXLOGFILES 50
  3      MAXLOGMEMBERS 5
  4      MAXDATAFILES 100
  5      MAXINSTANCES 1
  6      MAXLOGHISTORY 226
  7  LOGFILE
  8    GROUP 1 '/u01/oracle/oradata/xifenfei/redo01.log'  SIZE 100M,
  9    GROUP 2 '/u01/oracle/oradata/xifenfei/redo02.log'  SIZE 100M,
 10    GROUP 3 '/u01/oracle/oradata/xifenfei/redo03.log'  SIZE 100M
 11  DATAFILE
 12    '/u01/oracle/oradata/xifenfei/system01.dbf',
 13    '/u01/oracle/oradata/xifenfei/undotbs01.dbf',
 14    '/u01/oracle/oradata/xifenfei/cwmlite01.dbf',
 15    '/u01/oracle/oradata/xifenfei/drsys01.dbf',
 16    '/u01/oracle/oradata/xifenfei/example01.dbf',
 17    '/u01/oracle/oradata/xifenfei/indx01.dbf',
 18    '/u01/oracle/oradata/xifenfei/odm01.dbf',
 19    '/u01/oracle/oradata/xifenfei/tools01.dbf',
 20    '/u01/oracle/oradata/xifenfei/users01.dbf',
 21    '/u01/oracle/oradata/xifenfei/xdb01.dbf',
 22    '/u01/oracle/oradata/xifenfei/bbed01.dbf',
 23    '/u01/oracle/oradata/xifenfei/bbed02.dbf'
 24  CHARACTER SET ZHS16GBK
 25  ;

Control file created.

SQL> recover database ;
ORA-00283: recovery session canceled due to errors
ORA-00264: no recovery required


SQL> alter database open;

Database altered.

至此通过拷贝相同表空间的datafile header修复损坏的datafile header