模拟redo丢失
对数据库的一个pdb模拟事务操作，然后abort库，并且删除所有redo，模拟生产环境redo丢失的case
[oracle@ora1221 oradata]$ ss
SQL*Plus: Release 12.2.0.0.3 Production on Wed Jun 15 10:13:20 2016
Copyright (c) 1982, 2016, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup
ORACLE instance started.
Total System Global Area 2516582400 bytes
Fixed Size 8260048 bytes
Variable Size 671090224 bytes
Database Buffers 1828716544 bytes
Redo Buffers 8515584 bytes
Database mounted.
Database opened.
SQL>
SQL>
SQL> set pages 100
SQL> show pdbs;
CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
3 PDB1 MOUNTED
4 PDB2 MOUNTED
SQL> select con_id,file#,checkpoint_change# from v$datafile_header order by 1;
CON_ID FILE# CHECKPOINT_CHANGE#
---------- ---------- ------------------
1 1 1500157
1 3 1500157
1 4 1500157
1 7 1500157
2 5 1371280
2 6 1371280
2 8 1371280
3 9 1499902
3 12 1499902
3 11 1499902
3 10 1499902
4 15 1499903
4 14 1499903
4 13 1499903
4 16 1499903
15 rows selected.
SQL> alter PLUGGABLE database pdb1 open;
Pluggable database altered.
SQL> show pdbs;
CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
3 PDB1 READ WRITE NO
4 PDB2 MOUNTED
SQL> alter session set container=pdb1;
Session altered.
SQL> create user chf identified by oracle;
User created.
SQL> grant dba to chf;
Grant succeeded.
SQL> create table chf.t_pdb1_xifenfei as select * from dba_objects;
Table created.
SQL> delete from chf.t_pdb1_xifenfei;
72426 rows deleted.
--另外一个节点
[oracle@ora1221 ~]$ ss
SQL*Plus: Release 12.2.0.0.3 Production on Wed Jun 15 10:19:21 2016
Copyright (c) 1982, 2016, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.0.3 - 64bit Production
SQL> shutdown abort
ORACLE instance shut down.
[oracle@ora1221 orcl12c2]$ ls redo*
redo01.log redo02.log redo03.log
[oracle@ora1221 orcl12c2]$ rm redo0*
[oracle@ora1221 orcl12c2]$ ls -l redo*
ls: cannot access redo*: No such file or directory
尝试启动数据库
[oracle@ora1221 orcl12c2]$ ss SQL*Plus: Release 12.2.0.0.3 Production on Wed Jun 15 10:26:20 2016 Copyright (c) 1982, 2016, Oracle. All rights reserved. Connected to an idle instance. SQL> startup mount; ORACLE instance started. Total System Global Area 2516582400 bytes Fixed Size 8260048 bytes Variable Size 671090224 bytes Database Buffers 1828716544 bytes Redo Buffers 8515584 bytes Database mounted. SQL> alter database open; alter database open * ERROR at line 1: ORA-00313: open failed for members of log group 1 of thread 1 ORA-00312: online log 1 thread 1: '/u01/app/oracle/oradata/orcl12c2/redo01.log' ORA-27037: unable to obtain file status Linux-x86_64 Error: 2: No such file or directory Additional information: 7
使用隐含参数启动
----pfile里面增加 _allow_error_simulation=TRUE _allow_resetlogs_corruption=true ~ SQL> shutdown immediate; ORA-01109: database not open Database dismounted. ORACLE instance shut down SQL> startup pfile='/tmp/pfile' mount ORACLE instance started. Total System Global Area 2516582400 bytes Fixed Size 8260048 bytes Variable Size 671090224 bytes Database Buffers 1828716544 bytes Redo Buffers 8515584 bytes Database mounted. SQL> alter database open resetlogs; alter database open resetlogs * ERROR at line 1: ORA-00603: ORACLE server session terminated by fatal error ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [], [], [], [], [], [], [], [] Process ID: 36797 Session ID: 16 Serial number: 24277
继续重启库
ORA-600 kcbzib_kcrsds_1错误尝试重启数据库,如果不想考了bbed修改文件头信息
SQL> startup mount pfile='/tmp/pfile' ORACLE instance started. Total System Global Area 2516582400 bytes Fixed Size 8260048 bytes Variable Size 671090224 bytes Database Buffers 1828716544 bytes Redo Buffers 8515584 bytes Database mounted. SQL> recover database until cancel; ORA-00283: recovery session canceled due to errors ORA-16433: The database or pluggable database must be opened in read/write mode. SQL> alter database backup controlfile to trace as '/tmp/ctl'; alter database backup controlfile to trace as '/tmp/ctl' * ERROR at line 1: ORA-16433: The database or pluggable database must be opened in read/write mode.
重建控制文件
SQL> startup nomount pfile='/tmp/pfile'
ORACLE instance started.
Total System Global Area 2516582400 bytes
Fixed Size 8260048 bytes
Variable Size 671090224 bytes
Database Buffers 1828716544 bytes
Redo Buffers 8515584 bytes
SQL> CREATE CONTROLFILE REUSE DATABASE "orcl12c2" RESETLOGS NOARCHIVELOG
2 MAXLOGFILES 50
3 MAXLOGMEMBERS 5
4 MAXDATAFILES 100
5 MAXINSTANCES 1
6 MAXLOGHISTORY 226
7 LOGFILE
8 GROUP 1 '/u01/app/oracle/oradata/orcl12c2/redo01.log' SIZE 200M,
9 GROUP 2 '/u01/app/oracle/oradata/orcl12c2/redo02.log' SIZE 200M,
10 GROUP 3 '/u01/app/oracle/oradata/orcl12c2/redo03.log' SIZE 200M
11 DATAFILE
12 '/u01/app/oracle/oradata/orcl12c2/system01.dbf',
13 '/u01/app/oracle/oradata/orcl12c2/sysaux01.dbf',
14 '/u01/app/oracle/oradata/orcl12c2/undotbs01.dbf',
15 '/u01/app/oracle/oradata/orcl12c2/pdbseed/system01.dbf',
16 '/u01/app/oracle/oradata/orcl12c2/pdbseed/sysaux01.dbf',
17 '/u01/app/oracle/oradata/orcl12c2/users01.dbf',
18 '/u01/app/oracle/oradata/orcl12c2/pdbseed/undotbs01.dbf',
19 '/u01/app/oracle/oradata/orcl12c2/pdb1/system01.dbf',
20 '/u01/app/oracle/oradata/orcl12c2/pdb1/sysaux01.dbf',
21 '/u01/app/oracle/oradata/orcl12c2/pdb1/undotbs01.dbf',
22 '/u01/app/oracle/oradata/orcl12c2/pdb1/users01.dbf',
23 '/u01/app/oracle/oradata/orcl12c2/pdb2/system01.dbf',
24 '/u01/app/oracle/oradata/orcl12c2/pdb2/sysaux01.dbf',
25 '/u01/app/oracle/oradata/orcl12c2/pdb2/undotbs01.dbf',
26 '/u01/app/oracle/oradata/orcl12c2/pdb2/users01.dbf'
27 CHARACTER SET AL32UTF8
28 ;
Control file created.
SQL> recover database until cancel;
ORA-00283: recovery session canceled due to errors
ORA-01610: recovery using the BACKUP CONTROLFILE option must be done
SQL> recover database until cancel using backup controlfile;
ORA-00279: change 1500161 generated at 06/15/2016 10:40:42 needed for thread 1
ORA-00289: suggestion :
/u01/app/oracle/product/12.2.0/db_2/dbs/arch1_1_914582438.dbf
ORA-00280: change 1500161 for thread 1 is in sequence #1
Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
/u01/app/oracle/oradata/orcl12c2/redo01.log
Log applied.
Media recovery complete.
SQL> alter database open resetlogs;
Database altered.
1
<strong>open过程alert日志</strong>
1
<strong>open pdb1</strong>
1
SQL> alter PLUGGABLE database pdb1 open;
Pluggable database altered.
pdb1 open alert日志
2016-06-15T11:13:39.423057+08:00 alter PLUGGABLE database pdb1 open PDB1(3):Autotune of undo retention is turned on. 2016-06-15T11:13:39.495559+08:00 PDB1(3):Endian type of dictionary set to little PDB1(3):[40547] Successfully onlined Undo Tablespace 2. PDB1(3):Undo initialization finished serial:0 start:371149831 end:371149872 diff:41 ms (0.0 seconds) PDB1(3):Database Characterset for PDB1 is AL32UTF8 PDB1(3):********************************************************************* PDB1(3):WARNING: The following temporary tablespaces in container(PDB1) PDB1(3): contain no files. PDB1(3): This condition can occur when a backup controlfile has PDB1(3): been restored. It may be necessary to add files to these PDB1(3): tablespaces. That can be done using the SQL statement: PDB1(3): PDB1(3): ALTER TABLESPACE <tablespace_name> ADD TEMPFILE PDB1(3): PDB1(3): Alternatively, if these temporary tablespaces are no longer PDB1(3): needed, then they can be dropped. PDB1(3): Empty temporary tablespace: TEMP PDB1(3):********************************************************************* PDB1(3):Opatch validation is skipped for PDB PDB1 (con_id=0) PDB1(3):Opening pdb with no Resource Manager plan active Pluggable database PDB1 opened read write Completed: alter PLUGGABLE database pdb1 open
open pdb2
SQL> alter PLUGGABLE database pdb2 open; alter PLUGGABLE database pdb2 open * ERROR at line 1: ORA-00600: internal error code, arguments: [17090], [], [], [], [], [], [], [], [], [], [], []
分析alert日志和trace文件
--alert日志部分 PDB1(3):alter PLUGGABLE database pdb2 open PDB1(3):ORA-65118 signalled during: alter PLUGGABLE database pdb2 open... 2016-06-15T11:28:57.439963+08:00 PDB1(3):Unified Audit: Audit record write to table failed due to ORA-25153. Writing the audit record to OS spillover file. Please grep in the trace files for ORA-25153 for more diagnostic information. Errors in file /u01/app/oracle/diag/rdbms/orcl12c2/orcl12c2/trace/orcl12c2_ora_40547.trc (incident=29073) (PDBNAME=PDB1): ORA-00600: internal error code, arguments: [17090], [], [], [], [], [], [], [], [], [], [], [] PDB1(3):Incident details in: /u01/app/oracle/diag/rdbms/orcl12c2/orcl12c2/incident/incdir_29073/orcl12c2_ora_40547_i29073.trc PDB1(3):***************************************************************** PDB1(3):An internal routine has requested a dump of selected redo. PDB1(3):This usually happens following a specific internal error, when PDB1(3):analysis of the redo logs will help Oracle Support with the PDB1(3):diagnosis. PDB1(3):It is recommended that you retain all the redo logs generated (by PDB1(3):all the instances) during the past 12 hours, in case additional PDB1(3):redo dumps are required to help with the diagnosis. PDB1(3):***************************************************************** 2016-06-15T11:28:59.123041+08:00 PDB1(3):Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. 2016-06-15T11:28:59.945667+08:00 Dumping diagnostic data in directory=[cdmp_20160615112859], requested by (instance=1, osid=40547), summary=[incident=29073]. 2016-06-15T11:35:59.987419+08:00 PDB1(3): alter PLUGGABLE database pdb2 open PDB1(3):ORA-65118 signalled during: alter PLUGGABLE database pdb2 open... --trace部分 PARSING IN CURSOR #0x7f051a3d7650 len=118 dep=1 uid=0 oct=3 lid=0 tim=372490287736 hv=1128335472 ad='0x6ca82f00' sqlid='gu930gd1n223h' select tablespace_name, tablespace_size, allocated_space, free_space, con_id from cdb_temp_free_space order by con_id END OF STMT EXEC #0x7f051a3d7650:c=0,e=144,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=4,plh=2538033465,tim=372490287732 FETCH #0x7f051a3d7650:c=0,e=290,p=0,cr=14,cu=0,mis=0,r=0,dep=1,og=4,plh=2538033465,tim=372490288109 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 52 FileOperation=2 fileno=0 filetype=36 obj#=402 tim=372490288373 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 17 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490288577 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 3 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490288655 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 690 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490289365 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 6 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490289470 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 445 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490289934 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 3 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490289983 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 375 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490290374 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 6 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490290453 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 367 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490290839 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 3 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490290882 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 355 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291252 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 4 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291298 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 276 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291590 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 1 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291614 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 256 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291879 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 2 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291903 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 261 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490292172 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 30 FileOperation=3 fileno=0 filetype=36 obj#=402 tim=372490292225 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 934 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490293171 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 3 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490293208 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 245 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490293465 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 262 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490293755 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 1 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490293780 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 250 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490294039 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 256 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490294323 WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 8 FileOperation=5 fileno=0 filetype=36 obj#=402 tim=372490294359 2016-06-15T11:36:00.055196+08:00 Incident 29074 created, dump file: /u01/app/oracle/diag/rdbms/orcl12c2/orcl12c2/incident/incdir_29074/orcl12c2_ora_40547_i29074.trc ORA-00600: internal error code, arguments: [17090], [], [], [], [], [], [], [], [], [], [], []
从中可以判断出来是由于CDB$ROOT的未增加tempfile导致
SQL> alter tablespace temp add tempfile '/u01/app/oracle/oradata/orcl12c2/temp01.dbf' reuse; Tablespace altered. SQL> alter PLUGGABLE database pdb2 open; Pluggable database altered.
查看数据库恢复情况
SQL> show pdbs;
CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
3 PDB1 READ WRITE NO
4 PDB2 READ WRITE NO
SQL> select con_id,file#,checkpoint_change#,resetlogs_change# from v$datafile_header;
CON_ID FILE# CHECKPOINT_CHANGE# RESETLOGS_CHANGE#
---------- ---------- ------------------ -----------------
1 1 2500167 1500164
1 3 2500167 1500164
1 4 2500167 1500164
2 5 1371280 1341067
2 6 1371280 1341067
1 7 2500167 1500164
2 8 1371280 1341067
3 9 2501017 1500164
3 10 2501017 1500164
3 11 2501017 1500164
3 12 2501017 1500164
4 13 2502748 1500164
4 14 2502748 1500164
4 15 2502748 1500164
4 16 2502748 1500164
15 rows selected.
至此基本上测试完成在在cdb环境中丢失redo的恢复。在生产中，需要把temp加全，并且建议重建数据库
