ORA-600 3020/ORA-600 2662故障

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:ORA-600 3020/ORA-600 2662故障

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

数据库重启之后报ORA-600 3020错误

Tue Dec 12 00:37:44 2023
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Media Recovery Log /data/oracle/flash_recovery_area/ORCL/archivelog/2023_11_30/o1_mf_1_3867_lpj3krsn_.arc
ORA-279 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Media Recovery Log /data/oracle/flash_recovery_area/ORCL/archivelog/2023_12_02/o1_mf_1_3868_lppkmyhf_.arc
Tue Dec 12 00:37:45 2023
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_pr01_57992.trc  (incident=31482):
ORA-00600: internal error code, arguments: [3020], [1], [87806], [4282110], [], [], [], [], [], [], [], []
ORA-10567: Redo is inconsistent with data block (file# 1, block# 87806, file offset is 719306752 bytes)
ORA-10564: tablespace SYSTEM
ORA-01110: data file 1: '/data/oracle/oradata/orcl/system01.dbf'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 5843
Incident details in: /data/oracle/diag/rdbms/orcl/orcl/incident/incdir_31482/orcl_pr01_57992_i31482.trc
Tue Dec 12 00:37:45 2023
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_pr08_58007.trc  (incident=31538):
ORA-00600: internal error code, arguments: [3020], [1], [61031], [4255335], [], [], [], [], []
ORA-10567:Redo is inconsistent with data block (file# 1, block# 61031, file offset is 499965952 bytes)
ORA-10564: tablespace SYSTEM
ORA-01110: data file 1: '/data/oracle/oradata/orcl/system01.dbf'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 39
Incident details in: /data/oracle/diag/rdbms/orcl/orcl/incident/incdir_31538/orcl_pr08_58007_i31538.trc

客户不太了解上述故障,直接使用隐含参数,强制拉库报ORA-600 2662错误

Tue Dec 12 00:45:11 2023
SMON: enabling cache recovery
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_58731.trc  (incident=32668):
ORA-00600: internal error code, arguments: [2662], [0], [120784398], [0], [121441601], [12583040]
Incident details in: /data/oracle/diag/rdbms/orcl/orcl/incident/incdir_32668/orcl_ora_58731_i32668.trc
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_58731.trc:
ORA-00600: internal error code, arguments: [2662], [0], [120784398], [0], [121441601], [12583040]
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_58731.trc:
ORA-00600: internal error code, arguments: [2662], [0], [120784398], [0], [121441601], [12583040]
Error 600 happened during db open, shutting down database
USER (ospid: 58731): terminating the instance due to error 600
Instance terminated by USER, pid = 58731

接手该故障之后,数据库已经被resetlogs,只能基于该现场进一步恢复,修改数据库scn,open数据库成功,导出数据,完成本次恢复.
20231213102213


MySQL数据库文件丢失恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:MySQL数据库文件丢失恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

客户服务器重启,mysql相关数据文件丢失
20231206095845


通过底层工具进行分析,无法正确恢复数据库名字,一个个单个ibd文件(而且很多本身是错误的)
20231210224010

对于这种情况,通过mysql block扫描恢复出来page文件
20231210224106

恢复出来客户需要数据
20231210224248
20231207215458

这个客户出现该故障的原因大概率是由于文件系统损坏导致.最终可以比较好的恢复,主要是故障之后第一时间保护了现场,没进行任何的写入覆盖,如果覆盖了神仙也没有办法.如果此类问题无法自行解决或者恢复效果不好,可以联系我们进行技术支持,最大限度抢救和数据,减少损失
电话/微信:17813235971    Q Q:107644445QQ咨询惜分飞    E-Mail:dba@xifenfei.com

Patch SCN工具一键恢复ORA-600 kcbzib_kcrsds_1

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:Patch SCN工具一键恢复ORA-600 kcbzib_kcrsds_1

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

一个19c库由于某种原因redo损坏强制打开库报ORA-600 kcbzib_kcrsds_1错误

SQL> startup mount pfile='?/database/pfile.txt';
ORACLE instance started.

Total System Global Area  859830696 bytes
Fixed Size                  9034152 bytes
Variable Size             566231040 bytes
Database Buffers          276824064 bytes
Redo Buffers                7741440 bytes
Database mounted.
SQL>
SQL>
SQL> alter database open resetlogs ;
alter database open resetlogs
*
ERROR at line 1:
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [],
[], [], [], [], [], [], []
Process ID: 7236
Session ID: 252 Serial number: 14807

alert日志报错

2023-12-07T21:27:40.176923+08:00
alter database open resetlogs upgrade
2023-12-07T21:27:40.223341+08:00
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 70591877 time 
.... (PID:7236): Clearing online redo logfile 1 C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO01.LOG
.... (PID:7236): Clearing online redo logfile 2 C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO02.LOG
.... (PID:7236): Clearing online redo logfile 3 C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO03.LOG
Clearing online log 1 of thread 1 sequence number 0
Clearing online log 2 of thread 1 sequence number 0
Clearing online log 3 of thread 1 sequence number 0
.... (PID:7236): Clearing online redo logfile 1 complete
.... (PID:7236): Clearing online redo logfile 2 complete
.... (PID:7236): Clearing online redo logfile 3 complete
Resetting resetlogs activation ID 1658522235 (0x62db0a7b)
Online log C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO01.LOG: Thread 1 Group 1 was previously cleared
Online log C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO02.LOG: Thread 1 Group 2 was previously cleared
Online log C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO03.LOG: Thread 1 Group 3 was previously cleared
2023-12-07T21:27:40.896383+08:00
Setting recovery target incarnation to 3
2023-12-07T21:27:40.927396+08:00
Ping without log force is disabled:
  instance mounted in exclusive mode.
Endian type of dictionary set to little
2023-12-07T21:27:40.943147+08:00
Assigning activation ID 1682724028 (0x644c54bc)
2023-12-07T21:27:40.943147+08:00
TT00 (PID:7180): Gap Manager starting
2023-12-07T21:27:40.958429+08:00
Redo log for group 1, sequence 1 is not located on DAX storage
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO01.LOG
Successful open of redo thread 1
2023-12-07T21:27:40.958429+08:00
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
stopping change tracking
2023-12-07T21:27:40.990460+08:00
TT03 (PID:1292): Sleep 5 seconds and then try to clear SRLs in 2 time(s)
Errors in file C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_7236.trc  (incident=12218):
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [], [], [], [], [], [], [], []
Incident details in: C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\incident\incdir_12218\orcl_ora_7236_i12218.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2023-12-07T21:27:41.021109+08:00
2023-12-07T21:27:41.848773+08:00
*****************************************************************
An internal routine has requested a dump of selected redo.
This usually happens following a specific internal error, when
analysis of the redo logs will help Oracle Support with the
diagnosis.
It is recommended that you retain all the redo logs generated (by
all the instances) during the past 12 hours, in case additional
redo dumps are required to help with the diagnosis.
*****************************************************************
2023-12-07T21:27:42.082978+08:00
Undo initialization recovery: err:600 start: 3673703 end: 3674796 diff: 1093 ms (1.1 seconds)
2023-12-07T21:27:42.082978+08:00
Errors in file C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_7236.trc:
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [], [], [], [], [], [], [], []
2023-12-07T21:27:42.082978+08:00
Errors in file C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_7236.trc:
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [], [], [], [], [], [], [], []
Error 600 happened during db open, shutting down database
Errors in file C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_7236.trc  (incident=12219):
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [], [], [], [], [], [], [], []
Incident details in: C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\incident\incdir_12219\orcl_ora_7236_i12219.trc
2023-12-07T21:27:42.881204+08:00
opiodr aborting process unknown ospid (7236) as a result of ORA-603
2023-12-07T21:27:42.881204+08:00
ORA-603 : opitsk aborting process
License high water mark = 4
USER (ospid: (prelim)): terminating the instance due to ORA error 
2023-12-07T21:27:46.052894+08:00
Instance terminated by USER(prelim), pid = 7236

通过patch scn工具快速修改数据库scn
patch_scn


实现数据库顺利打开
20231207213822

数据库打patch建议设置nls_language为英文

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:数据库打patch建议设置nls_language为英文

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

昨晚给客户11.2.0.4打比较新的psu和ojvm patch(主要为了修复安全扫描漏洞)
20231203185616


结果今天早上客户那边反馈应用有报ORA-29548错
ORA-29548

该错误比较明显应该是和java有关系,在本次变更中最大可能就是ojvm有关系,查看ojvm的postinstall.sql执行日志,发现如下问题

 34  -- Check the validity of JAVAVM and let the registry be updated accordingly.
 35  
 36      initjvmaux.validate_javavm;
 37  
 38  -- Add a row in registry$history to indicate this script was run.
 39  
 40      EXECUTE IMMEDIATE 'insert into registry$history
 41                         (action_time, action, namespace, version, id, comments)
 42                         values(SYSTIMESTAMP, ''jvmpsu.sql'', ''SERVER'',
 43                                ''11.2.0.4.221018OJVMPSU'', 0, ''RAN jvmpsu.sql'')';
 44  
 45    END IF;
 46  
 47    EXECUTE IMMEDIATE 'alter system set java_jit_enabled = ' || :jitstate;
 48  
 49  END;
 50  /
BEGIN
*
第 1 行出现错误:
ORA-01843: 无效的月份
ORA-06512: 在 line 8

查询组件有效性

SQL> select comp_name,version,status from dba_registry order by 1;

COMP_NAME
--------------------------------------------------------------------------------
VERSION                        STATUS
------------------------------ ----------------------
JServer JAVA Virtual Machine
11.2.0.4.0                     INVALID

OLAP Analytic Workspace
11.2.0.4.0                     VALID

OLAP Catalog
11.2.0.4.0                     VALID


COMP_NAME
--------------------------------------------------------------------------------
VERSION                        STATUS
------------------------------ ----------------------
OWB
11.2.0.1.0                     VALID

Oracle Application Express
3.2.1.00.10                    VALID

Oracle Database Catalog Views
11.2.0.4.0                     VALID


COMP_NAME
--------------------------------------------------------------------------------
VERSION                        STATUS
------------------------------ ----------------------
Oracle Database Java Packages
11.2.0.4.0                     VALID

Oracle Database Packages and Types
11.2.0.4.0                     VALID

Oracle Enterprise Manager
11.2.0.4.0                     VALID


COMP_NAME
--------------------------------------------------------------------------------
VERSION                        STATUS
------------------------------ ----------------------
Oracle Expression Filter
11.2.0.4.0                     VALID

Oracle Multimedia
11.2.0.4.0                     VALID

Oracle OLAP API
11.2.0.4.0                     VALID


COMP_NAME
--------------------------------------------------------------------------------
VERSION                        STATUS
------------------------------ ----------------------
Oracle Rules Manager
11.2.0.4.0                     VALID

Oracle Text
11.2.0.4.0                     VALID

Oracle Workspace Manager
11.2.0.4.0                     VALID


COMP_NAME
--------------------------------------------------------------------------------
VERSION                        STATUS
------------------------------ ----------------------
Oracle XDK
11.2.0.4.0                     VALID

Oracle XML Database
11.2.0.4.0                     VALID

Spatial
11.2.0.4.0                     VALID


已选择18行。

确认JServer JAVA Virtual Machine组件无效,很可能是ojvm的postinstall.sql执行失败有关系,而引起这个失败的原因是由于“ORA-01843: 无效的月份”,根据经验,引起这个问题的可能是由于win汉语环境中日期/时间的显示格式导致

SQL> select sysdate from dual;

SYSDATE
--------------
03-12月-23

设置为英文格式显示

SQL> alter session set nls_language='American';

Session altered.

SQL> select sysdate from dual;

SYSDATE
------------
03-DEC-23

重新执行postdeinstall和postinstall脚本之后,组件状态恢复正常,软件功能也恢复正常

SQL> select comp_name,version,status from dba_registry order by 1;

JServer JAVA Virtual Machine
11.2.0.4.0                     VALID

OLAP Analytic Workspace
11.2.0.4.0                     VALID

OLAP Catalog
11.2.0.4.0                     VALID

OWB
11.2.0.1.0                     VALID

Oracle Application Express
3.2.1.00.10                    VALID

Oracle Database Catalog Views
11.2.0.4.0                     VALID

Oracle Database Java Packages
11.2.0.4.0                     VALID

Oracle Database Packages and Types
11.2.0.4.0                     VALID

Oracle Enterprise Manager
11.2.0.4.0                     VALID

Oracle Expression Filter
11.2.0.4.0                     VALID

Oracle Multimedia
11.2.0.4.0                     VALID

Oracle OLAP API
11.2.0.4.0                     VALID

Oracle Rules Manager
11.2.0.4.0                     VALID

Oracle Text
11.2.0.4.0                     VALID

Oracle Workspace Manager
11.2.0.4.0                     VALID

Oracle XDK
11.2.0.4.0                     VALID

Oracle XML Database
11.2.0.4.0                     VALID

Spatial
11.2.0.4.0                     VALID


18 rows selected.

自此提醒由于oracle 环境语言默认显示的问题导致某些patch不能正常打成功,建议在执行数据库patch或者升级之时,把数据库语言环境调整为英文nls_language=’American’,以避免本次出现的不必要的麻烦(以前由于大部分客户没有是使用jvm这个功能组件因此没有暴露该问题)