ORA-00600[3689]引起MRP进程异常

今天检查数据库发现一数据库因为添加数据文件导致ORA-00600[3689]错误,进入引起MRP进程异常,从而使得数据库可以正常接收归档日志,但是不能在备库应用归档日志
数据库版本

SQL> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bi
PL/SQL Release 10.2.0.1.0 - Production
CORE    10.2.0.1.0      Production
TNS for Solaris: Version 10.2.0.1.0 - Production
NLSRTL Version 10.2.0.1.0 – Production

检查日志

Fri Oct 19 12:00:22 2012
RFS[1]: No standby redo logfiles created
RFS[1]: Archived Log: '/baceldba/archive/1_30374_730819916.dbf'
Fri Oct 19 12:00:28 2012
Media Recovery Log /baceldba/archive/1_30374_730819916.dbf
WARNING: File being created with same name as in Primary
Existing file may be overwritten
Fri Oct 19 12:00:50 2012
Recovery created file /data/oracle/oradata/vodapp/NETTVCLPS02.dbf
Successfully added datafile 32 to media recovery
Datafile #32: '/data/oracle/oradata/vodapp/NETTVCLPS02.dbf '
WARNING: File being created with same name as in Primary
Existing file may be overwritten
Fri Oct 19 12:01:15 2012
Recovery created file /data/oracle/oradata/vodapp/UCICMS01_DATA04.dbf
Successfully added datafile 33 to media recovery
Datafile #33: '/data/oracle/oradata/vodapp/UCICMS01_DATA04.dbf'
Fri Oct 19 12:01:16 2012
Errors in file /data/oracle/admin/vodapp/bdump/vodapp_mrp0_21304.trc:
ORA-00600: internal error code, arguments: [3689], [33], [], [], [], [], [], []
Errors with log /baceldba/archive/1_30374_730819916.dbf
MRP0: Background Media Recovery terminated with error 600  <--mrp进程异常
Fri Oct 19 12:01:16 2012
Errors in file /data/oracle/admin/vodapp/bdump/vodapp_mrp0_21304.trc:
ORA-00600: internal error code, arguments: [3689], [33], [], [], [], [], [], []
Some recovered datafiles maybe left media fuzzy
Media recovery may continue but open resetlogs may fail
Fri Oct 19 12:01:18 2012
Errors in file /data/oracle/admin/vodapp/bdump/vodapp_mrp0_21304.trc:
ORA-00600: internal error code, arguments: [3689], [33], [], [], [], [], [], []
Fri Oct 19 12:01:18 2012
Errors in file /data/oracle/admin/vodapp/bdump/vodapp_mrp0_21304.trc:
ORA-00600: internal error code, arguments: [3689], [33], [], [], [], [], [], []

通过观察日志发现,数据库的mrp0进程因为ora-600的错误导致异常终止,从而使得dg备库无法正常使用归档日志

分析trace文件得出

*** 2012-10-19 12:01:16.327
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [3689], [33], [], [], [], [], [], []
----- Call Stack Trace -----
calling              call     entry                argument values in hex      
location             type     point                (? means dubious value)     
-------------------- -------- -------------------- ----------------------------
ksedmp()+744         CALL     ksedst()             000000840 ?
                                                   FFFFFFFF7FFFA08C ?
                                                   000000000 ?
                                                   FFFFFFFF7FFF6B80 ?
                                                   FFFFFFFF7FFF58E8 ?
                                                   FFFFFFFF7FFF62E8 ?
kgeriv()+220         PTR_CALL 0000000000000000     000106000 ? 106323304 ?
                                                   106323000 ? 000106323 ?
                                                   000106000 ? 106323304 ?
kgesiv()+112         CALL     kgeriv()             10631DCD0 ? 000000000 ?
                                                   000000E69 ? 000000001 ?
                                                   FFFFFFFF7FFFA5E8 ?
                                                   000001430 ?
ksesic1()+96         CALL     kgesiv()             10631DCD0 ? 10646AD00 ?
                                                   000000E69 ? 000000001 ?
                                                   FFFFFFFF7FFFA5E8 ?
                                                   000001420 ?
kcvadc1_10gr2()+265  CALL     ksesic1()            000000E69 ? 00010631D ?
2                                                  1063232F8 ? 000106000 ?
                                                   106323000 ? 000106323 ?
kcoapl()+5664        PTR_CALL 0000000000000000     FFFFFFFF7A5570F8 ?
                                                   FFFFFFFF7FFFB320 ?
                                                   FFFFFFFF76667E18 ?
                                                   000000000 ? 000000021 ?
                                                   000000020 ?
kcramr()+5352        CALL     kcoapl()             FFFFFFFF76667E00 ?
                                                   000000003 ? 1055136B0 ?
                                                   105513340 ? 00000001B ?
                                                   000000017 ?
krddmr()+2688        CALL     kcramr()             000000000 ?
                                                   FFFFFFFF7A55FF78 ?
                                                   000002000 ? 000106000 ?
                                                   0000001F6 ? 464897868 ?
krsmdp()+2332        CALL     krddmr()             105502000 ?
                                                   FFFFFFFF7A5570F8 ?
                                                   000000001 ? 10646C760 ?
                                                   464897868 ? 000000000 ?
ksbrdp()+896         PTR_CALL 0000000000000000     000380000 ? 000380000 ?
                                                   105517000 ? 000106328 ?
                                                   000106000 ? 00000FFFF ?
opirip()+824         CALL     ksbrdp()             000106320 ? 380007734 ?
                                                   000380000 ? 00038000E ?
                                                   000106000 ? 1005DCF80 ?
opidrv()+1200        CALL     opirip()             10632A000 ? 000106000 ?
                                                   000106332 ? 380007000 ?
                                                   00010632A ? 1063D73A0 ?
sou2o()+80           CALL     opidrv()             10632D460 ? 000000001 ?
                                                   000000032 ? 000000000 ?
                                                   000000032 ? 000106000 ?
opimai_real()+268    CALL     sou2o()              FFFFFFFF7FFFF968 ?
                                                   000000032 ? 000000004 ?
                                                   FFFFFFFF7FFFF990 ?
                                                   105C1F000 ? 000105C1F ?
main()+152           CALL     opimai_real()        000000003 ?
                                                   FFFFFFFF7FFFFA68 ?
                                                   000000000 ? 000000000 ?
                                                   0023F9764 ? 000014400 ?
_start()+380         CALL     main()               000000001 ?
                                                   FFFFFFFF7FFFFB78 ?
                                                   000000000 ?
                                                   FFFFFFFF7FFFFA70 ?
                                                   FFFFFFFF7FFFFB80 ?
                                                   FFFFFFFF7CF00200 ?
 
--------------------- Binary Stack Dump ---------------------

查询mos发现
Bug 5230162 : ORA-600[3689][42] ON PHYSICAL STANDBY PREVENT MEDIA RECOVERY描述相符

处理建议
1.10.2.0.1数据库不稳定,bug较多,建议升级数据库到10.2.0.4或者10.2.0.5
2.当前standby_file_management为true,建议修改为false,每次手工两边增加数据文件
3.主库添加数据文件后,观察备库的mrp进程是否正常,如果异常
1)尝试启动mrp进程,如果因为新增加的数据文件导致异常,那先使用rman或者其他方法拷贝主库新建立数据文件到备库,然后启动mrp进程
2)如果因为某种原因导致归档日志出现gap,可以从备份中找出归档日志,如果备份中没有,那可以使用基于scn的备份方式来修复该故障。
3)如果scn相差较大,建议直接重建备库

expdp遭遇ORA-39006/ORA-39213故障解决

expdp导出数据遇到ORA-39006/ORA-39213错误,通过执行执行dbms_metadata_util.load_stylesheets解决
expdp工作异常

--导出awr信息
SQL> @?/rdbms/admin/awrextr.sql
…………
Exception encountered in AWR_EXTRACT
ORA-39006: internal error
ORA-39213: Metadata processing is not available
begin
*
ERROR at line 1:
ORA-31623: a job is not attached to this session via the specified handle
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 79
ORA-06512: at "SYS.DBMS_DATAPUMP", line 911
ORA-06512: at "SYS.DBMS_DATAPUMP", line 4710
ORA-06512: at "SYS.DBMS_SWRF_INTERNAL", line 656
ORA-06512: at "SYS.DBMS_SWRF_INTERNAL", line 962
ORA-06512: at line 3

--导出一个表
$ expdp "'/ as sysdba'" dumpfile=xifenfei.dmp tables=scott.t_xifenfei

Export: Release 10.2.0.1.0 - 64bit Production on Wednesday, 31 October, 2012 13:03:20

Copyright (c) 2003, 2005, Oracle.  All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options
ORA-39006: internal error
ORA-39213: Metadata processing is not available

错误提示

$ oerr ora 39006
39006, 00000, "internal error"
// *Cause:  An unexpected error occurred while processing a Data Pump job.
//          Subsequent messages supplied by DBMS_DATAPUMP.GET_STATUS 
//          will further describe the error.
// *Action: Contact Oracle Customer Support.
$ oerr ora 39213
39213, 00000, "Metadata processing is not available"
// *Cause:  The Data Pump could not use the Metadata API.  Typically,
//          this is caused by the XSL stylesheets not being set up properly.
// *Action: Connect AS SYSDBA and execute dbms_metadata_util.load_stylesheets
//          to reload the stylesheets.

解决ORA-39006/ORA-39213问题

--查询数据库已经安装组件
SQL> col COMP_NAME for a35
SQL> select comp_name, version, status from dba_registry;

COMP_NAME                           VERSION                        STATUS
----------------------------------- ------------------------------ ----------------------
Oracle Database Catalog Views       10.2.0.1.0                     VALID
Oracle Database Packages and Types  10.2.0.1.0                     VALID
Oracle Workspace Manager            10.2.0.1.0                     VALID
JServer JAVA Virtual Machine        10.2.0.1.0                     VALID
Oracle XDK                          10.2.0.1.0                     VALID
Oracle Database Java Packages       10.2.0.1.0                     VALID
Oracle Expression Filter            10.2.0.1.0                     VALID
Oracle Data Mining                  10.2.0.1.0                     VALID
Oracle Text                         10.2.0.1.0                     VALID
Oracle XML Database                 10.2.0.1.0                     VALID
Oracle Rules Manager                10.2.0.1.0                     VALID
Oracle interMedia                   10.2.0.1.0                     VALID
OLAP Analytic Workspace             10.2.0.1.0                     VALID
Oracle OLAP API                     10.2.0.1.0                     VALID
OLAP Catalog                        10.2.0.1.0                     VALID
Spatial                             10.2.0.1.0                     VALID
Oracle Enterprise Manager           10.2.0.1.0                     VALID

17 rows selected.

--如果缺少下面组件,使用下面对应的程序安装
Oracle Database Catalog Views
Oracle Database Packages and Types 
JServer JAVA Virtual Machine
Oracle XDK    
Oracle Database Java Packages

--使用下面脚本安装(根据组件选择)
SQL> connect / as sysdba
SQL> @$ORACLE_HOME/javavm/install/initjvm.sql
 
SQL> connect / as sysdba
SQL> @$ORACLE_HOME/xdk/admin/initxml.sql
 
SQL> connect / as sysdba
SQL> @$ORACLE_HOME/rdbms/admin/catjava.sql
 
SQL> connect / as sysdba
SQL> @$ORACLE_HOME/rdbms/admin/utlrp.sql

--执行sys.dbms_metadata_util.load_stylesheets
SQL> execute sys.dbms_metadata_util.load_stylesheets;

PL/SQL procedure successfully completed.

测试expdp导出

$ expdp "'/ as sysdba'" dumpfile=xifenfei.dmp tables=scott.t_xifenfei  Directory=AWR_DIR

Export: Release 10.2.0.1.0 - 64bit Production on Wednesday, 31 October, 2012 14:18:04

Copyright (c) 2003, 2005, Oracle.  All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options
Starting "SYS"."SYS_EXPORT_TABLE_01":  '/******** AS SYSDBA' dumpfile=xifenfei.dmp 
tables=scott.t_xifenfei Directory=AWR_DIR 
Estimate in progress using BLOCKS method...
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
Total estimation using BLOCKS method: 7 MB
Processing object type TABLE_EXPORT/TABLE/TABLE
. . exported "SCOTT"."T_XIFENFEI"                        5.374 MB   57376 rows
Master table "SYS"."SYS_EXPORT_TABLE_01" successfully loaded/unloaded
******************************************************************************
Dump file set for SYS.SYS_EXPORT_TABLE_01 is:
  /data/enmotech/xifenfei.dmp
Job "SYS"."SYS_EXPORT_TABLE_01" successfully completed at 14:18:11

测试证明,在不缺少相关组件的情况下,使用dbms_metadata_util.load_stylesheets可以解决expdp导出报ORA-39006/ORA-39213错误;如果缺少组件,需要先安装对应组件,然后再执行dbms_metadata_util.load_stylesheets解决该问题

dual 缺少同义词故障解决

在最近的一个客户案例中,因为缺少dual同义词,导致ddl语句无法执行。这里_system_trig_enabled参数和upgrade模式两种来解决该问题,整体上来说_system_trig_enabled不用重启数据库终止业务,更加人性化.
缺少dual同义词后果

SQL> create table t_xifenfei_dual as
  2  select * from dba_objects;
select * from dba_objects
              *
ERROR at line 2:
ORA-00604: error occurred at recursive SQL level 1
ORA-00942: table or view does not exist

SQL> alter session set events '942 trace name errorstack level 3';

Session altered.

SQL> create table t_xifenfei_dual as  select * from dba_objects;
create table t_xifenfei_dual as  select * from dba_objects
                                               *
ERROR at line 1:
ORA-00604: error occurred at recursive SQL level 1
ORA-00942: table or view does not exist

--trace文件
*** 2012-09-29 12:37:05.156
ksedmp: internal or fatal error
ORA-00942: table or view does not exist
Current SQL statement for this session:
select dummy from dual where  ora_dict_obj_type = 'SYNONYM' 
AND ora_dict_obj_owner = 'PUBLIC'

--dual 对象
SQL> select object_type,owner from dba_objects where object_name='DUAL';

OBJECT_TYPE         OWNER
------------------- ------------------------------
TABLE               SYS

尝试重建同义词

SQL> create public synonym dual for dual;
create public synonym dual for dual
*
ERROR at line 1:
ORA-00604: error occurred at recursive SQL level 1
ORA-00942: table or view does not exist

_system_trig_enabled参数

SQL> select a.ksppinm name,b.ksppstvl value,a.ksppdesc description
  2    from x$ksppi a,x$ksppcv b
  3   where a.inst_id = USERENV ('Instance')
  4     and b.inst_id = USERENV ('Instance')
  5     and a.indx = b.indx
  6     and upper(a.ksppinm) LIKE upper('%&param%')
  7  order by name
  8  /
Enter value for param: SYSTEM_TRIG_ENABLED
old   6:    and upper(a.ksppinm) LIKE upper('%&param%')
new   6:    and upper(a.ksppinm) LIKE upper('%SYSTEM_TRIG_ENABLED%')

NAME                             VALUE                    DESCRIPTION
-------------------------------- ------------------------ -----------------------------
_system_trig_enabled             TRUE                     are system triggers enabled

设置_SYSTEM_TRIG_ENABLED重建dual同义词

SQL> ALTER SYSTEM SET "_SYSTEM_TRIG_ENABLED"=FALSE SCOPE=MEMORY;

System altered.

SQL> create public synonym dual for dual;

Synonym created.


SQL> ALTER SYSTEM SET "_SYSTEM_TRIG_ENABLED"=true SCOPE=MEMORY;

System altered.

使用upgrade模式创建

SQL> drop PUBLIC SYNONYM dual;

Synonym dropped.

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area  318767104 bytes
Fixed Size                  1267236 bytes
Variable Size             109054428 bytes
Database Buffers          201326592 bytes
Redo Buffers                7118848 bytes
Database mounted.
Database opened.
SQL> create public synonym dual for dual;
create public synonym dual for dual
*
ERROR at line 1:
ORA-00604: error occurred at recursive SQL level 1
ORA-00942: table or view does not exist


SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup upgrage;
SP2-0714: invalid combination of STARTUP options
SQL> startup upgrape;
SP2-0714: invalid combination of STARTUP options
SQL> startup upgrade
ORACLE instance started.

Total System Global Area  318767104 bytes
Fixed Size                  1267236 bytes
Variable Size             109054428 bytes
Database Buffers          201326592 bytes
Redo Buffers                7118848 bytes
Database mounted.
Database opened.
SQL> create public synonym dual for dual;

Synonym created.

SQL> startup force
ORACLE instance started.

Total System Global Area  318767104 bytes
Fixed Size                  1267236 bytes
Variable Size             109054428 bytes
Database Buffers          201326592 bytes
Redo Buffers                7118848 bytes
Database mounted.
Database opened.

类此9430ms (rw) file: kct.c line: 7800 count: 13 total: 26874ms time: 1296124原因分析

ckpt进程的trace文件中出现类似:9430ms (rw) file: kct.c line: 7800 count: 13 total: 26874ms time: 1296124
ckpt进程的trace文件中报如下错误

Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
ORACLE_HOME = /oradb/11.2.0/db
System name:	AIX
Node name:	offondb2
Release:	1
Version:	6
Machine:	00CC83664C00
Instance name: offon2
Redo thread mounted by this instance: 2
Oracle process number: 20
Unix process pid: 19660878, image: oracle@offondb2 (CKPT)


*** 2012-10-26 13:25:00.971
  1: 9430ms (rw) file: kct.c line: 7800 count: 13 total: 26874ms time: 1296124
  2: 1777ms (rw) file: krse.c line: 1800 count: 164 total: 122140ms time: 364176
  3: 1450ms (rw) file: kct.c line: 1011 count: 1 total: 1450ms time: 1594117
  4: 1230ms (rw) file: kcrf.c line: 10012 count: 135 total: 90995ms time: 2630737
  5: 1173ms (ro) file: kcrr.c line: 3525 count: 13 total: 8842ms time: 3645916
  6: 890ms (rw) file: kcrf.c line: 9959 count: 8 total: 4812ms time: 3578222
  7: 830ms (ro) file: kcf.c line: 5306 count: 1 total: 830ms time: 1594116
  8: 677ms (rw) file: kcv.c line: 11783 count: 8 total: 4499ms time: 416869
Control file enqueue hold time tracking dump at time: 2092019

*** 2012-10-26 15:25:14.789
  1: 9430ms (rw) file: kct.c line: 7800 count: 13 total: 26874ms time: 1296124
  2: 1777ms (rw) file: krse.c line: 1800 count: 165 total: 122832ms time: 364176
  3: 1450ms (rw) file: kct.c line: 1011 count: 1 total: 1450ms time: 1594117
  4: 1230ms (rw) file: kcrf.c line: 10012 count: 135 total: 90995ms time: 2630737
  5: 1173ms (ro) file: kcrr.c line: 3525 count: 13 total: 8842ms time: 3645916
  6: 890ms (rw) file: kcrf.c line: 9959 count: 8 total: 4812ms time: 3578222
  7: 830ms (ro) file: kcf.c line: 5306 count: 1 total: 830ms time: 1594116
  8: 677ms (rw) file: kcv.c line: 11783 count: 8 total: 4499ms time: 416869

原因分析并解决

New controlfile enqueue hold time tracking statistics have been added in 11.2 to 
aid diagnosis of controlfile transaction related performance related issues.

To disable the trace set _controlfile_enqueue_holding_time_tracking_size to 0:

- spfile
alter system set "_controlfile_enqueue_holding_time_tracking_size"=0 scope=spfile;

- pfile
_controlfile_enqueue_holding_time_tracking_size=0

A database restart is required.

查询_controlfile_enqueue_holding_time_tracking_size

SQL> col name for a32
SQL> col value for a24
SQL> col description for a70
SQL> set linesize 150
SQL> select a.ksppinm name,b.ksppstvl value,a.ksppdesc description
  2    from x$ksppi a,x$ksppcv b
  3   where a.inst_id = USERENV ('Instance')
  4     and b.inst_id = USERENV ('Instance')
  5     and a.indx = b.indx
  6     and upper(a.ksppinm) LIKE upper('%&param%')
  7  order by name
  8  /
Enter value for param: _controlfile_enqueue_holding_time_tracking_size
old   6:    and upper(a.ksppinm) LIKE upper('%&param%')
new   6:    and upper(a.ksppinm) LIKE upper('%_controlfile_enqueue_holding_time_tracking_size%')
</strong>
NAME                             VALUE      DESCRIPTION
-------------------------------- ---------- ------------------------------------------------
_controlfile_enqueue_holding_tim 10         control file enqueue holding time tracking size
e_tracking_size

补充MOS说明[791417.1]

New controlfile enqueue hold time tracking statistics have been added in 11.2 to aid 
diagnosis of controlfile transaction related performance related issues:

Control File Enqueue AWR Statistics:
max cf enq hold time - The maximum amount of time in milliseconds a client has held the control file enqueue.
total cf enq hold time - The total amount of time in milliseconds all clients have held the control file enqueue.
total number of cf enq holders - The total number of times clients have held the control file enqueue.

Periodically, the CKPT process dumps statistics for the top N control file enqueue holders. 
N defaults to 10, but can be modified with the static hidden 
parameter: _controlfile_enqueue_holding_time_tracking_size.The dump looks like the following:

Preface: "Control file enqueue hold time tracking dump at time: [relative time]".
a. Time the client has held the control file enqueue.
b. Type of client's control file enqueue transaction - rw or ro.
c. File name where the client obtained control file enqueue.
d. Line number where the client obtained control file enqueue.
e. Number of times the client has held the control file enqueue since it became a member of the top N.
f. Total time the client has held the control file in all those times from [e].
g. Relative time the client obtained the control file enqueue from [a].

rman备份出现ORA-19625/ORA-27054解决

RAC环境NFS挂载归档日志使用rman备份出现ORA-19625/ORA-27054错误分析
系统运行环境

OS:AIX 6100-06
DB:11.1.0.6.0 RAC
归档:挂载NFS

rman执行archive log时候报错

sql statement: alter system archive log  current
Starting backup at 25-OCT-11
current log archived
released channel: c1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup command at 02/17/2012 13:03:51
RMAN-06059: expected archived log not found, lost of archived log compromises recoverability
ORA-19625: error identifying file /rarchlogA/1_13775_764866137.dbf
ORA-27054: NFS file system where the file is created or resides is not mounted with correct options
Additional information: 6

这里由于RAC存放归档使用了NFS文件系统,在使用rman备份归档日志执行alter system archive log current的时候发生如下错误.

相关目录挂载情况

$ df -g
Filesystem    GB blocks      Free %Used    Iused %Iused Mounted on
…………
/dev/lv_arch      50.00     43.24   14%      166     1% /arch1
oradb2:/arch2     50.00     35.31   30%      463     1% /arch2
/dev/baklv        90.00     83.82    7%       10     1% /backup


$ mount:
…………
        /dev/lv_oracle   /oracle          jfs2   Oct 14 11:27 rw,log=/dev8
         /dev/lv_arch     /arch1           jfs2   Oct 14 11:27 rw,log=/dev8
     oradb2   /arch2           /arch2           nfs3   Oct 14 11:47

通过这里可以知道,这里使用默认的参数挂载NFS,从而使得NFS在rman工作时候不能正常工作。

错误原因

From  Oracle 10G R2 , Oracle checks the options with which a NFS mount is mounted on the filesystem. 
and this is done to ensure that no corruption of the database can happen as incorrectly 
mounted NFS volumes can result in data corruption. 

There are no single set of NFS mount options that work across all Oracle platforms 
Please ensure that you have the proper mount options specified by the NAS vendor /Vendor user guide 


The exact checks used for an NFS mounted disk vary between platforms but in general 
the basic checks will include the following checks
 

a) The mount table (eg; /etc/mnttab) can be read to check the mount options 
b) The NFS mount is mounted with the &quot;hard&quot; option 
c) The mount options include rsize&gt;=32768 and wsize&gt;=32768 
d) For RAC environments, where NFS disks are supported, the &quot;noac&quot; mount option is used. 

解决方案
1.临时解决方案
As suggested in the bug the workaround recommended is to use the Event 10298.
alter system set events ‘10298 trace name context forever, level 32’;

2.永久解决方案
具体见:http://www.orasos.com/3269.html