主库突然报ORA-16191错，主库无法传输日志到备库

at Apr 08 19:56:26 2023
Thread 1 advanced to log sequence 32548 (LGWR switch)
  Current log# 1 seq# 32548 mem# 0: /u01/app/oracle/oradata/orcl/redo01_a.rdo
  Current log# 1 seq# 32548 mem# 1: /oracle/fast_recovery_area/orcl/redo01_b.rdo
Sat Apr 08 19:56:26 2023
Archived Log entry 61796 added for thread 1 sequence 32547 ID 0x5fc1b26f dest 1:
Sat Apr 08 19:56:27 2023
LNS: Standby redo logfile selected for thread 1 sequence 32548 for destination LOG_ARCHIVE_DEST_2
Sat Apr 08 20:21:29 2023
NSA:  Error 3135 archiving log 1 to 'orcldg'
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_nsa2_26205.trc:
ORA-03135: connection lost contact
LNS: Failed to archive log 1 thread 1 sequence 32548 (3135)
Sat Apr 08 20:26:26 2023
Thread 1 advanced to log sequence 32549 (LGWR switch)
  Current log# 2 seq# 32549 mem# 0: /u01/app/oracle/oradata/orcl/redo02_a.rdo
  Current log# 2 seq# 32549 mem# 1: /oracle/fast_recovery_area/orcl/redo02_b.rdo
Sat Apr 08 20:26:26 2023
Archived Log entry 61798 added for thread 1 sequence 32548 ID 0x5fc1b26f dest 1:
Sat Apr 08 20:26:55 2023
Error 1017 received logging on to the standby
------------------------------------------------------------
Check that the primary and standby are using a password file
and remote_login_passwordfile is set to SHARED or EXCLUSIVE,
and that the SYS password is same in the password files.
      returning error ORA-16191
------------------------------------------------------------
PING[ARC2]: Heartbeat failed to connect to standby 'orcldg'. Error is 16191.
Sat Apr 08 20:27:57 2023
Error 1017 received logging on to the standby
------------------------------------------------------------
Check that the primary and standby are using a password file
and remote_login_passwordfile is set to SHARED or EXCLUSIVE,
and that the SYS password is same in the password files.
      returning error ORA-16191
------------------------------------------------------------
PING[ARC2]: Heartbeat failed to connect to standby 'orcldg'. Error is 16191.

备库报错

Sat Apr 08 19:29:09 2023
Media Recovery Waiting for thread 1 sequence 32548 (in transit)
Recovery of Online Redo Log: Thread 1 Group 12 Seq 32548 Reading mem 0
  Mem# 0: /u01/app/oracle/oradata/orcl/std_redo12.log
Sat Apr 08 21:52:19 2023
RFS[9]: Possible network disconnect with primary database

检查主备库remote_login_passwordfile参数,都是为EXCLUSIVE,确认没有问题

--备库
SQL> show parameter pass;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
remote_login_passwordfile            string      EXCLUSIVE
SQL> select open_mode from v$database;

OPEN_MODE
--------------------
READ ONLY WITH APPLY

--主库
SQL> select database_role from v$database;

DATABASE_ROLE
----------------
PRIMARY

SQL> show parameter pass;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
remote_login_passwordfile            string      EXCLUSIVE

检查主备库密码文件,证明主备库密码文件一致,也不是密码文件引起不同步

--主库
[oracle@oracle1 trace]$ cd $ORACLE_HOME/dbs
[oracle@oracle1 dbs]$ echo $ORACLE_SID
orcl
[oracle@oracle1 dbs]$ ls -ltr orapw*
-rw-r-----. 1 oracle oinstall 1536 Sep 16  2022 orapworcl
[oracle@oracle1 dbs]$ md5sum orapworcl 
d27c234b5131b4d2e13b1eeb4388f0eb  orapworcl

--备库
[oracle@oracle2 trace]$ cd $ORACLE_HOME/dbs
[oracle@oracle2 dbs]$ echo $ORACLE_SID
orcl
[oracle@oracle2 dbs]$ ls -l orapw*
-rw-r----- 1 oracle oinstall 1536 Sep 16  2022 orapworcl
[oracle@oracle2 dbs]$ md5sum orapworcl 
d27c234b5131b4d2e13b1eeb4388f0eb  orapworcl

查看数据库有logon触发器

SQL> select owner,trigger_name from dba_triggers where triggering_event LIKE '%LOGON%';

OWNER                          TRIGGER_NAME
------------------------------ ------------------------------
DBTOOLS                        TRIGGER_LOGINLIMIT

根据Error 1017 / ORA-16191 In Standby Alertlog (Doc ID 2225190.1)中描述logon触发器可能导致dg传输日志失败.