PostgreSQL查询一个表相关的所有oid

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:PostgreSQL查询一个表相关的所有oid

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

在pg中查询一个表相关的所有oid(表OID、TOAST表OID、索引OID、TOAST表的索引OID、约束OID)

-- 替换这里的表名和模式名
\set search_path = public;
\set table_name '\'AO_6384AB_DISCOVERED\''
\set schema_name '\'public\''

WITH table_info AS (
    -- 基础表信息:表OID、TOAST表OID
    SELECT
        c.oid AS table_oid,
        c.relname AS table_name,
        c.reltoastrelid AS toast_oid,
        n.nspname AS schema_name
    FROM
        pg_catalog.pg_class c
    JOIN
        pg_catalog.pg_namespace n ON c.relnamespace = n.oid
    WHERE
        c.relname = :table_name
        AND n.nspname = :schema_name
        AND c.relkind = 'r'
),
index_info AS (
    -- 索引信息:索引OID
    SELECT
        t.table_oid,
        i.oid AS object_oid,
        i.relname AS object_name,
        'index' AS object_type
    FROM
        pg_catalog.pg_class i
    JOIN
        pg_catalog.pg_index idx ON i.oid = idx.indexrelid
    JOIN
        table_info t ON idx.indrelid = t.table_oid
    WHERE
        i.relkind = 'i'
),
toast_info AS (
    -- TOAST表信息:TOAST表OID
    SELECT
        t.table_oid,
        t.toast_oid AS object_oid,
        c.relname AS object_name,
        'toast_table' AS object_type
    FROM
        table_info t
    JOIN
        pg_catalog.pg_class c ON t.toast_oid = c.oid
    WHERE
        t.toast_oid <> 0
),
toast_index_info AS (
    -- TOAST索引信息:TOAST表的索引OID
    SELECT
        t.table_oid,
        i.oid AS object_oid,
        i.relname AS object_name,
        'toast_index' AS object_type
    FROM
        table_info t
    JOIN
        pg_catalog.pg_class c ON t.toast_oid = c.oid
    JOIN
        pg_catalog.pg_index idx ON c.oid = idx.indrelid
    JOIN
        pg_catalog.pg_class i ON idx.indexrelid = i.oid
    WHERE
        t.toast_oid <> 0
        AND i.relkind = 'i'
),
constraint_info AS (
    -- 约束信息:约束OID
    SELECT
        t.table_oid,
        con.oid AS object_oid,
        con.conname AS object_name,
        CASE con.contype
            WHEN 'p' THEN 'primary_key'
            WHEN 'u' THEN 'unique_constraint'
            WHEN 'f' THEN 'foreign_key'
            WHEN 'c' THEN 'check_constraint'
            ELSE 'other_constraint'
        END AS object_type
    FROM
        pg_catalog.pg_constraint con
    JOIN
        table_info t ON con.conrelid = t.table_oid
)
-- 合并所有结果,先输出表本身的信息,再输出其他关联对象
SELECT
    t.table_oid AS object_oid,
    t.table_name AS object_name,
    'table' AS object_type,
    t.schema_name AS schema_name
FROM
    table_info t
UNION ALL
SELECT
    object_oid,
    object_name,
    object_type,
    (SELECT schema_name FROM table_info) AS schema_name
FROM
    index_info
UNION ALL
SELECT
    object_oid,
    object_name,
    object_type,
    (SELECT schema_name FROM table_info) AS schema_name
FROM
    toast_info
UNION ALL
SELECT
    object_oid,
    object_name,
    object_type,
    (SELECT schema_name FROM table_info) AS schema_name
FROM
    toast_index_info
UNION ALL
SELECT
    object_oid,
    object_name,
    object_type,
    (SELECT schema_name FROM table_info) AS schema_name
FROM
    constraint_info
ORDER BY
    object_type;

PostgreSQL oid文件替换实现数据访问

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:PostgreSQL oid文件替换实现数据访问

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

对于postgresql数据库,我们都知道他的数据在独立的oid文件里面,而字典主要在pg_type/1247,pg_class/1259,pg_attribute/1260等对象中,而数据是存储在独立的oid和toast oid文件中.如果通过提供oid文件,可以实现狸猫换太子的恢复数据.这里做一个简单测试,大概思路是这样的库里面有一个表a,然后创建一个空表b,然后把a表相关的oid文件复制成b表的oid文件,然后验证数据是否可以正常查询
创建一个空的b表,结构和a表一致

[postgres@xifenfei 13676]$ psql
psql (12.8)
Type "help" for help.

postgres=# CREATE TABLE "public"."AO_6384AB_DISCOVERED1" (
postgres(#   "DATE" "pg_catalog"."timestamp",
postgres(#   "ID" "pg_catalog"."int4",
postgres(#   "KEY" "pg_catalog"."varchar" COLLATE "pg_catalog"."default",
postgres(#   "PLUGIN_KEY" "pg_catalog"."varchar" COLLATE "pg_catalog"."default",
postgres(#   "USER_KEY" "pg_catalog"."varchar" COLLATE "pg_catalog"."default"
postgres(# )
postgres-# ;
CREATE TABLE

检查a/b的情况

postgres=# select count(1) from "AO_6384AB_DISCOVERED";
 count 
-------
   544
(1 row)

postgres=# select count(1) from "AO_6384AB_DISCOVERED1";
 count 
-------
     0
(1 row)

postgres=# \d "AO_6384AB_DISCOVERED"
                    Table "public.AO_6384AB_DISCOVERED"
   Column   |            Type             | Collation | Nullable | Default 
------------+-----------------------------+-----------+----------+---------
 DATE       | timestamp without time zone |           |          | 
 ID         | integer                     |           |          | 
 KEY        | character varying           |           |          | 
 PLUGIN_KEY | character varying           |           |          | 
 USER_KEY   | character varying           |           |          | 

postgres=# \d "AO_6384AB_DISCOVERED1"
                   Table "public.AO_6384AB_DISCOVERED1"
   Column   |            Type             | Collation | Nullable | Default 
------------+-----------------------------+-----------+----------+---------
 DATE       | timestamp without time zone |           |          | 
 ID         | integer                     |           |          | 
 KEY        | character varying           |           |          | 
 PLUGIN_KEY | character varying           |           |          | 
 USER_KEY   | character varying           |           |          | 

存在数据的a/b表相关oid信息



postgres=# select oid,relname,relfilenode from pg_class where relname like '%AO_6384AB_DISCOVERED%';
  oid  |        relname        | relfilenode 
-------+-----------------------+-------------
 16516 | AO_6384AB_DISCOVERED  |       16516
 17069 | AO_6384AB_DISCOVERED1 |       17069
(2 rows)

postgres=# select oid,relname,relfilenode from pg_class where relname like '%16516%' or  relname like '%17069%'  ;
  oid  |       relname        | relfilenode 
-------+----------------------+-------------
 16519 | pg_toast_16516       |       16519
 16521 | pg_toast_16516_index |       16521
 17072 | pg_toast_17069       |       17072
 17074 | pg_toast_17069_index |       17074
(4 rows)

关闭数据库把a表相关的oid文件拷贝替换b表的oid文件

[postgres@xifenfei 13676]$ pg_ctl stop -D /pgdata
waiting for server to shut down....2025-12-12 20:00:58.804 HKT [21445] LOG:  received fast shutdown request
2025-12-12 20:00:58.805 HKT [21445] LOG:  aborting any active transactions
2025-12-12 20:00:58.805 HKT [21471] FATAL:  terminating connection due to administrator command
2025-12-12 20:00:58.805 HKT [21473] FATAL:  terminating connection due to administrator command
2025-12-12 20:00:58.805 HKT [21856] FATAL:  terminating connection due to administrator command
2025-12-12 20:00:58.806 HKT [21472] FATAL:  terminating connection due to administrator command
2025-12-12 20:00:58.806 HKT [21445] LOG: background worker"logical replication launcher"(PID 252)exited with exit code 1
2025-12-12 20:00:58.807 HKT [21447] LOG:  shutting down
2025-12-12 20:00:58.811 HKT [21445] LOG:  database system is shut down
 done
server stopped
[postgres@xifenfei 13676]$ cp  16516 17069
[postgres@xifenfei 13676]$ cp  16519 17072
[postgres@xifenfei 13676]$ cp  16521 17074
[postgres@xifenfei 13676]$ pg_ctl start -D /pgdata
waiting for server to start....2025-12-12 20:02:05.985 HKT [22241] LOG:  
starting PostgreSQL 12.8 on x86_64-pc-linux-gnu, 
compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-26.0.1), 64-bit
2025-12-12 20:02:05.985 HKT [22241] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2025-12-12 20:02:05.986 HKT [22241] LOG:  listening on IPv6 address "::", port 5432
2025-12-12 20:02:05.986 HKT [22241] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2025-12-12 20:02:05.993 HKT [22242] LOG:  database system was shut down at 2025-12-12 20:00:58 HKT
2025-12-12 20:02:05.995 HKT [22241] LOG:  database system is ready to accept connections
 done
server started

查询a/b表数据

[postgres@xifenfei 13676]$ psql
psql (12.8)
Type "help" for help.

postgres=# select count(1) from "AO_6384AB_DISCOVERED1";
 count 
-------
   544
(1 row)

postgres=# select count(1) from "AO_6384AB_DISCOVERED";
 count 
-------
   544
(1 row)

通过上述从测试,证明创建一个新的表结构,完全可以访问老的oid文件中的数据

模拟sql server故障备份完成恢复实现数据0丢失

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:模拟sql server故障备份完成恢复实现数据0丢失

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

在sql server数据库中,使用备份还原,可以用来恢复的备份文件只能是全备、增量备份、事务日志备份,在很多情况下,sql server生产库异常,最后的事务日志没有来得及备份(但是ldf文件存在),这种情况下如何最大限度恢复数据实现数据0丢失,其实最重要的关键点就是对最后异常库的ldf文件的事务日志进行备份,在sql server里面给这个操作起了一个专业的词叫做:备份结尾日志(Tail of log).我这边通过一个实验来模拟整个操作过程.
1. 创建一个库,恢复模式为完整
s1


2.创建表并插入一条记录,然后进行全备
s2
s3

3. 再插入一条记录,并做事务日志备份
s4
s5

4. 再插入一条记录,并在脱机情况下删除掉数据库的mdf文件(模拟故障),在实际生产中如果原机器损坏,可以把在新机器上安装同版本sql,然后创建同名库,把ldf文件替换
s6
s8

5. 进行最后的ldf(事务日志的备份),在sql server中这个备份操作叫做“备份结尾日志(Tail of log)”,先尝试直接使用图形化工具进行备份
s9

6. 图形化这种情况无法直接备份,使用命令行形式成功备份
s10

这里备份命令带WITH NORECOVERY、WITH NO_TRUNCATE、WITH CONTINUE_AFTER_ERROR相关内容官方描述:

如果数据库处于联机状态并且您计划对数据库执行还原操作,则从备份日志结尾开始。 
要避免联机数据库出错,必须使用 BACKUP Transact-SQL 语句的 WITH NORECOVERY 选项。

如果数据库处于脱机状态而无法启动,则需要还原数据库,从备份日志结尾开始。 
由于此时不会发生任何事务,因此请使用 WITH NO_TRUNCATE 选项。 
NO_TRUNCATE 实际上与仅复制事务日志备份相同。 由于此时不会发生任何事务,因此可以选择使用 WITH NORECOVERY。

如果数据库损坏,则尝试使用 WITH CONTINUE_AFTER_ERROR 语句的 BACKUP 选项执行结尾日志备份。

在损坏的数据库上,仅当日志文件未受损、数据库处于支持结尾日志备份的状态并且数据库不包含任何大容量日志更改时,
日志尾部备份才会成功。 如果无法创建结尾日志备份,则最新日志备份后提交的任何事务都将丢失。

7. 进行数据库还原操作(这里先删除了故障库和对应ldf文件),也可以还原成其他库名和不同路径,也可以通过命令行进行还原(全备和除最后一个事务日志之前的备份需要使用WITH NORECOVERY)
s11
s12
s13


基于上述测试,完美的恢复故障之前的所有数据

sql server 事务日志备份异常恢复案例

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:sql server 事务日志备份异常恢复案例

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有客户的sql server数据库运行在双机环境中,由于心跳网络异常导致双机频繁切换最终数据库损坏DBCC检查报大量错误

DBCC CHECKDB('OLTP') WITH NO_INFOMSGS, ALL_ERRORMSGS

Msg 8909, Level 16, State 1, Line 1
Table error: Object ID 0, index ID -1, partition ID 0, alloc unit ID 28147935764938752 (type Unknown), page ID (1:33059984) contains an incorrect page ID in its page header. The PageId in the page header = (68:3276868).
Msg 8909, Level 16, State 1, Line 1
Table error: Object ID 0, index ID -1, partition ID 0, alloc unit ID 11540680206712832 (type Unknown), page ID (1:33059985) contains an incorrect page ID in its page header. The PageId in the page header = (102:6488116).
Msg 8909, Level 16, State 1, Line 1
Table error: Object ID 0, index ID -1, partition ID 0, alloc unit ID 16888988233302016 (type Unknown), page ID (1:33059986) contains an incorrect page ID in its page header. The PageId in the page header = (93:6619252).
Msg 8909, Level 16, State 1, Line 1
Table error: Object ID 0, index ID -1, partition ID 0, alloc unit ID 16888988233302016 (type Unknown), page ID (1:33059987) contains an incorrect page ID in its page header. The PageId in the page header = (93:6619252).
Msg 8909, Level 16, State 1, Line 1
Table error: Object ID 0, index ID -1, partition ID 0, alloc unit ID 16888988233302016 (type Unknown), page ID (1:33059988) contains an incorrect page ID in its page header. The PageId in the page header = (93:6619252).
Msg 8909, Level 16, State 1, Line 1
Table error: Object ID 0, index ID -1, partition ID 0, alloc unit ID 28147836977938432 (type Unknown), page ID (1:33059989) contains an incorrect page ID in its page header. The PageId in the page header = (73:6619248).
Msg 8909, Level 16, State 1, Line 1
……………………
Object ID 1961110077, index ID 0, partition ID 72057594217627648, alloc unit ID 72057594256687104 (type In-row data): Page (1:36535484) could not be processed.  See other errors for details.
Msg 8928, Level 16, State 1, Line 1
Object ID 1961110077, index ID 0, partition ID 72057594217627648, alloc unit ID 72057594256687104 (type In-row data): Page (1:36535485) could not be processed.  See other errors for details.
Msg 8928, Level 16, State 1, Line 1
Object ID 1961110077, index ID 0, partition ID 72057594217627648, alloc unit ID 72057594256687104 (type In-row data): Page (1:36535486) could not be processed.  See other errors for details.
Msg 8928, Level 16, State 1, Line 1
Object ID 1961110077, index ID 0, partition ID 72057594217627648, alloc unit ID 72057594256687104 (type In-row data): Page (1:36535487) could not be processed.  See other errors for details.
CHECKDB found 0 allocation errors and 24 consistency errors in table 'CIOMessage' (object ID 1961110077).
CHECKDB found 0 allocation errors and 17955 consistency errors in database 'OLTP'.

Completion time: 2025-11-19T17:13:03.2762122+08:00

客户每天做全库备份,每4小时做事务日志备份,备份类似这样的情况
sql0


客户尝试使用全备进行恢复,结果发现只有13日的全备是好的,可以还原出来数据库,其他备份还原直接报错,基于这样的情况,可以希望把数据恢复到11月19日.我接手这个故障之后,先尝试还原13日的备份
sql6

然后尝试人工应用事务日志备份,类似命令

RESTORE LOG OLTP1121 FROM DISK = 'D:\share\OLTP_backup_2025_11_13_030001_7745248.trn' WITH NORECOVERY
RESTORE LOG OLTP1121 FROM DISK = 'D:\share\OLTP_backup_2025_11_13_060001_3581210.trn' WITH NORECOVERY
RESTORE LOG OLTP1121 FROM DISK = 'D:\share\OLTP_backup_2025_11_13_090001_2856408.trn' WITH NORECOVERY
RESTORE LOG OLTP1121 FROM DISK = 'D:\share\OLTP_backup_2025_11_13_120002_0713663.trn' WITH NORECOVERY
RESTORE LOG OLTP1121 FROM DISK = 'D:\share\OLTP_backup_2025_11_13_150001_7305524.trn' WITH NORECOVERY
RESTORE LOG OLTP1121 FROM DISK = 'D:\share\OLTP_backup_2025_11_13_180000_9123036.trn' WITH NORECOVERY
RESTORE LOG OLTP1121 FROM DISK = 'D:\share\OLTP_backup_2025_11_13_210001_3663138.trn' WITH NORECOVERY
RESTORE LOG OLTP1121 FROM DISK = 'D:\share\OLTP_backup_2025_11_14_000001_1605695.trn' WITH NORECOVERY
RESTORE LOG OLTP1121 FROM DISK = 'D:\share\OLTP_backup_2025_11_14_030001_7280782.trn' WITH NORECOVERY
………………
RESTORE LOG OLTP1121 FROM DISK = 'D:\share\OLTP_backup_2025_11_17_180001_1343952.trn' WITH NORECOVERY

结果在OLTP_backup_2025_11_17_180001_1343952文件位置报错

Processed 0 pages for database 'OLTP_1121', file 'OLTP' on file 1.
Processed 10388 pages for database 'OLTP1121', file 'OLTP_log' on file 1.
Msg 9004, Level 16, State 3, Line 1
An error occurred while processing the log for database 'OLTP_1121'.  If possible, restore from backup. 
If a backup is not available, it might be necessary to rebuild the log.
Msg 3013, Level 16, State 1, Line 1
RESTORE LOG is terminating abnormally.

Completion time: 2025-11-21T13:41:54.2352031+08:00

通过图形化界面进行事务日志恢复也报错
sql7


基于这样的情况,数据库层面的正常恢复途径只能恢复到11月17日18时左右数据,因为后面的日志发生了损坏,无法继续正常恢复,对于这种情况,我们这边使用日志解析工具对剩余事务日志备份进行解析,生成.sql文件
sql3
sql1

然后客户把解析出来的.sql文件依次在会到11月17日18时的库上面去执行,这样顺利吧客户整体数据库恢复到最新状态,完成本次恢复任务(注意后续可能一些类似序列值需要调整)

win平台挂起Oracle数据库启动进程

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:win平台挂起Oracle数据库启动进程

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

对于oracle的有些故障,如果数据库完全open会报错,导致启动失败,这个时候,我们可以尝试在数据库open到一半的过程中把启动过程挂起,然后登录数据库对字典进行修复,然后就可以正常启动数据库。非win平台可以通过gdb挂起数据库启动过程,因为win系统是进程模式,无法通过gdb挂起来实现,通过自己写程序修改oracle内存来实现这个功能

Oracle Recovery Tools工具中,点击修改内存—>检索Oracle数据库进程—>选择需要操作的数据库进程—>挂起Oracle进程 就可以实现oracle数据库在open的过程中挂住
QQ20251208-180202


然后其他会话登录数据库,进行需要的操作,再重新点击取消挂起数据库可以继续open成功
QQ20251208-174237