MySQL数据库文件丢失恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:MySQL数据库文件丢失恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

客户服务器重启,mysql相关数据文件丢失
20231206095845


通过底层工具进行分析,无法正确恢复数据库名字,一个个单个ibd文件(而且很多本身是错误的)
20231210224010

对于这种情况,通过mysql block扫描恢复出来page文件
20231210224106

恢复出来客户需要数据
20231210224248
20231207215458

这个客户出现该故障的原因大概率是由于文件系统损坏导致.最终可以比较好的恢复,主要是故障之后第一时间保护了现场,没进行任何的写入覆盖,如果覆盖了神仙也没有办法.如果此类问题无法自行解决或者恢复效果不好,可以联系我们进行技术支持,最大限度抢救和数据,减少损失
电话/微信:17813235971    Q Q:107644445QQ咨询惜分飞    E-Mail:dba@xifenfei.com

unknown variable ‘default-character-set=utf8′ 处理

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:unknown variable ‘default-character-set=utf8′ 处理

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

分享一例由于mysql参数配置不当,当时mysql无法正常启动的case

[root@XIFENFEI ~]# service mysql start
Redirecting to /bin/systemctl start mysql.service
Job for mysqld.service failed because the control process exited with error code. 
See "systemctl status mysqld.service" and "journalctl -xe" for details.
[root@XIFENFEI ~]# systemctl status mysqld.service
 mysqld.service - LSB: start and stop MySQL
   Loaded: loaded (/etc/rc.d/init.d/mysqld; bad; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2023-05-22 16:09:57 CST; 13s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 101951 ExecStop=/etc/rc.d/init.d/mysqld stop (code=exited, status=0/SUCCESS)
  Process: 102545 ExecStart=/etc/rc.d/init.d/mysqld start (code=exited, status=1/FAILURE)

May 22 16:09:53 RHAA08-0901 systemd[1]: Starting LSB: start and stop MySQL...
May 22 16:09:57 RHAA08-0901 mysqld[102545]: Starting MySQL.... ERROR! 
            The server quit without updating PID file (/www/se....pid).
May 22 16:09:57 RHAA08-0901 systemd[1]: mysqld.service: control process exited, code=exited status=1
May 22 16:09:57 RHAA08-0901 systemd[1]: Failed to start LSB: start and stop MySQL.
May 22 16:09:57 RHAA08-0901 systemd[1]: Unit mysqld.service entered failed state.
May 22 16:09:57 RHAA08-0901 systemd[1]: mysqld.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

mysql日志

2023-05-22T06:59:57.646266Z 0 [Warning] option 'max_allowed_packet': unsigned value 107374182400 adjusted to 1073741824
2023-05-22T06:59:57.646480Z 0 [Note] --secure-file-priv is set to NULL. 
          Operations related to importing and exporting data are disabled
2023-05-22T06:59:57.646548Z 0 [Note] /www/server/mysql/bin/mysqld (mysqld 5.7.41-log) starting as process 79451 ...
2023-05-22T06:59:57.726167Z 0 [Note] InnoDB: PUNCH HOLE support available
2023-05-22T06:59:57.726257Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2023-05-22T06:59:57.726269Z 0 [Note] InnoDB: Uses event mutexes
2023-05-22T06:59:57.726286Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2023-05-22T06:59:57.726296Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.12
2023-05-22T06:59:57.726306Z 0 [Note] InnoDB: Using Linux native AIO
2023-05-22T06:59:57.726975Z 0 [Note] InnoDB: Number of pools: 1
2023-05-22T06:59:57.727161Z 0 [Note] InnoDB: Using CPU crc32 instructions
2023-05-22T06:59:57.733999Z 0 [Note] InnoDB: Initializing buffer pool, total size = 4G, instances = 8, chunk size = 128M
2023-05-22T06:59:58.051369Z 0 [Note] InnoDB: Completed initialization of buffer pool
2023-05-22T06:59:58.087559Z 0 [Note] InnoDB: If the mysqld execution user is authorized,
 page cleaner thread priority can be changed. See the man page of setpriority().
2023-05-22T06:59:58.099727Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2023-05-22T06:59:58.202034Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2023-05-22T06:59:58.202150Z 0 [Note] InnoDB: Setting file '/www/server/data/ibtmp1' size to 12 MB. 
Physically writing the file full; Please wait ...
2023-05-22T06:59:58.242233Z 0 [Note] InnoDB: File '/www/server/data/ibtmp1' size is now 12 MB.
2023-05-22T06:59:58.244128Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2023-05-22T06:59:58.244163Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2023-05-22T06:59:58.244910Z 0 [Note] InnoDB: Waiting for purge to start
2023-05-22T06:59:58.295153Z 0 [Note] InnoDB: 5.7.41 started; log sequence number 1184000067
2023-05-22T06:59:58.295727Z 0 [Note] InnoDB: Loading buffer pool(s) from /www/server/data/ib_buffer_pool
2023-05-22T06:59:58.296135Z 0 [Note] Plugin 'FEDERATED' is disabled.
2023-05-22T06:59:58.296288Z 0 [Note] InnoDB: Buffer pool(s) load completed at 230522 14:59:58
2023-05-22T06:59:58.299095Z 0 [ERROR] unknown variable 'default-character-set=utf8'
2023-05-22T06:59:58.299131Z 0 [ERROR] Aborting

2023-05-22T06:59:58.299179Z 0 [Note] Binlog end
2023-05-22T06:59:58.299530Z 0 [Note] Shutting down plugin 'ngram'
2023-05-22T06:59:58.299560Z 0 [Note] Shutting down plugin 'partition'
2023-05-22T06:59:58.299572Z 0 [Note] Shutting down plugin 'BLACKHOLE'
2023-05-22T06:59:58.299588Z 0 [Note] Shutting down plugin 'ARCHIVE'
2023-05-22T06:59:58.299599Z 0 [Note] Shutting down plugin 'MyISAM'
2023-05-22T06:59:58.299626Z 0 [Note] Shutting down plugin 'MRG_MYISAM'
2023-05-22T06:59:58.299641Z 0 [Note] Shutting down plugin 'MEMORY'
2023-05-22T06:59:58.299657Z 0 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2023-05-22T06:59:58.299797Z 0 [Note] Shutting down plugin 'CSV'
2023-05-22T06:59:58.299811Z 0 [Note] Shutting down plugin 'INNODB_SYS_VIRTUAL'
2023-05-22T06:59:58.299822Z 0 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2023-05-22T06:59:58.299832Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
2023-05-22T06:59:58.299843Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
2023-05-22T06:59:58.299853Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2023-05-22T06:59:58.299863Z 0 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2023-05-22T06:59:58.299873Z 0 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2023-05-22T06:59:58.299883Z 0 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2023-05-22T06:59:58.299893Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
2023-05-22T06:59:58.299904Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2023-05-22T06:59:58.299914Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
2023-05-22T06:59:58.299924Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
2023-05-22T06:59:58.299934Z 0 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2023-05-22T06:59:58.299944Z 0 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
2023-05-22T06:59:58.299954Z 0 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2023-05-22T06:59:58.299964Z 0 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
2023-05-22T06:59:58.299974Z 0 [Note] Shutting down plugin 'INNODB_METRICS'
2023-05-22T06:59:58.299984Z 0 [Note] Shutting down plugin 'INNODB_TEMP_TABLE_INFO'
2023-05-22T06:59:58.299995Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
2023-05-22T06:59:58.300005Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
2023-05-22T06:59:58.300015Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2023-05-22T06:59:58.300025Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET'
2023-05-22T06:59:58.300035Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2023-05-22T06:59:58.300045Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2023-05-22T06:59:58.300055Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM'
2023-05-22T06:59:58.300066Z 0 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2023-05-22T06:59:58.300075Z 0 [Note] Shutting down plugin 'INNODB_CMP'
2023-05-22T06:59:58.300085Z 0 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2023-05-22T06:59:58.300095Z 0 [Note] Shutting down plugin 'INNODB_LOCKS'
2023-05-22T06:59:58.300105Z 0 [Note] Shutting down plugin 'INNODB_TRX'
2023-05-22T06:59:58.300116Z 0 [Note] Shutting down plugin 'InnoDB'
2023-05-22T06:59:58.300271Z 0 [Note] InnoDB: FTS optimize thread exiting.
2023-05-22T06:59:58.300437Z 0 [Note] InnoDB: Starting shutdown...
2023-05-22T06:59:58.400764Z 0 [Note] InnoDB: Dumping buffer pool(s) to /www/server/data/ib_buffer_pool
2023-05-22T06:59:58.401153Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 230522 14:59:58
2023-05-22T07:00:00.340582Z 0 [Note] InnoDB: Shutdown completed; log sequence number 1184000086
2023-05-22T07:00:00.343128Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2023-05-22T07:00:00.343157Z 0 [Note] Shutting down plugin 'sha256_password'
2023-05-22T07:00:00.343165Z 0 [Note] Shutting down plugin 'mysql_native_password'
2023-05-22T07:00:00.343347Z 0 [Note] Shutting down plugin 'binlog'
2023-05-22T07:00:00.344547Z 0 [Note] /www/server/mysql/bin/mysqld: Shutdown complete

提示比较明显由于default-character-set=utf8参数设置不当当时,检查my.cnf配置,发现
在mysqld中配置了default-character-set=utf8,该参数正确配置为:character-set-server = utf8,设置正确参数值之后,mysql启动正常

[root@XIFENFEI data]# service mysql start
Redirecting to /bin/systemctl start mysql.service
[root@XIFENFEI data]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.7.41-log Source distribution

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| 365xxxy             |
| xxxxhu         |
| xxxiuye              |
| mysql              |
| performance_schema |
| xxxxchegn         |
| sxxxxp               |
| sys                |
| xxxxc               |
+--------------------+
10 rows in set (0.00 sec)

修改MySQL的ib_logfile大小和组数

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:修改MySQL的ib_logfile大小和组数

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

在某些情况下,需要修改MySQL的ib_logfile大小和组数(类似oracle redo),以下演示在MySQL 8.0修改ib_logfile大小和组数操作
查看当前ib_logfile情况

mysql> select version();
+-----------+
| version() |
+-----------+
| 8.0.21    |
+-----------+
1 row in set (0.00 sec)

mysql> show variables like '%innodb_log_file%';
+---------------------------+-----------+
| Variable_name             | Value     |
+---------------------------+-----------+
| innodb_log_file_size      | 134217728 |
| innodb_log_files_in_group | 2         |
+---------------------------+-----------+
2 rows in set, 1 warning (0.00 sec)

mysql> show variables like '%innodb_log_group_home_dir%';
+---------------------------+-------+
| Variable_name             | Value |
+---------------------------+-------+
| innodb_log_group_home_dir | .\    |
+---------------------------+-------+
1 row in set, 1 warning (0.00 sec)

mysql> show variables like '%datadir%';
+---------------+---------------+
| Variable_name | Value         |
+---------------+---------------+
| datadir       | E:\MySQL\8.0\ |
+---------------+---------------+
1 row in set, 1 warning (0.00 sec)

C:\Users\XFF>dir E:\MySQL\8.0\ib_log*
 驱动器 E 中的卷是 SSD
 卷的序列号是 98A5-7F8E

 E:\MySQL\8.0 的目录

2022-07-16  14:14       134,217,728 ib_logfile0
2022-07-03  17:30       134,217,728 ib_logfile1
               2 个文件    268,435,456 字节
               0 个目录 807,501,471,744 可用字节

当前ib_logfile成员为2个,每个128M,后续计划修改为成为3个,每个256M

修改my.ini参数

innodb_log_files_in_group=3
innodb_log_file_size=256M

重启mysql服务

C:\Users\XFF>dir E:\MySQL\8.0\ib_log*
 驱动器 E 中的卷是 SSD
 卷的序列号是 98A5-7F8E

 E:\MySQL\8.0 的目录

2022-07-16  14:19       268,435,456 ib_logfile0
2022-07-16  14:19       268,435,456 ib_logfile1
2022-07-16  14:19       268,435,456 ib_logfile2
               3 个文件    805,306,368 字节
               0 个目录 806,964,072,448 可用字节


mysql> show variables like '%innodb_log_file%';
+---------------------------+-----------+
| Variable_name             | Value     |
+---------------------------+-----------+
| innodb_log_file_size      | 268435456 |
| innodb_log_files_in_group | 3         |
+---------------------------+-----------+
2 rows in set, 1 warning (0.00 sec)

在8.0版本中直接修改成功,如果是以前MySQL版本,修改过程可能遭遇InnoDB: Error: log file ./ib_logfile0 is of different size 0 268435456 bytes错误,类似这样的需要删除ib_logfile文件,启动MySQL服务重新生成ib_logfile文件(在8.0版本中直接自动删除并重建)

mysql drop table 恢复

对于MySQL数据库的innodb引擎的数据库中,由于误操作删除表,或者由于sqldump自动生成语句含drop table create table语句导致数据丢失,在没有覆盖的情况下,可以实现完美恢复
创建测试表

mysql> CREATE TABLE recover.`t_drop` (
    ->   `messageId` varchar(30) NOT NULL,
    ->   `msgContent` varchar(1000) default NULL,
    ->   `scheduleDate` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
    ->   `deliverState` int(1) default NULL,
    ->   PRIMARY KEY  (`messageId`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.00 sec)

mysql> insert into recover.t_drop select messageId,msgContent,scheduleDate,deliverState from sms_send_record;
Query OK, 11 rows affected (0.00 sec)
Records: 11  Duplicates: 0  Warnings: 0

mysql> select * from recover.`t_drop`;
+--------------------+----------------------------------------------------------------------------------+---------------------+--------------+
| messageId          | msgContent                                                                       | scheduleDate        | deliverState |
+--------------------+----------------------------------------------------------------------------------+---------------------+--------------+
| 10235259536125650  | 尊敬的用户您好:您的手机验证码为474851如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
| 10235353811295807  | 尊敬的用户您好:您的手机验证码为444632如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
| 102354211240398235 | 尊敬的用户您好:您的手机验证码为478503如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
| 102354554052884567 | 尊敬的用户您好:您的手机验证码为216825如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
| 132213454294519126 | 尊敬的用户您好:您的手机验证码为854812如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
| 82329022242584577  | 尊敬的用户您好:您的手机验证码为253127如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
| 82329022242584581  | 尊敬的用户您好:您的手机验证码为253127如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
| 8233400415607376   | 尊敬的用户您好:您的手机验证码为338470如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
| 82334502212106951  | 尊敬的用户您好:您的手机验证码为916515如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
| 82339012756833423  | 尊敬的用户您好:您的手机验证码为396108如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
| 8234322198577796   | 尊敬的用户您好:您的手机验证码为935297如非本人操作,请拨打奥斯卡客服:400-620-757    | 2010-01-01 00:00:00 |            0 | 
+--------------------+----------------------------------------------------------------------------------+---------------------+--------------+
11 rows in set (0.00 sec)

mysql> checksum table t_drop;
+-----------------+-----------+
| Table           | Checksum  |
+-----------------+-----------+
| recover.t_drop  | 920719058 | 
+-----------------+-----------+
1 row in set (0.00 sec)

删除测试表,后续用来恢复

mysql> drop table recover.t_drop;
Query OK, 0 rows affected (0.00 sec)

解析ibdata文件

[root@web103 mysql_recovery]# ./stream_parser -f /var/lib/mysql/ibdata1 
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         463312
time of last access:            1440825416 Sat Aug 29 13:16:56 2015
time of last modification:      1440855835 Sat Aug 29 21:43:55 2015
time of last status change:     1440855835 Sat Aug 29 21:43:55 2015
total size, in bytes:            236978176 (226.000 MiB)

Size to process:                 236978176 (226.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         463312
time of last access:            1440825416 Sat Aug 29 13:16:56 2015
time of last modification:      1440855835 Sat Aug 29 21:43:55 2015
time of last status change:     1440855835 Sat Aug 29 21:43:55 2015
total size, in bytes:            236978176 (226.000 MiB)

Size to process:                 236978176 (226.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         463312
time of last access:            1440825416 Sat Aug 29 13:16:56 2015
time of last modification:      1440855835 Sat Aug 29 21:43:55 2015
time of last status change:     1440855835 Sat Aug 29 21:43:55 2015
total size, in bytes:            236978176 (226.000 MiB)

Size to process:                 236978176 (226.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         463312
time of last access:            1440825416 Sat Aug 29 13:16:56 2015
time of last modification:      1440855835 Sat Aug 29 21:43:55 2015
time of last status change:     1440855835 Sat Aug 29 21:43:55 2015
total size, in bytes:            236978176 (226.000 MiB)

Size to process:                 236978176 (226.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         463312
time of last access:            1440825416 Sat Aug 29 13:16:56 2015
time of last modification:      1440855835 Sat Aug 29 21:43:55 2015
time of last status change:     1440855835 Sat Aug 29 21:43:55 2015
total size, in bytes:            236978176 (226.000 MiB)

Size to process:                 236978176 (226.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         463312
time of last access:            1440825416 Sat Aug 29 13:16:56 2015
time of last modification:      1440855835 Sat Aug 29 21:43:55 2015
time of last status change:     1440855835 Sat Aug 29 21:43:55 2015
total size, in bytes:            236978176 (226.000 MiB)

Size to process:                 236978176 (226.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         463312
time of last access:            1440855928 Sat Aug 29 21:45:28 2015
time of last modification:      1440855835 Sat Aug 29 21:43:55 2015
time of last status change:     1440855835 Sat Aug 29 21:43:55 2015
total size, in bytes:            236978176 (226.000 MiB)

Size to process:                 236978176 (226.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         463312
time of last access:            1440855928 Sat Aug 29 21:45:28 2015
time of last modification:      1440855835 Sat Aug 29 21:43:55 2015
time of last status change:     1440855835 Sat Aug 29 21:43:55 2015
total size, in bytes:            236978176 (226.000 MiB)

Size to process:                 236978176 (226.000 MiB)
Worker(1): 56.64% done. 2015-08-29 21:45:30 ETA(in 00:00:01). Processing speed: 8.000 MiB/sec
Worker(0): 56.64% done. 2015-08-29 21:45:30 ETA(in 00:00:01). Processing speed: 8.000 MiB/sec
Worker(4): 56.64% done. 2015-08-29 21:45:30 ETA(in 00:00:01). Processing speed: 8.000 MiB/sec
Worker(3): 56.64% done. 2015-08-29 21:45:30 ETA(in 00:00:01). Processing speed: 8.000 MiB/sec
Worker(2): 56.64% done. 2015-08-29 21:45:30 ETA(in 00:00:01). Processing speed: 8.000 MiB/sec
Worker(7): 56.64% done. 2015-08-29 21:45:30 ETA(in 00:00:01). Processing speed: 8.000 MiB/sec
Worker(5): 56.64% done. 2015-08-29 21:45:30 ETA(in 00:00:01). Processing speed: 8.000 MiB/sec
Worker(6): 56.64% done. 2015-08-29 21:45:30 ETA(in 00:00:01). Processing speed: 8.000 MiB/sec
All workers finished in 1 sec

恢复mysql字典

[root@web103 mysql_recovery]# ./recover_dictionary.sh 
Generating dictionary tables dumps... OK
Creating test database ... OK
Creating dictionary tables in database test:
SYS_TABLES ... OK
SYS_COLUMNS ... OK
SYS_INDEXES ... OK
SYS_FIELDS ... OK
All OK
Loading dictionary tables data:
SYS_TABLES ... 162 recs OK
SYS_COLUMNS ... 1247 recs OK
SYS_INDEXES ... 216 recs OK
SYS_FIELDS ... 411 recs OK
All OK
[root@web103 mysql_recovery]# mysql test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 10162
Server version: 5.0.95 Source distribution

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> select * from SYS_TABLES WHERE name like 'recover%';
+-------------------------+-----+--------+------+--------+---------+--------------+-------+
| NAME                    | ID  | N_COLS | TYPE | MIX_ID | MIX_LEN | CLUSTER_NAME | SPACE |
+-------------------------+-----+--------+------+--------+---------+--------------+-------+
| recover/#sql-64a9_1f6d  | 120 |      8 |    1 |      0 |       0 |              |     0 | 
| recover/#sql2-64a9-1f6d | 115 |      8 |    1 |      0 |       0 |              |     0 | 
| recover/t_delete        |  74 |      2 |    1 |      0 |       0 |              |     0 | 
| recover/t_delete1       |  84 |      2 |    1 |      0 |       0 |              |     0 | 
| recover/t_drop          | 125 |      4 |    1 |      0 |       0 |              |     0 | 
| recover/t_truncate      | 120 |      8 |    1 |      0 |       0 |              |     0 | 
| recover/t_xifenfei      |  75 |      2 |    1 |      0 |       0 |              |     0 | 
| recover/zx_users        |  89 |     85 |    1 |      0 |       0 |              |     0 | 
+-------------------------+-----+--------+------+--------+---------+--------------+-------+
8 rows in set (0.00 sec)

mysql> select * from SYS_INDEXES WHERE TABLE_ID=125;
+----------+-----+---------+----------+------+-------+------------+
| TABLE_ID | ID  | NAME    | N_FIELDS | TYPE | SPACE | PAGE_NO    |
+----------+-----+---------+----------+------+-------+------------+
|      125 | 142 | PRIMARY |        1 |    3 |     0 | 4294967295 | 
+----------+-----+---------+----------+------+-------+------------+
1 row in set (0.00 sec)

恢复被删除表记录

[root@web103 mysql_recovery]# ./c_parser -5f pages-ibdata1/FIL_PAGE_INDEX/0000000000000142.page  -t dictionary/t_drop.sql >dumps/default/t_drop 2>2.sql
[root@web103 mysql_recovery]# more dumps/default/t_drop
-- Page id: 9860, Format: COMPACT, Records list: Valid, Expected records: (11 11)
00000099F9F2    80000026800110  t_drop  "10235259536125650"     "尊敬的用户您好:您的手机验证码为474851如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
00000099F9F2    80000026800129  t_drop  "10235353811295807"     "尊敬的用户您好:您的手机验证码为444632如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
00000099F9F2    80000026800142  t_drop  "102354211240398235"    "尊敬的用户您好:您的手机验证码为478503如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
00000099F9F2    8000002680015C  t_drop  "102354554052884567"    "尊敬的用户您好:您的手机验证码为216825如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
00000099F9F2    80000026800176  t_drop  "132213454294519126"    "尊敬的用户您好:您的手机验证码为854812如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
00000099F9F2    80000026800190  t_drop  "82329022242584577"     "尊敬的用户您好:您的手机验证码为253127如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
00000099F9F2    800000268001A9  t_drop  "82329022242584581"     "尊敬的用户您好:您的手机验证码为253127如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
00000099F9F2    800000268001C2  t_drop  "8233400415607376"      "尊敬的用户您好:您的手机验证码为338470如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
00000099F9F2    800000268001DA  t_drop  "82334502212106951"     "尊敬的用户您好:您的手机验证码为916515如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
00000099F9F2    800000268001F3  t_drop  "82339012756833423"     "尊敬的用户您好:您的手机验证码为396108如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
00000099F9F2    8000002680020C  t_drop  "8234322198577796"      "尊敬的用户您好:您的手机验证码为935297如非本人操作,请拨打奥斯卡客服:400-620-7575。"  "2010-01-01 00:00:00"   0
-- Page id: 9860, Found records: 11, Lost records: NO, Leaf page: YES

恢复数据入库

mysql> source dictionary/t_drop.sql
Query OK, 0 rows affected (0.00 sec)

mysql> source 2.sql
Query OK, 0 rows affected (0.00 sec)

Query OK, 11 rows affected (0.00 sec)
Records: 11  Deleted: 0  Skipped: 0  Warnings: 0

mysql> checksum table t_drop;
+-----------------+-----------+
| Table           | Checksum  |
+-----------------+-----------+
| recover.t_drop  | 920719058 | 
+-----------------+-----------+
1 row in set (0.00 sec)

至此实现删除数据完美恢复

mysql delete 恢复

在mysql(innodb引擎)中,有些时候犹豫误操作导致表中数据被删除,从而导致不可挽回的损失,本文模拟在数据库被误delete的情况下,实现较为完美删除,当然在实际中可能有少量不覆盖或者无法恢复回来,但是在覆盖不多或者未覆盖的情况下,可以实现绝大多数甚至全部恢复.因此在发生误操作时候,应当第一时间保护现场,尽可能防止复写导致不可挽回的损失.在测试恢复过程中,由于mysql和操作系统编码问题,折腾了很久,感谢Lunar的指点
创建模拟表并插入数据

mysql> CREATE TABLE `sms_send_record_del` (
    ->   `messageId` varchar(30) NOT NULL,
    ->   `tokenId` varchar(20) NOT NULL,
    ->   `mobile` varchar(14) default NULL,
    ->   `msgFormat` int(1) NOT NULL,
    ->   `msgContent` varchar(1000) default NULL,
    ->   `scheduleDate` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
    ->   `deliverState` int(1) default NULL,
    ->   `deliverdTime` timestamp NOT NULL default '0000-00-00 00:00:00',
    ->   PRIMARY KEY  (`messageId`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.00 sec)

mysql> insert into sms_send_record_del select * from sms_send_record;
Query OK, 11 rows affected (0.00 sec)
Records: 11  Duplicates: 0  Warnings: 0

mysql> checksum table sms_send_record_del;
+---------------------------------+------------+
| Table                           | Checksum   |
+---------------------------------+------------+
| sms_service.sms_send_record_del | 2258631583 | 
+---------------------------------+------------+
1 row in set (0.00 sec)

mysql> checksum table sms_send_record;    
+-----------------------------+------------+
| Table                       | Checksum   |
+-----------------------------+------------+
| sms_service.sms_send_record | 2258631583 | 
+-----------------------------+------------+
1 row in set (0.00 sec)

mysql-delete-recovery-1


确定innodb文件对应位置

mysql> SHOW VARIABLES LIKE 'datadir';
+---------------+-----------------+
| Variable_name | Value           |
+---------------+-----------------+
| datadir       | /var/lib/mysql/ | 
+---------------+-----------------+
1 row in set (0.00 sec)

mysql> SHOW VARIABLES LIKE 'innodb_file_per_table';
+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| innodb_file_per_table | OFF   | 
+-----------------------+-------+
1 row in set (0.00 sec)

mysql> SHOW VARIABLES LIKE 'innodb_data_file_path';
+-----------------------+------------------------+
| Variable_name         | Value                  |
+-----------------------+------------------------+
| innodb_data_file_path | ibdata1:10M:autoextend | 
+-----------------------+------------------------+
1 row in set (0.00 sec)

删除表记录
模拟错误操作,误删除表所有数据

mysql> delete from sms_send_record_del;
Query OK, 11 rows affected (0.00 sec)

解析ibdata1文件

[root@web103 mysql_recovery]# ./stream_parser -f /var/lib/mysql/ibdata1 
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         315712
time of last access:            1440599559 Wed Aug 26 22:32:39 2015
time of last modification:      1440601853 Wed Aug 26 23:10:53 2015
time of last status change:     1440601853 Wed Aug 26 23:10:53 2015
total size, in bytes:            161480704 (154.000 MiB)

Size to process:                 161480704 (154.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         315712
time of last access:            1440599559 Wed Aug 26 22:32:39 2015
time of last modification:      1440601853 Wed Aug 26 23:10:53 2015
time of last status change:     1440601853 Wed Aug 26 23:10:53 2015
total size, in bytes:            161480704 (154.000 MiB)

Size to process:                 161480704 (154.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         315712
time of last access:            1440599559 Wed Aug 26 22:32:39 2015
time of last modification:      1440601853 Wed Aug 26 23:10:53 2015
time of last status change:     1440601853 Wed Aug 26 23:10:53 2015
total size, in bytes:            161480704 (154.000 MiB)

Size to process:                 161480704 (154.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         315712
time of last access:            1440599559 Wed Aug 26 22:32:39 2015
time of last modification:      1440601853 Wed Aug 26 23:10:53 2015
time of last status change:     1440601853 Wed Aug 26 23:10:53 2015
total size, in bytes:            161480704 (154.000 MiB)

Size to process:                 161480704 (154.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         315712
Opening file: /var/lib/mysql/ibdata1
time of last access:            1440599559 Wed Aug 26 22:32:39 2015
time of last modification:      1440601853 Wed Aug 26 23:10:53 2015
time of last status change:     1440601853 Wed Aug 26 23:10:53 2015
File information:

total size, in bytes:            161480704 (154.000 MiB)

ID of device containing file:         2049
Size to process:                 161480704 (154.000 MiB)
Opening file: /var/lib/mysql/ibdata1
File information:

ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         315712
time of last access:            1440599559 Wed Aug 26 22:32:39 2015
time of last modification:      1440601853 Wed Aug 26 23:10:53 2015
time of last status change:     1440601853 Wed Aug 26 23:10:53 2015
total size, in bytes:            161480704 (154.000 MiB)

Size to process:                 161480704 (154.000 MiB)
Opening file: /var/lib/mysql/ibdata1
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
File information:

blocksize for filesystem I/O:         4096
number of blocks allocated:         315712
ID of device containing file:         2049
inode number:                      1344553
protection:                         100660 (regular file)
number of hard links:                    1
user ID of owner:                       27
group ID of owner:                      27
device ID (if special file):             0
blocksize for filesystem I/O:         4096
number of blocks allocated:         315712
time of last access:            1440599559 Wed Aug 26 22:32:39 2015
time of last modification:      1440601853 Wed Aug 26 23:10:53 2015
time of last status change:     1440601853 Wed Aug 26 23:10:53 2015
total size, in bytes:            161480704 (154.000 MiB)

Size to process:                 161480704 (154.000 MiB)
time of last access:            1440601884 Wed Aug 26 23:11:24 2015
time of last modification:      1440601853 Wed Aug 26 23:10:53 2015
time of last status change:     1440601853 Wed Aug 26 23:10:53 2015
total size, in bytes:            161480704 (154.000 MiB)

Size to process:                 161480704 (154.000 MiB)
All workers finished in 0 sec

分析数据字典

mysql> show tables
    -> ;
+----------------+
| Tables_in_test |
+----------------+
| SYS_COLUMNS    | 
| SYS_FIELDS     | 
| SYS_INDEXES    | 
| SYS_TABLES     | 
+----------------+
4 rows in set (0.00 sec)

mysql> select * from SYS_TABLES;
+----------------------------------------+----+-------------+------+--------+---------+--------------+-------+
| NAME                                   | ID | N_COLS      | TYPE | MIX_ID | MIX_LEN | CLUSTER_NAME | SPACE |
+----------------------------------------+----+-------------+------+--------+---------+--------------+-------+
| recover/t_delete                       | 74 |           2 |    1 |      0 |       0 |              |     0 | 
| recover/t_delete1                      | 84 |           2 |    1 |      0 |       0 |              |     0 | 
| recover/t_xifenfei                     | 75 |           2 |    1 |      0 |       0 |              |     0 | 
| recover/zx_users                       | 89 |          85 |    1 |      0 |       0 |              |     0 | 
| sms_service/sms_send_record            | 36 |           8 |    1 |      0 |       0 |              |     0 | 
| sms_service/sms_send_record_del        | 90 |           8 |    1 |      0 |       0 |              |     0 | 
| SYS_FOREIGN                            | 11 | -2147483644 |    1 |      0 |       0 |              |     0 | 
| SYS_FOREIGN_COLS                       | 12 | -2147483644 |    1 |      0 |       0 |              |     0 | 
| test/SYS_COLUMNS                       | 86 |           7 |    1 |      0 |       0 |              |     0 | 
| test/SYS_FIELDS                        | 88 |           3 |    1 |      0 |       0 |              |     0 | 
| test/SYS_INDEXES                       | 87 |           7 |    1 |      0 |       0 |              |     0 | 
| test/SYS_TABLES                        | 85 |           8 |    1 |      0 |       0 |              |     0 | 
| test/zx_users                          | 43 |          85 |    1 |      0 |       0 |              |     0 | 
| xifenfei/t_delete                      | 44 |           8 |    1 |      0 |       0 |              |     0 | 
| xifenfei/t_xifenfei                    | 59 |           2 |    1 |      0 |       0 |              |     0 | 
+----------------------------------------+----+-------------+------+--------+---------+--------------+-------+
39 rows in set (0.00 sec)

mysql> select * from SYS_INDEXES WHERE TABLE_ID=90;
+----------+-----+---------+----------+------+-------+---------+
| TABLE_ID | ID  | NAME    | N_FIELDS | TYPE | SPACE | PAGE_NO |
+----------+-----+---------+----------+------+-------+---------+
|       90 | 110 | PRIMARY |        1 |    3 |     0 |    2955 | 
+----------+-----+---------+----------+------+-------+---------+
1 row in set (0.00 sec)

找回被删除记录

[root@web103 mysql_recovery]# ./c_parser -5Df pages-ibdata1/FIL_PAGE_INDEX/0000000000000110.page \
[root@web103 mysql_recovery]#    -t dictionary/sms_send_record_del.sql >/tmp/t_1.txt 2>/tmp/t_1.sql

mysql-delete-recovery-2


加载数据并验证

mysql> use sms_service;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> source /tmp/t_1.sql
Query OK, 0 rows affected (0.00 sec)

Query OK, 11 rows affected, 8 warnings (0.01 sec)
Records: 11  Deleted: 0  Skipped: 0  Warnings: 8

mysql> checksum table sms_send_record_del;
+---------------------------------+------------+
| Table                           | Checksum   |
+---------------------------------+------------+
| sms_service.sms_send_record_del | 2258631583 | 
+---------------------------------+------------+
1 row in set (0.00 sec)

发生误操作之时,请尽可能保护现场,防止覆盖导致无可挽回的损失.