mongodb的热备工具pbm-agent

 

        pbm全称percona backup mongodb，是percona公司提供的mongodb的备份恢复工具，支持percona mongodb,community mongodb，

       支持版本>v3.6，pbm底层是调用mongodump/mongorestore进行转储备份，由于该工具开发时间还不到1年，目前对分片集群的支持力度较弱，

       且不支持增量备份。

 

 

 

/bin/sh -c /home/work/backup/pbm-agent  --mongodb-uri=mongodb://mongo_bakup:123456@127.0.0.1:27027/?replicaSet=myset  &> /home/work/backup/pbm-agent.27027.log

 

 副本集主库上创建pbm备份的账号和密码：

db.createRole({ "role": "perconabakcupAnyAction",
                "privileges": [{ "resource": { "anyResource": true },"actions": [ "anyAction" ]}],
                "roles": []}
);

db.createUser({user: "backupuser","pwd": "123456",
            "roles" : [
                { "db" : "pbm", "role" : "readWrite", "collection": "" },
                { "db" : "admin", "role" : "backup" },
                { "db" : "admin", "role" : "clusterMonitor" },
                { "db" : "admin", "role" : "restore" },
                { "db" : "admin", "role" : "perconabakcupAnyAction" }
            ]
    }); 



db.getSiblingDB("admin").createRole({ "role": "pbmAnyAction",
"privileges": [
{ "resource": { "anyResource": true },
"actions": [ "anyAction" ]
}
],
"roles": []
});
db.getSiblingDB("admin").createUser({user: "pbmuser",
"pwd": "secretpwd",
"roles" : [
{ "db" : "admin", "role" : "readWrite", "collection": "" },
{ "db" : "admin", "role" : "backup" },
{ "db" : "admin", "role" : "clusterMonitor" },
{ "db" : "admin", "role" : "restore" },
{ "db" : "admin", "role" : "pbmAnyAction" }
]
});
 

 

 

 

 

 查看所有历史备份：pbm list  --mongodb-uri="mongodb://username:password@127.0.0.1:27000/?replicaSet=setname"

[work@xxx pbm]$ ./pbm  list          --mongodb-uri="mongodb://mongodb_dba:123456@127.0.0.1:28042/?replicaSet=glc-test"
Backup history:
  2020-11-09T11:49:33+08:00
  2020-11-09T13:35:52+08:00
  2020-11-11T00:04:20+08:00
  2020-11-13T00:04:06+08:00
  2020-11-15T00:04:10+08:00
  2020-11-17T00:04:25+08:00
  2020-11-19T00:04:05+08:00
  2020-11-21T00:04:04+08:00
  2020-11-23T00:04:04+08:00
  2020-11-25T19:19:13+08:00
  2020-11-27T14:37:57+08:00     In progress [running] (Launched at 2020-11-27T14:37:58+08:00)
[work@xxx pbm]$

 

 

 备份命令：pbm-agent --mongodb-uri=mongodb://admin:123456@127.0.0.1:28042/?replicaSet=glc-test

[work@xxx pbm]$ cat pbm-agent.28042.log
pbm agent is listening for the commands
2021/01/19 11:49:29 Got command resyncBcpList
2021/01/19 11:49:29 [INFO] resync_list: started
&{BucketName:mongo.owt.dba Prefix:glc-test/ MaxKeys:10 Marker: Truncated:false NextMarker: Delimiter:/ ObjectSummaries:[] CommonPrefixes:[]}
2021/01/19 11:49:29 [INFO] resync_list: succeed
2021/01/19 11:49:34 Got command backup 2021-01-19T11:49:33+08:00
2021/01/19 11:49:34 Backup 2021-01-19T11:49:33+08:00 started on node glc-test/xxx:28042
2021-01-19T11:49:37.672+0800    writing admin.system.users to archive on stdout
2021-01-19T11:49:37.677+0800    done dumping admin.system.users (15 documents)
2021-01-19T11:49:37.678+0800    writing admin.system.roles to archive on stdout
2021-01-19T11:49:37.683+0800    done dumping admin.system.roles (3 documents)
2021-01-19T11:49:37.684+0800    writing admin.system.version to archive on stdout
2021-01-19T11:49:37.708+0800    done dumping admin.system.version (2 documents)
2021-01-19T11:49:37.709+0800    writing ai_open_caps.ai_caps_skill_info to archive on stdout
2021-01-19T11:49:37.733+0800    done dumping ai_open_caps.ai_caps_skill_info (12 documents)
2021-01-19T11:49:37.734+0800    writing apple.cas_users to archive on stdout
2021-01-19T11:49:40.074+0800    done dumping apple.cas_users (201551 documents)
2021-01-19T11:49:40.076+0800    writing apple.my3 to archive on stdout
2021-01-19T11:49:40.083+0800    done dumping apple.my3 (1 document)
2021-01-19T11:49:40.085+0800    writing apple.inventory to archive on stdout
2021-01-19T11:49:40.101+0800    done dumping apple.inventory (39 documents)
2021-01-19T11:49:40.103+0800    writing apple.testdata2 to archive on stdout
2021-01-19T11:50:05.245+0800    done dumping apple.testdata2 (9999999 documents)
2021-01-19T11:50:05.247+0800    writing apple.iphone to archive on stdout
2021-01-19T11:50:05.264+0800    done dumping apple.iphone (1 document)
2021-01-19T11:50:05.266+0800    writing apple.glc to archive on stdout
2021-01-19T11:50:05.283+0800    done dumping apple.glc (11 documents)
2021-01-19T11:50:05.284+0800    writing apple.user to archive on stdout
2021-01-19T11:50:07.236+0800    done dumping apple.user (597941 documents)
2021-01-19T11:50:07.238+0800    writing apple.iphonr to archive on stdout
2021-01-19T11:50:07.245+0800    done dumping apple.iphonr (0 documents)
2021-01-19T11:50:07.247+0800    writing apple.my2 to archive on stdout
2021-01-19T11:50:07.255+0800    done dumping apple.my2 (1 document)
2021-01-19T11:50:07.256+0800    writing apple.home to archive on stdout
2021-01-19T11:50:07.265+0800    done dumping apple.home (1 document)
2021-01-19T11:50:07.266+0800    writing apple.ai_caps_skill_info to archive on stdout
2021-01-19T11:50:07.273+0800    done dumping apple.ai_caps_skill_info (12 documents)
2021-01-19T11:50:07.275+0800    writing eagle.comment_statistics_grab to archive on stdout
2021-01-19T11:50:32.449+0800    done dumping eagle.comment_statistics_grab (3099821 documents)2021-01-19T13:05:49.233+0800    writing eagle.col to archive on stdout
2021-01-19T13:05:49.240+0800    done dumping eagle.col (4 documents)2021-01-19T13:12:16.343+0800    writing gallery.stars to archive on stdout
2021-01-19T13:12:16.353+0800    done dumping gallery.stars (107 documents)
2021-01-19T13:12:16.354+0800    writing gallery.gallerypublish_en to archive on stdout
2021-01-19T13:12:16.362+0800    done dumping gallery.gallerypublish_en (1 document)
2021-01-19T13:12:16.363+0800    writing gallery.gallerycp to archive on stdout
2021-01-19T13:12:16.381+0800    done dumping gallery.gallerycp (133 documents)
2021-01-19T13:12:16.382+0800    writing gallery.report to archive on stdout
2021-01-19T13:12:16.398+0800    done dumping gallery.report (8 documents)
2021-01-19T13:12:16.400+0800    writing gallery.id_generator_tw to archive on stdout
2021-01-19T13:12:16.416+0800    done dumping gallery.id_generator_tw (1 document)
2021-01-19T13:12:16.417+0800    writing gallery.cp_auth to archive on stdout
2021-01-19T13:12:16.432+0800    done dumping gallery.cp_auth (25 documents)
2021-01-19T13:12:16.433+0800    writing google.items to archive on stdout
2021-01-19T13:12:16.457+0800    done dumping google.items (17 documents)
2021-01-19T13:12:16.459+0800    writing google.chrome to archive on stdout
2021-01-19T13:12:16.467+0800    done dumping google.chrome (1 document)
2021-01-19T13:12:16.469+0800    writing pbm.pbmConfig to archive on stdout
2021-01-19T13:12:16.475+0800    done dumping pbm.pbmConfig (1 document)
2021-01-19T13:12:16.477+0800    writing pbm.pbmLock to archive on stdout
2021-01-19T13:12:16.483+0800    done dumping pbm.pbmLock (1 document)
2021-01-19T13:12:16.484+0800    writing pbm.pbmBackups to archive on stdout
2021-01-19T13:12:16.494+0800    done dumping pbm.pbmBackups (1 document)
2021-01-19T13:12:16.496+0800    writing pbm.pbmBackups.old to archive on stdout
2021-01-19T13:12:16.511+0800    done dumping pbm.pbmBackups.old (9 documents)
2021-01-19T13:12:16.513+0800    writing pbm.pbmCmd to archive on stdout
2021-01-19T13:12:16.519+0800    done dumping pbm.pbmCmd (13 documents)
2021-01-19T13:12:16.521+0800    writing test.mycol to archive on stdout
2021-01-19T13:12:16.528+0800    done dumping test.mycol (4 documents)
2021-01-19T13:12:16.529+0800    writing test.inventory to archive on stdout
2021-01-19T13:12:16.536+0800    done dumping test.inventory (1 document)
2021-01-19T13:12:16.537+0800    writing test.items to archive on stdout
2021-01-19T13:12:16.544+0800    done dumping test.items (1 document)
2021-01-19T13:12:16.546+0800    writing test.mycollection to archive on stdout
2021-01-19T13:12:16.569+0800    done dumping test.mycollection (0 documents)
2021-01-19T13:12:16.571+0800    writing test.newcollection to archive on stdout
2021-01-19T13:12:16.578+0800    done dumping test.newcollection (1 document)
2021/01/19 13:12:19 [lzd debug2] remote part: 72 size: 125438232
2021/01/19 13:12:20 mongodump finished, waiting for the oplog
2021/01/19 13:12:23 [lzd debug2] remote part: 1 size: 52889
2021/01/19 13:12:24 [lzd debug2] remote part: 1 size: 1435
2021/01/19 13:12:24 Backup 2021-01-19T11:49:33+08:00 finished
[work@xxx pbm]$ ls

 

 

 

pbm restore恢复:只能在主库恢复，且保持主库服务启动，且在恢复过程中需要禁止业务写入数据

pbm恢复底层调用mongorestore，在从库执行恢复命令会提示Node in not suitable for restore，并自动将恢复任务切至主库：

 

 

 

 

 

 

Percona Backup for MongoDB支持通过增量备份和oplog来实现恢复到任意时间点的功能。具体步骤如下：

  1. 恢复最新的全量备份

使用pbm-restore命令恢复最新的全量备份，例如：

pbm-restore --backup-dir=/data/backup/2022-03-01T01-00-00 --target-dir=/data/mongodb

其中backup-dir为备份目录，target-dir为目标目录。

  1. 应用增量备份

使用pbm-apply命令应用增量备份，例如：

pbm-apply --oplog=/data/backup/2022-03-01T01-00-00/oplog.bson --target-dir=/data/mongodb

其中oplog为增量备份的oplog文件，target-dir为目标目录。

  1. 恢复到指定时间点

使用pbm-restore-time命令恢复到指定时间点，例如：

pbm-restore-time --target-dir=/data/mongodb --backup-time="2022-03-01T02:00:00Z"

其中target-dir为目标目录，backup-time为指定的时间点。

  1. 检查恢复结果

使用pbm-list命令检查恢复结果，例如：

pbm-list --target-dir=/data/mongodb

可以查看恢复后的数据是否正确。

需要注意的是，恢复到任意时间点的功能需要使用增量备份和oplog，因此在备份时需要开启oplog，并定期进行增量备份。

 

 

 

 

 ########################

 

 

###############################################################################

posted @ 2020-11-23 11:27  igoodful  阅读(694)  评论(0)  编辑  收藏  举报