etcd备份还原实战

备份

1、 ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key snapshot save snapshotdb

2 、mv /var/lib/etcd /var/lib/etcd.bak

 

ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key snapshot status snapshotdb --write-out=table

+----------+----------+------------+------------+

| HASH | REVISION | TOTAL KEYS | TOTAL SIZE |

+----------+----------+------------+------------+

| 6e87b0a8 | 45935245 | 1882 | 39 MB |

+----------+----------+------------+------------+

默认的数据目录中db是无法验证的

 

还原

停止集群

systemctl stop kube-apiserver

systemctl stop etcd

mv /etc/kubernetes/manifests /etc/kubernetes/manifests.bak 停止api集群和etcd

rm -rf /var/lib/etcd/ （上面已经备份了）

拷贝 ETCD 备份快照

scp /data/etcd_backup_dir/etcd-snapshot-20191222.db root@k8s-master2:/data/etcd_backup_dir/

scp /data/etcd_backup_dir/etcd-snapshot-20191222.db root@k8s-master3:/data/etcd_backup_dir/

在所有master节点执行

 

ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key snapshot restore /mnt/snapshotdb --data-dir=/var/lib/etcd/

 

{"level":"info","ts":1658296213.7700896,"caller":"snapshot/v3_snapshot.go:287","msg":"restoring snapshot","path":"/mnt/snapshotdb","wal-dir":"/var/lib/etcd/member/wal","data-dir":"/var/lib/etcd/","snap-dir":"/var/lib/etcd/member/snap"}

{"level":"info","ts":1658296214.9195104,"caller":"mvcc/kvstore.go:378","msg":"restored last compact revision","meta-bucket-name":"meta","meta-bucket-name-key":"finishedCompactRev","restored-compact-revision":45933969}

{"level":"info","ts":1658296214.9418955,"caller":"membership/cluster.go:392","msg":"added member","cluster-id":"cdf818194e3a8c32","local-member-id":"0","added-peer-id":"8e9e05c52164694d","added-peer-peer-urls":["http://localhost:2380"]}

{"level":"info","ts":1658296214.9568207,"caller":"snapshot/v3_snapshot.go:300","msg":"restored snapshot","path":"/mnt/snapshotdb","wal-dir":"/var/lib/etcd/member/wal","data-dir":"/var/lib/etcd/","snap-dir":"/var/lib/etcd/member/snap"}

启动集群

mv /etc/kubernetes/manifests.bak /etc/kubernetes/manifests

检查集群健康状态

 

[root@master ~]# ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key endpoint health

https://127.0.0.1:2379 is healthy: successfully committed proposal: took = 13.961901ms

 

systemctl start kube-apiserver

kubectl get cs