k8s使用的etcd空间清理

 

1.查看当前k8s的情况

[root@master snap]# kubectl get pods -n kube-system -o wide
NAME                                       READY   STATUS    RESTARTS      AGE   IP              NODE     NOMINATED NODE   READINESS GATES
calico-kube-controllers-855f469468-lh7kp   1/1     Running   4 (91m ago)   26d   10.244.219.85   master   <none>           <none>
calico-node-757ps                          1/1     Running   8 (91m ago)   27d   192.168.1.108   master   <none>           <none>
calico-node-jbsd7                          1/1     Running   7 (91m ago)   27d   192.168.1.105   node1    <none>           <none>
calico-node-x6ks8                          1/1     Running   7 (91m ago)   27d   192.168.1.106   node2    <none>           <none>
coredns-66f779496c-rg7w7                   1/1     Running   8 (91m ago)   27d   10.244.219.86   master   <none>           <none>
coredns-66f779496c-tzbbn                   1/1     Running   8 (91m ago)   27d   10.244.219.87   master   <none>           <none>
etcd-master                                1/1     Running   4 (91m ago)   25d   192.168.1.108   master   <none>           <none>
kube-apiserver-master                      1/1     Running   4 (91m ago)   25d   192.168.1.108   master   <none>           <none>
kube-controller-manager-master             1/1     Running   9 (91m ago)   27d   192.168.1.108   master   <none>           <none>
kube-proxy-76c84                           1/1     Running   9 (91m ago)   27d   192.168.1.108   master   <none>           <none>
kube-proxy-8qdh6                           1/1     Running   7 (91m ago)   27d   192.168.1.106   node2    <none>           <none>
kube-proxy-vh2bh                           1/1     Running   7 (91m ago)   27d   192.168.1.105   node1    <none>           <none>
kube-scheduler-master                      1/1     Running   9 (91m ago)   27d   192.168.1.108   master   <none>           <none>

 

可以看到这里的etcd部署在 192.168.1.108 机器上
kubectl describe pods etcd-master -n kube-system
查看找到如下项目
--listen-client-urls=https://127.0.0.1:2379,https://192.168.1.108:2379

 

2.查看etcd使用情况
export ETCDCTL_API=3
export ETCDCTL_ENDPOINTS="https://127.0.0.1:2379"
export ETCDCTL_CACERT="/etc/kubernetes/pki/etcd/ca.crt"
export ETCDCTL_CERT="/etc/kubernetes/pki/etcd/server.crt"
export ETCDCTL_KEY="/etc/kubernetes/pki/etcd/server.key"

[root@master ~]# etcdctl --write-out=table endpoint status
+------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|        ENDPOINT        |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://127.0.0.1:2379 | 6fe6caf9a934dbb7 |   3.5.9 |  4.7 MB |      true |      false |        11 |     230832 |             230832 |        |
+------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

 

3.查看etcd存储目录

[root@master snap]#kubectl describe pods etcd-master -n kube-system
    Mounts:
      /etc/kubernetes/pki/etcd from etcd-certs (rw)
      /var/lib/etcd from etcd-data (rw)
[root@master snap]# pwd
/var/lib/etcd/member/snap
[root@master snap]# ls -al
total 4688
drwx------ 2 root root      246 Sep 30 10:51 .
drwx------ 4 root root       29 Sep 30 09:00 ..
-rw-r--r-- 1 root root    10018 Sep 29 13:46 000000000000000a-000000000002e643.snap
-rw-r--r-- 1 root root    10018 Sep 29 15:16 000000000000000a-0000000000030d54.snap
-rw-r--r-- 1 root root    10018 Sep 29 16:54 000000000000000a-0000000000033465.snap
-rw-r--r-- 1 root root    10380 Sep 30 09:15 000000000000000b-0000000000035b76.snap
-rw-r--r-- 1 root root    10380 Sep 30 10:51 000000000000000b-0000000000038287.snap
-rw------- 1 root root 20709376 Sep 30 11:03 db

 

4.清理空间
4.1 查看旧版本号

export ETCDCTL_API=3
export ETCDCTL_ENDPOINTS="https://127.0.0.1:2379"
export ETCDCTL_CACERT="/etc/kubernetes/pki/etcd/ca.crt"
export ETCDCTL_CERT="/etc/kubernetes/pki/etcd/server.crt"
export ETCDCTL_KEY="/etc/kubernetes/pki/etcd/server.key"

[root@master snap]# etcdctl endpoint status --write-out="json" | egrep -o '"revision":[0-9]*' | egrep -o '[0-9].*'
204156

 

4.2 执行如下命令进行压缩
[root@master snap]# etcdctl compact 204156
compacted revision 204156

 

4.3 整理多余的空间
[root@master snap]# etcdctl defrag
Finished defragmenting etcd member[https://127.0.0.1:2379]

 

执行这个命令后数据明显变少了

[root@master snap]# ls -al
total 1144
drwx------ 2 root root      246 Sep 30 11:07 .
drwx------ 4 root root       29 Sep 30 09:00 ..
-rw-r--r-- 1 root root    10018 Sep 29 13:46 000000000000000a-000000000002e643.snap
-rw-r--r-- 1 root root    10018 Sep 29 15:16 000000000000000a-0000000000030d54.snap
-rw-r--r-- 1 root root    10018 Sep 29 16:54 000000000000000a-0000000000033465.snap
-rw-r--r-- 1 root root    10380 Sep 30 09:15 000000000000000b-0000000000035b76.snap
-rw-r--r-- 1 root root    10380 Sep 30 10:51 000000000000000b-0000000000038287.snap
-rw------- 1 root root 17862656 Sep 30 11:07 db
[root@master snap]# pwd
/var/lib/etcd/member/snap

 

4.4 取消告警信息
[root@master snap]# etcdctl alarm disarm