etcd日常维护

 

1.查看etcd使用空间大小

docker ps 找到etcd对应的容器id

 

[root@hxl-milvus-prd ~]# docker ps
CONTAINER ID   IMAGE                                                 COMMAND                  CREATED         STATUS                   PORTS                                                                                      NAMES
dde5b6f0e299   zilliz/attu:v2.4.6                                    "docker-entrypoint.s…"   2 months ago    Up 2 months              0.0.0.0:8000->3000/tcp, :::8000->3000/tcp                                                  attu
b18cfae7e2ba   milvusdb/milvus:v2.5.14                               "/tini -- milvus run…"   2 months ago    Up 2 months (healthy)    0.0.0.0:9091->9091/tcp, :::9091->9091/tcp, 0.0.0.0:19530->19530/tcp, :::19530->19530/tcp   milvus-standalone
79d5efdec85f   minio/minio:RELEASE.2023-03-20T20-16-18Z              "/usr/bin/docker-ent…"   2 months ago    Up 2 months (healthy)    0.0.0.0:9000-9001->9000-9001/tcp, :::9000-9001->9000-9001/tcp                              milvus-minio
3823a524a8a3   quay.io/coreos/etcd:v3.5.5                            "etcd -advertise-cli…"   2 months ago    Up 2 months (healthy)    2379-2380/tcp                                                                              milvus-etcd

 

 

milvus安装的etcd默认没有外放2379端口，需要在容器里面执行

 

 

[root@hxl-milvus-prd ~]# docker exec -it 3823a524a8a3 etcdctl --write-out=table endpoint status
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|    ENDPOINT    |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 127.0.0.1:2379 | 8e9e05c52164694d |   3.5.5 |  2.4 MB |      true |      false |         7 |   13656175 |           13656175 |        |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

 

 

  

 

2.配额设置

services:
  etcd:
    container_name: milvus-etcd
    image: quay.io/coreos/etcd:v3.5.5
    environment:
      - ETCD_AUTO_COMPACTION_MODE=revision
      - ETCD_AUTO_COMPACTION_RETENTION=1000
      - ETCD_QUOTA_BACKEND_BYTES=4294967296
      - ETCD_SNAPSHOT_COUNT=50000
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd
    command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
    healthcheck:
      test: ["CMD", "etcdctl", "endpoint", "health"]
      interval: 30s
      timeout: 20s
      retries: 3

 

quota-backend-bytes默认是2GB,这里调整为4GB,最大是8GB(8589934592 bytes)

 

3.监控检查

 

[root@localhost milvus]# docker exec -it 4b22b6205f7e etcdctl --write-out=table endpoint health
+----------------+--------+------------+-------+
|    ENDPOINT    | HEALTH |    TOOK    | ERROR |
+----------------+--------+------------+-------+
| 127.0.0.1:2379 |   true | 7.173068ms |       |
+----------------+--------+------------+-------+
[root@localhost milvus]# 

 

 

#################################ETCD空间不足的处理办法##################################

若是docker部署的，需要登录容器里面再执行

 

 

登录容器
docker exec -it 22e987f71758 /bin/bash

 

 

 

# 获取当前版本
$ rev=$(ETCDCTL_API=3 etcdctl --endpoints=http://127.0.0.1:2379 endpoint status --write-out="json" | egrep -o '"revision":[0-9]*' | egrep -o '[0-9].*')

# 压缩所有旧版本
$ ETCDCTL_API=3 etcdctl --endpoints=http://127.0.0.1:2379 compact $rev

 

# 整理多余的空间
$ ETCDCTL_API=3 etcdctl --endpoints=http://127.0.0.1:2379 defrag

 

# 取消告警信息
$ ETCDCTL_API=3 etcdctl --endpoints=http://127.0.0.1:2379 alarm disarm

 

# 再次查看ETCD存储使用情况
$ ETCDCTL_API=3 etcdctl --endpoints="http://127.0.0.1:2379" --write-out=table endpoint status