使用Docker部署Redis Sentinel 高可用测试环境
背景：
之前通过Docker部署了Redis单实例。本文要实现的通过docker来部署3个redis数据节点和3个sentinel节点的测试环境。
环境及配置：
1. 建立相关目录
[root@localhost dir_redis_sentinel]# tree . ├── dir_1 │ ├── data6379 │ │ └── redis_1.sock │ ├── haconfig.sh │ ├── redis_1.cnf │ ├── sentinel_1.cnf │ └── sentinel6379 │ ├── failover.log │ └── sentinel_1.log ├── dir_2 │ ├── data6379 │ │ └── redis_2.sock │ ├── haconfig.sh │ ├── redis_2.cnf │ ├── sentinel_2.cnf │ └── sentinel6379 │ ├── failover.log │ └── sentinel_2.log └── dir_3 ├── data6379 │ └── redis_3.sock ├── haconfig.sh ├── redis_3.cnf ├── sentinel_3.cnf └── sentinel6379 ├── failover.log └── sentinel_3.log 9 directories, 30 files
2. 修改redis配置文件redis.cnf：
[root@localhost dir_redis_sentinel]# cat dir_1/redis_1.cnf daemonize no protected-mode yes pidfile "/data/data6379/redis_1.pid" port 6379 tcp-backlog 511 bind 0.0.0.0 unixsocket "/data/data6379/redis_1.sock" timeout 0 tcp-keepalive 0 loglevel notice logfile "/data/data6379/redis_1.log" databases 16 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum no dbfilename "dump.rdb" dir "/data/data6379" masterauth "redis" slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-ping-slave-period 5 repl-timeout 60 repl-disable-tcp-nodelay no repl-backlog-size 32mb repl-backlog-ttl 3600 slave-priority 100 requirepass "redis" rename-command FLUSHDB REDIS_FLUSHDB rename-command FLUSHALL REDIS_FLUSHALL rename-command KEYS REDIS_KEYS maxmemory 128mb maxmemory-policy allkeys-lru appendonly yes appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite yes auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 1000 latency-monitor-threshold 0 notify-keyspace-events "e" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-entries 512 list-max-ziplist-value 64 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes # Generated by CONFIG REWRITE
3. 修改 sentinel配置文件：
[root@localhost dir_redis_sentinel]# cat dir_1/sentinel_1.cnf port 16379 dir "/data/sentinel6379" logfile "/data/sentinel6379/sentinel_1.log" daemonize no protected-mode no sentinel myid d516c83477b7a7fd38f9ceff70391c1e0d8ed28c sentinel monitor mysent1 162.29.0.6 6379 2 sentinel down-after-milliseconds mysent1 15000 sentinel failover-timeout mysent1 120000 #发生切换之后执行的一个自定义脚本：如发邮件、vip切换等 #sentinel notification-script <master-name> <script-path> sentinel client-reconfig-script mysent1 /data/haconfig.sh # Generated by CONFIG REWRITE sentinel auth-pass mysent1 redis
5. 创建docker-compose配置文件：
[root@localhost data]# cat compose_redis_sentinel.yaml version: '2' networks: redisnet: external: true # ipam: # driver: default # config: # - subnet: 162.29.0.0/24 services: redis_1: #build: # context: . # dockerfile: Dockerfile image: redis:4.0.8 container_name: redis_1 command: - sh - -c - | redis-server /data/redis_1.cnf & redis-sentinel /data/sentinel_1.cnf #restart: always #environment: # MYSQL_ROOT_PASSWORD: 12345678 ports: - 63791:6379 - 16371:16379 volumes: - /data/data/dir_redis_sentinel/dir_1:/data networks: redisnet: ipv4_address: 162.29.0.6 redis_2: #build: # context: . # dockerfile: Dockerfile image: redis:4.0.8 container_name: redis_2 command: - sh - -c - | redis-server /data/redis_2.cnf & redis-sentinel /data/sentinel_2.cnf #restart: always #environment: # MYSQL_ROOT_PASSWORD: 12345678 ports: - 63792:6379 - 16372:16379 volumes: - /data/data/dir_redis_sentinel/dir_2:/data networks: redisnet: ipv4_address: 162.29.0.7 redis_3: #build: # context: . # dockerfile: Dockerfile image: redis:4.0.8 container_name: redis_3 command: - sh - -c - | redis-server /data/redis_3.cnf & redis-sentinel /data/sentinel_3.cnf #restart: always #environment: # MYSQL_ROOT_PASSWORD: 12345678 ports: - 63793:6379 - 16373:16379 volumes: - /data/data/dir_redis_sentinel/dir_3:/data networks: redisnet: ipv4_address: 162.29.0.8
安装
docker安装以及配置自定义网络请参考前文
1. 安装docker-compose
pip install docker-compose
2.启动
docker-compose -f compose_redis_sentinel.yaml up -d
查看是否启动成功
[root@localhost data]# docker-compose -f compose_redis_sentinel.yaml ps Name Command State Ports ------------------------------------------------------------------------------------------------------- redis_1 docker-entrypoint.sh sh -c ... Exit 255 0.0.0.0:16371->16379/tcp, 0.0.0.0:63791->6379/tcp redis_2 docker-entrypoint.sh sh -c ... Exit 255 0.0.0.0:16372->16379/tcp, 0.0.0.0:63792->6379/tcp redis_3 docker-entrypoint.sh sh -c ... Exit 255 0.0.0.0:16373->16379/tcp, 0.0.0.0:63793->6379/tcp
重启docker实例
docker-compose -f compose_redis_sentinel.yaml restart
3. 实例启动后，执行slaveof 配置主从关系
5. 登录其中一个sentinel，查看sentinel的状态
[root@localhost data]# redis-cli -h 162.29.0.6 -p 16379 info sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mysent1,status=ok,address=162.29.0.8:6379,slaves=2,sentinels=3
6. 查看master节点的信息
### sentinel masters 展示所有被监控的主节点状态以及相关的统计信息 ### sentinel master mysent1 展示指定<master name>的主节点状态以及相关的统计信息 ### sentinel slaves mysent1 展示指定<master name>的从节点状态以及相关的统计信息 ### sentinel sentinels mysent1 展示指定<master name>的Sentinel节点集合（不包含当前Sentinel节点） ### sentinel get-master-addr-by-name mysent1 返回指定<master name>主节点的IP地址和端口 162.29.0.6:16379> sentinel get-master-addr-by-name mysent1 1) "162.29.0.8" 2) "6379"
7. 手动故障转移测试
sentinel failover mysent1 对指定<master name>主节点进行强制故障转移（没有和其他Sentinel节 点“协商”），当故障转移完成后，其他Sentinel节点按照故障转移的结果更 新自身配置 162.29.0.6:16379> sentinel failover mysent1 OK 8:X 27 Jul 16:48:16.187 # Executing user requested FAILOVER of 'mysent1' 8:X 27 Jul 16:48:16.187 # +new-epoch 2 8:X 27 Jul 16:48:16.187 # +try-failover master mysent1 162.29.0.8 6379 8:X 27 Jul 16:48:16.238 # +vote-for-leader d516c83477b7a7fd38f9ceff70391c1e0d8ed28c 2 8:X 27 Jul 16:48:16.238 # +elected-leader master mysent1 162.29.0.8 6379 8:X 27 Jul 16:48:16.238 # +failover-state-select-slave master mysent1 162.29.0.8 6379 8:X 27 Jul 16:48:16.296 # +selected-slave slave 162.29.0.6:6379 162.29.0.6 6379 @ mysent1 162.29.0.8 6379 8:X 27 Jul 16:48:16.296 * +failover-state-send-slaveof-noone slave 162.29.0.6:6379 162.29.0.6 6379 @ mysent1 8:X 27 Jul 16:48:16.373 * +failover-state-wait-promotion slave 162.29.0.6:6379 162.29.0.6 6379 @ mysent1 162.29.0.8 8:X 27 Jul 16:48:17.279 # +promoted-slave slave 162.29.0.6:6379 162.29.0.6 6379 @ mysent1 162.29.0.8 6379 8:X 27 Jul 16:48:17.279 # +failover-state-reconf-slaves master mysent1 162.29.0.8 6379 8:X 27 Jul 16:48:17.348 * +slave-reconf-sent slave 162.29.0.7:6379 162.29.0.7 6379 @ mysent1 162.29.0.8 6379 8:X 27 Jul 16:48:18.326 * +slave-reconf-inprog slave 162.29.0.7:6379 162.29.0.7 6379 @ mysent1 162.29.0.8 6379 8:X 27 Jul 16:48:18.326 * +slave-reconf-done slave 162.29.0.7:6379 162.29.0.7 6379 @ mysent1 162.29.0.8 6379 8:X 27 Jul 16:48:18.418 # +failover-end master mysent1 162.29.0.8 6379 8:X 27 Jul 16:48:18.418 # +switch-master mysent1 162.29.0.8 6379 162.29.0.6 6379 8:X 27 Jul 16:48:18.419 * +slave slave 162.29.0.7:6379 162.29.0.7 6379 @ mysent1 162.29.0.6 6379 8:X 27 Jul 16:48:18.419 * +slave slave 162.29.0.8:6379 162.29.0.8 6379 @ mysent1 162.29.0.6 6379 162.29.0.6:16379> info sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mysent1,status=ok,address=162.29.0.6:6379,slaves=2,sentinels=3
这样，Docker部署Redis Sentinel 环境就搭建好了，可以做各种自己想要的测试了。
