Redis集群搭建

一、搭建集群

Redis Cluster集群要求必须有3个master才构成一个集群，所以我这里采用3主3从的方式，实际生产中的高可用至少3主6从

（1）使用docker-compose启动6个redis容器，使用 host network，不然jedis内部重定向ip地址不正确。

version: '3.7'

services:
  redis7001:
    image: 'redis:6.2.4'
    container_name: redis7001
    network_mode: "host"
    command:
      ["redis-server","/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7001/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7001/data:/data
    ports:
      - "7001:7001"
      - "17001:17001"
    environment:
      - TZ=Asia/Shanghai
  redis7002:
    image: 'redis:6.2.4'
    container_name: redis7002
    network_mode: "host"
    command:
      ["redis-server", "/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7002/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7002/data:/data
    ports:
      - "7002:7002"
      - "17002:17002"
    environment:
      - TZ=Asia/Shanghai
  redis7003:
    image: 'redis:6.2.4'
    container_name: redis7003
    network_mode: "host"
    command:
      ["redis-server", "/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7003/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7003/data:/data
    ports:
      - "7003:7003"
      - "17003:17003"
    environment:
      - TZ=Asia/Shanghai
  redis7004:
    image: 'redis:6.2.4'
    container_name: redis7004
    network_mode: "host"
    command:
      ["redis-server", "/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7004/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7004/data:/data
    ports:
      - "7004:7004"
      - "17004:17004"
    environment:
      - TZ=Asia/Shanghai 
  redis7005:
    image: 'redis:6.2.4'
    container_name: redis7005
    network_mode: "host"
    command:
      ["redis-server", "/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7005/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7005/data:/data
    ports:
      - "7005:7005"
      - "17005:17005"
    environment:
      - TZ=Asia/Shanghai 
  redis7006:
    image: 'redis:6.2.4'
    container_name: redis7006
    network_mode: "host"
    command:
      ["redis-server", "/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7006/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7006/data:/data
    ports:
      - "7006:7006"
      - "17006:17006"
    environment:
      - TZ=Asia/Shanghai 

[root@shang redis]# docker ps
CONTAINER ID   IMAGE         COMMAND                  CREATED          STATUS         PORTS                                                                                                NAMES
e3018a083b37   redis:6.2.4   "docker-entrypoint.s…"   13 seconds ago   Up 4 seconds   0.0.0.0:7003->7003/tcp, :::7003->7003/tcp, 6379/tcp, 0.0.0.0:17003->17003/tcp, :::17003->17003/tcp   redis7003
911dc36fe35a   redis:6.2.4   "docker-entrypoint.s…"   13 seconds ago   Up 4 seconds   0.0.0.0:7005->7005/tcp, :::7005->7005/tcp, 6379/tcp, 0.0.0.0:17005->17005/tcp, :::17005->17005/tcp   redis7005
efabccd7769c   redis:6.2.4   "docker-entrypoint.s…"   13 seconds ago   Up 6 seconds   0.0.0.0:7004->7004/tcp, :::7004->7004/tcp, 6379/tcp, 0.0.0.0:17004->17004/tcp, :::17004->17004/tcp   redis7004
db48c12796cb   redis:6.2.4   "docker-entrypoint.s…"   13 seconds ago   Up 4 seconds   0.0.0.0:7002->7002/tcp, :::7002->7002/tcp, 6379/tcp, 0.0.0.0:17002->17002/tcp, :::17002->17002/tcp   redis7002
c717621d2c7b   redis:6.2.4   "docker-entrypoint.s…"   13 seconds ago   Up 4 seconds   0.0.0.0:7006->7006/tcp, :::7006->7006/tcp, 6379/tcp, 0.0.0.0:17006->17006/tcp, :::17006->17006/tcp   redis7006
10fd3e24f10f   redis:6.2.4   "docker-entrypoint.s…"   13 seconds ago   Up 4 seconds   0.0.0.0:7001->7001/tcp, :::7001->7001/tcp, 6379/tcp, 0.0.0.0:17001->17001/tcp, :::17001->17001/tcp   redis7001

（2）查看redis集群 [cluster]表示为集群，但此时它们之间并没有建立联系，也没有分配slot。

[root@shang 7001]# ps -ef | grep redis
polkitd    3888   3867  0 09:10 ?        00:00:02 redis-server *:7001 [cluster]
polkitd    4143   4120  0 09:14 ?        00:00:01 redis-server *:7002 [cluster]
polkitd    4257   4235  0 09:17 ?        00:00:01 redis-server *:7003 [cluster]
polkitd    4373   4352  0 09:20 ?        00:00:00 redis-server *:7004 [cluster]
polkitd    4490   4469  0 09:22 ?        00:00:00 redis-server *:7005 [cluster]
polkitd    4604   4581  0 09:23 ?        00:00:00 redis-server *:7006 [cluster]
root       4667   1576  0 09:24 pts/0    00:00:00 grep --color=auto redis

[root@shang 7001]# docker exec -it redis7001 redis-cli -p 7001 # 进入端口为7001的redis7001容器
127.0.0.1:7001> cluster nodes # 查看集群节点信息
87346232e7666c772d304518fa4215804be3f576 :7001@17001 myself,master - 0 0 0 connected

（3）建立联系,并分配slot

# 查看帮助命令
[root@shang 7001]# docker exec -it redis7001 redis-cli --cluster help 
# meet
create         host1:port1 ... hostN:portN
                 --cluster-replicas <arg>

docker exec -it redis7001 redis-cli --cluster create 192.168.16.3:7001 192.168.16.3:7002 192.168.16.3:7003 192.168.16.3:7004 192.168.16.3:7005 192.168.16.3:7006 --cluster-replicas 1
建立cluster联系，按照 1:1 的方式（3:3），前三个为master，后三个为slave。
如果 --cluster-replicas 2，表示按照 1:2 的方式（3:6)，前三个为master，后三个为slave

（4）查看是否搭建成功

[root@shang 7001]# docker exec -it redis7001 redis-cli -p 7001 # 进入端口为7001的redis7001容器
127.0.0.1:7001> cluster nodes # 查看集群节点信息

二、集群扩容

docker exec -it redis7001 redis-cli --cluster help # 查看帮助命令
add-node  new_host:new_port existing_host:existing_port # 新redis   集群中任一redis
          --cluster-slave  # 标志为从机
          --cluster-master-id <arg> # 指定从属于哪个主机，master-id
# 扩容一个slave				 
docker exec -it redis7001 redis-cli --cluster add-node 192.168.16.3:7007 192.168.16.3:7001 --cluster-slave --cluster-master-id 87346232e7666c772d304518fa4215804be3f576

# 扩容一个master，并分配slot
docker exec -it redis7001 redis-cli --cluster add-node 192.168.16.3:7008 192.168.16.3:7001

reshard  host:port # 集群任一节点
         --cluster-from <arg> # 带有slot的master节点id，多个以 , 隔开
         --cluster-to <arg> # 分配给哪个master节点id，多个以 , 隔开
         --cluster-slots <arg> # 总共分配多少slot
         --cluster-yes # 跳过确认
         --cluster-timeout <arg> # 
         --cluster-pipeline <arg> # 控制每次批量迁移键的数量，默认为10
         --cluster-replace
		
docker exec -it redis7001 redis-cli --cluster reshard 192.168.16.3:7007 --cluster-from 87346232e7666c772d304518fa4215804be3f576,4979d1b009f30576c4ba7587c01176275cf81628 --cluster-to 90afcac7725cac92e9b064f3a71348b9913c7eb3  --cluster-slots 1000 --cluster-yes

三、集群缩容

先移除某一个master节点的所有slave，避免选举为master。
移除master之前，要将master上的slot分配给其它存活的master，然后再删除当前master，避免数据丢失。

# 删除slave
 del-node host:port node_id # 任一集群节点   需要删除的节点id
 
 docker exec -it redis7001 redis-cli --cluster del-node 192.168.16.3:7001 3a29bd1d4e885cdfe5e03185183b88da2cfd5115
# 分离slot
docker exec -it redis7001 redis-cli --cluster reshard 192.168.16.3:7001 --cluster-from 90afcac7725cac92e9b064f3a71348b9913c7eb3 --cluster-to 05abce1703ec1aca5e38548ca2c252c4713c5840 --cluster-slots 1000 --cluster-yes

 docker exec -it redis7001 redis-cli --cluster del-node 192.168.16.3:7001 90afcac7725cac92e9b064f3a71348b9913c7eb3 # 移除master 7008

四、记录点

由于宿主机器（192.168.xx.xx）和Docker容器（170.17.0.x）的网络不同，导致Jedis操作Redis集群无法获取连接。根本原因在于JedisFactory中的makeObject中的HostAndPort被修改为docker网络，而不是我们宿主机器的外网ip，所以无法创建连接，获取失败。