Redis多机多节点集群实验
第一步：环境准备
我们搞两台虚拟机
局域网IP 分别是 192.168.1.109和192.168.1.110
我们约定把192.168.1.109作为集群控制端，需要安装redis-trib.rb
第二步：安装Redis
前面已经安装过了 不解释，
192.168.1.109虚拟机里，Reids安装包里有个集群工具，要复制到/usr/local/bin里去
cp redis-3.2.9/src/redis-trib.rb /usr/local/bin
第三步：修改配置，创建节点
首先我们在192.168.1.109虚拟机里创建三个节点，端口分别是7001，7002，7003
我们先在root目录下新建一个redis_cluster目录，然后该目录下再创建3个目录，
分别是7001，7002，7003，用来存redis配置文件；
这里我们要使用redis集群，要先修改redis的配置文件redis.conf
mkdir redis_cluster 新建目录
[root@localhost ~]# cd redis_cluster/
[root@localhost redis_cluster]# mkdir 7001 7002 7003
[root@localhost redis_cluster]# ll
总用量 0
drwxr-xr-x. 2 root root 6 7月 27 19:49 7001
drwxr-xr-x. 2 root root 6 7月 27 19:49 7002
drwxr-xr-x. 2 root root 6 7月 27 19:49 7003
[root@localhost redis_cluster]#
先复制一份配置文件到7001目录下
[root@localhost redis_cluster]# cd
[root@localhost ~]# cp redis-3.2.9/redis.conf redis_cluster/7001/
我们修改下这个配置文件
vi redis_cluster/7001/redis.conf
修改一下几个
port 7001 //六个节点配置文件分别是7001-7003
bind 192.168.1.109 //默认ip为127.0.0.1 需要改为其他节点机器可访问的ip 否则创建集群时无法访，和单机集群有区别
daemonize yes //redis后台运行
pidfile /var/run/redis_7001.pid //pidfile文件对应7001-7003
cluster-enabled yes //开启集群
cluster-config-file nodes_7001.conf //保存节点配置，自动创建，自动更新对应7001-7003
cluster-node-timeout 5000 //集群超时时间，节点超过这个时间没反应就断定是宕机
appendonly yes //存储方式，aof，将写操作记录保存到日志中
7001下的修改完后，我们把7001下的配置分别复制到7002-7003 然后对应的再修改下配置即可；
[root@localhost ~]# cp redis_cluster/7001/redis.conf redis_cluster/7002/
[root@localhost ~]# cp redis_cluster/7001/redis.conf redis_cluster/7003/
[root@localhost ~]# vi redis_cluster/7002/redis.conf
[root@localhost ~]# vi redis_cluster/7003/redis.conf
[root@localhost ~]# vi redis_cluster/7004/redis.conf
编辑后面5个配置文件，把 port ，pidfile，cluster-config-file 分别修改下即可；
同理 192.168.1.110机器，也搞一个redis_cluster目录，然后再新建7004，7005，7006目录，
复制三份配置文件 其他一样，就端口和bind地址改下即可；这里不啰嗦了；
第四步：启动两台机器的六个节点
192.168.1.109机器
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7001/redis.conf
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7002/redis.conf
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7003/redis.conf
[root@localhost ~]# ps -ef | grep redis
root 2242 1 0 19:55 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.109:7001 [cluster]
root 2252 1 0 19:59 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.109:7002 [cluster]
root 2256 1 0 19:59 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.109:7003 [cluster]
root 2260 2214 0 19:59 pts/0 00:00:00 grep --color=auto redis
192.168.1.110机器
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7004/redis.conf
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7005/redis.conf
[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7006/redis.conf
[root@localhost ~]# ps -ef | grep redis
root 2347 1 0 20:31 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.110:7004 [cluster]
root 2351 1 0 20:31 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.110:7005 [cluster]
root 2355 1 0 20:31 ? 00:00:00 /usr/local/redis/bin/redis-server 192.168.1.110:7006 [cluster]
root 2363 2270 0 20:32 pts/0 00:00:00 grep --color=auto redis
说明都启动OK
第五步：设置防火墙，开放集群端口
两台机器的防火墙我们直接关掉
systemctl stop firewalld.service
第六步：创建集群
192.168.1.109机器作为集群控制端
redis官方提供了redis-trib.rb工具，第一步里已经房到里bin下 ；
但是在使用之前 需要安装ruby，以及redis和ruby连接
yum -y install ruby ruby-devel rubygems rpm-build
gem install redis
redis-trib.rb create --replicas 1 192.168.1.109:7001 192.168.1.109:7002 192.168.1.109:7003 192.168.1.110:7004 192.168.1.110:7005 192.168.1.110:7006
创建集群
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.1.109:7001
192.168.1.110:7004
192.168.1.109:7002
Adding replica 192.168.1.110:7005 to 192.168.1.109:7001
Adding replica 192.168.1.109:7003 to 192.168.1.110:7004
Adding replica 192.168.1.110:7006 to 192.168.1.109:7002
M: 08d5f59fa053d79e3cc71fc5bb8759bec191ff41 192.168.1.109:7001
slots:0-5460 (5461 slots) master
M: 4bda83bae0aeb9988aeb7d34eff2cc75226a0edd 192.168.1.109:7002
slots:10923-16383 (5461 slots) master
S: b782f527ead6f63c8278c1b0713b92a9fd4fe995 192.168.1.109:7003
replicates 86c8cb7975f85a859147fef9672d44b24c5a718b
M: 86c8cb7975f85a859147fef9672d44b24c5a718b 192.168.1.110:7004
slots:3671,5461-10922,11797,15924 (5465 slots) master
S: 2d44accacb1bb4cd1eef5f98f7af64a43f01544d 192.168.1.110:7005
replicates 08d5f59fa053d79e3cc71fc5bb8759bec191ff41
S: 2c42433953bcd3ad47015d9f8ac72e7a7cf447c5 192.168.1.110:7006
replicates 4bda83bae0aeb9988aeb7d34eff2cc75226a0edd
Can I set the above configuration? (type 'yes' to accept):
从运行结果看 主节点就是7001 7004 7002 从节点分别是7005 7003 7006
7001分配到的哈希槽是 0-5460
7004分配到的哈希槽是 5461-10922
7002分配到的哈希槽是 10923-16383
最后问我们是否接受上面的设置，输入yes 就表示接受，我们输入yes
然后显示：
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 192.168.1.109:7001)
M: d06aebf92e99169864893decc371e34cd321640f 192.168.1.109:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 189dc189dd743f9fc61c439d44cc3379d8cbe535 192.168.1.110:7004
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: fe2b0038304c02bc807c38fe006338bd5dd85a70 192.168.1.110:7006
slots: (0 slots) slave
replicates 25a6be3b62fa675a3835e6c62641c3784ec62257
M: 25a6be3b62fa675a3835e6c62641c3784ec62257 192.168.1.109:7002
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: df28512d63af4a3e6dc1739e1c267ba916f3a579 192.168.1.110:7005
slots: (0 slots) slave
replicates d06aebf92e99169864893decc371e34cd321640f
S: aaa3310d3c30cc9a7f235db27e3c395a5e180a52 192.168.1.109:7003
slots: (0 slots) slave
replicates 189dc189dd743f9fc61c439d44cc3379d8cbe535
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
显示配置哈希槽，以及集群创建成功，可以用了；
第七步：集群数据测试
我们先连接任意一个节点，然后添加一个key：
redis-cli是redis默认的客户端工具，启动时加上｀-c｀参数，`-p`指定端口，就可以连接到集群。
这里还得加-h 指定机器IP
连接任意一个节点端口：
[root@localhost ~]# /usr/local/redis/bin/redis-cli -h 192.168.1.109 -c -p 7002
192.168.1.109:7002>
连接到7002节点
192.168.1.109:7002> set xxx 'fadfa'
-> Redirected to slot [4038] located at 192.168.1.109:7001
OK
前面说过Redis Cluster值分配规则，所以分配key的时候，它会使用CRC16(‘my_name’)%16384算法，来计算，将这个key 放到哪个节点，这里分配到了4038slot 就分配到了7001(0-5460)这个节点上。所以有：
Redirected to slot [4038] located at 192.168.1.109:7001
我们从其他集群节点 ，都可以获取到数据
[root@localhost ~]# /usr/local/redis/bin/redis-cli -h 192.168.1.110 -c -p 7005
192.168.1.110:7005>
192.168.1.110:7005> get xxx
-> Redirected to slot [4038] located at 192.168.1.109:7001
"fadfa"
192.168.1.109:7001>
第八步：集群宕机测试
假如我们干掉一个节点，比如7002 这个主节点
[root@localhost ~]# ps -ef | grep redis
root 9799 1 0 21:33 ? 00:00:02 /usr/local/redis/bin/redis-server 192.168.1.109:7001 [cluster]
root 9803 1 0 21:33 ? 00:00:02 /usr/local/redis/bin/redis-server 192.168.1.109:7002 [cluster]
root 9807 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.109:7003 [cluster]
root 9830 2214 0 21:53 pts/0 00:00:00 grep --color=auto redis
[root@localhost ~]# kill -9 9803
[root@localhost ~]# ps -ef | grep redis
root 9799 1 0 21:33 ? 00:00:02 /usr/local/redis/bin/redis-server 192.168.1.109:7001 [cluster]
root 9807 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.109:7003 [cluster]
root 9832 2214 0 21:54 pts/0 00:00:00 grep --color=auto redis
[root@localhost ~]#
然后我们查看下集群情况
[root@localhost ~]# redis-trib.rb check 192.168.1.109:7001
>>> Performing Cluster Check (using node 192.168.1.109:7001)
M: d06aebf92e99169864893decc371e34cd321640f 192.168.1.109:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 189dc189dd743f9fc61c439d44cc3379d8cbe535 192.168.1.110:7004
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: fe2b0038304c02bc807c38fe006338bd5dd85a70 192.168.1.110:7006
slots:10923-16383 (5461 slots) master
0 additional replica(s)
S: df28512d63af4a3e6dc1739e1c267ba916f3a579 192.168.1.110:7005
slots: (0 slots) slave
replicates d06aebf92e99169864893decc371e34cd321640f
S: aaa3310d3c30cc9a7f235db27e3c395a5e180a52 192.168.1.109:7003
slots: (0 slots) slave
replicates 189dc189dd743f9fc61c439d44cc3379d8cbe535
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
我们发现 7006本来是从节点，由于他对应的主节点挂了，就自动变成主节点master，所有会有最后一个说明
All 16384 slots covered. 所有哈希槽都可覆盖了； 集群可以正常使用；
假如我们把7006也干掉，试试看
[root@localhost ~]# ps -ef | grep redis
root 2578 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7004 [cluster]
root 2582 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7005 [cluster]
root 2586 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7006 [cluster]
root 2623 2270 0 21:56 pts/0 00:00:00 grep --color=auto redis
[root@localhost ~]# kill -9 2586
[root@localhost ~]# ps -ef | grep redis
root 2578 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7004 [cluster]
root 2582 1 0 21:33 ? 00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7005 [cluster]
root 2625 2270 0 21:56 pts/0 00:00:00 grep --color=auto redis
[root@localhost ~]#
[root@localhost ~]# redis-trib.rb check 192.168.1.109:7001
这里我们发现 出事了，因为主从节点都挂了 所以有一部分哈希槽没得分配，最后一句
[ERR] Not all 16384 slots are covered by nodes. 没有安全覆盖；
所以不能正常使用集群
