centos6搭建redis集群搭建（单机多节点）
一、安装redis
1、安装gcc环境
yum install gcc-c++
2、下载源码包并解压
wget http://download.redis.io/releases/redis-3.2.4.tar.gz
tar -zxvf redis-3.2.4.tar.gz
cd redis-3.2.4
3、安装ruby2.4.1并编译（一定要安装大于2.2.2的ruby，要不然redis在make的时候会报错）
wget http://ftp.ruby-lang.org/pub/ruby/2.4/ruby-2.4.1.tar.gz
tar -zxvf ruby-2.4.1.tar.gz
cd ruby-2.4.1
mkdir -p /usr/local/ruby
./configure --prefix=/usr/local/ruby
make && make install
ln -s /usr/local/ruby/bin/ruby /usr/bin/ruby
4、安装
make
make install PREFIX=/usr/local/redis
cp redis.conf /usr/local/redis/bin/redis.conf
chmod /usr/local/redis/bin/redis.conf
5、启动redis
./redis-server redis.conf
6、验证
[root@host-172-16-80-177 bin]# ./redis-cli -p 6379
127.0.0.1:6379> get
(error) ERR wrong number of arguments for 'get' command
127.0.0.1:6379> set aa hzb
OK
127.0.0.1:6379> get aa
"hzb"
二、搭建redis集群（集群模式）
1、在/usr/local/下面创建redis-cluster目录
mkdir -p /usr/local/redis-cluster
2、复制/usr/local/redis/bin目录到redis-cluster里面并重命名为redis1
cd /usr/local
cp -r redis/bin redis-cluster/redis1
3、将redis1在同目录复制6份分别起名redis2,redis3,redis4,redis5,redis6（必须6个节点以上才能创建集群）
[root@host-172-16-80-177 redis-cluster]# pwd
/usr/local/redis-cluster
[root@host-172-16-80-177 redis-cluster]# cp -r redis1 redis2
[root@host-172-16-80-177 redis-cluster]# cp -r redis1 redis3
[root@host-172-16-80-177 redis-cluster]# cp -r redis1 redis4
[root@host-172-16-80-177 redis-cluster]# cp -r redis1 redis5
[root@host-172-16-80-177 redis-cluster]# cp -r redis1 redis6
把rendis.conf文件复制到redisx文件夹下
分别修改为redis-7001.conf,redis-7002.conf 后面依次到redis-7006.conf
以redis-7001为例：
daemonize yes
Port 700x
bind #服务器ip
logfile "./redis-700x.log"
protected-mode no
pidfile /var/run/redis_700x.pid
cluster-enabled yes
4、将redis-trib.rb复到制redis-cluster目录里面
[root@host-172-16-80-177 redis-cluster]# cp /root/redis-3.2.4/src/redis-trib.rb .
[root@host-172-16-80-177 redis-cluster]# ll
total 76
drwxr-xr-x 2 root root 4096 Oct 19 22:37 redis1
drwxr-xr-x 2 root root 4096 Oct 19 22:48 redis2
drwxr-xr-x 2 root root 4096 Oct 19 22:49 redis3
drwxr-xr-x 2 root root 4096 Oct 19 22:49 redis4
drwxr-xr-x 2 root root 4096 Oct 19 22:49 redis5
drwxr-xr-x 2 root root 4096 Oct 19 22:49 redis6
-rwxr-xr-x 1 root root 60852 Oct 19 22:52 redis-trib.rb
5、安装gem
yum install rubygems -y
6、安装ruby的redis包
gem install redis
上面这一步可能会出现：
ERROR: Loading command: install (LoadError)
cannot load such file -- zlib
ERROR: While executing gem ... (NoMethodError)
undefined method `invoke_with_build_args' for nil:NilClass
解决办法是：
进入ruby源码文件夹
安装ruby自身提供的zlib包
#cd ext/zlib
#ruby ./extconf.rb
#make
#make install
进入ruby源码文件夹
安装ruby自身提供的openssl包
#cd ext/openssl
#ruby ./extconf.rb
#make
#make install
如果执行make的时候报错：make: *** No rule to make target ，
错误原因：makefile里面的文件依赖关系有问题，导致头文件找不到。
解决办法：打开编译相关模块的makefile文件，找到.o文件的依赖关系，找到.h文件的路径，查看是否出错并改正。
原因分析：这种情况有时候是因为工程所在的目录改变，例如原来在C盘，现在移到了D盘，而makefile里面的.h文件路径没有改变
以上问题解决后再执行：
[root@host-172-16-80-177 include]# gem install redis
Fetching: redis-4.0.1.gem (100%)
Successfully installed redis-4.0.1
Parsing documentation for redis-4.0.1
Installing ri documentation for redis-4.0.1
Done installing documentation for redis after 1 seconds
1 gem installed
表示ruby的redis依赖包安装成功。
7、分别进入redis1,redis2,redis3,redis4,redis5,redis6里面执行
./redis-server redis-7001.conf
......
将6个节点启动
[root@host-172-16-80-177 redis4]# ps -ef|grep redis
root 3167 1 0 02:49 ? 00:00:00 ./redis-server 127.0.0.1:7001 [cluster]
root 3176 1 0 02:50 ? 00:00:00 ./redis-server 127.0.0.1:7002 [cluster]
root 3187 1 0 02:51 ? 00:00:00 ./redis-server 127.0.0.1:7003 [cluster]
root 3191 1 0 02:52 ? 00:00:00 ./redis-server 127.0.0.1:7004 [cluster]
root 3191 1 0 02:52 ? 00:00:00 ./redis-server 127.0.0.1:7005 [cluster]
root 3191 1 0 02:52 ? 00:00:00 ./redis-server 127.0.0.1:7006 [cluster]
root 3195 1852 0 02:52 pts/0 00:00:00 grep redis
8、用redis-trib.rb构建集群
cd /usr/local/redis-cluster/
[root@host-172-16-80-177 redis-cluster]# ./redis-trib.rb create --replicas 1 172.16.80.177:7001 172.16.80.177:7002 172.16.80.177:7003 172.16.80.177:7004 172.16.80.177:7005 172.16.80.177:7006
>>> Creating cluster
[ERR] Sorry, can't connect to node 172.16.80.177:7001
以上错误原因是redis.conf里面未开起远程访问
打开每个redis里面的.conf文件，找到
# JUST COMMENT THE FOLLOWING LINE.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bind 127.0.0.1
# Protected mode is a layer of security protection, in order to avoid that
# Redis instances left open on the internet are accessed and exploited.
将上面的bind 127.0.0.1改成bind 172.16.80.177(外部可以访问的ip)，然后重启各服务
root 3383 1 0 03:12 ? 00:00:00 ./redis-server 172.16.80.177:7001 [cluster]
root 3458 1 0 03:20 ? 00:00:00 ./redis-server 172.16.80.177:7002 [cluster]
root 3464 1 0 03:20 ? 00:00:00 ./redis-server 172.16.80.177:7003 [cluster]
root 3470 1 0 03:20 ? 00:00:00 ./redis-server 172.16.80.177:7004 [cluster]
root 3506 1 0 03:25 ? 00:00:00 ./redis-server 172.16.80.177:7005 [cluster]
root 3518 1 0 03:25 ? 00:00:00 ./redis-server 172.16.80.177:7006 [cluster]
root 3523 1852 0 03:26 pts/0 00:00:00 grep redis
再用redis-trib.rb创建集群：
[root@host-172-16-80-177 redis-cluster]# ./redis-trib.rb create --replicas 1 172.16.80.177:7001 172.16.80.177:7002 172.16.80.177:7003 172.16.80.177:7004 172.16.80.177:7005 172.16.80.177:7006
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
172.16.80.177:7001
172.16.80.177:7002
172.16.80.177:7003
Adding replica 172.16.80.177:7004 to 172.16.80.177:7001
Adding replica 172.16.80.177:7005 to 172.16.80.177:7002
Adding replica 172.16.80.177:7006 to 172.16.80.177:7003
M: 40bbe8a979fe39f1f0941cc8e118443436fc71d8 172.16.80.177:7001
slots:0-5460 (5461 slots) master
M: e6b76ee1540c9be6c44abc3dec42378f8cfd0191 172.16.80.177:7002
slots:5461-10922 (5462 slots) master
M: fdc248e24bd9f8690c652213a5781f4e567208e8 172.16.80.177:7003
slots:10923-16383 (5461 slots) master
S: 38faede9dd348983b1b8b80edc975e8d34105e5c 172.16.80.177:7004
replicates 40bbe8a979fe39f1f0941cc8e118443436fc71d8
S: 38faede9dd348983b1b8b80edc975e8d34105e5c 172.16.80.177:7005
replicates e6b76ee1540c9be6c44abc3dec42378f8cfd0191
S: 38faede9dd348983b1b8b80edc975e8d34105e5c 172.16.80.177:7006
replicates fdc248e24bd9f8690c652213a5781f4e567208e8
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 172.16.80.177:7001)
M: 40bbe8a979fe39f1f0941cc8e118443436fc71d8 172.16.80.177:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: e6b76ee1540c9be6c44abc3dec42378f8cfd0191 172.16.80.177:7002
slots:5461-10922 (5462 slots) master
0 additional replica(s)
S: 38faede9dd348983b1b8b80edc975e8d34105e5c 172.16.80.177:7004
slots: (0 slots) slave
replicates 40bbe8a979fe39f1f0941cc8e118443436fc71d8
M: fdc248e24bd9f8690c652213a5781f4e567208e8 172.16.80.177:7003
slots:10923-16383 (5461 slots) master
0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
至此，redis集群创建成功
9、验证集群是否成功
[root@host-172-16-80-177 redis2]# ./redis-cli -h 172.16.80.177 -c -p 7002
172.16.80.177:7002> set hello helloword
-> Redirected to slot [866] located at 172.16.80.177:7001
OK
172.16.80.177:7006> get hello
-> Redirected to slot [866] located at 172.16.80.177:7001
"helloword"
检查节点状态命令：
./redis-cli check ip:port
./redis-cli check 192.168.1.88:7001
三、搭建redis集群（哨兵模式）
1、在/usr/local/下面创建redis-sentinel目录
mkdir -p /usr/local/redis-sentinel
2、将一大节中的/usr/local/redis/bin复制到redis-sentinel目录底下起名叫redis1,redis2,redis3,redis4,sentinel1,sentinel2
修改4个redis的配置文件（以redis1为例，将redis.conf重命名为redis-7001.conf）
daemonize yes
Port 7001
#Bind 127.0.0.1
logfile "./redis-7001.log"
protected-mode no
cluster-config-file nodes-7001.conf
其他三个redis的端口号为7002,7003,7004
3、将redis解压包里面的sentinel.conf复制到sentinel1和sentinel2里面并重命名为sentinel-7005.conf,sentinel-7006.conf，并赋予可执行权限
修改配置（以sentinel-7005.conf为例）
port 7005
sentinel monitor mymaster 172.16.80.177 7001 2
protected-mode no
注：我们稍后要启动四个redis实例，其中端口为7001的redis设为master，其他三个设为slave 。所以my mymaster 后跟的是master的ip和端口，最后一个’2’代表我要启动只要有2个sentinel认为master下线，就认为该master客观下线，启动failover并选举产生新的master。通常最后一个参数不能多于启动的sentinel实例数。
4.启动4个redis实例
./redis-server redis-7001.conf
......
[root@host-172-16-80-177 redis4]# ps -ef|grep redis
root 29967 1 0 22:02 ? 00:00:00 ./redis-server 172.16.80.177:7001
root 29971 1 0 22:02 ? 00:00:00 ./redis-server 172.16.80.177:7002
root 29976 1 0 22:02 ? 00:00:00 ./redis-server 172.16.80.177:7003
root 29985 1 0 22:02 ? 00:00:00 ./redis-server 172.16.80.177:7004
然后分别登陆7002 7003 7004三个实例，动态改变主从关系，成为7005的slave:
./redis-cli -h 172.16.80.177 -p 7002
172.16.80.177:7002> SLAVEOF 172.16.80.177 7001
5、以后台启动模式启动两个sentinel（哨兵）:
[root@host-172-16-80-177 sentinel1]# ./redis-sentinel sentinel-7005.conf &
[1] 30038
[root@host-172-16-80-177 sentinel1]# 30038:X 22 Oct 22:10:49.712 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.4 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 7005
| `-._ `._ / _.-' | PID: 30038
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
30038:X 22 Oct 22:10:49.713 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
30038:X 22 Oct 22:10:49.744 # Sentinel ID is cbd44de127d84fd7d21f9e4a73e62b94c3b0ea00
30038:X 22 Oct 22:10:49.744 # +monitor master mymaster 172.16.80.177 7001 quorum 2
30038:X 22 Oct 22:10:49.745 * +slave slave 172.16.80.177:7004 172.16.80.177 7004 @ mymaster 172.16.80.177 7001
30038:X 22 Oct 22:10:49.768 * +slave slave 172.16.80.177:7002 172.16.80.177 7002 @ mymaster 172.16.80.177 7001
30038:X 22 Oct 22:10:49.792 * +slave slave 172.16.80.177:7003 172.16.80.177 7003 @ mymaster 172.16.80.177 7001
[root@host-172-16-80-177 redis-3.2.4]# ps -ef|grep sentinel
root 30038 29505 0 22:10 pts/0 00:00:00 ./redis-sentinel *:7005 [sentinel]
root 30059 29505 0 22:13 pts/0 00:00:00 ./redis-sentinel *:7006 [sentinel]
6、验证
sentinel的基本命令
① INFO
sentinel的基本状态信息
②SENTINEL masters
列出所有被监视的主服务器，以及这些主服务器的当前状态
③ SENTINEL slaves
列出给定主服务器的所有从服务器，以及这些从服务器的当前状态
④SENTINEL get-master-addr-by-name
返回给定名字的主服务器的 IP 地址和端口号
⑤SENTINEL reset
重置所有名字和给定模式 pattern 相匹配的主服务器。重置操作清除主服务器目前的所有状态， 包括正在执行中的故障转移， 并移除目前已经发现和关联的， 主服务器的所有从服务器和 Sentinel 。
⑥SENTINEL failover
当主服务器失效时， 在不询问其他 Sentinel 意见的情况下， 强制开始一次自动故障迁移，但是它会给其他sentinel发送一个最新的配置，其他sentinel会根据这个配置进行更新
[root@host-172-16-80-177 redis1]# ./redis-cli -h 172.16.80.177 -p 7001
172.16.80.177:7001>
172.16.80.177:7001> set name hzb
OK
172.16.80.177:7001> get name
"hzb"
172.16.80.177:7001> exit
[root@host-172-16-80-177 redis1]# cd ../redis2
[root@host-172-16-80-177 redis2]# ./redis-cli -h 172.16.80.177 -p 7002
172.16.80.177:7002> get name
"hzb"
172.16.80.177:7002> set age 32
(error) READONLY You can't write against a read only slave.
上面可以看到slave是只读的
登录一个sentinel，用sentinel masters命令查看哪个是master
[root@host-172-16-80-177 sentinel1]# ./redis-cli -h 172.16.80.177 -p 7005
172.16.80.177:7005>
172.16.80.177:7005>
172.16.80.177:7005> sentinel masters
1) 1) "name"
2) "mymaster"
3) "ip"
4) "172.16.80.177"
5) "port"
6) "7001"
7) "runid"
8) "8b5d064e4195a1ee6279f5fb06f3fa88e4605854"
9) "flags"
10) "master"
11) "link-pending-commands"
12) "0"
13) "link-refcount"
14) "1"
15) "last-ping-sent"
16) "0"
17) "last-ok-ping-reply"
18) "624"
19) "last-ping-reply"
20) "624"
21) "down-after-milliseconds"
22) "30000"
23) "info-refresh"
24) "5427"
25) "role-reported"
26) "master"
27) "role-reported-time"
28) "85727"
29) "config-epoch"
30) "0"
31) "num-slaves"
32) "3"
33) "num-other-sentinels"
34) "1"
35) "quorum"
36) "2"
37) "failover-timeout"
38) "180000"
39) "parallel-syncs"
40) "1"
目前7001是master, 我们强制kill掉 7001 的进程以后，可以看到sentinel打出的信息：
30325:X 22 Oct 22:51:24.138 # +sdown master mymaster 172.16.80.177 7001
30325:X 22 Oct 22:51:24.557 # +new-epoch 1
30325:X 22 Oct 22:51:24.599 # +vote-for-leader 3948b4c445549c51d67bf829553372bca8bc67c8 1
30325:X 22 Oct 22:51:25.174 # +config-update-from sentinel 3948b4c445549c51d67bf829553372bca8bc67c8 172.16.80.177 7006 @ mymaster 172.16.80.177 7001
30325:X 22 Oct 22:51:25.174 # +switch-master mymaster 172.16.80.177 7001 172.16.80.177 7003
30325:X 22 Oct 22:51:25.175 * +slave slave 172.16.80.177:7004 172.16.80.177 7004 @ mymaster 172.16.80.177 7003
30325:X 22 Oct 22:51:25.175 * +slave slave 172.16.80.177:7002 172.16.80.177 7002 @ mymaster 172.16.80.177 7003
30325:X 22 Oct 22:51:25.175 * +slave slave 172.16.80.177:7001 172.16.80.177 7001 @ mymaster 172.16.80.177 7003
30325:X 22 Oct 22:51:55.202 # +sdown slave 172.16.80.177:7001 172.16.80.177 7001 @ mymaster 172.16.80.177 7003
[root@host-172-16-80-177 sentinel1]# ./redis-cli -h 172.16.80.177 -p 7005
172.16.80.177:7005> sentinel masters
1) 1) "name"
2) "mymaster"
3) "ip"
4) "172.16.80.177"
5) "port"
6) "7003"
7) "runid"
8) "31721c8d1371697d78bf3c6acc2f8d4ba829825a"
9) "flags"
10) "master"
11) "link-pending-commands"
12) "0"
13) "link-refcount"
14) "1"
15) "last-ping-sent"
16) "0"
17) "last-ok-ping-reply"
18) "454"
19) "last-ping-reply"
20) "454"
21) "down-after-milliseconds"
22) "30000"
23) "info-refresh"
24) "7168"
25) "role-reported"
26) "master"
27) "role-reported-time"
28) "27254"
29) "config-epoch"
30) "1"
31) "num-slaves"
32) "3"
33) "num-other-sentinels"
34) "1"
35) "quorum"
36) "2"
37) "failover-timeout"
38) "180000"
39) "parallel-syncs"
40) "1"
上面显示了，master自动从7001变成了7003节点
再次将7001启动起来
30385:X 22 Oct 22:55:15.544 * +convert-to-slave slave 172.16.80.177:7001 172.16.80.177 7001 @ mymaster 172.16.80.177 7003
可以看到自动将7001变成了7003的slave
下面加一个批量启动和停止的脚本：
#!/bin/bash
start(){
./redis1/redis-server ./redis1/redis.conf &
./redis2/redis-server ./redis2/redis.conf &
./redis3/redis-server ./redis3/redis.conf &
./redis4/redis-server ./redis4/redis.conf &
./sentinel1/redis-sentinel ./sentinel1/sentinel.conf &
./sentinel2/redis-sentinel ./sentinel2/sentinel.conf &
}
stop(){
ps -ef | grep "redis" | grep -v "grep" |awk '{print $2}'| while read pid
do
C_PID=$(ps --no-heading $pid | wc -l)
echo "当前PID=$pid"
if [[ $C_PID == "1" ]]; then
kill -9 $pid
echo "PID=$pid 已经结束"
else
echo "PID=$pid 不存在"
fi
done
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
*)
printf 'Usage: %s {start|stop|restart}\n'"$prog"
exit 1
;;
esac
