Redis复制和哨兵部署 - 一泽涟漪

环境准备

主机名	sht-sgmhadoopdn-01	sht-sgmhadoopdn-02	sht-sgmhadoopdn-03
OS	CentOS Linux release 7.6.1810 (Core)
IP地址	172.16.101.58	172.16.101.59	172.16.101.60
redis安装路径	/usr/local/redis
redis角色	master	slave	slave
sentinel	yes	yes	yes

redis各版本下载地址

http://download.redis.io/releases/

此处我们选择3.2.12版本

http://download.redis.io/releases/redis-3.2.12.tar.gz

一. 各节点系统环境配置

1. 新建系统用户redis

# groupadd -r dba
# useradd -r -g dba -G root redis
# cat /home/redis/.bash_profile
................
PATH=/usr/local/redis/src:$PATH:$HOME/.local/bin:$HOME/bin
................

2. 内核参数优化

# cat /etc/sysctl.conf
.......................
vm.overcommit_memory=1
vm.swappiness=0
fs.aio-max-nr=1048576
fs.file-max= 7672460
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.ip_local_port_range=9000 65500
net.core.rmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_default=262144
net.core.wmem_max=1048586
net.core.somaxconn = 4096
kernel.sem= 50100 64128000 50100 1280
kernel.shmall=5242880
kernel.shmmax=12884901888
.......................

3. 系统资源使用优化

redis               soft    nproc           65536
redis               hard    nproc           65536
redis               soft    nofile          65536
redis               hard    nofile          65536

4. 大内存页优化

# echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled" >> /etc/rc.local

二. 各节点redis安装

$ cd /usr/local/redis
$ tar -zxf redis-3.2.12.tar.gz 
$ mv redis-3.2.12/* .
$ rm -rf redis-3.2.12*
$ make
$ mkdir {data,log,conf}
$ mv redis.conf sentinel.conf conf

三.修改redis主配置文件redis.conf，这里仅列出需要的参数

bind 0.0.0.0
protected-mode no
tcp-backlog 1024
tcp-keepalive 0
supervised systemd
daemonize yes
logfile "/usr/local/redis/log/redis.log"
dir "/usr/local/redis/data"
maxmemory 4gb
maxmemory-policy allkeys-lru
maxclients 65000
appendonly yes
appendfilename "appendonly.aof"
rename-command flushall ""
rename-command flushdb ""
rename-command save ""
rename-command keys ""
repl-ping-slave-period 3
repl-timeout 15
min-slaves-to-write 1
min-slaves-max-lag 10

注意：

maxmemory内存为系统总内存的2/3左右
在slave节点配置文件需要添加“slaveof 172.16.101.58 6379”

四. 在各节点将redis配置为systemd服务，并启动redis服务

[Unit]
Description=Redis In-Memory Data Store
Documentation=https://redis.io/documentation
After=syslog.target
After=network.target

[Service]
Type=notify
User=redis
Group=dba
Restart=no

# Disable OOM kill on the Redis
OOMScoreAdjust=-1000
Environment=PG_OOM_ADJUST_FILE=/proc/self/oom_score_adj
Environment=PG_OOM_ADJUST_VALUE=0

ExecStart=/usr/local/redis/src/redis-server /usr/local/redis/conf/redis.conf
ExecStop=/bin/kill -15 $MAINPID
LimitNOFILE=65536
 

[Install]
WantedBy=multi-user.target

# systemctl start redis
# systemctl enable redis
# systemctl status redis

● redis.service - Redis In-Memory Data Store
   Loaded: loaded (/usr/lib/systemd/system/redis.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2020-03-28 21:15:36 CST; 1s ago
     Docs: https://redis.io/documentation
  Process: 12897 ExecStop=/bin/kill -15 $MAINPID (code=exited, status=1/FAILURE)
 Main PID: 12945 (redis-server)
   CGroup: /system.slice/redis.service
           └─12945 /usr/local/redis/src/redis-server 0.0.0.0:6379

Mar 28 21:15:36 sht-sgmhadoopdn-01 systemd[1]: Starting Redis In-Memory Data Store...
Mar 28 21:15:36 sht-sgmhadoopdn-01 systemd[1]: Started Redis In-Memory Data Store.

redis master启动成功的log如下

12945:C 28 Mar 21:15:36.238 * supervised by systemd, will signal readiness
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.12 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 12945
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

12945:M 28 Mar 21:15:36.241 # Server started, Redis version 3.2.12
12945:M 28 Mar 21:15:36.241 * DB loaded from disk: 0.000 seconds
12945:M 28 Mar 21:15:36.242 * The server is now ready to accept connections on port 6379

redis slave节点启动成功的log如下

5116:C 28 Mar 21:45:36.336 * supervised by systemd, will signal readiness
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.12 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 5116
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

5116:S 28 Mar 21:45:36.342 # Server started, Redis version 3.2.12
5116:S 28 Mar 21:45:36.342 * DB loaded from disk: 0.000 seconds
5116:S 28 Mar 21:45:36.342 * The server is now ready to accept connections on port 6379
5116:S 28 Mar 21:45:36.342 * Connecting to MASTER 172.16.101.58:6379
5116:S 28 Mar 21:45:36.342 * MASTER <-> SLAVE sync started
5116:S 28 Mar 21:45:36.343 * Non blocking connect for SYNC fired the event.
5116:S 28 Mar 21:45:36.344 * Master replied to PING, replication can continue...
5116:S 28 Mar 21:45:36.344 * Partial resynchronization not possible (no cached master)
5116:S 28 Mar 21:45:36.345 * Full resync from master: e6eb0ae9701e18de4080032853185db5c748267a:1093
5116:S 28 Mar 21:45:36.406 * MASTER <-> SLAVE sync: receiving 77 bytes from master
5116:S 28 Mar 21:45:36.406 * MASTER <-> SLAVE sync: Flushing old data
5116:S 28 Mar 21:45:36.406 * MASTER <-> SLAVE sync: Loading DB in memory
5116:S 28 Mar 21:45:36.406 * MASTER <-> SLAVE sync: Finished with success

五，验证master、slave节点复制情况

1. 查看主从状态

在master节点查看

$ redis-cli info replication

# Replication
role:master
connected_slaves:2
min_slaves_good_slaves:2
slave0:ip=172.16.101.60,port=6379,state=online,offset=1877,lag=1
slave1:ip=172.16.101.59,port=6379,state=online,offset=1877,lag=1
master_repl_offset:1877
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:1876

在slave节点1查看

# Replication
role:slave
master_host:172.16.101.58
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:2185
slave_priority:100
slave_read_only:1
connected_slaves:0
min_slaves_good_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

在slave2节点查看

# Replication
role:slave
master_host:172.16.101.58
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:2451
slave_priority:100
slave_read_only:1
connected_slaves:0
min_slaves_good_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

2. 测试复制情况

在主节点写入数据

172.16.101.58:6379> set string "This is Redis Replication."
OK
172.16.101.58:6379> get string
"This is Redis Replication."

分别在从节点查看

172.16.101.59:6379> scan 10
1) "0"
2) 1) "string"
172.16.101.59:6379> get string
"This is Redis Replication."

172.16.101.60:6379> scan 10
1) "0"
2) 1) "string"
172.16.101.60:6379> get string
"This is Redis Replication."

六.哨兵部署

1. 修改各节点sentinel.conf文件，内容如下，并将该文件复制到所有节点

protected-mode no
port 26379
supervised systemd
dir "/usr/local/redis/data"
logfile "/usr/local/redis/log/sentinel.log"
sentinel monitor mymaster 172.16.101.58 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000

2. 将sentinel配置为systemd服务并启动

[Unit]
Description=Redis Sentinel provides high availability for Redis.
Documentation=https://redis.io/topics/sentinel
After=syslog.target
After=network.target
After=redis.target

[Service]
Type=fork
User=redis
Group=dba
Restart=always

# Disable OOM kill on the Redis
OOMScoreAdjust=-1000
Environment=PG_OOM_ADJUST_FILE=/proc/self/oom_score_adj
Environment=PG_OOM_ADJUST_VALUE=0

ExecStart=/usr/local/redis/src/redis-sentinel /usr/local/redis/conf/sentinel.conf
ExecStop=/bin/kill -15 $MAINPID
LimitNOFILE=65536
 

[Install]
WantedBy=multi-user.target

# systemctl daemon-reload
# systemctl start sentinel
# systemctl status sentinel

● sentinel.service - Redis Sentinel provides high availability for Redis.
   Loaded: loaded (/usr/lib/systemd/system/sentinel.service; disabled; vendor preset: disabled)
   Active: active (running) since Sat 2020-03-28 23:43:02 CST; 4s ago
     Docs: https://redis.io/topics/sentinel
 Main PID: 14942 (redis-sentinel)
   CGroup: /system.slice/sentinel.service
           └─14942 /usr/local/redis/src/redis-sentinel 0.0.0.0:26379 [sentinel]

Mar 28 23:43:02 sht-sgmhadoopdn-03 systemd[1]: Starting Redis Sentinel provides high availability for Redis....
Mar 28 23:43:02 sht-sgmhadoopdn-03 systemd[1]: Started Redis Sentinel provides high availability for Redis..

3. 在各节点查看sentinel运行情况

$ redis-cli -h 172.16.101.58 -p 26379 info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=172.16.101.58:6379,slaves=2,sentinels=1

$ redis-cli -h 172.16.101.59 -p 26379 info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=172.16.101.58:6379,slaves=2,sentinels=1

$ redis-cli -h 172.16.101.60 -p 26379 info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=172.16.101.58:6379,slaves=2,sentinels=1

posted on 2020-03-28 17:28 一泽涟漪阅读(700) 评论(0) 收藏举报

刷新页面返回顶部