1、Redis cluster实现（6节点）
2、Redis cluster slot重分配
3、Redis及Memcached对比
----------------------------------------------------------------------------------------------------
1、Redis cluster实现（6节点）
#原理： 1）各redis node通过ping机制互联，半数以上node检测失败，才认为是客观宕机 2）所有redis master node被平均映射到2^14（16384）个槽位 3）客户端写操作时，CRC16('key')%16384，根据取模运算结果决定写往哪个redis master node。（解决单机无法实现高并发） 4）每个master有各自的slave，当master宕机后，slave提升为主，继续提供服务。（解决高可用） #环境： redis版本：4.0.14 redis node：10.0.0.7、10.0.0.27、10.0.0.37、10.0.0.47、10.0.0.57、10.0.0.67
1
1）各redis节点初始化：版本一致、密码相同、无数据。主配置文件配置：cluster-enabled yes和cluster-config-file nodes-6380.conf，各自启动为master 2）redis3和4版本使用redis-trib.rb工具管理集群，须升级ruby版本，gem安装redis模块 3）使用redis-trib.rb命令创建集群 4）验证集群状态
1.2 各redis节点初始化
#编译安装 #准备编译基础环境 # yum -y install wget gcc jemalloc-devel #下载源码并解压 # cd /usr/local/src/ # wget https://download.redis.io/releases/redis-4.0.14.tar.gz # tar xf redis-4.0.14.tar.gz -C /usr/local/ #编译安装Redis # cd /usr/local/redis-4.0.14/ # make install PREFIX=/usr/local/redis_4.0.14 #软链接命令 # ln -s /usr/local/redis_4.0.14/ /usr/local/redis # ln -s /usr/local/redis/bin/* /usr/bin/ #准备相关文件 # mkdir /usr/local/redis/{etc,logs,data,run} # cat > /usr/local/redis/etc/redis.conf <<EOF bind 0.0.0.0 protected-mode yes port 6379 tcp-backlog 511 timeout 0 tcp-keepalive 300 daemonize yes supervised systemd pidfile /usr/local/redis/run/redis_6379.pid loglevel notice logfile "/usr/local/redis/logs/redis_6379.log" databases 16 always-show-logo no save "" stop-writes-on-bgsave-error no rdbcompression yes rdbchecksum yes dbfilename dump_6379.rdb dir /usr/local/redis/data slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 0 repl-ping-slave-period 10 repl-timeout 60 repl-disable-tcp-nodelay no repl-backlog-size 512mb repl-backlog-ttl 3600 slave-priority 100 requirepass root #node节点连接密码 rename-command FLUSHDB "LqYtt96iKVUP" rename-command FLUSHALL "+QAmxR9Q/YIL" maxclients 10000 maxmemory 1073741824 lazyfree-lazy-eviction no lazyfree-lazy-expire no lazyfree-lazy-server-del no slave-lazy-flush no appendonly yes appendfilename appendonly_6379.aof appendfsync always no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes aof-use-rdb-preamble no lua-time-limit 5000 cluster-enabled yes #开启集群模式 cluster-config-file /usr/local/redis/etc/nodes-6379.conf #由node节点自动生成集群配置文件，无需改动 cluster-node-timeout 15000 cluster-slave-validity-factor 10 cluster-migration-barrier 1 cluster-require-full-coverage no slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes masterauth root #slave连接master使用的密码 EOF #准备service文件 # cat > /usr/lib/systemd/system/redis.service <<EOF [Unit] Description=Redis persistent key-value database After=network.target After=network-online.target Wants=network-online.target [Service] LimitNOFILE=64000 ExecStart=/usr/local/redis/bin/redis-server /usr/local/redis/etc/redis.conf --supervised systemd ExecReload=/bin/kill -s HUP $MAINPID ExecStop=/bin/kill -s QUIT $MAINPID Type=notify User=redis Group=redis RuntimeDirectory=redis RuntimeDirectoryMode=0755 [Install] WantedBy=multi-user.target EOF #创建redis用户 # useradd -r -s /bin/false redis #授权对应目录 # chown -R redis.redis /usr/local/redis/ #清理源码及编译目录 for i in `find /usr/local/src -type f` /usr/local/redis-4.0.14; do rm -rf $i done #解决启动过程的三个警告 #1：tcp-backlog，三次握手队列值，须大于128 #2：vm.overcommit_memory，允许redis内存申请，须为1。 #为0，检查内存是否足够APP内存申请，足够则允许；不够返回错误给APP。 #为1，允许分配所有物理内存，不论当前内存状态如何。 #为2，允许分配所有物理内存和SWAP分区总和的内存。 #3：transparent HugePages，不开启大叶内存动态分配，让redis负责内存管理 # echo -e 'net.core.somaxconn = 512\nvm.overcommit_memory = 1' >> /etc/sysctl.conf # sysctl -p # echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.d/rc.local # [ -x /etc/rc.d/rc.local ] || chmod +x /etc/rc.d/rc.local # echo never > /sys/kernel/mm/transparent_hugepage/enabled #启动服务 # systemctl enable --now redis
1.3 升级ruby版本，gem安装redis模块
#redis3和4版本使用redis-trib.rb工具管理集群，须升级ruby版本，gem安装redis模块 #复制redis-trib.rb工具 # cp /usr/local/redis-4.0.14/src/redis-trib.rb /usr/bin/ #升级ruby版本 # yum -y install wget gcc zlib-devel openssl-devel # cd /usr/local/src # wget https://cache.ruby-lang.org/pub/ruby/2.5/ruby-2.5.5.tar.gz # tar xf ruby-2.5.5.tar.gz -C /usr/local # cd /usr/local/ruby-2.5.5/ # ./configure # make -j 2 && make install #gem安装redis模块 # cd /usr/local/src/ # wget https://rubygems.org/downloads/redis-4.1.0.gem # gem install -l /usr/local/src/redis-4.1.0.gem #设置redis-trib.rb连接redis使用的密码 # sed -ri '/:password =>/s#(.*=> ).*#\1"root",#' /usr/local/lib/ruby/gems/2.5.0/gems/redis-4.1.0/lib/redis/client.rb
1.4 使用redis-trib.rb命令创建集群
#redis 4版本创建集群 # redis-trib.rb create --replicas 1 10.0.0.7:6379 10.0.0.27:6379 10.0.0.37:6379 10.0.0.47:6379 10.0.0.57:6379 10.0.0.67:6379 >>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 10.0.0.7:6379 10.0.0.27:6379 10.0.0.37:6379 Adding replica 10.0.0.57:6379 to 10.0.0.7:6379 Adding replica 10.0.0.67:6379 to 10.0.0.27:6379 Adding replica 10.0.0.47:6379 to 10.0.0.37:6379 M: d5b6ecf3d6dab01f0411f4440f7139517da36cdb 10.0.0.7:6379 slots:0-5460 (5461 slots) master M: 395ff128d02a3d64ae95013df7c139aac302456e 10.0.0.27:6379 slots:5461-10922 (5462 slots) master M: 6cf40571e95dc5ba821a1575069f5ad5a35f6dc3 10.0.0.37:6379 slots:10923-16383 (5461 slots) master S: 8f4c09a553d0e7c2cc1cc438aecc9d08f89edb4c 10.0.0.47:6379 replicates 6cf40571e95dc5ba821a1575069f5ad5a35f6dc3 S: c2ffcbaa4a4f9b7461345859f2c36047695745d5 10.0.0.57:6379 replicates d5b6ecf3d6dab01f0411f4440f7139517da36cdb S: 02b0b116d8f4d91d6519ae63eec350ffeed29f16 10.0.0.67:6379 replicates 395ff128d02a3d64ae95013df7c139aac302456e Can I set the above configuration? (type 'yes' to accept): yes #输入yes确认 >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join...... >>> Performing Cluster Check (using node 10.0.0.7:6379) M: d5b6ecf3d6dab01f0411f4440f7139517da36cdb 10.0.0.7:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: 395ff128d02a3d64ae95013df7c139aac302456e 10.0.0.27:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s) S: 02b0b116d8f4d91d6519ae63eec350ffeed29f16 10.0.0.67:6379 slots: (0 slots) slave replicates 395ff128d02a3d64ae95013df7c139aac302456e M: 6cf40571e95dc5ba821a1575069f5ad5a35f6dc3 10.0.0.37:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s) S: c2ffcbaa4a4f9b7461345859f2c36047695745d5 10.0.0.57:6379 slots: (0 slots) slave replicates d5b6ecf3d6dab01f0411f4440f7139517da36cdb S: 8f4c09a553d0e7c2cc1cc438aecc9d08f89edb4c 10.0.0.47:6379 slots: (0 slots) slave replicates 6cf40571e95dc5ba821a1575069f5ad5a35f6dc3 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. #redis 5版本创建集群 # redis-cli -a root --cluster create 10.0.0.7:6379 10.0.0.27:6379 10.0.0.37:6379 10.0.0.47:6379 10.0.0.57:6379 10.0.0.67:6379 --cluster-replicas 1
1
#验证Redis集群状态，主从建立 #验证集群状态 # redis-cli -h 10.0.0.7 -p 6379 -a root Warning: Using a password with '-a' option on the command line interface may not be safe. 10.0.0.7:6379> cluster info cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:1 cluster_stats_messages_ping_sent:208 cluster_stats_messages_pong_sent:192 cluster_stats_messages_sent:400 cluster_stats_messages_ping_received:187 cluster_stats_messages_pong_received:208 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:400 #查看集群node对应关系 # redis-cli -h 10.0.0.7 -p 6379 -a root Warning: Using a password with '-a' option on the command line interface may not be safe. 10.0.0.7:6379> cluster nodes 8f4c09a553d0e7c2cc1cc438aecc9d08f89edb4c 10.0.0.47:6379@16379 slave 6cf40571e95dc5ba821a1575069f5ad5a35f6dc3 0 1678958593342 4 connected c2ffcbaa4a4f9b7461345859f2c36047695745d5 10.0.0.57:6379@16379 slave d5b6ecf3d6dab01f0411f4440f7139517da36cdb 0 1678958595000 5 connected 395ff128d02a3d64ae95013df7c139aac302456e 10.0.0.27:6379@16379 master - 0 1678958596400 2 connected 5461-10922 02b0b116d8f4d91d6519ae63eec350ffeed29f16 10.0.0.67:6379@16379 slave 395ff128d02a3d64ae95013df7c139aac302456e 0 1678958595380 6 connected d5b6ecf3d6dab01f0411f4440f7139517da36cdb 10.0.0.7:6379@16379 myself,master - 0 1678958592000 1 connected 0-5460 6cf40571e95dc5ba821a1575069f5ad5a35f6dc3 10.0.0.37:6379@16379 master - 0 1678958594367 3 connected 10923-16383 #验证集群写入key #经过算法计算，当前key需要写入指定node的槽位，槽位不在当前node所以无法写入 # redis-cli -h 10.0.0.7 -p 6379 -a root Warning: Using a password with '-a' option on the command line interface may not be safe. 10.0.0.7:6379> set k1 v1 (error) MOVED 12706 10.0.0.37:6379 10.0.0.7:6379> exit #以集群模式连接redis，写入成功 # redis-cli -h 10.0.0.7 -p 6379 -a root -c Warning: Using a password with '-a' option on the command line interface may not be safe. 10.0.0.7:6379> set k1 v1 -> Redirected to slot [12706] located at 10.0.0.37:6379 OK 10.0.0.37:6379> get k1 "v1" #redis-trib.rb命令验证集群状态与监控 #redis 4版本 # redis-trib.rb info 10.0.0.7:6379 10.0.0.7:6379 (d5b6ecf3...) -> 0 keys | 5461 slots | 1 slaves. 10.0.0.27:6379 (395ff128...) -> 0 keys | 5462 slots | 1 slaves. 10.0.0.37:6379 (6cf40571...) -> 1 keys | 5461 slots | 1 slaves. [OK] 1 keys in 3 masters. 0.00 keys per slot on average. # redis-trib.rb check 10.0.0.7:6379 >>> Performing Cluster Check (using node 10.0.0.7:6379) M: d5b6ecf3d6dab01f0411f4440f7139517da36cdb 10.0.0.7:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) S: 8f4c09a553d0e7c2cc1cc438aecc9d08f89edb4c 10.0.0.47:6379 slots: (0 slots) slave replicates 6cf40571e95dc5ba821a1575069f5ad5a35f6dc3 S: c2ffcbaa4a4f9b7461345859f2c36047695745d5 10.0.0.57:6379 slots: (0 slots) slave replicates d5b6ecf3d6dab01f0411f4440f7139517da36cdb M: 395ff128d02a3d64ae95013df7c139aac302456e 10.0.0.27:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s) S: 02b0b116d8f4d91d6519ae63eec350ffeed29f16 10.0.0.67:6379 slots: (0 slots) slave replicates 395ff128d02a3d64ae95013df7c139aac302456e M: 6cf40571e95dc5ba821a1575069f5ad5a35f6dc3 10.0.0.37:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. #redis 5版本验证集群状态与监控 # redis-cli -a root --cluster check 10.0.0.7:6379
2、Redis cluster slot重分配
2.1 热添加一主一从
#添加新master # redis-trib.rb add-node 10.0.0.17:6379 10.0.0.7:6379 #为新master分配槽位 # redis-trib.rb reshard 10.0.0.7:6379 #输入分配槽位数，被分配node_id，提供槽位node_id(all表示在有槽位的所有master上均匀抽取)，yes确认 #添加新slave，为新master分配新slave # redis-trib.rb add-node 10.0.0.77:6379 10.0.0.7:6379 #进入新slave，指向新master_id cluster replicate d1bab0a048f312285be04eb31707cb9ab2bc9647
3、Redis及Memcached对比
| redis | memcached | |
| 持久化 | 支持（RDB、AOF） | 不支持 |
| 数据类型 | 字符串、列表、集合、有序集合、哈希 | 字符串 |
| 线程 | 单线程 | 多线程 |
| 集群节点数据 | 不同 | 相同 |
