Redis Cluster分布式集群详解

Redis Cluster分布式集群介绍

Redis Cluster 是 redis 的分布式解决方案，支持分片，Redis3.0 版本之前，可以通过 Redis Sentinel（哨兵）来实现高可用 (HA)，从 3.0 版本之后，官方推出了 Redis Cluster，它的主要用途是实现数据分片 (Data Sharding)，不过同样可以实现 HA，是官方当前推荐的方案，有效地解决了 Redis 分布式的需求，当一个服务挂了可以快速的切换到另外一个服务。

分布式数据库把整个数据按分区规则映射到多个节点，即把数据划分到多个节点上，每个节点负责整体数据的一个子集。

比如我们库有 3000 条用户数据，有 3 个 redis 节点，将 3000 条分成 3 份，分别存入到 3 个 redis 节点
20250828211156

分区规则

哈希分布

数据分散度高，键值分布与业务无关，无法顺序访问，支持批量操作

哈希分布产品：一致性哈希memcache，redis cluster等

原理：节点取余，假设三台机器，hash（key）%3 ，落到不同节点

redis 集群使用了哈希分布的 “虚拟槽分区” 方式（槽：slot），所有的键根据哈希函数 (CRC16 [key]&16383)映射到 0－16383 槽内，共 16384 个槽位，每个节点维护部分槽及槽所映射的键值数据。
20250828211238

Redis 用虚拟槽分区原因：

  1. 解耦数据与节点关系
  2. 节点自身维护槽映射关系
  3. 分布式存储

顺序分布

数据分散度易倾斜，键值分布与业务相关，可顺序访问，支持批量操作

顺序分布产品：BigTable，Hbase等

原理：100个数据分到三个节点上，1-33的数据落到第一个节点；34-66的数据落到第二个节点；67-100的数据落到第三个节点。

Redis Cluster 的缺陷

  1. 键的批量操作支持有限，比如 mset, mget，如果多个键映射在不同的槽，就不支持了
  2. 不支持多数据库，只有 0，select 0
  3. 复制结构只支持单层结构，不支持树型结构。
  4. 键事务支持有限，当多个 key 分布在不同节点时无法使用事务，同一个节点是支持事务
  5. 键是数据分区的最小粒度，不能将一个很大的键值对映射到不同的节点

部署Redis Cluster

架构

主机	       角色
10.0.0.11	master01
10.0.0.12	slave01
10.0.0.13	master02
10.0.0.14	slave02
10.0.0.15	master03
10.0.0.16	slave03

6台机器安装redis

可以参考《linux部署redis7.0.7版本》章节：https://www.cnblogs.com/zqfstack/p/18999667

配置redis cluster集群

配置redis cluster有两种方式，以下我们两种都介绍

  1. 自动分配角色方式：主节点和从节点由集群自行控制，槽位平均分
  2. 手动分配角色方式：可以手动指定主节点和从节点，手动自定义分配槽位

自动配置方式

  1. 6个节点的配置文件一样：
cat  /data/redis/conf/redis.conf 
# 基础配置
bind 0.0.0.0
port 6379
loglevel notice
logfile "/data/redis/log/redis.log"
pidfile /data/redis/redis.pid
daemonize yes
tcp-backlog 600 
timeout 0
tcp-keepalive 0

# 持久化配置
dir /data/redis/data
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite yes
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 1024mb
aof-load-truncated yes
aof-rewrite-incremental-fsync yes
save 900 1
save 300 10
save 60 10000
dbfilename dump.rdb
rdbcompression yes
stop-writes-on-bgsave-error yes
rdbchecksum yes 

# 安全配置
requirepass 1qaz@WSX

# 慢查询配置
slowlog-log-slower-than 1000000 
slowlog-max-len 50 

#参数优化
maxclients 10000
maxmemory 2G
maxmemory-policy allkeys-lru
maxmemory-samples 5 
lua-time-limit 5000 

# 集群配置
databases 16
masterauth 1qaz@WSX
cluster-enabled yes
cluster-config-file /data/redis/conf/cluster-config.conf
cluster-node-timeout 15000
cluster-slave-validity-factor 10
cluster-require-full-coverage no
  1. 启动6个节点的redis：
systemctl restart redis
systemctl enable  redis
systemctl status redis
  1. 创建redis分布式集群：3主3从（1:1）
redis-cli -a 1qaz@WSX --cluster create  --cluster-replicas 1 10.0.0.11:6379 10.0.0.12:6379 10.0.0.13:6379 10.0.0.14:6379 10.0.0.15:6379 10.0.0.16:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 10.0.0.15:6379 to 10.0.0.11:6379
Adding replica 10.0.0.16:6379 to 10.0.0.12:6379
Adding replica 10.0.0.14:6379 to 10.0.0.13:6379
M: 9a614911c24eb31a3fd374c227cfdfa11d3b8d3a 10.0.0.11:6379
   slots:[0-5460] (5461 slots) master
M: a8eb7445b86f72b544fa10bd94b38aaedccab79d 10.0.0.12:6379
   slots:[5461-10922] (5462 slots) master
M: e3d647ba6cdfc5a824be906852b02e3037b645b0 10.0.0.13:6379
   slots:[10923-16383] (5461 slots) master
S: 49429abf2fd5db9c92e4012d4c7e335241d44a13 10.0.0.14:6379
   replicates e3d647ba6cdfc5a824be906852b02e3037b645b0
S: a7b010a5732d3b1a4171a837a76fb0e3efdf8e53 10.0.0.15:6379
   replicates 9a614911c24eb31a3fd374c227cfdfa11d3b8d3a
S: f2b8bdd8a1be10ec5c7ac79c736b92bf0d2fd0d8 10.0.0.16:6379
   replicates a8eb7445b86f72b544fa10bd94b38aaedccab79d
Can I set the above configuration? (type 'yes' to accept): yes     # 确认上述自动分配的主从关系
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 10.0.0.11:6379)
M: 9a614911c24eb31a3fd374c227cfdfa11d3b8d3a 10.0.0.11:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 49429abf2fd5db9c92e4012d4c7e335241d44a13 10.0.0.14:6379
   slots: (0 slots) slave
   replicates e3d647ba6cdfc5a824be906852b02e3037b645b0
M: e3d647ba6cdfc5a824be906852b02e3037b645b0 10.0.0.13:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: a7b010a5732d3b1a4171a837a76fb0e3efdf8e53 10.0.0.15:6379
   slots: (0 slots) slave
   replicates 9a614911c24eb31a3fd374c227cfdfa11d3b8d3a
S: f2b8bdd8a1be10ec5c7ac79c736b92bf0d2fd0d8 10.0.0.16:6379
   slots: (0 slots) slave
   replicates a8eb7445b86f72b544fa10bd94b38aaedccab79d
M: a8eb7445b86f72b544fa10bd94b38aaedccab79d 10.0.0.12:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


# 可以看到有3个主3个从，且是一一对应的关系。

# 扩展：
# 如果是3主6从（1:2）：
redis-cli -a 1qaz@WSX --cluster create --cluster-replicas 2  +redis集群实例地址
  1. 检查集群和查看状态
redis-cli -c -a 1qaz@WSX   #指定-c以集群方式登录
# 查看集群状态：  
127.0.0.1:6379> cluster nodes
49429abf2fd5db9c92e4012d4c7e335241d44a13 10.0.0.14:6379@16379 slave e3d647ba6cdfc5a824be906852b02e3037b645b0 0 1751517390309 3 connected
e3d647ba6cdfc5a824be906852b02e3037b645b0 10.0.0.13:6379@16379 master - 0 1751517389298 3 connected 10923-16383
a7b010a5732d3b1a4171a837a76fb0e3efdf8e53 10.0.0.15:6379@16379 slave 9a614911c24eb31a3fd374c227cfdfa11d3b8d3a 0 1751517388290 1 connected
f2b8bdd8a1be10ec5c7ac79c736b92bf0d2fd0d8 10.0.0.16:6379@16379 slave a8eb7445b86f72b544fa10bd94b38aaedccab79d 0 1751517388000 2 connected
a8eb7445b86f72b544fa10bd94b38aaedccab79d 10.0.0.12:6379@16379 master - 0 1751517386270 2 connected 5461-10922
9a614911c24eb31a3fd374c227cfdfa11d3b8d3a 10.0.0.11:6379@16379 myself,master - 0 1751517386000 1 connected 0-5460

127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:119
cluster_stats_messages_pong_sent:129
cluster_stats_messages_sent:248
cluster_stats_messages_ping_received:124
cluster_stats_messages_pong_received:119
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:248
total_cluster_links_buffer_limit_exceeded:0

# 检察集群
[root@redis-master01 src]# redis-cli --cluster check 10.0.0.11:6379  -a 1qaz@WSX
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.0.11:6379 (9a614911...) -> 0 keys | 5461 slots | 1 slaves.
10.0.0.13:6379 (e3d647ba...) -> 0 keys | 5461 slots | 1 slaves.
10.0.0.12:6379 (a8eb7445...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.11:6379)
M: 9a614911c24eb31a3fd374c227cfdfa11d3b8d3a 10.0.0.11:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 49429abf2fd5db9c92e4012d4c7e335241d44a13 10.0.0.14:6379
   slots: (0 slots) slave
   replicates e3d647ba6cdfc5a824be906852b02e3037b645b0
M: e3d647ba6cdfc5a824be906852b02e3037b645b0 10.0.0.13:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: a7b010a5732d3b1a4171a837a76fb0e3efdf8e53 10.0.0.15:6379
   slots: (0 slots) slave
   replicates 9a614911c24eb31a3fd374c227cfdfa11d3b8d3a
S: f2b8bdd8a1be10ec5c7ac79c736b92bf0d2fd0d8 10.0.0.16:6379
   slots: (0 slots) slave
   replicates a8eb7445b86f72b544fa10bd94b38aaedccab79d
M: a8eb7445b86f72b544fa10bd94b38aaedccab79d 10.0.0.12:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

手动配置方式

  1. 6个节点的配置文件一样：
cat  /data/redis/conf/redis.conf 
# 基础配置
bind 0.0.0.0
port 6379
loglevel notice
logfile "/data/redis/log/redis.log"
pidfile /data/redis/redis.pid
daemonize yes
tcp-backlog 600 
timeout 0
tcp-keepalive 0

# 持久化配置
dir /data/redis/data
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite yes
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 1024mb
aof-load-truncated yes
aof-rewrite-incremental-fsync yes
save 900 1
save 300 10
save 60 10000
dbfilename dump.rdb
rdbcompression yes
stop-writes-on-bgsave-error yes
rdbchecksum yes 

# 安全配置
requirepass 1qaz@WSX

# 慢查询配置
slowlog-log-slower-than 1000000 
slowlog-max-len 50 

#参数优化
maxclients 10000
maxmemory 2G
maxmemory-policy allkeys-lru
maxmemory-samples 5 
lua-time-limit 5000 

# 集群配置
databases 16
masterauth 1qaz@WSX
cluster-enabled yes
cluster-config-file /data/redis/conf/cluster-config.conf
cluster-node-timeout 15000
cluster-slave-validity-factor 10
cluster-require-full-coverage no
  1. 启动6个节点的redis：
systemctl restart redis
systemctl enable  redis
systemctl status redis
  1. 配置节点握手，组成网络
redis-cli -a 1qaz@WSX  # 我们选择master01机器进入redis

127.0.0.1:6379> cluster meet 10.0.0.12 6379
OK
127.0.0.1:6379> cluster meet 10.0.0.13 6379
OK
127.0.0.1:6379> cluster meet 10.0.0.14 6379
OK
127.0.0.1:6379> cluster meet 10.0.0.15 6379
OK
127.0.0.1:6379> cluster meet 10.0.0.16 6379
OK
  1. 查看节点信息
127.0.0.1:6379> cluster nodes
54394b240d3b8374a3145e92fd4934bfe67688ad 10.0.0.15:6379@16379 master - 0 1751541145311 4 connected 10923-16383
52cdf1e8df84d9a57820c9e203daf80e241b1113 10.0.0.16:6379@16379 master - 0 1751541144301 5 connected
517e5630bfb0a3273b3cf0dd810f11997f2493dc 10.0.0.11:6379@16379 myself,master - 0 1751541142000 2 connected 0-5461
8af4a3fb041f8225780748bd0c559b5604fecb6a 10.0.0.14:6379@16379 master - 0 1751541143000 0 connected
8ac8123b45f85b097dde14c1947335be7b3229f1 10.0.0.13:6379@16379 master - 0 1751541144000 3 connected 5462-10922
24acc1a09525df3c209e0ea75ab5aa24ac3bf047 10.0.0.12:6379@16379 master - 0 1751541142000 1 connected

可以看到master01节点已经感知到了所有其他节点，但是注意此时所有节点均为master节点。

  1. 分配slot槽

集群有16384个槽，槽是数据管理和迁移的基本单位。当数据库中的16384个槽都分配到节点时，集群处于上线状态（ok）；如果有任意一个槽没有分配节点，则集群处于下线状态（fail）

# 分配卡槽到我们所指定的3个master节点，master01：10.0.0.11，master02：10.0.0.13，master03：10.0.0.15
redis-cli -h 10.0.0.11 -p 6379 -a 1qaz@WSX cluster addslots {0..5461}
redis-cli -h 10.0.0.13 -p 6379 -a 1qaz@WSX cluster addslots {5462..10922} 
redis-cli -h 10.0.0.15 -p 6379 -a 1qaz@WSX cluster addslots {10923..16383}
  1. 查看分配槽位情况
redis-cli --cluster check 10.0.0.11:6379  -a 1qaz@WSX
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.0.11:6379 (517e5630...) -> 0 keys | 5462 slots | 0 slaves.
10.0.0.15:6379 (54394b24...) -> 0 keys | 5461 slots | 0 slaves.
10.0.0.16:6379 (52cdf1e8...) -> 0 keys | 0 slots | 0 slaves.
10.0.0.14:6379 (8af4a3fb...) -> 0 keys | 0 slots | 0 slaves.
10.0.0.13:6379 (8ac8123b...) -> 0 keys | 5461 slots | 0 slaves.
10.0.0.12:6379 (24acc1a0...) -> 0 keys | 0 slots | 0 slaves.
[OK] 0 keys in 6 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.11:6379)
M: 517e5630bfb0a3273b3cf0dd810f11997f2493dc 10.0.0.11:6379
   slots:[0-5461] (5462 slots) master
M: 54394b240d3b8374a3145e92fd4934bfe67688ad 10.0.0.15:6379
   slots:[10923-16383] (5461 slots) master
M: 52cdf1e8df84d9a57820c9e203daf80e241b1113 10.0.0.16:6379
   slots: (0 slots) master
M: 8af4a3fb041f8225780748bd0c559b5604fecb6a 10.0.0.14:6379
   slots: (0 slots) master
M: 8ac8123b45f85b097dde14c1947335be7b3229f1 10.0.0.13:6379
   slots:[5462-10922] (5461 slots) master
M: 24acc1a09525df3c209e0ea75ab5aa24ac3bf047 10.0.0.12:6379
   slots: (0 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:5
cluster_my_epoch:2
cluster_stats_messages_ping_sent:22675
cluster_stats_messages_pong_sent:24131
cluster_stats_messages_meet_sent:10
cluster_stats_messages_sent:46816
cluster_stats_messages_ping_received:24131
cluster_stats_messages_pong_received:22685
cluster_stats_messages_received:46816
total_cluster_links_buffer_limit_exceeded:0

可以看到已经16384个槽位已经均匀的分配到了我们指定的3个master节点。并且集群已经进入到了上线状态。

  1. 指定主从关系

查看各个节点的节点ID：

127.0.0.1:6379> cluster nodes
54394b240d3b8374a3145e92fd4934bfe67688ad 10.0.0.15:6379@16379 master - 0 1751541234049 4 connected 10923-16383
52cdf1e8df84d9a57820c9e203daf80e241b1113 10.0.0.16:6379@16379 master - 0 1751541235058 5 connected
517e5630bfb0a3273b3cf0dd810f11997f2493dc 10.0.0.11:6379@16379 myself,master - 0 1751541235000 2 connected 0-5461
8af4a3fb041f8225780748bd0c559b5604fecb6a 10.0.0.14:6379@16379 master - 0 1751541236063 0 connected
8ac8123b45f85b097dde14c1947335be7b3229f1 10.0.0.13:6379@16379 master - 0 1751541234000 3 connected 5462-10922
24acc1a09525df3c209e0ea75ab5aa24ac3bf047 10.0.0.12:6379@16379 master - 0 1751541237069 1 connected

# 登录到从节点10.0.0.12：指定主库为10.0.0.11
redis-cli -c  -h 10.0.0.12  -a 1qaz@WSX
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> CLUSTER REPLICATE 517e5630bfb0a3273b3cf0dd810f11997f2493dc
OK

# 登录到从节点10.0.0.14：指定主库为10.0.0.13
redis-cli -c  -h 10.0.0.14  -a 1qaz@WSX
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.0.14:6379> CLUSTER REPLICATE 8ac8123b45f85b097dde14c1947335be7b3229f1
OK

# 登录到从节点10.0.0.16：指定主库为10.0.0.15
redis-cli -c  -h 10.0.0.16  -a 1qaz@WSX
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.0.16:6379> CLUSTER REPLICATE 54394b240d3b8374a3145e92fd4934bfe67688ad
OK
  1. 查看主从关系是否正常
127.0.0.1:6379> cluster nodes
54394b240d3b8374a3145e92fd4934bfe67688ad 10.0.0.15:6379@16379 master - 0 1751541594296 4 connected 10923-16383
52cdf1e8df84d9a57820c9e203daf80e241b1113 10.0.0.16:6379@16379 slave 54394b240d3b8374a3145e92fd4934bfe67688ad 0 1751541591000 4 connected
517e5630bfb0a3273b3cf0dd810f11997f2493dc 10.0.0.11:6379@16379 myself,master - 0 1751541593000 2 connected 0-5461
8af4a3fb041f8225780748bd0c559b5604fecb6a 10.0.0.14:6379@16379 slave 8ac8123b45f85b097dde14c1947335be7b3229f1 0 1751541592000 3 connected
8ac8123b45f85b097dde14c1947335be7b3229f1 10.0.0.13:6379@16379 master - 0 1751541592000 3 connected 5462-10922
24acc1a09525df3c209e0ea75ab5aa24ac3bf047 10.0.0.12:6379@16379 slave 517e5630bfb0a3273b3cf0dd810f11997f2493dc 0 1751541593287 2 connected

可以看到主从建立成功。

  1. 查看插槽信息
127.0.0.1:6379> cluster slots
1) 1) (integer) 0
   2) (integer) 5461
   3) 1) "10.0.0.11"
      2) (integer) 6379
      3) "517e5630bfb0a3273b3cf0dd810f11997f2493dc"
      4) (empty array)
   4) 1) "10.0.0.12"
      2) (integer) 6379
      3) "24acc1a09525df3c209e0ea75ab5aa24ac3bf047"
      4) (empty array)
2) 1) (integer) 5462
   2) (integer) 10922
   3) 1) "10.0.0.13"
      2) (integer) 6379
      3) "8ac8123b45f85b097dde14c1947335be7b3229f1"
      4) (empty array)
   4) 1) "10.0.0.14"
      2) (integer) 6379
      3) "8af4a3fb041f8225780748bd0c559b5604fecb6a"
      4) (empty array)
3) 1) (integer) 10923
   2) (integer) 16383
   3) 1) "10.0.0.15"
      2) (integer) 6379
      3) "54394b240d3b8374a3145e92fd4934bfe67688ad"
      4) (empty array)
   4) 1) "10.0.0.16"
      2) (integer) 6379
      3) "52cdf1e8df84d9a57820c9e203daf80e241b1113"
      4) (empty array)

一切正常，搭建完毕。

redis cluster集群常用检查命令

查看集群状态

redis-cli -c -a 1qaz@WSX --cluster check 10.0.0.91:6379

查看集群节点信息

redis-cli -c -a 1qaz@WSX -h 10.0.0.91 -p 6379 cluster nodes

查看集群信息

redis-cli -c -a 1qaz@WSX -h 10.0.0.91 -p 6379 cluster info

java应用连接cluster集群配置

  # redis 配置
  redis:
    cluster:
      nodes: # 指定集群中至少一个或多个节点地址（主或从都可以）
        - 10.0.0.91:6379
        - 10.0.0.92:6379
        - 10.0.0.93:6379
        - 10.0.0.94:6379
        - 10.0.0.95:6379
        - 10.0.0.96:6379
      max-redirects: 3 # 当遇到重定向错误时，最大重试次数（通常保持默认即可）
    database: 0
    password: 1qaz@WSX # 集群所有节点的密码必须一致
    timeout: 10s
    lettuce:
      pool:
        min-idle: 0
        max-idle: 8
        max-active: 8
        max-wait: -1ms
        # Cluster 模式下的读取策略配置 (非常重要!)
        read-from: REPLICA_PREFERRED
posted @ 2025-08-28 21:31  阿峰博客站  阅读(114)  评论(0)    收藏  举报