第九次作业-20230910
一、详细总结Redis 主从同步过程
Redis 主从复制分为全量同步和增量同步
1.1 全量同步
Redis 服务器首次同步数据是进行全量同步，当 Master 收到从服务器的 psync（2.8版本之前是SYNC）命令，会 fork 出一个子进程在后台执行 bgsave 命令，此时新写入的数据会被写入到缓冲区中，bgsave 执行完成之后，将生成的 RDB 文件发送给 Slave，Slave 收到后会先删除旧的数据，再将 RDB 文件载入到自己的内存；之后当 Slave 收到 Master 数据缓冲区的内容时，再将缓冲区中的内容加载至内存，从而完成一次完整的数据同步。
1.2 增量同步
全量同步完成之后，后续 Master 中数据的变化，从服务器会同 Master 进行增量同步。从服务器只需要发送当前的 offset 位置（相当于 MySQL 的 binlog 位置）给主服务器，然后将主服务器根据相应的位置将之后的数据（包括在缓冲区中积压的数据）发送给从服务器，从服务器在写入至内存即可。
二、基于配置文件实现Redis的主从模式
实验环境
# Redis 版本相同
root@ubuntu18-server11:~# redis-cli -v
redis-cli 7.2.0
2.1 编辑从节点配置文件
# 查看从节点数据为空
root@ubuntu18-server:~# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> KEYS *
(empty array)
# 编辑Master的IP和端口以及密码
root@ubuntu18-server:~# vim /apps/redis/etc/redis.conf
...
replicaof 192.168.119.171 6379
# If the master is password protected (using the "requirepass" configuration
# directive below) it is possible to tell the replica to authenticate before
# starting the replication synchronization process, otherwise the master will
# refuse the replica request.
#
# masterauth <master-password>
masterauth 123456
...
root@ubuntu18-server:~# systemctl restart redis-server.service
2.2 查看从节点状态
root@ubuntu18-server:~# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.119.171
master_port:6379
master_link_status:up
master_last_io_seconds_ago:7
master_sync_in_progress:0
slave_read_repl_offset:42
slave_repl_offset:42
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:57d8d33fe921658a8763b1afddb763be41493547
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:42
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:536870912
repl_backlog_first_byte_offset:1
repl_backlog_histlen:42
# 查看数据同步
127.0.0.1:6379> KEYS *
1) "key87"
2) "key88"
...
98) "key12"
99) "key70"
100) "key32"
三、基于Redis sentinel实现Redis 服务的高可用
环境准备
3.1 准备好主从关系
1、master 节点
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=192.168.119.161,port=6379,state=online,offset=23366,lag=0
slave1:ip=192.168.119.162,port=6379,state=online,offset=23366,lag=0
master_failover_state:no-failover
master_replid:37ef554e18a0776bd1f52d5694c6a53fad96f753
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:23366
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:536870912
repl_backlog_first_byte_offset:1
repl_backlog_histlen:23366
2、slave 节点（两个从节点都以192.168.119.171为主节点）
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.119.171
master_port:6379
master_link_status:up
master_last_io_seconds_ago:4
master_sync_in_progress:0
slave_read_repl_offset:23450
slave_repl_offset:23450
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:37ef554e18a0776bd1f52d5694c6a53fad96f753
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:23450
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:536870912
repl_backlog_first_byte_offset:1
repl_backlog_histlen:23450
3.2 编辑哨兵配置文件
1、在主节点编辑好配置文件
root@ubuntu18-server11:~# cp /usr/local/src/redis-7.2.0/sentinel.conf /apps/redis/etc/
root@ubuntu18-server11:~# cd /apps/redis/etc/
root@ubuntu18-server11:/apps/redis/etc# vim sentinel.conf
root@ubuntu18-server11:/apps/redis/etc# grep -Ev "^#|^$" sentinel.conf
protected-mode no
port 26379
daemonize yes
pidfile /apps/redis/run/redis-sentinel.pid
loglevel notice
logfile "/apps/redis/logs/redis-sentinel.log"
dir /apps/redis/data
sentinel monitor mymaster 192.168.119.171 6379 2
sentinel auth-pass mymaster 123456
sentinel down-after-milliseconds mymaster 10000
acllog-max-len 128
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
SENTINEL resolve-hostnames no
SENTINEL announce-hostnames no
SENTINEL master-reboot-down-after-period mymaster 0
2、将配置文件分发到各从节点
root@ubuntu18-server11:/apps/redis/etc# scp sentinel.conf root@192.168.119.161:/apps/redis/etc/
root@ubuntu18-server11:/apps/redis/etc# scp sentinel.conf root@192.168.119.162:/apps/redis/etc/
3、配置启动sentinel服务
root@ubuntu18-server11:/apps/redis/etc# redis-sentinel /apps/redis/etc/sentinel.conf
root@ubuntu18-server:/apps/redis/etc# redis-sentinel /apps/redis/etc/sentinel.conf
root@ubuntu18-server2:/apps/redis/etc# redis-sentinel /apps/redis/etc/sentinel.conf
3.3 查看日志文件
1、master 节点查看
root@ubuntu18-server11:/apps/redis/etc# tail -f /apps/redis/logs/redis-sentinel.log
2648:X 08 Mar 2024 14:40:59.570 * Sentinel ID is 821c3649dc7695ce1049e7250d3620365bc4a081
2648:X 08 Mar 2024 14:40:59.570 # +monitor master mymaster 192.168.119.171 6379 quorum 2
2648:X 08 Mar 2024 14:40:59.580 * +slave slave 192.168.119.161:6379 192.168.119.161 6379 @ mymaster 192.168.119.171 6379
2648:X 08 Mar 2024 14:40:59.582 * Sentinel new configuration saved on disk
2648:X 08 Mar 2024 14:40:59.582 * +slave slave 192.168.119.162:6379 192.168.119.162 6379 @ mymaster 192.168.119.171 6379
2648:X 08 Mar 2024 14:40:59.593 * Sentinel new configuration saved on disk
2648:X 08 Mar 2024 14:41:01.590 * +sentinel sentinel 8daef2c63f976764a3e16815fccc1abe8595e84b 192.168.119.162 26379 @ mymaster 192.168.119.171 6379
2648:X 08 Mar 2024 14:41:01.592 * Sentinel new configuration saved on disk
2648:X 08 Mar 2024 14:41:01.603 * +sentinel sentinel 9213f768242779d8ec6ffb97ab233103d5bdf09c 192.168.119.161 26379 @ mymaster 192.168.119.171 6379
2648:X 08 Mar 2024 14:41:01.604 * Sentinel new configuration saved on disk
2、slave1 节点查看
root@ubuntu18-server:~# tail -f /apps/redis/logs/redis-sentinel.log
4812:X 08 Mar 2024 06:40:59.553 * Sentinel ID is 9213f768242779d8ec6ffb97ab233103d5bdf09c
4812:X 08 Mar 2024 06:40:59.553 # +monitor master mymaster 192.168.119.171 6379 quorum 2
4812:X 08 Mar 2024 06:40:59.563 * +slave slave 192.168.119.161:6379 192.168.119.161 6379 @ mymaster 192.168.119.171 6379
4812:X 08 Mar 2024 06:40:59.564 * Sentinel new configuration saved on disk
4812:X 08 Mar 2024 06:40:59.564 * +slave slave 192.168.119.162:6379 192.168.119.162 6379 @ mymaster 192.168.119.171 6379
4812:X 08 Mar 2024 06:40:59.576 * Sentinel new configuration saved on disk
4812:X 08 Mar 2024 06:41:01.573 * +sentinel sentinel 821c3649dc7695ce1049e7250d3620365bc4a081 192.168.119.171 26379 @ mymaster 192.168.119.171 6379
4812:X 08 Mar 2024 06:41:01.574 * Sentinel new configuration saved on disk
4812:X 08 Mar 2024 06:41:01.574 * +sentinel sentinel 8daef2c63f976764a3e16815fccc1abe8595e84b 192.168.119.162 26379 @ mymaster 192.168.119.171 6379
4812:X 08 Mar 2024 06:41:01.585 * Sentinel new configuration saved on disk
3、slave2 节点查看
root@ubuntu18-server2:/apps/redis/etc# tail -f /apps/redis/logs/redis-sentinel.log
14602:X 08 Mar 2024 06:40:59.538 * Sentinel ID is 8daef2c63f976764a3e16815fccc1abe8595e84b
14602:X 08 Mar 2024 06:40:59.538 # +monitor master mymaster 192.168.119.171 6379 quorum 2
14602:X 08 Mar 2024 06:40:59.539 * +slave slave 192.168.119.161:6379 192.168.119.161 6379 @ mymaster 192.168.119.171 6379
14602:X 08 Mar 2024 06:40:59.549 * Sentinel new configuration saved on disk
14602:X 08 Mar 2024 06:40:59.549 * +slave slave 192.168.119.162:6379 192.168.119.162 6379 @ mymaster 192.168.119.171 6379
14602:X 08 Mar 2024 06:40:59.559 * Sentinel new configuration saved on disk
14602:X 08 Mar 2024 06:41:01.566 * +sentinel sentinel 821c3649dc7695ce1049e7250d3620365bc4a081 192.168.119.171 26379 @ mymaster 192.168.119.171 6379
14602:X 08 Mar 2024 06:41:01.567 * Sentinel new configuration saved on disk
14602:X 08 Mar 2024 06:41:01.578 * +sentinel sentinel 9213f768242779d8ec6ffb97ab233103d5bdf09c 192.168.119.161 26379 @ mymaster 192.168.119.171 6379
14602:X 08 Mar 2024 06:41:01.580 * Sentinel new configuration saved on disk
3.4 查看 sentinel 集群状态
1、查看master的IP，从节点的数量以及集群的数量信息
root@ubuntu18-server11:~# redis-cli -h 192.168.119.171 -p 26379
192.168.119.171:26379> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_tilt_since_seconds:-1
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.119.171:6379,slaves=2,sentinels=3
2、配置文件中将会自动生成sentinel集群的相关信息（自己的id信息，其它节点的信息）
（1）master 节点
（2）其中一个slave节点
3.5 模拟故障转移
1、停止 master 的 redis 服务
root@ubuntu18-server11:~# systemctl stop redis-server.service
2、查看 sentinel 日志
3、在新的节点上检查主从信息
# 161节点的角色已经变为 master，只有一个从节点
root@ubuntu18-server:~# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.119.162,port=6379,state=online,offset=737954,lag=0
master_failover_state:no-failover
master_replid:1c0a4d080609e759baa61f8d2c4ddb84532412a3
master_replid2:37ef554e18a0776bd1f52d5694c6a53fad96f753
master_repl_offset:737954
second_repl_offset:676930
repl_backlog_active:1
repl_backlog_size:536870912
repl_backlog_first_byte_offset:1
repl_backlog_histlen:737954
# 162节点的master更改为161节点
root@ubuntu18-server2:~# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.119.161
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_read_repl_offset:761743
slave_repl_offset:761743
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:1c0a4d080609e759baa61f8d2c4ddb84532412a3
master_replid2:37ef554e18a0776bd1f52d5694c6a53fad96f753
master_repl_offset:761743
second_repl_offset:676930
repl_backlog_active:1
repl_backlog_size:536870912
repl_backlog_first_byte_offset:5027
repl_backlog_histlen:756717
3.6 故障恢复
1、修改故障节点的Redis配置文件，配置新的master节点信息
root@ubuntu18-server11:/apps/redis/etc# systemctl restart redis-server.service
2、查看主从信息
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.119.161
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_read_repl_offset:1338829
slave_repl_offset:1338829
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:1c0a4d080609e759baa61f8d2c4ddb84532412a3
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1338829
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:536870912
repl_backlog_first_byte_offset:788302
repl_backlog_histlen:550528
3、主节点写入数据，查看从节点的同步
# 主节点写入数据
root@ubuntu18-server:~# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> set name Jerry
OK
# 查看从节点是否成功同步数据
root@ubuntu18-server11:/apps/redis/etc# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> GET name
"Jerry"
四、总结Redis sentinel的实现机制
Redis Sentinel（哨兵）是一种官方提供的针对Redis高可用性的解决方案，它的核心功能是监控Redis主从集群的健康状况，并在检测到主节点故障时自动完成故障转移操作，确保系统的连续可用性。以下是Redis Sentinel实现机制的关键要点：
- 监控: Sentinel是一个独立运行的进程，它可以监视一个或多个Redis主从集群。Sentinel通过向Redis节点发送PING命令来检测它们是否可达和在线。
- 集群化: 为了提高Sentinel本身的可用性，通常会部署成一个Sentinel集群，每个Sentinel节点都能监控集群状态，避免单点故障。Sentinel节点之间通过流言协议（gossip protocol）来传播主从节点的状态信息。
- 主观下线与客观下线: 当一个Sentinel认为主节点不可达时（例如超过一定次数的PING请求未响应），会标记主节点为“主观下线”。若足够数量的Sentinel（根据配置的quorum参数确定）也报告主节点不可达，则主节点被标记为“客观下线”。
- 自动故障转移: 当主节点被标记为客观下线后，Sentinel集群内部通过投票协议来选举出一个Sentinel作为领导者，由领导者负责执行故障转移操作。领导者 Sentinel会选择一个合适的从节点晋升为主节点，并通知其他从节点改为复制新的主节点。
- 配置更新与通知: Sentinel不仅负责故障转移，还能动态更新Redis客户端关于新主节点的配置信息。当故障转移完成后，Sentinel会通过发布与订阅机制通知所有已连接的客户端主节点已经变更，客户端可以据此重新配置连接指向新的主节点。
- 监控与恢复: Sentinel会持续监控整个Redis集群，确保故障转移后的主从关系正确。当原主节点恢复后，可以根据配置将其设置为新的主节点或者从节点。
- 配置持久化与自动重启: Sentinel有自己的配置文件，并且能够持久化其对Redis集群状态的监控信息。在Sentinel进程重启时，可以从配置文件中恢复其对集群的认知和监控状态。
五、实现Redis Cluster高可用集群，并验证基于python向Redis Cluster写入数据
虽然在哨兵 sentinel 机制中，可以解决 Redis 高可用的问题，即当 master 故障后可以自动将 slave 提升为 master 从而可以保证 Redis 服务的正常使用，但是无法解决 Redis 单机写入的瓶颈问题。单机的 Redis 写入性能受限于单机的内存大小、并发数量、网卡速率等因素，因此 Redis 官方在 Redis 3.0版本之后推出了无中心架构的 Redis Cluster 机制，在无中心的 Redis 集群当中，其每个节点保存当前节点数据和整个集群状态，每个节点都和其它所有节点连接，特点如下：
1、所有Redis节点互联（PING机制）
2、集群中某个节点的失效，是整个集群中超过半数的节点监测都失效才算真正的失效
3、客户端可以不需要proxy即可直接连接Redis，应用程序需要写全部的Redis服务器IP
4、Redis Cluster把所有的Redis node映射到0~16383个槽位上（slot），读写需要到指定的Redis node上进行操作，因此有多少个Redis node相当于Redis并发扩展了多少倍
5、Redis Cluster预先分配16384个槽位，当需要再Redis集群中写入一个key-value的时候，会使用CRC16(key) mod 16384之后的值，然后决定将key写入到哪一个槽位从而决定写入哪一个Redis节点上，从而有效解决单机瓶颈
6、集群最大内存空间是master节点的内存累计空间
7、写入性能是master的累计写入总和
Redis Cluster 数据读写流程：https://developer.redis.com/operate/redis-at-scale/scalability/redis-cluster-and-client-libraries/
5.1 环境准备
系统环境
root@ubuntu18-server:~# cat /etc/issue
Ubuntu 18.04.6 LTS \n \l
Redis version: 7.2.0
节点分配
192.168.119.161
192.168.119.162
192.168.119.163
192.168.119.164
192.168.119.165
192.168.119.166
192.168.119.167 # 预留扩容
192.168.119.168 # 预留扩容
5.1.1 安装 Redis7.2.0 版本并开启集群功能
使用脚本自动化安装，其余节点相同
root@ubuntu18-server:~# tar xvf redis7InstallScript.tar.gz
root@ubuntu18-server:~# cd redis7InstallScript/
root@ubuntu18-server:~/redis7InstallScript# ls
redis-7.2.0_install.sh redis-7.2.0.tar.gz redis.conf
root@ubuntu18-server:~/redis7InstallScript# bash redis-7.2.0_install.sh
# 开启Redis Cluster功能
root@ubuntu18-server:~# vim /apps/redis/etc/redis.conf
cluster-enabled yes
cluster-config-file nodes-6379.conf
masterauth 123456
requirepass 123456
root@ubuntu18-server:~# systemctl restart redis-server.service
# 查看集群功能是否开启
root@ubuntu18-server:~# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info cluster
# Cluster
cluster_enabled:1
5.1.2 集群命令使用
root@ubuntu18-server:~# redis-cli --cluster help
Cluster Manager Commands:
create host1:port1 ... hostN:portN # 创建集群并执行节点数
--cluster-replicas <arg> # 指定master的副本数量
check <host:port> or <host> <port> - separated by either colon or space
--cluster-search-multiple-owners # 检查是否有槽位被分配给了多个节点
info <host:port> or <host> <port> - separated by either colon or space # 查看集群主机信息
fix <host:port> or <host> <port> - separated by either colon or space # 修复集群
--cluster-search-multiple-owners # 修复槽位被同时分配给多个节点的问题
--cluster-fix-with-unreachable-masters # 修复不可达主机
reshard <host:port> or <host> <port> - separated by either colon or space # 手动热迁移集群指定主机的slots数据到新的目的主机
--cluster-from <arg>
--cluster-to <arg>
--cluster-slots <arg>
--cluster-yes
--cluster-timeout <arg>
--cluster-pipeline <arg>
--cluster-replace
rebalance <host:port> or <host> <port> - separated by either colon or space # 如果各master的slot槽位差异很大，可以使用rebalance实现自动平衡集群中各主机的slot数量
--cluster-weight <node1=w1...nodeN=wN>
--cluster-use-empty-masters
--cluster-timeout <arg>
--cluster-simulate
--cluster-pipeline <arg>
--cluster-threshold <arg>
--cluster-replace
add-node new_host:new_port existing_host:existing_port # 添加节点到集群
--cluster-slave
--cluster-master-id <arg>
del-node host:port node_id # 从集群删除主机
call host:port command arg arg .. arg # 在集群的所有节点上执行命令
--cluster-only-masters
--cluster-only-replicas
set-timeout host:port milliseconds # 设置节点的超时时间
import host:port # 将外部redis服务器的数据导入当前集群
--cluster-from <arg>
--cluster-from-user <arg>
--cluster-from-pass <arg>
--cluster-from-askpass
--cluster-copy
--cluster-replace
backup host:port backup_directory # 备份集群数据
help
For check, fix, reshard, del-node, set-timeout, info, rebalance, call, import, backup you can specify the host and port of any working node in the cluster.
Cluster Manager Options:
--cluster-yes Automatic yes to cluster commands prompts
5.2 集群初始化
在任意一节点可执行初始化操作
root@ubuntu18-server:~# redis-cli -a 123456 --cluster create 192.168.119.161:6379 192.168.119.162:6379 192.168.119.163:6379 192.168.119.164:6379 192.168.119.165:6379 192.168.119.166:6379 --cluster-replicas 1
5.3 验证集群状态
root@ubuntu18-server:~# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
# 主从状态信息
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.119.165,port=6379,state=online,offset=714,lag=1
master_failover_state:no-failover
master_replid:851739edde74e60519b830540c5299448a589c2d
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:728
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:536870912
repl_backlog_first_byte_offset:1
repl_backlog_histlen:728
# 集群状态信息
127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:523
cluster_stats_messages_pong_sent:545
cluster_stats_messages_sent:1068
cluster_stats_messages_ping_received:540
cluster_stats_messages_pong_received:523
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:1068
total_cluster_links_buffer_limit_exceeded:0
# 集群节点信息，Slave节点除了显示自己的ID信息外，Master的ID信息也会展示
127.0.0.1:6379> cluster nodes
fdee09e80ceda3bb7fc11c8eedf3833ac09931ec 192.168.119.165:6379@16379 slave 5fcc191932ea0660f0b9a636d34e22aa926824d2 0 1710233373696 1 connected
b7d52d2dd34812c34b37fa8ad08ac83cf22cf40f 192.168.119.163:6379@16379 master - 0 1710233372684 3 connected 10923-16383
da2b2b032b96e25a97bc41e99d12ff0f56009c52 192.168.119.162:6379@16379 master - 0 1710233371000 2 connected 5461-10922
4873b095054dc04fc0ae0975dfae2df2d1756d9d 192.168.119.166:6379@16379 slave da2b2b032b96e25a97bc41e99d12ff0f56009c52 0 1710233373000 2 connected
83db779dbfc60c79983fbe22427f870f2add700c 192.168.119.164:6379@16379 slave b7d52d2dd34812c34b37fa8ad08ac83cf22cf40f 0 1710233371663 3 connected
5fcc191932ea0660f0b9a636d34e22aa926824d2 192.168.119.161:6379@16379 myself,master - 0 1710233370000 1 connected 0-5460
5.4 基于python向集群写入数据
准备Python脚本文件并写入数据
# 查看每个节点分配了多少key（主从复制，故总共200个）
root@ubuntu18-server:~# redis-cli -a 123456 --cluster call 192.168.119.161:6379 DBSIZE
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Calling DBSIZE
192.168.119.161:6379: 32
192.168.119.162:6379: 32
192.168.119.163:6379: 36
192.168.119.166:6379: 32
192.168.119.165:6379: 32
192.168.119.164:6379: 36
六、基于redis-shake实现不同Redis环境的数据迁移
RedisShark：https://github.com/tair-opensource/RedisShake
6.1 功能简介
rdb_reader: 支持读取解析本地rdb文件，向目的redis/redis cluster恢复数据
sync_reader: 支持对源redis 和目的redis 进行在线数据同步，需要支持 psync(redis 2.8开始支持)，psync 具备了数据全量同步和增量同步模式
scan_reader: 支持基于dump从源主机导出key，然后在目的主机使用RESTORE导入key，通过SCAN命令遍历源端数据库中的所有 key，不推荐
6.2 准备 redis 源主机
将主机ubuntu18-server7: 192.168.119.117 的数据导入到 Redis Cluster，源主机与 Redis Cluster 的 key 不要冲突
# 为了方便观察，先删除我们的集群中的数据
root@ubuntu18-server3:~# redis-cli -a 123456 --cluster call 192.168.119.161:6379 flushall
root@ubuntu18-server3:~# redis-cli -a 123456 --no-auth-warning --cluster call 192.168.119.161:6379 dbsize
>>> Calling dbsize
192.168.119.161:6379: 0
192.168.119.162:6379: 0
192.168.119.166:6379: 0
192.168.119.163:6379: 0
192.168.119.164:6379: 0
192.168.119.165:6379: 0
# Redis 服务已安装完成
● redis-server.service - Redis data structure server
Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2024-03-13 01:38:08 UTC; 4h 35min ago
Docs: https://redis.io/documentation
Main PID: 1005 (redis-server)
Status: "Ready to accept connections"
Tasks: 5 (limit: 4623)
CGroup: /system.slice/redis-server.service
└─1005 /apps/redis/bin/redis-server 0.0.0.0:6379
Mar 13 01:38:08 ubuntu18-server7 systemd[1]: Starting Redis data structure server...
Mar 13 01:38:08 ubuntu18-server7 systemd[1]: Started Redis data structure server.
# 写入数据
root@ubuntu18-server7:~# cat redis-client.sh
#!/bin/bash
for i in `seq 1 100000`;do
redis-cli -h 127.0.0.1 -a 123456 --no-auth-warning set key-${i} value-${i} &
# echo "key-${i} value-${i} 写入完成"
done
echo "十万个key写入完成!"
root@ubuntu18-server7:~# bash redis-client.sh
6.3部署 Redis-shake并同步数据
参考文档：
https://tair-opensource.github.io/RedisShake/zh/reader/scan_reader.html
https://github.com/tair-opensource/RedisShake/blob/v4/shake.toml
root@ubuntu18-server7:/usr/local/src# cd /usr/local/src/
root@ubuntu18-server7:/usr/local/src# wget https://github.com/tair-opensource/RedisShake/releases/download/v4.0.5/redis-shake-linux-amd64.tar.gz
root@ubuntu18-server7:/usr/local/src# tar xvf redis-shake-linux-amd64.tar.gz
./redis-shake
./shake.toml
# 编辑配置文件
root@ubuntu18-server7:/usr/local/src# vim shake.toml
root@ubuntu18-server7:/usr/local/src# ./redis-shake shake.toml
# 写入一个数据
root@ubuntu18-server7:~# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> set name Jerry
OK
# 观察同步状态，read_count 和 write_count 的数量加1
2024-03-13 07:03:22 INF read_count=[100001], read_ops=[0.00], write_count=[100001], write_ops=[0.00], syncing aof, diff=[0]
2024-03-13 07:03:27 INF read_count=[100001], read_ops=[0.00], write_count=[100001], write_ops=[0.00], syncing aof, diff=[0]
6.4 集群节点验证数据
# Master 节点数量一共100001个
root@ubuntu18-server3:~# redis-cli -a 123456 --no-auth-warning --cluster call 192.168.119.161:6379 dbsize
>>> Calling dbsize
192.168.119.161:6379: 33365
192.168.119.162:6379: 33280
192.168.119.166:6379: 33280
192.168.119.163:6379: 33356
192.168.119.164:6379: 33356
192.168.119.165:6379: 33365
七、基于mysql和Redis,实现PHP页面的数据缓存效果
环境准备
7.1 部署 Redis 及 MySQL
7.1.1 安装 MySQL
root@ubuntu18-server11:~# tar xf runtime-docker24.0.2-containerd1.6.21-binary-install.tar.gz
root@ubuntu18-server11:~# bash runtime-install.sh containerd
# 部署MySQL
root@ubuntu18-server11:~# mkdir -p /data/mysql
root@ubuntu18-server11:~# nerdctl run -d -p 3306:3306 --name mysql-container-test -v /data/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD="mypass@123" registry.cn-hangzhou.aliyuncs.com/zhangshijie/mysql:5.7.36
# 初始化数据库
root@ubuntu18-server11:~# apt -y install mysql-client
root@ubuntu18-server11:~# mysql -uroot -h 192.168.119.171 -pmypass@123
mysql> CREATE database test_store;
Query OK, 1 row affected (0.00 sec)
mysql> USE test_store;
Database changed
mysql> CREATE table products
-> (
-> product_id BIGINT PRIMARY KEY AUTO_INCREMENT,
-> product_name VARCHAR(50),
-> price DOUBLE
-> ) Engine = InnoDB;
Query OK, 0 rows affected (0.06 sec)
mysql> INSERT INTO products(product_name, price) VALUES ('Virtual Private Servers','5.00');
Query OK, 1 row affected (0.01 sec)
mysql> INSERT INTO products(product_name, price) VALUES ('Managed Databases', '15.00');
Query OK, 1 row affected (0.00 sec)
mysql> INSERT INTO products(product_name, price) VALUES ('Block Storage', '10.00');
Query OK, 1 row affected (0.01 sec)
mysql> INSERT INTO products(product_name, price) VALUES ('Managed Kubernetes','60.00');
Query OK, 1 row affected (0.01 sec)
mysql> INSERT INTO products(product_name, price) VALUES ('Load Balancer', '10.00');
Query OK, 1 row affected (0.00 sec)
mysql> SELECT * FROM products;
+------------+-------------------------+-------+
| product_id | product_name | price |
+------------+-------------------------+-------+
| 1 | Virtual Private Servers | 5 |
| 2 | Managed Databases | 15 |
| 3 | Block Storage | 10 |
| 4 | Managed Kubernetes | 60 |
| 5 | Load Balancer | 10 |
+------------+-------------------------+-------+
5 rows in set (0.00 sec)
7.1.2 安装 Redis
root@ubuntu18-server11:~# tar xf redis7InstallScript.tar.gz -C /usr/local/src
root@ubuntu18-server11:~# cd /usr/local/src/redis7InstallScript/
root@ubuntu18-server11:/usr/local/src/redis7InstallScript# bash redis-7.2.0_install.sh
7.2 准备web服务器
root@ubuntu20-server1-111:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal
root@ubuntu20-server1-111:~# apt update
root@ubuntu20-server1-111:~# apt -y install apache2 php-fpm libapache2-mod-php php7.4-mysql php7.4-redis
# 测试php页面
root@ubuntu20-server1-111:/var/www/html# cat index.php
<?php
phpinfo();
?>
7.3 测试 Redis 数据访问
root@ubuntu20-server1-111:/var/www/html# cat redis.php
<?php
$redis = new Redis();
$redis->connect('192.168.119.171',6379);
$redis->auth('123456');
echo $redis->get("name"); //提前写入一个测试数据
?>
7.4 部署实际web界面
root@ubuntu20-server1-111:/var/www/html# cat redis-mysql.php
<?php
$redis = new Redis();
$redis->connect('192.168.119.171',6379);
$redis->auth('123456');
$key='PRODUCTS';
if (!$redis->get($key)) {
$source = '未命中缓存：数据来源MySQL Server';
$database_name = 'test_store';
$database_user= 'root';
$database_password = 'mypass@123';
$mysql_host = '192.168.119.171';
$pdo = new PDO('mysql:host=' . $mysql_host . '; dbname=' . $database_name, $database_user, $database_password);
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$sql = "SELECT * FROM products";
$stmt = $pdo->prepare($sql);
$stmt->execute();
while ($row = $stmt->fetch(PDO::FETCH_ASSOC)) {
$products[] = $row;
}
$redis->set($key, serialize($products));
$redis->expire($key, 60);
} else {
$source = '成功命中缓存：数据来源 Redis Server';
$products = unserialize($redis->get($key));
}
echo $source . ': <br>';
print_r($products);
?>
7.5 测试访问
7.5.1 首次访问未命中
7.5.2 再次访问命中
7.5.2 验证Redis数据
root@ubuntu18-server11:~# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> KEYS *
1) "name"
2) "PRODUCTS"
127.0.0.1:6379> TTL PRODUCTS
(integer) 46
127.0.0.1:6379> get PRODUCTS
"a:5:{i:0;a:3:{s:10:\"product_id\";s:1:\"1\";s:12:\"product_name\";s:23:\"Virtual Private Servers\";s:5:\"price\";s:1:\"5\";}i:1;a:3:{s:10:\"product_id\";s:1:\"2\";s:12:\"product_name\";s:17:\"Managed Databases\";s:5:\"price\";s:2:\"15\";}i:2;a:3:{s:10:\"product_id\";s:1:\"3\";s:12:\"product_name\";s:13:\"Block Storage\";s:5:\"price\";s:2:\"10\";}i:3;a:3:{s:10:\"product_id\";s:1:\"4\";s:12:\"product_name\";s:18:\"Managed Kubernetes\";s:5:\"price\";s:2:\"60\";}i:4;a:3:{s:10:\"product_id\";s:1:\"5\";s:12:\"product_name\";s:13:\"Load Balancer\";s:5:\"price\";s:2:\"10\";}}"
