Redis4.0数据库（二）：Redis4.0之持久化存储（下）

十，Redis的RDB存储方式

10.1 redis的运行方式说明

redis如果提供缓存服务，可以关闭所有持久化存储，如此一来redis重启后所有数据会丢失
开启rdb或aof持久化存储，能把redis中的数据持久化到磁盘中。
rdb和aof对性能都有影响，所以建议持久化的操作在从库上进行

10.2 redis rdb存储方式，使用save配置开启rdb存储或者关闭rdb存储

  1. #与rdb相关的配置文件信息
  2. dir /data/redis/ #dir为rdb存储的路径
  3. dbfilename dump.rdb #rdb存储文件的名字
  4. save 60 10000 #60s改变10000key，触发rdb存储
  5. save 300 10 #300s改变10个key，触发rdb存储
  6. save 900 1 #900s改变1个key触发rdb存储
  7. rdbcompression no #rdb压缩最好关闭，影响cpu

10.3 设置开启或者关闭rdb存储

提示：默认情况下rdb持久化存储是开启的

  1. [root@Redis01 ~]# redis-cli config set save "" #关闭rdb存储
  2. OK
  3. [root@Redis01 ~]# redis-cli config rewrite #配置保存
  4. OK
  5. [root@Redis01 ~]# redis-cli config set save "180 1 120 10 60 10000" #开启rdb
  6. OK
  7. [root@Redis01 ~]# redis-cli config rewrite #配置保存
  8. OK

10.4 进行数据写入，观察rdb存储日志

  1. #输入1万条数据
  2. for line in `seq -w 10000`;do redis-cli set key_${line} value1_${line};done
  1. #查看日志信息
  2. 51876:M 11 Aug 09:36:24.049 * 10000 changes in 60 seconds. Saving... #发生了1万条数据改变60秒内，触发rdb存储
  3. 51876:M 11 Aug 09:36:24.050 * Background saving started by pid 80855
  4. 80855:C 11 Aug 09:36:24.058 * DB saved on disk
  5. 80855:C 11 Aug 09:36:24.059 * RDB: 0 MB of memory used by copy-on-write
  6. 51876:M 11 Aug 09:36:24.152 * Background saving terminated with success

10.5 redis提供的bgsave命令能够立刻触发rdb存储，观察存储日志

  1. [root@Redis01 ~]# redis-cli save #会阻塞前端客户数据输入
  2. OK
  3. [root@Redis01 ~]# redis-cli bgsave #后台启动新进程进行rdb存储
  4. Background saving started
  1. #查看日志
  2. 51876:M 11 Aug 09:45:15.100 * DB saved on disk #save触发的日志信息
  3. 51876:M 11 Aug 09:45:19.641 * Background saving started by pid 81313 #bgsave触发的信息
  4. 81313:C 11 Aug 09:45:19.646 * DB saved on disk #bgsave触发的信息
  5. 81313:C 11 Aug 09:45:19.646 * RDB: 0 MB of memory used by copy-on-write #bgsave触发的信息
  6. 51876:M 11 Aug 09:45:19.743 * Background saving terminated with success #bgsave触发的信息

十一，Redis的AOF存储方式

redis的appendonly（aof）持久化存储会把用户每次的操作都记录到文件中（类似mysqlbinlog）

11.1 动态开启或者关闭aof

  1. [root@Redis01 ~]# redis-cli config set appendonly yes #开启
  2. OK
  3. [root@Redis01 ~]# redis-cli config rewrite
  4. OK
  5. [root@Redis01 ~]# redis-cli config set appendonly no #关闭
  6. OK
  7. [root@Redis01 ~]# redis-cli config rewrite
  8. OK

11.2 写入数据，观察aof。多次运行，aof文件不断增大，rdb文件大小不变

  1. #查看aof和rdb文件大小
  2. [root@Redis01 redis]# du -sh appendonly.aof
  3. 512K appendonly.aof
  4. [root@Redis01 redis]# du -sh dump.rdb
  5. 260K dump.rdb
  7. #写入数据
  8. [root@Redis01 ~]# for line in `seq -w 100`;do redis-cli set key_${line} value_${line};done
  10. #查看aof和rdb文件大小
  11. [root@Redis01 redis]# du -sh appendonly.aof
  12. 1.5M appendonly.aof
  13. [root@Redis01 redis]# du -sh dump.rdb
  14. 260K dump.rdb

11.3 重写aof文件，整理相同的key，写入最后的有效值

BGREWRITEAOF
执行一个AOF文件重写操作。重写会创建一个当前AOF文件的体积优化版本
即使BGREWRITEAOF执行失败，也不会有任何数据丢失，因为旧的AOF文件在BGREWRITEAOF成功之前不会被修改。
重写操作只会在没有其他持久化工作在后台执行时被触发。
从Redis2.4开始，AOF重写由Redis自行触发，BGREWRITEAOF仅仅用于手动触发重写操作。

  1. #清空aof文件
  2. [root@Redis01 redis]# >appendonly.aof
  3. [root@Redis01 redis]# ls
  4. appendonly.aof dump.rdb redis.log redis.pid
  5. [root@Redis01 redis]# du -sh appendonly.aof
  6. 0 appendonly.aof
  7. [root@Redis01 ~]# redis-cli bgrewriteaof #手动触发AOF重写
  8. Background append only file rewriting started
  9. [root@Redis01 redis]# du -sh appendonly.aof #redis里所有数据被重写入aof
  10. 512K appendonly.aof
  12. #清空aof文件
  13. [root@Redis01 redis]# >appendonly.aof
  14. [root@Redis01 redis]# du -sh appendonly.aof
  15. 0 appendonly.aof
  16. [root@Redis01 redis]# redis-cli set yunjisuan benet
  17. OK
  18. [root@Redis01 redis]# du -sh appendonly.aof
  19. 4.0K appendonly.aof
  20. [root@Redis01 redis]# cat appendonly.aof
  21. *2
  22. $6
  23. SELECT #select 0 表示切换到db0
  24. $1
  25. 0
  26. *3
  27. $3
  28. set #执行set yunjisuan benet
  29. $9
  30. yunjisuan
  31. $5
  32. benet
  33. [root@Redis01 redis]# redis-cli del yunjisuan benet
  34. (integer) 1
  35. [root@Redis01 redis]# cat appendonly.aof
  36. *2
  37. $6
  38. SELECT
  39. $1
  40. 0
  41. *3
  42. $3
  43. set
  44. $9
  45. yunjisuan
  46. $5
  47. benet
  48. *3
  49. $3
  50. del #执行del yunjisuan benet
  51. $9
  52. yunjisuan
  53. $5
  54. benet

我们发现虽然我们向redis添加了一个key，又删除了这个key。redis数据库从本质上来说并没有新增任何数据。但是aof文件仍旧把操作都给记录了。这样就会导致aof文件最终会非常大。所以aof文件的优化，就是让aof文件进行重写，只记录数据的增量部分。如此aof文件就小很多了。

11.4 aof配置自动rewrite机制

  1. #在默认配置文件里，默认存在
  2. auto-aof-rewrite-percentage 100 #默认100%，也就是aof增加一倍后考虑rewrite，两个条件要同时满足
  3. auto-aof-rewrite-min-size 64mb #默认64mb，也就是aof达到64M后考虑rewirte，两个条件要同时满足
  5. #获取aof-rewrite配置
  6. [root@Redis01 redis]# redis-cli config get auto-aof-rewrite*
  7. 1) "auto-aof-rewrite-percentage"
  8. 2) "100"
  9. 3) "auto-aof-rewrite-min-size"
  10. 4) "67108864" #64MB（单位字节）
  12. #进行aof自动重写测试
  13. [root@Redis01 redis]# redis-cli config set auto-aof-rewrite-min-size 100000
  14. [root@Redis01 redis]# redis-cli config get auto-aof-rewrite*
  15. 1) "auto-aof-rewrite-percentage"
  16. 2) "100"
  17. 3) "auto-aof-rewrite-min-size"
  18. 4) "100000"
  19. [root@Redis01 redis]# redis-cli config rewrite
  20. OK
  21. [root@Redis01 redis]# > appendonly.aof
  22. [root@Redis01 redis]# du -sh appendonly.aof
  23. 0 appendonly.aof
  24. [root@Redis01 ~]# for line in `seq -w 1000`;do redis-cli set key2_${line} value2_${line};done
  25. [root@Redis01 redis]# du -sh appendonly.aof
  26. 48K appendonly.aof
  27. [root@Redis01 ~]# for line in `seq -w 1000`;do redis-cli del key2_${line} value2_${line};done
  28. [root@Redis01 redis]# du -sh appendonly.aof
  29. 128K appendonly.aof
  30. [root@Redis01 redis]# du -sh appendonly.aof
  31. 128K appendonly.aof
  32. [root@Redis01 redis]# du -sh appendonly.aof
  33. 92K appendonly.aof #自动触发了aof重写机制

十二，Redis最大内存设置和删除算法

redis-cli flushall #手动清空redis里所有数据

12.1 redis的键设置有效期，过期自动删除

  1. [root@Redis01 ~]# redis-cli set name yunjisuan
  2. OK
  3. [root@Redis01 ~]# redis-cli ttl name
  4. (integer) -1 #-1代表key永久有效
  5. [root@Redis01 ~]# redis-cli expire name 10 #设定key 10s有效
  6. (integer) 1
  7. [root@Redis01 ~]# redis-cli ttl name #查看key存活剩余时间
  8. (integer) 8
  9. [root@Redis01 ~]# redis-cli ttl name
  10. (integer) 7
  11. [root@Redis01 ~]# redis-cli ttl name
  12. (integer) 6
  13. [root@Redis01 ~]# redis-cli ttl name
  14. (integer) 6
  15. [root@Redis01 ~]# redis-cli ttl name
  16. (integer) 5
  17. [root@Redis01 ~]# redis-cli ttl name
  18. (integer) 4
  19. [root@Redis01 ~]# redis-cli ttl name
  20. (integer) 4
  21. [root@Redis01 ~]# redis-cli ttl name
  22. (integer) 3
  23. [root@Redis01 ~]# redis-cli ttl name
  24. (integer) 2
  25. [root@Redis01 ~]# redis-cli ttl name
  26. (integer) 1
  27. [root@Redis01 ~]# redis-cli ttl name
  28. (integer) 0
  29. [root@Redis01 ~]# redis-cli ttl name
  30. (integer) -2
  31. [root@Redis01 ~]# redis-cli get name #key已经被过期清除了
  32. (nil)

12.2 查看设置最大内存

  1. #查看和设定最大内存限制
  2. [root@Redis01 ~]# redis-cli config get maxmemory
  3. 1) "maxmemory"
  4. 2) "0" #默认对内存无限制
  5. [root@Redis01 ~]# redis-cli config set maxmemory 1M #限制1M
  6. OK
  7. [root@Redis01 ~]# redis-cli config get maxmemory
  8. 1) "maxmemory"
  9. 2) "1000000"

12.3 可选择的删除算法

  • volatile-lru:
    • 使用LRU算法删除键（key需要设置过期时间）
  • volatile-random:
    • 随机删除键（key需要设置过期时间）
  • volatile-ttl:
    • 删除ttl最小的键（key需要设置过期时间）
  • allkeys-lru:
    • 使用LRU算法删除键（所有key）
  • allkeys-random:
    • 随机删除键（所有key）
  • noeviction:
    • 不进行任何的操作，只返回错误，默认
  1. [root@Redis01 ~]# redis-cli config get maxmemory-policy #内存清理算法
  2. 1) "maxmemory-policy"
  3. 2) "noeviction" #默认noeviction

12.4 模拟超过内存

  1. [root@Redis01 ~]# for line in `seq -w 10000`;do redis-cli set key_${line} value_${line};done
  2. [root@Redis01 ~]# redis-cli set name benet
  3. (error) OOM command not allowed when used memory > 'maxmemory'.
  4. [root@Redis01 ~]# redis-cli
  5. 127.0.0.1:6379> set name benet
  6. (error) OOM command not allowed when used memory > 'maxmemory'.

测试会发现报错
(error) OOM command not allowed when used memory > 'maxmemory'.

12.5 设置删除算法

  1. #将删除算法设置为volatile-lru
  2. [root@Redis01 ~]# redis-cli config get maxmemory-policy
  3. 1) "maxmemory-policy"
  4. 2) "noeviction"
  5. [root@Redis01 ~]# redis-cli config set maxmemory-policy volatile-lru
  6. OK
  7. [root@Redis01 ~]# redis-cli config get maxmemory-policy
  8. 1) "maxmemory-policy"
  9. 2) "volatile-lru"
  10. [root@Redis01 ~]# redis-cli config rewrite
  11. OK
  13. #算法测试
  14. [root@Redis01 ~]# redis-cli get key_00111
  15. "value_00111"
  16. [root@Redis01 ~]# redis-cli expire key_00111 3600
  17. (integer) 1
  18. [root@Redis01 ~]# redis-cli ttl key_00111
  19. (integer) -2
  20. [root@Redis01 ~]# redis-cli get key_00111
  21. (nil)
  23. 说明：由上述测试可以发现
  24. volatile-lru算法
  25. 当内存到了最大值以后，会优先删除有过期时间的key。

十三，Redis禁用屏蔽危险命令

13.1 redis禁用的命令

FLUSHALL和FLUSHDB会清除redis的数据，比较危险
KEYS在键过多的时候使用会阻塞业务请求

13.2 redis禁用危险命令配置代码如下（写入配置文件即可，此配置无法平滑更新）

  1. rename-command FLUSHALL "" #将命令改名成空
  2. rename-command FLUSHDB "" #将命令改名成空
  3. rename-command KEYS "" #将命令改名成空

将配置加入redis.conf配置文件

  1. [root@Redis01 ~]# echo 'rename-command FLUSHALL ""' >> /usr/local/redis/conf/redis.conf
  2. [root@Redis01 ~]# echo 'rename-command FLUSHDB ""' >> /usr/local/redis/conf/redis.conf
  3. [root@Redis01 ~]# echo 'rename-command KEYS ""' >> /usr/local/redis/conf/redis.conf
  4. [root@Redis01 ~]# tail -3 /usr/local/redis/conf/redis.conf
  5. rename-command FLUSHALL ""
  6. rename-command FLUSHDB ""
  7. rename-command KEYS ""

13.3 登陆redis，运行禁止命令进行测试

  1. #重启redis-server
  2. [root@Redis01 ~]# redis-cli shutdown
  3. [root@Redis01 ~]# redis-server /usr/local/redis/conf/redis.conf
  5. #测试被屏蔽的危险命令
  6. [root@Redis01 ~]# redis-cli flushall
  7. (error) ERR unknown command `flushall`, with args beginning with:
  8. [root@Redis01 ~]# redis-cli flushdb
  9. (error) ERR unknown command `flushdb`, with args beginning with:
  10. [root@Redis01 ~]# redis-cli
  11. 127.0.0.1:6379> keys *
  12. (error) ERR unknown command `keys`, with args beginning with: `*`,

十四，Redis主从服务器环境的搭建

主机名IP用途
redis01 192.168.200.165 redis-master
redis02 192.168.200.164 redis-slaveA
redis03 192.168.200.163 redis-slaveB

14.1 环境要求与redis基础编译部署调优

  1. #操作系统环境要求
  2. [root@Redis01 ~]# cat /etc/redhat-release
  3. CentOS Linux release 7.5.1804 (Core)
  4. [root@Redis01 ~]# uname -r
  5. 3.10.0-862.3.3.el7.x86_64
  6. [root@Redis01 ~]# systemctl stop firewalld
  7. [root@Redis01 ~]# systemctl disable firewalld
  8. [root@Redis01 ~]# setenforce 0
  9. setenforce: SELinux is disabled
  10. [root@Redis01 ~]# sestatus
  11. SELinux status: disabled
  14. #三台redis都进行如下编译过程
  15. [root@Redis01 ~]# yum -y install wget gcc gcc-c++ make tar openssl openssl-devel cmake
  16. [root@Redis01 ~]# tar xf redis-4.0.11.tar.gz -C /usr/src/
  17. [root@Redis01 ~]# cd /usr/src/redis-4.0.11/
  18. [root@Redis01 redis-4.0.11]# make
  19. [root@Redis01 redis-4.0.11]# make MALLOC=jemalloc
  20. [root@Redis01 redis-4.0.11]# make PREFIX=/usr/local/redis install
  21. [root@Redis01 redis-4.0.11]# cd /usr/local/redis/
  22. [root@Redis01 redis]# ls
  23. bin
  24. [root@Redis01 redis]# mkdir -p /usr/local/redis/conf
  25. [root@Redis01 redis]# cp /usr/src/redis-4.0.11/redis.conf /usr/local/redis/conf/
  26. [root@Redis01 redis]# cp /usr/src/redis-4.0.11/sentinel.conf /usr/local/redis/conf/
  27. [root@Redis01 src]# ln -s /usr/local/redis/bin/* /usr/local/bin/
  28. [root@Redis01 src]# which redis-server
  29. /usr/local/bin/redis-server
  31. #三台都进行配置文件优化和简单的基础调优
  32. [root@Redis01 redis]# pwd
  33. /usr/local/redis
  34. [root@Redis01 redis]# cp conf/redis.conf{,.bak}
  35. [root@Redis01 redis]# egrep -v "^$|^#" conf/redis.conf.bak > conf/redis.conf
  36. [root@Redis01 redis]# mkdir -p /data/redis/ #创建redis数据目录
  38. #修改配置文件
  39. [root@Redis01 redis]# cat -n conf/redis.conf | sed -n '1p;3p;4p;7p;9p;11p;21p'
  40. 1 bind 127.0.0.1
  41. 3 port 6379
  42. 4 tcp-backlog 511
  43. 7 daemonize no
  44. 9 pidfile /var/run/redis_6379.pid
  45. 11 logfile ""
  46. 21 dir ./
  47. #修改成以下设置
  48. [root@Redis01 redis]# cat -n conf/redis.conf | sed -n '1p;3p;4p;7p;9p;11p'
  49. 1 bind 0.0.0.0 #监听地址
  50. 3 port 6379 #监听端口
  51. 4 tcp-backlog 1024 #tcp连接数
  52. 7 daemonize yes #是否后台启动
  53. 9 pidfile /data/redis/redis.pid #pid存放目录
  54. 11 logfile "/data/redis/redis.log" #日志存放目录
  55. 21 dir /data/redis/ #工作目录
  57. #进行基础调优设置
  58. echo "* - nofile 10240" >> /etc/security/limits.conf
  59. echo "net.core.somaxconn = 10240" >> /etc/sysctl.conf
  60. echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
  61. sysctl -p
  62. echo never > /sys/kernel/mm/transparent_hugepage/enabled
  63. echo never > /sys/kernel/mm/transparent_hugepage/defrag
  64. echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local
  65. echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local
  67. #三台的redis-server都启动
  68. [root@Redis01 ~]# redis-server /usr/local/redis/conf/redis.conf
  69. [root@Redis01 ~]# netstat -antup | grep redis
  70. tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 24733/redis-server
  71. [root@Redis02 ~]# redis-server /usr/local/redis/conf/redis.conf
  72. [root@Redis02 ~]# netstat -antup | grep redis
  73. tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 18289/redis-server
  74. [root@Redis03 ~]# redis-server /usr/local/redis/conf/redis.conf
  75. [root@Redis03 ~]# netstat -antup | grep redis
  76. tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 18300/redis-server

14.2 redis主从同步服务器搭建

redis的主从同步，不用修改master任何配置
只需要在redis-slave上指定master的IP地址即可

  1. #先启动redis-master，然后再在两个redis-slave上都进行如下操作
  2. [root@Redis02 ~]# redis-cli shutdown
  3. [root@Redis02 ~]# echo "SLAVEOF 192.168.200.165 6379" >> /usr/local/redis/conf/redis.conf
  4. [root@Redis02 ~]# > /data/redis/redis.log
  5. [root@Redis02 ~]# redis-server /usr/local/redis/conf/redis.conf
  6. [root@Redis02 ~]# netstat -antup | grep redis
  7. tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 18342/redis-server
  8. tcp 0 0 192.168.200.164:37373 192.168.200.165:6379 ESTABLISHED 18342/redis-server

14.3 主从同步日志分析（全量同步）

  1. #查看redis-slave同步日志
  2. [root@Redis02 ~]# cat /data/redis/redis.log
  3. 18341:C 11 Aug 13:56:54.895 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
  4. 18341:C 11 Aug 13:56:54.895 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=18341, just started
  5. 18341:C 11 Aug 13:56:54.895 # Configuration loaded
  6. _._
  7. _.-``__ ''-._
  8. _.-`` `. `_. ''-._ Redis 4.0.11 (00000000/0) 64 bit
  9. .-`` .-```. ```\/ _.,_ ''-._
  10. ( ' , .-` | `, ) Running in standalone mode
  11. |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
  12. | `-._ `._ / _.-' | PID: 18342
  13. `-._ `-._ `-./ _.-' _.-'
  14. |`-._`-._ `-.__.-' _.-'_.-'|
  15. | `-._`-._ _.-'_.-' | http://redis.io
  16. `-._ `-._`-.__.-'_.-' _.-'
  17. |`-._`-._ `-.__.-' _.-'_.-'|
  18. | `-._`-._ _.-'_.-' |
  19. `-._ `-._`-.__.-'_.-' _.-'
  20. `-._ `-.__.-' _.-'
  21. `-._ _.-'
  22. `-.__.-'
  24. 18342:S 11 Aug 13:56:54.897 # Server initialized #服务器初始化
  25. 18342:S 11 Aug 13:56:54.897 * DB loaded from disk: 0.000 seconds #数据从磁盘加载0秒
  26. 18342:S 11 Aug 13:56:54.897 * Ready to accept connections #准备接受连接
  27. 18342:S 11 Aug 13:56:54.897 * Connecting to MASTER 192.168.200.165:6379 #链接到主192.168.200.165：6379
  28. 18342:S 11 Aug 13:56:54.897 * MASTER <-> SLAVE sync started #主从同步开始
  29. 18342:S 11 Aug 13:56:54.897 * Non blocking connect for SYNC fired the event. #非阻塞同步连接触发事件
  30. 18342:S 11 Aug 13:56:54.898 * Master replied to PING, replication can continue... #主应答，复制可以继续
  31. 18342:S 11 Aug 13:56:54.898 * Partial resynchronization not possible (no cached master) #部分同步不能（本机无缓存的主文件）
  32. 18342:S 11 Aug 13:56:54.899 * Full resync from master: e3adc85bd644e66bd1ee17b49c25e5e0491084d5:0 #进行全同步
  33. 18342:S 11 Aug 13:56:54.917 * MASTER <-> SLAVE sync: receiving 43606 bytes from master #从主接收43606字节
  34. 18342:S 11 Aug 13:56:54.917 * MASTER <-> SLAVE sync: Flushing old data #刷新旧数据
  35. 18342:S 11 Aug 13:56:54.917 * MASTER <-> SLAVE sync: Loading DB in memory #加载数据到内存
  36. 18342:S 11 Aug 13:56:54.918 * MASTER <-> SLAVE sync: Finished with success #同步完成
  38. #查看redis-master同步日志
  39. [root@Redis01 ~]# cat /data/redis/redis.log
  40. #从192.168.200.163:6379请求同步
  41. 26003:M 11 Aug 14:14:55.342 * Slave 192.168.200.163:6379 asks for synchronization
  42. #从192.168.200.163:6379请求完整的重新同步
  43. 26003:M 11 Aug 14:14:55.342 * Full resync requested by slave 192.168.200.163:6379
  44. #master启动bgsave与目标的磁盘进行同步
  45. 26003:M 11 Aug 14:14:55.342 * Starting BGSAVE for SYNC with target: disk
  46. #后台保存rdb的进程的pid号为26128
  47. 26003:M 11 Aug 14:14:55.342 * Background saving started by pid 26128
  48. #rdb文件已经保存到了磁盘
  49. 26128:C 11 Aug 14:14:55.344 * DB saved on disk
  50. #rdb写时复制使用了0MB的内存
  51. 26128:C 11 Aug 14:14:55.344 * RDB: 0 MB of memory used by copy-on-write
  52. #后台保存成功
  53. 26003:M 11 Aug 14:14:55.414 * Background saving terminated with success
  54. #与从192.168.200.163:6379同步成功
  55. 26003:M 11 Aug 14:14:55.415 * Synchronization with slave 192.168.200.163:6379 succeeded

14.4 主从同步日志分析（部分同步）

  1. #清空master日志
  2. [root@Redis01 ~]# > /data/redis/redis.log
  3. #清空slave日志，并shutdown
  4. [root@Redis02 ~]# > /data/redis/redis.log
  5. [root@Redis02 ~]# redis-cli shutdown
  6. #启动slave
  7. [root@Redis02 ~]# redis-server /usr/local/redis/conf/redis.conf
  9. #查看redis-slave日志
  10. [root@Redis02 ~]# cat /data/redis/redis.log
  11. 18342:S 11 Aug 14:23:27.678 # User requested shutdown...
  12. 18342:S 11 Aug 14:23:27.678 * Saving the final RDB snapshot before exiting.
  13. 18342:S 11 Aug 14:23:27.681 * DB saved on disk
  14. 18342:S 11 Aug 14:23:27.681 * Removing the pid file.
  15. 18342:S 11 Aug 14:23:27.681 # Redis is now ready to exit, bye bye...
  16. 18417:C 11 Aug 14:24:41.432 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
  17. 18417:C 11 Aug 14:24:41.432 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=18417, just started
  18. 18417:C 11 Aug 14:24:41.432 # Configuration loaded
  19. _._
  20. _.-``__ ''-._
  21. _.-`` `. `_. ''-._ Redis 4.0.11 (00000000/0) 64 bit
  22. .-`` .-```. ```\/ _.,_ ''-._
  23. ( ' , .-` | `, ) Running in standalone mode
  24. |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
  25. | `-._ `._ / _.-' | PID: 18418
  26. `-._ `-._ `-./ _.-' _.-'
  27. |`-._`-._ `-.__.-' _.-'_.-'|
  28. | `-._`-._ _.-'_.-' | http://redis.io
  29. `-._ `-._`-.__.-'_.-' _.-'
  30. |`-._`-._ `-.__.-' _.-'_.-'|
  31. | `-._`-._ _.-'_.-' |
  32. `-._ `-._`-.__.-'_.-' _.-'
  33. `-._ `-.__.-' _.-'
  34. `-._ _.-'
  35. `-.__.-'
  37. #服务器初始化
  38. 18418:S 11 Aug 14:24:41.434 # Server initialized
  39. #从磁盘加载旧数据用时0.001秒
  40. 18418:S 11 Aug 14:24:41.434 * DB loaded from disk: 0.001 seconds
  41. #由于之前是一个从库，利用主的参数合成一个主的缓存，这样就可以和主仅仅进行一部分的数据同步
  42. 18418:S 11 Aug 14:24:41.434 * Before turning into a slave, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
  43. #准备接受连接
  44. 18418:S 11 Aug 14:24:41.434 * Ready to accept connections
  45. #连接到master192.168.200.165:6379
  46. 18418:S 11 Aug 14:24:41.435 * Connecting to MASTER 192.168.200.165:6379
  47. #主从同步开始
  48. 18418:S 11 Aug 14:24:41.435 * MASTER <-> SLAVE sync started
  49. #非阻塞同步连接触发事件
  50. 18418:S 11 Aug 14:24:41.435 * Non blocking connect for SYNC fired the event.
  51. #master应答，复制可以继续
  52. 18418:S 11 Aug 14:24:41.435 * Master replied to PING, replication can continue...
  53. #尝试进行部分同步（要求646f44ea31d2b057e919e08493f94e97e92007f2:911）
  54. 18418:S 11 Aug 14:24:41.435 * Trying a partial resynchronization (request 646f44ea31d2b057e919e08493f94e97e92007f2:911).
  55. #成功进行部分同步
  56. 18418:S 11 Aug 14:24:41.436 * Successful partial resynchronization with master.
  57. #master应答接受一个部分同步的请求
  58. 18418:S 11 Aug 14:24:41.436 * MASTER <-> SLAVE sync: Master accepted a Partial Resynchronization.
  60. #查看redis-master日志
  61. [root@Redis01 ~]# cat /data/redis/redis.log
  62. #一个从库192.168.200.164：6379请求进行同步
  63. 26003:M 11 Aug 14:24:41.424 * Slave 192.168.200.164:6379 asks for synchronization
  64. #一个部分同步的请求来自192.168.200.164:6379，master已经接受请求，从偏移量为911处开始发送98字节的剩余数据
  65. 26003:M 11 Aug 14:24:41.424 * Partial resynchronization request from 192.168.200.164:6379 accepted. Sending 98 bytes of backlog starting from offset 911.

14.5 主从同步的停止

  1. #清空从库日志
  2. [root@Redis02 ~]# > /data/redis/redis.log
  4. #停止从库主从同步（只能在从库上执行）
  5. [root@Redis02 ~]# redis-cli slaveof no one
  6. OK
  8. #查看日志
  9. [root@Redis02 ~]# cat /data/redis/redis.log
  10. #将第二次复制的ID设置为646f44ea31d2b057e919e08493f94e97e92007f2，有效偏移量:3571。新的复制ID为efb9d2d393bb0bb4570a2f3144e7b2fbfc60b892
  11. 18418:M 11 Aug 14:55:15.723 # Setting secondary replication ID to 646f44ea31d2b057e919e08493f94e97e92007f2, valid up to offset: 3571. New replication ID is efb9d2d393bb0bb4570a2f3144e7b2fbfc60b892
  12. #与主库失去联系
  13. 18418:M 11 Aug 14:55:15.723 # Connection with master lost.
  14. #主从状态缓存断开
  15. 18418:M 11 Aug 14:55:15.723 * Caching the disconnected master state.
  16. #丢弃之前缓存的主的状态
  17. 18418:M 11 Aug 14:55:15.723 * Discarding previously cached master state.
  18. #主模式启用
  19. 18418:M 11 Aug 14:55:15.723 * MASTER MODE enabled (user request from 'id=4 addr=127.0.0.1:39500 fd=8 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=slaveof')
  21. #清空redis日志并恢复从库主从同步
  22. [root@Redis02 ~]# > /data/redis/redis.log
  23. [root@Redis02 ~]# redis-cli slaveof 192.168.200.165 6379
  24. OK
  26. #查看slave日志
  27. [root@Redis02 ~]# cat /data/redis/redis.log
  28. 18418:S 11 Aug 15:02:40.330 * Before turning into a slave, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
  29. 18418:S 11 Aug 15:02:40.330 * SLAVE OF 192.168.200.165:6379 enabled (user request from 'id=5 addr=127.0.0.1:39502 fd=7 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=slaveof')
  30. 18418:S 11 Aug 15:02:40.928 * Connecting to MASTER 192.168.200.165:6379
  31. 18418:S 11 Aug 15:02:40.928 * MASTER <-> SLAVE sync started
  32. 18418:S 11 Aug 15:02:40.928 * Non blocking connect for SYNC fired the event.
  33. 18418:S 11 Aug 15:02:40.929 * Master replied to PING, replication can continue...
  34. 18418:S 11 Aug 15:02:40.929 * Trying a partial resynchronization (request efb9d2d393bb0bb4570a2f3144e7b2fbfc60b892:3571).
  35. 18418:S 11 Aug 15:02:40.929 * Full resync from master: 646f44ea31d2b057e919e08493f94e97e92007f2:4186
  36. 18418:S 11 Aug 15:02:40.929 * Discarding previously cached master state.
  37. 18418:S 11 Aug 15:02:40.967 * MASTER <-> SLAVE sync: receiving 43607 bytes from master
  38. 18418:S 11 Aug 15:02:40.968 * MASTER <-> SLAVE sync: Flushing old data
  39. 18418:S 11 Aug 15:02:40.968 * MASTER <-> SLAVE sync: Loading DB in memory
  40. 18418:S 11 Aug 15:02:40.970 * MASTER <-> SLAVE sync: Finished with success

14.6 加密的主从同步

（1）为redis-master平滑设置连接密码

  1. [root@Redis01 ~]# redis-cli config get requirepass
  2. 1) "requirepass"
  3. 2) ""
  4. [root@Redis01 ~]# redis-cli config set requirepass 'yunjisuan'
  5. OK
  6. [root@Redis01 ~]# redis-cli config get requirepass
  7. (error) NOAUTH Authentication required.
  8. [root@Redis01 ~]# redis-cli -a yunjisuan config get requirepass
  9. Warning: Using a password with '-a' option on the command line interface may not be safe.
  10. 1) "requirepass"
  11. 2) "yunjisuan"
  12. [root@Redis01 ~]# redis-cli -a yunjisuan config rewrite
  13. Warning: Using a password with '-a' option on the command line interface may not be safe.
  14. OK
  16. #查看从库日志信息
  17. [root@Redis02 ~]# cat /data/redis/redis.log
  18. #主从同步需要进行端口验证请求
  19. 18418:S 11 Aug 15:08:40.440 * (Non critical) Master does not understand REPLCONF listening-port: -NOAUTH Authentication required.
  20. 18418:S 11 Aug 15:08:40.440 * (Non critical) Master does not
  21. understand REPLCONF capa: -NOAUTH Authentication required.
  22. #部分同步不能，没有主的缓存
  23. 18418:S 11 Aug 15:08:40.440 * Partial resynchronization not possible (no cached master)
  24. #从主发来的意外回复：需要身份验证
  25. 18418:S 11 Aug 15:08:40.441 # Unexpected reply to PSYNC from master: -NOAUTH Authentication required.
  26. #进行同步重试
  27. 18418:S 11 Aug 15:08:40.441 * Retrying with SYNC...
  28. #主从复制终止：需要身份验证
  29. 18418:S 11 Aug 15:08:40.441 # MASTER aborted replication with an error: NOAUTH Authentication required.

（2）为从库提供主从同步密码验证

  1. #从服务器需要设置主从同步的认证密码
  2. [root@Redis02 ~]# redis-cli config get masterauth
  3. 1) "masterauth"
  4. 2) ""
  5. [root@Redis02 ~]# redis-cli config set masterauth "yunjisuan"
  6. OK
  7. [root@Redis02 ~]# redis-cli config get masterauth
  8. 1) "masterauth"
  9. 2) "yunjisuan"
  10. [root@Redis02 ~]# redis-cli config rewrite
  11. OK
  12. [root@Redis02 ~]# tail -1 /usr/local/redis/conf/redis.conf
  13. masterauth "yunjisuan"
  15. #查看从服务器日志
  16. [root@Redis02 ~]# cat /data/redis/redis.log
  17. 18418:S 11 Aug 15:14:07.854 * Connecting to MASTER 192.168.200.165:6379
  18. 18418:S 11 Aug 15:14:07.854 * MASTER <-> SLAVE sync started
  19. 18418:S 11 Aug 15:14:07.855 * Non blocking connect for SYNC fired the event.
  20. 18418:S 11 Aug 15:14:07.856 * Master replied to PING, replication can continue...
  21. 18418:S 11 Aug 15:14:07.858 * Partial resynchronization not possible (no cached master)
  22. 18418:S 11 Aug 15:14:07.860 * Full resync from master: 646f44ea31d2b057e919e08493f94e97e92007f2:4648
  23. 18418:S 11 Aug 15:14:07.886 * MASTER <-> SLAVE sync: receiving 43607 bytes from master
  24. 18418:S 11 Aug 15:14:07.888 * MASTER <-> SLAVE sync: Flushing old data
  25. 18418:S 11 Aug 15:14:07.889 * MASTER <-> SLAVE sync: Loading DB in memory
  26. 18418:S 11 Aug 15:14:07.893 * MASTER <-> SLAVE sync: Finished with success
  27. 18418:S 11 Aug 15:14:18.778 # CONFIG REWRITE executed with success.

十五，使用Python操作Redis单例

15.1 Python安装redis扩展

  1. [root@redis-master ~]# yum -y install epel-release
  2. [root@redis-master ~]# yum -y install python2-pip
  3. [root@redis-master ~]# pip install redis

15.2 利用python进行redis数据的读写

    1. [root@Redis01 ~]# python
    2. Python 2.7.5 (default, Apr 11 2018, 07:36:10)
    3. [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
    4. Type "help", "copyright", "credits" or "license" for more information.
    5. >>> import redis
    6. >>> r = redis.Redis(host='127.0.0.1',port=6379,password='yunjisuan',db=0)
    7. >>> r.set('key_test','value_test')
    8. True
    9. >>> value = r.get('key_test')
    10. >>> print (value)
    11. value_test
    12. >>> exit()
    13. [root@Redis01 ~]# redis-cli -a yunjisuan get key_test
    14. Warning: Using a password with '-a' option on the command line interface may not be safe.
    15. "value_test"
posted @ 2021-09-23 21:11  你好我叫阿成  阅读(1142)  评论(0)    收藏  举报