Redis4.0之持久化存储（下）

十，Redis的RDB存储方式

10.1 redis的运行方式说明

redis如果提供缓存服务，可以关闭所有持久化存储，如此一来redis重启后所有数据会丢失
开启rdb或aof持久化存储，能把redis中的数据持久化到磁盘中。
rdb和aof对性能都有影响，所以建议持久化的操作在从库上进行

10.2 redis rdb存储方式，使用save配置开启rdb存储或者关闭rdb存储

#与rdb相关的配置文件信息
dir /data/redis/ #dir为rdb存储的路径
dbfilename dump.rdb #rdb存储文件的名字
save 60 10000 #60s改变10000key，触发rdb存储
save 300 10 #300s改变10个key，触发rdb存储
save 900 1 #900s改变1个key触发rdb存储
rdbcompression no #rdb压缩最好关闭，影响cpu

10.3 设置开启或者关闭rdb存储

提示：默认情况下rdb持久化存储是开启的

redis-cli config set save "" #关闭rdb存储

redis-cli config rewrite #配置保存

redis-cli config set save "180 1 120 10 60 10000" #开启rdb

redis-cli config rewrite #配置保存

image_1d01k0lqck84gt01p89e7615em9.png-23.1kB

10.4 进行数据写入，观察rdb存储日志

输入1万条数据

for line in `seq -w 10000`;do redis-cli set key_${line} value1_${line};done

image_1d01k1u0m1rldmt61nu13iq3o6m.png-9.9kB

查看日志信息

cat /data/redis/redis.log

1224:M 31 Dec 23:51:32.345 # CONFIG REWRITE executed with success.
1224:M 31 Dec 23:52:04.342 # CONFIG REWRITE executed with success.
1224:M 31 Dec 23:53:23.603 * 1 changes in 180 seconds. Saving...
#发生了1万条数据改变180秒内，触发rdb存储
1224:M 31 Dec 23:53:23.613 * Background saving started by pid 1246
1246:C 31 Dec 23:53:23.636 * DB saved on disk
1246:C 31 Dec 23:53:23.637 * RDB: 6 MB of memory used by copy-on-write
1224:M 31 Dec 23:53:23.713 * Background saving terminated with success

image_1d01k3hgiuj61m1q1pkn1o6c17oo13.png-40.5kB

查看redis里有多少个键值（keys）

redis-cli info

image_1d01k5vgb3e41hp29mj15ri140o1g.png-76kB

redis占用了多少内存

redis-cli info memory

used_memory:848480 --->数据占用内存大小
used_memory_human:828.59K --->人性化的方式显示数据占用内存大小
used_memory_rss:2179072 --->数据和进程占用大小
used_memory_rss_human:2.08M --->人性化的方式显示数据和进程占用大小
used_memory_peak:849456
used_memory_peak_human:829.55K
used_memory_peak_perc:99.89%
used_memory_overhead:836222
used_memory_startup:786592
used_memory_dataset:12258
used_memory_dataset_perc:19.81%
total_system_memory:1021906944
total_system_memory_human:974.57M
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:2.57 --->redis碎片率(频繁的写入删除,就会产生碎片,删的越多碎片就会越高),1-2之间代表没有内存碎片,小于1说明已经占用虚拟缓存,分配的内存不够用了
mem_allocator:jemalloc-4.0.3
active_defrag_running:0
lazyfree_pending_objects:0

image_1d01k6mg5s8nctp1rkq1tmji5s1t.png-58kB

10.5 redis提供的bgsave命令能够立刻触发rdb存储，观察存储日志

redis-cli save #会阻塞前端客户数据输入

redis-cli bgsave #后台启动新进程进行rdb存储(不影响前端输入)

image_1d01k9trg11p36nf4f01td81ic72q.png-13.7kB

查看日志

cat /data/redis/redis.log

1224:M 31 Dec 23:57:31.841 * DB saved on disk #save触发的日志信息
1224:M 31 Dec 23:57:39.821 * Background saving started by pid 11243 #bgsave触发的信息
11243:C 31 Dec 23:57:39.920 * DB saved on disk #bgsave触发的信息
11243:C 31 Dec 23:57:39.920 * RDB: 8 MB of memory used by copy-on-write #bgsave触发的信息
1224:M 31 Dec 23:57:39.985 * Background saving terminated with success #bgsave触发的信息

image_1d01kb0mf5nu10sn8d3aht8v537.png-31.4kB

十一，Redis的AOF存储方式

redis的appendonly（aof）持久化存储会把用户每次的操作都记录到文件中（类似mysqlbinlog）

11.1 动态开启或者关闭aof

redis-cli config set appendonly yes #开启

redis-cli config rewrite

redis-cli config get appendonly #查询状态

redis-cli config set appendonly no #关闭

redis-cli config rewrite

image_1d01m3r3np1t336kr8igo1gtg41.png-34.1kB

11.2 写入数据，观察aof。多次运行，aof文件不断增大，rdb文件大小不变

查看aof和rdb文件大小

du -sh /data/redis/appendonly.aof

du -sh /data/redis/dump.rdb

image_1d01m4s291hms1ping45lk517754e.png-17.8kB

写入数据

for line in `seq -w 100`;do redis-cli set key_${line} value_${line};done

image_1d01m6tb2q7emfq8ia44d1d055e.png-26kB

查看aof和rdb文件大小

redis-cli config set appendonly yes #开启

redis-cli config rewrite

du -sh /data/redis/appendonly.aof

du -sh /data/redis/dump.rdb

image_1d01mb67a16oit91efir9p1vgj6b.png-28kB

11.3 重写aof文件，整理相同的key，写入最后的有效值

BGREWRITEAOF
执行一个AOF文件重写操作。重写会创建一个当前AOF文件的体积优化版本
即使BGREWRITEAOF执行失败，也不会有任何数据丢失，因为旧的AOF文件在BGREWRITEAOF成功之前不会被修改。
重写操作只会在没有其他持久化工作在后台执行时被触发。
从Redis2.4开始，AOF重写由Redis自行触发，BGREWRITEAOF仅仅用于手动触发重写操作。

清空aof文件

> /data/redis/appendonly.aof

du -sh /data/redis/appendonly.aof

redis-cli bgrewriteaof #手动触发AOF重写

du -sh /data/redis/appendonly.aof #redis里所有数据被重写入aof

image_1d01mg8h2rk44t114slsuq132678.png-27.4kB

清空aof文件

> /data/redis/appendonly.aof

du -sh /data/redis/appendonly.aof

redis-cli set yunjisuan benet

du -sh /data/redis/appendonly.aof

cat /data/redis/appendonly.aof

*2
$6
SELECT #select 0 表示切换到db0
$1
0
*3
$3
set #执行set yunjisuan benet
$9
yunjisuan
$5
benet

image_1d01mrbsrqal1kmme7e1h24gqf7o.png-35.8kB

redis-cli del yunjisuan benet

cat /data/redis/appendonly.aof

*2
$6
SELECT
$1
0
*3
$3
set
$9
yunjisuan
$5
benet
*3
$3
del #执行del yunjisuan benet
$9
yunjisuan
$5
benet

image_1d01msgkgach176rks911g42p18o.png-30.7kB

重要提示

我们发现虽然我们向redis添加了一个key，又删除了这个key。redis数据库从本质上来说并没有新增任何数据。但是aof文件仍旧把操作都给记录了。这样就会导致aof文件最终会非常大。所以aof文件的优化，就是让aof文件进行重写，只记录数据的增量部分。如此aof文件就小很多了。

11.4 aof配置自动rewrite机制

在默认配置文件里，默认存在

redis-cli config get auto-aof-rewrite*

auto-aof-rewrite-percentage 100 #默认100%，也就是aof增加一倍后考虑rewrite，两个条件要同时满足
auto-aof-rewrite-min-size 64mb #默认64mb，也就是aof达到64M后考虑rewirte，两个条件要同时满足

image_1d01n1icc1shv1e1h1jdi11slvoa19.png-16.6kB

获取aof-rewrite配置

redis-cli config get auto-aof-rewrite*

1) "auto-aof-rewrite-percentage"
2) "100"
3) "auto-aof-rewrite-min-size"
4) "67108864" #64MB（单位字节）

image_1d01n1icc1shv1e1h1jdi11slvoa19.png-16.6kB

进行aof自动重写测试

redis-cli config set auto-aof-rewrite-min-size 100000

redis-cli config get auto-aof-rewrite*

redis-cli config rewrite

> /data/redisappendonly.aof

du -sh /data/redis/appendonly.aof

for line in `seq -w 1000`;do redis-cli set key2_${line} value2_${line};done

du -sh /data/redis/appendonly.aof

for line in `seq -w 1000`;do redis-cli del key2_${line} value2_${line};done

du -sh appendonly.aof

du -sh appendonly.aof

du -sh appendonly.aof

[root@redis01 ~]# du -sh /data/redis/appendonly.aof
128K /data/redis/appendonly.aof
[root@redis01 ~]# du -sh /data/redis/appendonly.aof
128K /data/redis/appendonly.aof
[root@redis01 ~]# du -sh /data/redis/appendonly.aof
128K /data/redis/appendonly.aof
[root@redis01 ~]# du -sh /data/redis/appendonly.aof
92K /data/redis/appendonly.aof #自动触发了aof重写机制

image_1d01n96dmrdnnnba991vtq3tc2p.png-55.9kB

image_1d01nqq1j1fgs1daotkb8su18k136.png-56.3kB

image_1d01nrjlk1aenfm9pc6l6pbcq3j.png-68.4kB

十二，Redis最大内存设置和删除算法

redis-cli flushall #手动清空redis里所有数据

image_1d01pqcaf1e5d1fj6b8g1sa11gm04g.png-14.7kB

12.1 redis的键设置有效期，过期自动删除

redis-cli set name yunjisuan

redis-cli ttl name
(integer) -1 #-1代表key永久有效

redis-cli expire name 10 #设定key 10s有效
(integer) 1

redis-cli ttl name #查看key存活剩余时间
(integer) 8

redis-cli ttl name
(integer) 6

redis-cli ttl name
(integer) 2

redis-cli ttl name
(integer) -2

redis-cli get name #key已经被过期清除了
(nil)

image_1d01ps1s2ltdd7o1alefu4b9d4t.png-34.5kB

12.2 查看设置最大内存

查看和设定最大内存限制

redis-cli config get maxmemory

1) "maxmemory"
2) "0" #默认对内存无限制

redis-cli config set maxmemory 1M #限制1M

redis-cli config get maxmemory

1) "maxmemory"
2) "1000000"

image_1d01q1kn81g1rgbj1hicl8n43g5a.png-21.6kB

12.3 可选择的删除算法

volatile-lru:
使用LRU算法删除键（key需要设置过期时间）
volatile-random:
随机删除键（key需要设置过期时间）
volatile-ttl:
删除ttl最小的键（key需要设置过期时间）
allkeys-lru:
使用LRU算法删除键（所有key）
allkeys-random:
随机删除键（所有key）
noeviction:
不进行任何的操作，只返回错误，默认

redis-cli config get maxmemory-policy #内存清理算法

1) "maxmemory-policy"
2) "noeviction" #默认noeviction

image_1d01q5s371ebf12hthpq177kf3g5n.png-12.8kB

12.4 模拟超过内存

for line in `seq -w 2000`;do redis-cli set key_${line} value_${line};done

测试会发现报错
(error) OOM command not allowed when used memory > 'maxmemory'.

image_1d01qgeug1gr3c971vk2euf14nn64.png-86.5kB

12.5 设置删除算法

将删除算法设置为volatile-lru

redis-cli config get maxmemory-policy

redis-cli config set maxmemory-policy volatile-lru

redis-cli config get maxmemory-policy

redis-cli config rewrite

image_1d01qkldcbppkrmt101f5l1uh56h.png-29.9kB

算法测试

redis-cli get key_0011

redis-cli expire key_0011 3600

redis-cli ttl key_0011

redis-cli get key_0011

说明：由上述测试可以发现
volatile-lru算法
当内存到了最大值以后，会优先删除有过期时间的key。

image_1d01qqav71r0m1li5to7ifk1pg46u.png-22.9kB

十三，Redis禁用屏蔽危险命令

13.1 redis禁用的命令

FLUSHALL和FLUSHDB会清除redis的数据，比较危险
KEYS在键过多的时候使用会阻塞业务请求

13.2 redis禁用危险命令配置代码如下（写入配置文件即可，此配置无法平滑更新）

rename-command FLUSHALL "" #将命令改名成空
rename-command FLUSHDB "" #将命令改名成空
rename-command KEYS "" #将命令改名成空

将配置加入redis.conf配置文件

echo 'rename-command FLUSHALL ""' >> /usr/local/redis/conf/redis.conf

echo 'rename-command FLUSHDB ""' >> /usr/local/redis/conf/redis.conf

echo 'rename-command KEYS ""' >> /usr/local/redis/conf/redis.conf

tail -3 /usr/local/redis/conf/redis.conf

image_1d01qu2u81huour01g5884ej3j7b.png-32.1kB

13.3 登陆redis，运行禁止命令进行测试

重启redis-server

redis-cli shutdown

redis-server /usr/local/redis/conf/redis.conf

netstat -antup | grep redis

image_1d01qvi0pr906uhduk1v5u1orc7o.png-21.6kB

测试被屏蔽的危险命令

redis-cli flushall

redis-cli flushdb

redis-cli

127.0.0.1:6379> keys *
(error) ERR unknown command `keys`, with args beginning with: `*`,

image_1d01r134u1hv27o1b271eb6t785.png-27.3kB

十四，Redis主从服务器环境的搭建

在工作中redis主库不开启任何持久化，rdb和aof肯定是全关闭的，任何影响主库性能全都关闭。
持久化都是在从上做。

主机名	IP	用途
redis01	192.168.200.70	redis-master
redis02	192.168.200.102	redis-slaveA
redis03	192.168.200.103	redis-slaveB

14.1 环境要求与redis基础编译部署调优

操作系统环境要求

cat /etc/redhat-release

uname -r

systemctl stop firewalld

systemctl disable firewalld

setenforce 0

sestatus

image_1d01tsag3u8qgqiukci8r1rag8i.png-20.9kB

image_1d01tsseu1gup1vh65531dflub18v.png-20.2kB

三台redis都进行如下编译过程(安装过程略)

yum -y install wget gcc gcc-c++ make tar openssl openssl-devel cmake

tar xf redis-4.0.11.tar.gz -C /usr/src/

cd /usr/src/redis-4.0.11/

make

make MALLOC=jemalloc

make PREFIX=/usr/local/redis install

cd /usr/local/redis/

ls

mkdir -p /usr/local/redis/conf

cp /usr/src/redis-4.0.11/redis.conf /usr/local/redis/conf/

cp /usr/src/redis-4.0.11/sentinel.conf /usr/local/redis/conf/

ln -s /usr/local/redis/bin/* /usr/local/bin/

which redis-server

三台都进行配置文件优化和简单的基础调优

cd /usr/local/redis

cp conf/redis.conf{,.bak}

egrep -v "^$|^#" conf/redis.conf.bak > conf/redis.conf

mkdir -p /data/redis/ #创建redis数据目录

image_1d01u5lii1rpudsumtles01s589c.png-20.8kB

修改配置文件

cat -n conf/redis.conf | sed -n '1p;3p;4p;7p;9p;11p;21p'

1 bind 127.0.0.1
3 port 6379
4 tcp-backlog 511
7 daemonize no
9 pidfile /var/run/redis_6379.pid
11 logfile ""
21 dir ./

image_1d01u6e6ql9p1fopk4u33n1uqfa6.png-23.7kB

修改成以下设置

vim conf/redis.conf

cat -n conf/redis.conf | sed -n '1p;3p;4p;7p;9p;11p'

1 bind 0.0.0.0 #监听地址
3 port 6379 #监听端口
4 tcp-backlog 1024 #tcp连接数
7 daemonize yes #是否后台启动
9 pidfile /data/redis/redis.pid #pid存放目录
11 logfile "/data/redis/redis.log" #日志存放目录
21 dir /data/redis/ #工作目录

image_1d01uhe8u1thktuk1e6k110e2qbb0.png-45.3kB

进行基础调优设置

echo "* - nofile 10240" >> /etc/security/limits.conf

echo "net.core.somaxconn = 10240" >> /etc/sysctl.conf

echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf

sysctl -p

echo never > /sys/kernel/mm/transparent_hugepage/enabled

echo never > /sys/kernel/mm/transparent_hugepage/defrag

echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local

echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local

image_1d01um5uj14ag15l613dru1p1av99.png-50.1kB

三台的redis-server都启动(上文实验已经启动过redis-master)

redis-server /usr/local/redis/conf/redis.conf

netstat -antup | grep redis

image_1d01une5qu9k17bq9u41mca11lam.png-19.6kB

image_1d01uo5v6n8qln6l3hhjm18fe13.png-18.5kB

14.2 redis主从同步服务器搭建

redis的主从同步，不用修改master任何配置
只需要在redis-slave上指定master的IP地址即可

先启动redis-master，然后再在两个redis-slave上都进行如下操作

redis-cli shutdown

echo "SLAVEOF 192.168.200.70 6379" >> /usr/local/redis/conf/redis.conf

> /data/redis/redis.log

redis-server /usr/local/redis/conf/redis.conf

netstat -antup | grep redis

image_1d01usnco1k0j1mfbp561nq71f501g.png-37.5kB

image_1d01uu9uv1ed6jq9106urt1pof1t.png-37.1kB

14.3 主从同步日志分析（全量同步）

查看redis-slave同步日志

cat /data/redis/redis.log

18341:C 11 Aug 13:56:54.895 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
18341:C 11 Aug 13:56:54.895 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=18341, just started
18341:C 11 Aug 13:56:54.895 # Configuration loaded
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 4.0.11 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
| `-._ `._ / _.-' | PID: 18342
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
18342:S 11 Aug 13:56:54.897 # Server initialized #服务器初始化
18342:S 11 Aug 13:56:54.897 * DB loaded from disk: 0.000 seconds #数据从磁盘加载0秒
18342:S 11 Aug 13:56:54.897 * Ready to accept connections #准备接受连接
18342:S 11 Aug 13:56:54.897 * Connecting to MASTER 192.168.200.165:6379 #链接到主192.168.200.165：6379
18342:S 11 Aug 13:56:54.897 * MASTER <-> SLAVE sync started #主从同步开始
18342:S 11 Aug 13:56:54.897 * Non blocking connect for SYNC fired the event. #非阻塞同步连接触发事件
18342:S 11 Aug 13:56:54.898 * Master replied to PING, replication can continue... #主应答，复制可以继续
18342:S 11 Aug 13:56:54.898 * Partial resynchronization not possible (no cached master) #部分同步不能（本机无缓存的主文件）
18342:S 11 Aug 13:56:54.899 * Full resync from master: e3adc85bd644e66bd1ee17b49c25e5e0491084d5:0 #进行全同步
18342:S 11 Aug 13:56:54.917 * MASTER <-> SLAVE sync: receiving 43606 bytes from master #从主接收43606字节
18342:S 11 Aug 13:56:54.917 * MASTER <-> SLAVE sync: Flushing old data #刷新旧数据
18342:S 11 Aug 13:56:54.917 * MASTER <-> SLAVE sync: Loading DB in memory #加载数据到内存
18342:S 11 Aug 13:56:54.918 * MASTER <-> SLAVE sync: Finished with success #同步完成

查看redis-master同步日志

cat /data/redis/redis.log

#从192.168.200.163:6379请求同步
26003:M 11 Aug 14:14:55.342 * Slave 192.168.200.163:6379 asks for synchronization
#从192.168.200.163:6379请求完整的重新同步
26003:M 11 Aug 14:14:55.342 * Full resync requested by slave 192.168.200.163:6379
#master启动bgsave与目标的磁盘进行同步
26003:M 11 Aug 14:14:55.342 * Starting BGSAVE for SYNC with target: disk
#后台保存rdb的进程的pid号为26128
26003:M 11 Aug 14:14:55.342 * Background saving started by pid 26128
#rdb文件已经保存到了磁盘
26128:C 11 Aug 14:14:55.344 * DB saved on disk
#rdb写时复制使用了0MB的内存
26128:C 11 Aug 14:14:55.344 * RDB: 0 MB of memory used by copy-on-write
#后台保存成功
26003:M 11 Aug 14:14:55.414 * Background saving terminated with success
#与从192.168.200.163:6379同步成功
26003:M 11 Aug 14:14:55.415 * Synchronization with slave 192.168.200.163:6379 succeeded

image_1d01v04g67iqbg71ear1mhhmgc2a.png-118.7kB

14.4 主从同步日志分析（部分同步）

清空master日志

> /data/redis/redis.log

image_1d01vb9066vq13451hmn19ro1v232n.png-8.4kB

清空slave日志，并shutdown在启动slave

> /data/redis/redis.log

redis-cli shutdown

redis-server /usr/local/redis/conf/redis.conf

netstat -antup | grep redis

image_1d01vfc7gh451uic1bfemsdmmu34.png-31.9kB

查看redis-slave日志

cat /data/redis/redis.log

18342:S 11 Aug 14:23:27.678 # User requested shutdown...
18342:S 11 Aug 14:23:27.678 * Saving the final RDB snapshot before exiting.
18342:S 11 Aug 14:23:27.681 * DB saved on disk
18342:S 11 Aug 14:23:27.681 * Removing the pid file.
18342:S 11 Aug 14:23:27.681 # Redis is now ready to exit, bye bye...
18417:C 11 Aug 14:24:41.432 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
18417:C 11 Aug 14:24:41.432 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=18417, just started
18417:C 11 Aug 14:24:41.432 # Configuration loaded
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 4.0.11 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
| `-._ `._ / _.-' | PID: 18418
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
#服务器初始化
18418:S 11 Aug 14:24:41.434 # Server initialized
#从磁盘加载旧数据用时0.001秒
18418:S 11 Aug 14:24:41.434 * DB loaded from disk: 0.001 seconds
#由于之前是一个从库，利用主的参数合成一个主的缓存，这样就可以和主仅仅进行一部分的数据同步
18418:S 11 Aug 14:24:41.434 * Before turning into a slave, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
#准备接受连接
18418:S 11 Aug 14:24:41.434 * Ready to accept connections
#连接到master192.168.200.165:6379
18418:S 11 Aug 14:24:41.435 * Connecting to MASTER 192.168.200.165:6379
#主从同步开始
18418:S 11 Aug 14:24:41.435 * MASTER <-> SLAVE sync started
#非阻塞同步连接触发事件
18418:S 11 Aug 14:24:41.435 * Non blocking connect for SYNC fired the event.
#master应答，复制可以继续
18418:S 11 Aug 14:24:41.435 * Master replied to PING, replication can continue...
#尝试进行部分同步（要求646f44ea31d2b057e919e08493f94e97e92007f2:911）
18418:S 11 Aug 14:24:41.435 * Trying a partial resynchronization (request 646f44ea31d2b057e919e08493f94e97e92007f2:911).
#成功进行部分同步
18418:S 11 Aug 14:24:41.436 * Successful partial resynchronization with master.
#master应答接受一个部分同步的请求
18418:S 11 Aug 14:24:41.436 * MASTER <-> SLAVE sync: Master accepted a Partial Resynchronization.

查看redis-master日志

cat /data/redis/redis.log

#一个从库192.168.200.164：6379请求进行同步
26003:M 11 Aug 14:24:41.424 * Slave 192.168.200.164:6379 asks for synchronization
#一个部分同步的请求来自192.168.200.164:6379，master已经接受请求，从偏移量为911处开始发送98字节的剩余数据
26003:M 11 Aug 14:24:41.424 * Partial resynchronization request from 192.168.200.164:6379 accepted. Sending 98 bytes of backlog starting from offset 911.

image_1d01viobnqts1m15ofm1ncoa5h3h.png-141kB

14.5 主从同步的停止

清空从库日志并停止从库主从同步（只能在从库上执行）

> /data/redis/redis.log

redis-cli slaveof no one

image_1d01vnh95hbscciiupgaf1fmg3u.png-13.2kB

在次查看日志

cat /data/redis/redis.log

#将第二次复制的ID设置为646f44ea31d2b057e919e08493f94e97e92007f2，有效偏移量:3571。新的复制ID为efb9d2d393bb0bb4570a2f3144e7b2fbfc60b892
18418:M 11 Aug 14:55:15.723 # Setting secondary replication ID to 646f44ea31d2b057e919e08493f94e97e92007f2, valid up to offset: 3571. New replication ID is efb9d2d393bb0bb4570a2f3144e7b2fbfc60b892
#与主库失去联系
18418:M 11 Aug 14:55:15.723 # Connection with master lost.
#主从状态缓存断开
18418:M 11 Aug 14:55:15.723 * Caching the disconnected master state.
#丢弃之前缓存的主的状态
18418:M 11 Aug 14:55:15.723 * Discarding previously cached master state.
#主模式启用
18418:M 11 Aug 14:55:15.723 * MASTER MODE enabled (user request from 'id=4 addr=127.0.0.1:39500 fd=8 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=slaveof')

image_1d01vpj7t26q15ht118i12bv1spd4b.png-52.1kB

清空redis日志并恢复从库主从同步

> /data/redis/redis.log

redis-cli slaveof 192.168.200.70 6379

image_1d01vs9k21e1l1cn11mu815sk1lgf4o.png-13.9kB

查看slave日志

cat /data/redis/redis.log

18418:S 11 Aug 15:02:40.330 * Before turning into a slave, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
18418:S 11 Aug 15:02:40.330 * SLAVE OF 192.168.200.165:6379 enabled (user request from 'id=5 addr=127.0.0.1:39502 fd=7 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=slaveof')
18418:S 11 Aug 15:02:40.928 * Connecting to MASTER 192.168.200.165:6379
18418:S 11 Aug 15:02:40.928 * MASTER <-> SLAVE sync started
18418:S 11 Aug 15:02:40.928 * Non blocking connect for SYNC fired the event.
18418:S 11 Aug 15:02:40.929 * Master replied to PING, replication can continue...
18418:S 11 Aug 15:02:40.929 * Trying a partial resynchronization (request efb9d2d393bb0bb4570a2f3144e7b2fbfc60b892:3571).
18418:S 11 Aug 15:02:40.929 * Full resync from master: 646f44ea31d2b057e919e08493f94e97e92007f2:4186
18418:S 11 Aug 15:02:40.929 * Discarding previously cached master state.
18418:S 11 Aug 15:02:40.967 * MASTER <-> SLAVE sync: receiving 43607 bytes from master
18418:S 11 Aug 15:02:40.968 * MASTER <-> SLAVE sync: Flushing old data
18418:S 11 Aug 15:02:40.968 * MASTER <-> SLAVE sync: Loading DB in memory
18418:S 11 Aug 15:02:40.970 * MASTER <-> SLAVE sync: Finished with success

image_1d01vso9b1m5b1vjs1gqgm6g11c955.png-102.6kB

14.6 加密的主从同步

（1）为redis-master平滑设置连接密码

redis-cli config get requirepass

redis-cli config set requirepass 'yunjisuan'

redis-cli config get requirepass

redis-cli -a yunjisuan config get requirepass

redis-cli -a yunjisuan config rewrite

image_1d020032t1p86o2s1jk915ee1qql5i.png-48.9kB

查看从库日志信息

cat /data/redis/redis.log

#主从同步需要进行端口验证请求
18418:S 11 Aug 15:08:40.440 * (Non critical) Master does not understand REPLCONF listening-port: -NOAUTH Authentication required.
18418:S 11 Aug 15:08:40.440 * (Non critical) Master does not
understand REPLCONF capa: -NOAUTH Authentication required.
#部分同步不能，没有主的缓存
18418:S 11 Aug 15:08:40.440 * Partial resynchronization not possible (no cached master)
#从主发来的意外回复：需要身份验证
18418:S 11 Aug 15:08:40.441 # Unexpected reply to PSYNC from master: -NOAUTH Authentication required.
#进行同步重试
18418:S 11 Aug 15:08:40.441 * Retrying with SYNC...
#主从复制终止：需要身份验证
18418:S 11 Aug 15:08:40.441 # MASTER aborted replication with an error: NOAUTH Authentication required.

image_1d0202eu31o8k1e831ld31lqtu995v.png-242.9kB

（2）为从库提供主从同步密码验证

从服务器需要设置主从同步的认证密码

redis-cli config get masterauth

redis-cli config set masterauth "yunjisuan"

redis-cli config get masterauth

redis-cli config rewrite

tail -1 /usr/local/redis/conf/redis.conf

image_1d0207fn41ai1520oin47k6v66c.png-33.7kB

查看从服务器日志

cat /data/redis/redis.log

18418:S 11 Aug 15:14:07.854 * Connecting to MASTER 192.168.200.165:6379
18418:S 11 Aug 15:14:07.854 * MASTER <-> SLAVE sync started
18418:S 11 Aug 15:14:07.855 * Non blocking connect for SYNC fired the event.
18418:S 11 Aug 15:14:07.856 * Master replied to PING, replication can continue...
18418:S 11 Aug 15:14:07.858 * Partial resynchronization not possible (no cached master)
18418:S 11 Aug 15:14:07.860 * Full resync from master: 646f44ea31d2b057e919e08493f94e97e92007f2:4648
18418:S 11 Aug 15:14:07.886 * MASTER <-> SLAVE sync: receiving 43607 bytes from master
18418:S 11 Aug 15:14:07.888 * MASTER <-> SLAVE sync: Flushing old data
18418:S 11 Aug 15:14:07.889 * MASTER <-> SLAVE sync: Loading DB in memory
18418:S 11 Aug 15:14:07.893 * MASTER <-> SLAVE sync: Finished with success
18418:S 11 Aug 15:14:18.778 # CONFIG REWRITE executed with success.

image_1d0208a9v18o1ob814k3ilrpl96p.png-226.2kB

十五，使用Python操作Redis单例

15.1 Python安装redis扩展

yum -y install epel-release

yum -y install python2-pip

pip install redis

image_1d020a22a11951gc81gru1ruf89876.png-95.2kB

image_1d020bbm15tqc166mruao10d17j.png-156kB

image_1d020fmqm841nfb19fb19mf1eo480.png-39.9kB

15.2 利用python进行redis数据的读写

python

Python 2.7.5 (default, Apr 11 2018, 07:36:10)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import redis
>>> r = redis.Redis(host='127.0.0.1',port=6379,password='yunjisuan',db=0)
>>> r.set('key_test','value_test')
True
>>> value = r.get('key_test')
>>> print (value)
value_test
>>> exit()

image_1d020oik9mag1ia61vojifn162e8d.png-36kB

redis-cli -a yunjisuan get key_test

Warning: Using a password with '-a' option on the command line interface may not be safe.
"value_test"

image_1d020p06h16rbtna11dr1vru19qs8q.png-16kB

posted @ 2019-07-24 15:20 L1n 阅读(4864) 评论(0) 编辑收藏举报

会员力量，点亮园子希望

刷新页面返回顶部

L1n

若不是生活所迫，谁愿意把自己弄的一身才华。

Redis4.0之持久化存储（下）

十，Redis的RDB存储方式

10.1 redis的运行方式说明

10.2 redis rdb存储方式，使用save配置开启rdb存储或者关闭rdb存储

10.3 设置开启或者关闭rdb存储

10.4 进行数据写入，观察rdb存储日志

输入1万条数据

查看日志信息

查看redis里有多少个键值（keys）

redis占用了多少内存

10.5 redis提供的bgsave命令能够立刻触发rdb存储，观察存储日志

查看日志

十一，Redis的AOF存储方式

11.1 动态开启或者关闭aof

11.2 写入数据，观察aof。多次运行，aof文件不断增大，rdb文件大小不变

查看aof和rdb文件大小

写入数据

查看aof和rdb文件大小

11.3 重写aof文件，整理相同的key，写入最后的有效值

清空aof文件

清空aof文件

重要提示

11.4 aof配置自动rewrite机制

在默认配置文件里，默认存在

获取aof-rewrite配置

进行aof自动重写测试

十二，Redis最大内存设置和删除算法

12.1 redis的键设置有效期，过期自动删除

12.2 查看设置最大内存

查看和设定最大内存限制

12.3 可选择的删除算法

12.4 模拟超过内存

12.5 设置删除算法

将删除算法设置为volatile-lru

算法测试

十三，Redis禁用屏蔽危险命令

13.1 redis禁用的命令

13.2 redis禁用危险命令配置代码如下（写入配置文件即可，此配置无法平滑更新）

将配置加入redis.conf配置文件

13.3 登陆redis，运行禁止命令进行测试

重启redis-server

测试被屏蔽的危险命令

十四，Redis主从服务器环境的搭建

14.1 环境要求与redis基础编译部署调优

操作系统环境要求

三台redis都进行如下编译过程(安装过程略)

三台都进行配置文件优化和简单的基础调优

修改配置文件

修改成以下设置

进行基础调优设置

三台的redis-server都启动(上文实验已经启动过redis-master)

14.2 redis主从同步服务器搭建

先启动redis-master，然后再在两个redis-slave上都进行如下操作

14.3 主从同步日志分析（全量同步）

查看redis-slave同步日志

查看redis-master同步日志

14.4 主从同步日志分析（部分同步）

清空master日志

清空slave日志，并shutdown在启动slave

查看redis-slave日志

查看redis-master日志

14.5 主从同步的停止

清空从库日志并停止从库主从同步（只能在从库上执行）

在次查看日志

清空redis日志并恢复从库主从同步

查看slave日志

14.6 加密的主从同步

（1）为redis-master平滑设置连接密码

查看从库日志信息

（2）为从库提供主从同步密码验证

从服务器需要设置主从同步的认证密码

查看从服务器日志

十五，使用Python操作Redis单例

15.1 Python安装redis扩展

15.2 利用python进行redis数据的读写

公告