SGwanmin

在学习C语言和数据结构，，， 日了，上学时候没感觉啥的，学的越多发现得调头翻大学教材了，啊，西八！

导航

Redis进阶

Redis进阶-avpt

Redis-EFCore-logger Code 使用Demo 已上传至github：https://github.com/Seth-Song/EFCore_Redis_logger

1.数据持久化-RDB&AOF

为了保证Redis数据不受宕机影响，所以Redis提供了RDB和AOF数据持久化的功能，它根据使用情况，可以每隔一定时间对现有数据做一个快照，并且导出到磁盘 ， 或者追加到命令日志中，redis持久化的加载顺序：
1） AOF持久化开启且存在AOF文件时，优先加载AOF文件，
2） AOF关闭或者AOF文件不存在时，加载RDB文件，
3） 加载AOF/RDB文件成功后，Redis启动成功。
4） AOF/RDB文件存在错误时，Redis启动失败并打印错误信息

1.RDB

RDB的原理是每间隔一段时间，就“fork”一份与原来进程内容一致的新的进程。这个新的进程的所有数据（变量、环境变量等）内容都与原进程一致，并作为原进程的子进程工作。这个新的进程会把数据快照先写入到一份临时文件中，在这个过程完成后再将这个临时文件替换到磁盘中的存储文件中。

下面是相关配置信息的注解:

################################ SNAPSHOTTING（快照）################################
## save 是指定执行rdb储存的条件， save time numer
# 下面的语句既是 15分钟保存了一次，执行rdb持久化
# 如果5分钟内有10个key的内容被修改，那么执行rdb持久化一次
# 如果一分钟内容redis更改了10000次数据，也会执行一次rdb持久化

## 如果要禁用RDB，不配置任何save或者写一个save "" 空字符串即可
save 900 1
save 300 10
save 60 10000

# 该配置默认是yes，见名思意即是在bgsave过程中如果出现了问题，
# 那么就会中断，后面的数据也就不再持久化了
stop-writes-on-bgsave-error no

# 当导出到 .rdb 数据库时是否用LZF压缩字符串对象。
# 默认设置为 "yes"，所以几乎总是生效的。
# 如果你想节省CPU的话你可以把这个设置为 "no"，但是如果你有可压缩的key的话，那数据文件就会更大了。
rdbcompression yes

# 对rdb数据进行校验，耗费CPU资源，默认为yes
rdbchecksum yes

# 数据文件的名字，可自定义
dbfilename dump.rdb

# 数据文件存储的地方，默认是 ./  代表会在启动redis的当前路径下执行持久化并存放dump.rdb文件
dir ./

原文件中的内容（英文版）：

################################ SNAPSHOTTING  ################################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#
#   Note: you can disable saving completely by commenting out all "save" lines.
#
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument
#   like in the following example:
#
#   save ""

save 900 1
save 300 10
save 60 10000

# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes

# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes

# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes

# The filename where to dump the DB
dbfilename dump.rdb

# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir ./

特点：体积小，性能高，恢复速度快,；故障丢失，大量更新可能造成CPU吃紧

2.AOF（APPEND ONLY MODE）

记录每次对服务器写的操作，当服务器重启的时候会重新执行这些命令来恢复原始的数据。Redis 还能对 AOF 文件进行后台重写，使得 AOF 文件的体积不至于过大。

下面是相关配置信息的注解:

appendonly yes
appendfilename "appendonly.aof"
#appendfsync always               #(慢<-->安全)
appendfsync everysec              #(default, 快，故障时只丢失一秒数据)
#appendfsync no
no-appendfsync-on-rewrite yes     #(aof重写期间主进程是否继续讲日志从缓冲区刷新到旧日志文件)
auto-aof-rewrite-percentage 100   #(文件增长率)
auto-aof-rewrite-min-size 64mb    #(aof重写需要的文件)

如果使用appendfsync always，就会在每次更新时都对AOF日志文件进行更新，严重影响Redis的存取效率，但是如果系统宕机不会丢失数据。

一般情况推荐使用appendfsync everysec，每秒钟触发一次更新，新的线程会将一秒内的数据更新写到日志文件中，既不会影响Redis的存取效率，但是如果系统宕机也最多丢失1秒数据。

特点：日志文件可读性搞，适合保存增量数据，数据不丢失；文件体积大，恢复时间长。

原文件中的内容（英文版）：

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.

appendonly no

# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log . Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yesh

混合持久化

混合持久化appendonly.aof文件由rdb格式和aof指令格式两大部分组成。AOF根据配置规则在后台自动重写aof文件时，会先把对应的RDB文件写入新的AOF文件中，然后再将RDB之后的增量日志写到AOF中，这样就可以发挥RDB和AOF各自的优势，RDB做全量持久化，AOF做增量持久化，既保证了存取效率，还保证数据安全不丢失。

使用混合持久化，需要开启以下设置，Redis 5.0之后这个设置默认开启，只需要同时开启RDB和AOF即可。

aof-use-rdb-preamble yes

2.Redis集群

使用Redis缓存服务器非常方便，但是一旦Redis宕机，对系统的影响也会很大，所以Redis服务器的稳定性也非常重要

主从复制

当从库和主库建立MS关系后,会向主数据库发送SYNC命令，主库接收到SYNC命令后会开始在后台保存快照,并将期间接收到的写命令缓存起来，当快照完成后,主Redis会将快照文件和所有缓存的写命令发送给从Redis，从Redis接收到后,会载入快照文件并且执行收到的缓存的命令，之后,主Redis每当接收到写命令时就会将命令发送从Redis，从而保证数据的一致。

#Master: 192.168.56.11
bind 192.168.56.11                 # 使得Redis服务器可以跨网络访问
port 6379
requirepass demo12!@               # 设置密码

#Slave01: 192.168.56.12
bind 192.168.56.12
port 6379
replicaof 192.168.56.11 6379       # 指定主服务器，注意：有关slaveof的配置只是配置从服务器，主服务器不需要配置
masterauth demo12!@                # 主服务器密码，注意：有关slaveof的配置只是配置从服务器，主服务器不需要配置

#Slave02:192.168.56.13
bind 192.168.56.13
port 6379
replicaof 192.168.56.11 6379
masterauth demo12!@

启动主从Redis之后，redis会自动将数据同步到从服务器。

主从复制虽然可以保证数据可以自动Sync到其他备用Redis服务器上，但是并不能做到主从的自动切换。当主服务器宕机后，需要手动把一台从服务器切换为主服务器，这就需要人工干预，费事费力，还会造成一段时间内服务不可用。这不是一种推荐的方式，更多时候，我们优先考虑哨兵模式。

哨兵模式

哨兵模式是一种特殊的模式，首先Redis提供了哨兵的命令，哨兵是一个独立的进程，作为进程，它会独立运行。其原理是哨兵通过发送命令，等待Redis服务器响应，从而监控运行的多个Redis实例。然而一个哨兵进程对全部的Redis服务器进行监控，可能会出现问题，为此，我们通常使用多个哨兵进行监控。各个哨兵之间还会进行监控，这样就形成了多哨兵模式。

 

 

 哨兵模式的原理是在主从复制结构上实现自动切换主从关系的方案，如下表：

 

 

 

首先，需要按照上边介绍的主从复制，开启主从复制的Redis服务器；

然后需要配置哨兵模式，哨兵的配置文件如下：

# 哨兵进程端口
port 26379
# 哨兵进程服务临时文件夹，默认为/tmp，要保证有可写入的权限
dir "/tmp"
# 禁止保护模式
protected-mode yes
# 配置监听的主服务器，这里sentinel monitor代表监控，mymaster代表服务器的名称，可以自定义，192.168.11.128代表监控的主服务器，6379代表端口，
# 2代表只有两个或两个以上的哨兵认为主服务器不可用的时候，才会进行failover操作。
sentinel monitor mymaster 192.168.11.128 6379 2
# sentinel author-pass定义服务的密码，mymaster是服务名称，123456是Redis服务器密码
# sentinel auth-pass <master-name> <password>
sentinel auth-pass mymaster 123456

# 指定哨兵在监控Redis服务时，当Redis服务在一个默认毫秒数内都无法回答时，单个哨兵认为的主观下线时间，默认为30000（30秒）
sentinel down-after-milliseconds 30000
# 指定可以有多少个Redis服务同步新的主机，一般而言，这个数字越小同步时间越长，而越大，则对网络资源要求越高
sentinel parallel-syncs mymaster 1
# 指定故障切换允许的毫秒数，超过这个时间，就认为故障切换失败，默认为3分钟
sentinel failover-timeout mymaster 180000
# 指定sentinel检测到该监控的redis实例指向的实例异常时，调用的报警脚本。该配置项可选，比较常用
sentinel deny-scripts-reconfig yes

sentinel down-after-milliseconds配置项只是一个哨兵在超过规定时间依旧没有得到响应后，会自己认为主机不可用。哨兵会记录这个消息，当拥有认为主观下线的哨兵达到sentinel monitor所配置的数量时，就会发起一次投票，进行failover，此时哨兵会重写Redis的哨兵配置文件，以适应新场景的需要。

虽然哨兵模式解决了主从自动切换的问题，保证了缓存服务器的稳定性。但是对于每一台Redis缓存，都需要保存完整的缓存信息，如果缓存数据量大或者并发请求很多时，对主服务器的负荷还是很大，所以如果要是可以分担主服务器的请求压力就好了，这时最好使用Redis集群模式。

集群模式

Redis Cluster是由多个同时服务于一个数据集合的Redis实例组成的整体，对于用户来说，用户只关注这个数据集合，而整个数据集合的某个数据子集存储在哪个节点对于用户来说是透明的。Redis Cluster具有分布式系统的特点，也具有分布式系统如何实现高可用性与数据一致性的难点。

Redis Cluster特点如下：

  • 所有的节点相互连接；
  • 集群消息通信通过集群总线通信，集群总线端口大小为客户端服务端口+10000，这个10000是固定值；
  • 节点与节点之间通过二进制协议进行通信；
  • 客户端和集群节点之间通信和通常一样，通过文本协议进行；

Redis Cluster采用了分布式系统的分片（分区）的思路，每个主节点为一个分片，这样也就意味着 存储的数据是分散在所有分片中的。当增加节点或删除主节点时，原存储在某个主节点中的数据 会自动再次分配到其他主节点。在Redis Cluster中有一个概念slot，我们翻译为槽。Slot数量是固定的，为16384个。这些slot会均匀地分布到各个 节点上。另外Redis的键和值会根据hash算法存储在对应的slot中。简单讲，对于一个键值对，存的时候在哪里是通过 hash算法算出来的，那么取得时候也会算一下，知道值在哪个slot上。根据slot编号再到对应的节点上去取。

以下是集群模式的具体配置方法：

 

 使用三台虚拟机，每台虚拟机运行两个Redis实例，分别给出其中一主一从的6379和6380的部分需要修改的配置，其他两个Server都是一样的：

#redis-master1配置文件
bind 192.168.56.11
port 6379
protected-mode yes
#开启redis的集群模式
cluster-enabled yes
#配置集群模式下的配置文件
cluster-config-file nodes-6379.conf
#集群内节点之间支持最长响应时间
cluster-node-timeout 15000


#redis-slave01配置文件
bind 192.168.56.11
port 6380
protected-mode yes
#开启redis的集群模式
cluster-enabled yes
#配置集群模式下的配置文件
cluster-config-file nodes-6380.conf
#集群内节点之间支持最长响应时间
cluster-node-timeout 15000

置文件修改完毕之后，启动6个Redis服务，然后就可以运行下面命令来创建Cluster了。　

创建集群可能出现的错误：

redis-cli --cluster create 192.168.56.11:6379 192.168.56.11:6380 192.168.56.12:6379 192.168.56.12:6380 192.168.56.13:6379 192.168.56.13:6380 
--cluster-replicas 1

#这是由于创建集群中的某一个服务中曾经插入过数据，并且已经产生了持久化文件，此时需要flushall命令清空所有数据
[ERR] Node 127.0.0.1:6380 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0

#这是由于之前创建集群遗留的配置文件导致的问题，使用命令cluster reset即可
redis-4.1.0/lib/redis/client.rb:124:in `call': ERR Slot 935 is already busy

以下是创建成功的信息：

# redis-cli --cluster create 192.168.56.11:6379 192.168.56.11:6380 192.168.56.12:6379 192.168.56.12:6380 192.168.56.13:6379 192.168.56.13:6380 
# --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460		#槽位的分配
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.56.12:6380 to 192.168.56.11:6379		#一主对应一从，一般情况下，相同IP不会互相主从
Adding replica 192.168.56.11:6380 to 192.168.56.12:6379
Adding replica 192.168.56.13:6380 to 192.168.56.13:6379
>>> Trying to optimize slaves allocation for anti-affinity
[OK] Perfect anti-affinity obtained!
M: 31886d2098fb1e627bd71b5af000957a1e252787 192.168.56.11:6379
   slots:[0-5460] (5461 slots) master
S: be0ef4a1b1a60cee781afe5c2b8b5cbd7b68b4e6 192.168.56.11:6380
   replicates 8cd40e6a31c12e0b9c01f20056b9ecaa4db51446
M: 8cd40e6a31c12e0b9c01f20056b9ecaa4db51446 192.168.56.12:6379
   slots:[5461-10922] (5462 slots) master
S: eb812dc6051776e151bf69cd328bd0a66a20de01 192.168.56.12:6380
   replicates 587adfa041d0c0a14aa1a875bdec219a56b10201
M: 587adfa041d0c0a14aa1a875bdec219a56b10201 192.168.56.13:6379
   slots:[10923-16383] (5461 slots) master
S: 55a6f654dcb87c6a8017c8619f0ce8763a92abd6 192.168.56.13:6380
   replicates 31886d2098fb1e627bd71b5af000957a1e252787
Can I set the above configuration? (type 'yes' to accept): yes	#是否接受这样的配置，填yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 192.168.56.11:6379)
M: 31886d2098fb1e627bd71b5af000957a1e252787 192.168.56.11:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 55a6f654dcb87c6a8017c8619f0ce8763a92abd6 192.168.56.13:6380
   slots: (0 slots) slave
   replicates 31886d2098fb1e627bd71b5af000957a1e252787
M: 8cd40e6a31c12e0b9c01f20056b9ecaa4db51446 192.168.56.12:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: be0ef4a1b1a60cee781afe5c2b8b5cbd7b68b4e6 192.168.56.11:6380
   slots: (0 slots) slave
   replicates 8cd40e6a31c12e0b9c01f20056b9ecaa4db51446
S: eb812dc6051776e151bf69cd328bd0a66a20de01 192.168.56.12:6380
   slots: (0 slots) slave
   replicates 587adfa041d0c0a14aa1a875bdec219a56b10201
M: 587adfa041d0c0a14aa1a875bdec219a56b10201 192.168.56.13:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

然后Redis集群就创建成功了，连任意节点都可以使用Redis服务了。期间任何主服务异常断开，都会自动将对应的主服务器和从服务器倒置来保证Redis服务器的正常使用。

集群使用过程中，也可以动态的添加或者删除节点，例如需要再添加一主一从服务器，而在生产环境当中，为了保证Redis的高可用性，通常会配置一主两从的模式，也就是说在创建集群时该参数提示的副本数为2，如：--cluster-replicas 2。

 

 首先修改配置文件，启动Redis服务，然后就可以运行以下命令来添加到集群中了：

#将192.168.56.14:6379节点增加为集群的主节点，命令如下：
redis-cli --cluster add-node 192.168.56.14:6379 192.168.56.11:6379

#将192.168.56.14:6380节点增加为集群的从节点，命令如下：
redis-cli --cluster add-node 192.168.56.14:6380 192.168.56.11:6379 --cluster-slave

#指定的master节点添加从节点，如将192.168.56.14:6380实例指定添加为192.168.56.14:6379实例的从节点
redis-cli --cluster add-node 192.168.56.14:6380 192.168.56.14:6379 
--cluster-slave --cluster-master-id d04ed6a7d3dbf0aaff0643b045fe22efc7c34500

#检查集群状态，可以看到192.168.56.11:6381已经添加为集群中的master
redis-cli --cluster check 192.168.56.11:6379
192.168.56.11:6379 (31886d20...) -> 2 keys | 5461 slots | 1 slaves.
192.168.56.12:6379 (8cd40e6a...) -> 0 keys | 5462 slots | 1 slaves.
192.168.56.14:6379 (d04ed6a7...) -> 0 keys | 0 slots | 1 slaves.
192.168.56.13:6379 (587adfa0...) -> 1 keys | 5461 slots | 1 slaves.
[OK] 3 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.56.11:6379)
M: 31886d2098fb1e627bd71b5af000957a1e252787 192.168.56.11:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 55a6f654dcb87c6a8017c8619f0ce8763a92abd6 192.168.56.13:6380
   slots: (0 slots) slave
   replicates 31886d2098fb1e627bd71b5af000957a1e252787
M: 8cd40e6a31c12e0b9c01f20056b9ecaa4db51446 192.168.56.12:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 91c803049aa0a12678e26521959aba25d4b06913 192.168.56.14:6380
   slots: (0 slots) slave
   replicates d04ed6a7d3dbf0aaff0643b045fe22efc7c34500
S: be0ef4a1b1a60cee781afe5c2b8b5cbd7b68b4e6 192.168.56.11:6380
   slots: (0 slots) slave
   replicates 8cd40e6a31c12e0b9c01f20056b9ecaa4db51446
S: eb812dc6051776e151bf69cd328bd0a66a20de01 192.168.56.12:6380
   slots: (0 slots) slave
   replicates 587adfa041d0c0a14aa1a875bdec219a56b10201
M: d04ed6a7d3dbf0aaff0643b045fe22efc7c34500 192.168.56.14:6379
   slots: (0 slots) master
   1 additional replica(s)
M: 587adfa041d0c0a14aa1a875bdec219a56b10201 192.168.56.13:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

从上面的操作上，我们可以看到新增的master 6381实例上的slot为0，那么这里我们先给这个master进行分配一些slot，需要注意的是，reshard后面跟的ip，可以是集群中任一master ip。　

redis-cli --cluster reshard 192.168.56.11:6379
>>> Performing Cluster Check (using node 192.168.56.11:6379)
M: 31886d2098fb1e627bd71b5af000957a1e252787 192.168.56.11:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 55a6f654dcb87c6a8017c8619f0ce8763a92abd6 192.168.56.13:6380
   slots: (0 slots) slave
   replicates 31886d2098fb1e627bd71b5af000957a1e252787
M: 8cd40e6a31c12e0b9c01f20056b9ecaa4db51446 192.168.56.12:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 91c803049aa0a12678e26521959aba25d4b06913 192.168.56.11:6382
   slots: (0 slots) slave
   replicates d04ed6a7d3dbf0aaff0643b045fe22efc7c34500
S: be0ef4a1b1a60cee781afe5c2b8b5cbd7b68b4e6 192.168.56.11:6380
   slots: (0 slots) slave
   replicates 8cd40e6a31c12e0b9c01f20056b9ecaa4db51446
S: eb812dc6051776e151bf69cd328bd0a66a20de01 192.168.56.12:6380
   slots: (0 slots) slave
   replicates 587adfa041d0c0a14aa1a875bdec219a56b10201
M: d04ed6a7d3dbf0aaff0643b045fe22efc7c34500 192.168.56.11:6381
   slots: (0 slots) master
   1 additional replica(s)
M: 587adfa041d0c0a14aa1a875bdec219a56b10201 192.168.56.13:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 100	#定义要分配多少slot，这里分配100个
What is the receiving node ID? d04ed6a7d3dbf0aaff0643b045fe22efc7c34500	#定义接收slot的node id，即新的master id
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: 31886d2098fb1e627bd71b5af000957a1e252787	#定义第一个源master的id，如果想在所有master上拿slot，直接敲all
Source node #2: 587adfa041d0c0a14aa1a875bdec219a56b10201	##定义第二个源master的id，如果不再继续有新的源，直接敲done
Source node #3: done
Ready to move 100 slots.
  Source nodes:
    M: 31886d2098fb1e627bd71b5af000957a1e252787 192.168.56.11:6379
       slots:[0-5460] (5461 slots) master
       1 additional replica(s)
    M: 587adfa041d0c0a14aa1a875bdec219a56b10201 192.168.56.13:6379
       slots:[10923-16383] (5461 slots) master
       1 additional replica(s)
  Destination node:
    M: d04ed6a7d3dbf0aaff0643b045fe22efc7c34500 192.168.56.11:6381
       slots: (0 slots) master
       1 additional replica(s)
  Resharding plan:
    Moving slot 0 from 31886d2098fb1e627bd71b5af000957a1e252787
    Moving slot 1 from 31886d2098fb1e627bd71b5af000957a1e252787
	......
Do you want to proceed with the proposed reshard plan (yes/no)? yes
Moving slot 0 from 192.168.56.11:6379 to 192.168.56.14:6379: 
Moving slot 1 from 192.168.56.11:6379 to 192.168.56.14:6379: 
Moving slot 2 from 192.168.56.11:6379 to 192.168.56.14:6379: 
Moving slot 3 from 192.168.56.11:6379 to 192.168.56.14:6379:

如果集群中有节点出现故障，也可以将对应节点从集群中移除，但是节点上的Redis必须不带有slot，如果有slot，需要先进行移除，否则会报错。被删除的node重启后，依然会记得集群中的其他节点，这就需要执行cluster forget nodeid来忘记其他节点。

#这里先删除192.168.56.14:6380这个从节点实例，因为该实例上没有slot，是可以直接移除的
redis-cli --cluster del-node 192.168.56.14:6380 91c803049aa0a12678e26521959aba25d4b06913
>>> Removing node 91c803049aa0a12678e26521959aba25d4b06913 from cluster 192.168.56.14:6380
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

#那么现在再来删除带有slot的192.168.56.14:6379实例，则会出现报错，如下
redis-cli --cluster del-node 192.168.56.14:6379 d04ed6a7d3dbf0aaff0643b045fe22efc7c34500
>>> Removing node d04ed6a7d3dbf0aaff0643b045fe22efc7c34500 from cluster 192.168.56.14:6379
[ERR] Node 192.168.56.14:6379 is not empty! Reshard data away and try again.

#提示该实例不为空，需要Reshard掉数据后再重试，那么现在先将slot进行迁移掉
redis-cli --cluster reshard 192.168.56.14:6379
>>> Performing Cluster Check (using node 192.168.56.14:6379)
M: d04ed6a7d3dbf0aaff0643b045fe22efc7c34500 192.168.56.14:6379
   slots:[0-49],[10923-10972] (100 slots) master
......
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 100
What is the receiving node ID? 587adfa041d0c0a14aa1a875bdec219a56b10201
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: d04ed6a7d3dbf0aaff0643b045fe22efc7c34500
Source node #2: done

#重新查看192.168.56.14:6379的slots已经是0了
redis-cli --cluster check 192.168.56.11:6379
192.168.56.11:6379 (31886d20...) -> 2 keys | 5411 slots | 1 slaves.
192.168.56.12:6379 (8cd40e6a...) -> 0 keys | 5462 slots | 1 slaves.
192.168.56.11:6381 (d04ed6a7...) -> 0 keys | 0 slots | 0 slaves.	#slots数量已经为0
192.168.56.13:6379 (587adfa0...) -> 1 keys | 5511 slots | 1 slaves.

# redis-cli --cluster del-node 192.168.56.14:6379 d04ed6a7d3dbf0aaff0643b045fe22efc7c34500
>>> Removing node d04ed6a7d3dbf0aaff0643b045fe22efc7c34500 from cluster 192.168.56.14:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

如果slot不均匀分配，也可以使用平衡功能，将各个节点的slot进行重新均衡分配：

redis-cli --cluster rebalance --cluster-threshold 1 192.168.56.11:6379
>>> Performing Cluster Check (using node 192.168.56.11:6379)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Rebalancing across 3 nodes. Total weight = 3.00
Moving 1046 slots from 192.168.56.13:6379 to 192.168.56.11:6379
###################################################################################################
Moving 1004 slots from 192.168.56.13:6379 to 192.168.56.12:6379
###################################################################################################

[root@redis-master ~]# redis-cli --cluster check 192.168.56.11:6379
192.168.56.11:6379 (31886d20...) -> 2 keys | 5462 slots | 1 slaves.
192.168.56.12:6379 (8cd40e6a...) -> 0 keys | 5461 slots | 1 slaves.
192.168.56.13:6379 (587adfa0...) -> 1 keys | 5461 slots | 1 slaves.
......

#需要注意的是，所有节点的平衡阈值在1%，当各个节点的slot数值相差不到1%时，进行balance是会报以下提示的：
*** No rebalancing needed! All nodes are within the 1.00% threshold.

也可以将外部集群数据导入集群中：

redis-cli --cluster import 192.168.56.11:6379 --cluster-from 192.168.56.14:6379 --cluster-copy
>>> Importing data from 192.168.56.11:6381 to cluster 192.168.56.11:6379
>>> Performing Cluster Check (using node 192.168.56.11:6379)
M: 31886d2098fb1e627bd71b5af000957a1e252787 192.168.56.11:6379
   slots:[0-5461] (5462 slots) master
   1 additional replica(s)
S: 55a6f654dcb87c6a8017c8619f0ce8763a92abd6 192.168.56.13:6380
   slots: (0 slots) slave
   replicates 31886d2098fb1e627bd71b5af000957a1e252787
M: 8cd40e6a31c12e0b9c01f20056b9ecaa4db51446 192.168.56.12:6379
   slots:[5462-10922] (5461 slots) master
   1 additional replica(s)
S: be0ef4a1b1a60cee781afe5c2b8b5cbd7b68b4e6 192.168.56.11:6380
   slots: (0 slots) slave
   replicates 8cd40e6a31c12e0b9c01f20056b9ecaa4db51446
S: eb812dc6051776e151bf69cd328bd0a66a20de01 192.168.56.12:6380
   slots: (0 slots) slave
   replicates 587adfa041d0c0a14aa1a875bdec219a56b10201
M: 587adfa041d0c0a14aa1a875bdec219a56b10201 192.168.56.13:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
*** Importing 2 keys from DB 0
Migrating k5 to 192.168.56.13:6379: OK
Migrating k0 to 192.168.56.12:6379: OK

如果只使用cluster-copy，则要导入集群中的key不能在，否则 如果集群中已有同样的key，如果需要替换，可以cluster-copy和cluster-replace联用，这样集群中的key就会被替换为外部的。　

redis的过期策略以及内存淘汰机制

比如一种情况，你redis只能存5G数据，可是你写了10G，那会删5G的数据。怎么删的，这个问题思考过么？还有，你的数据已经设置了过期时间，但是时间到了，内存占用率还是比较高，有思考过原因么?
redis采用的是定期删除+惰性删除策略。
为什么不用定时删除策略?
定时删除,用一个定时器来负责监视key,过期则自动删除。虽然内存及时释放，但是十分消耗CPU资源。在大并发请求下，CPU要将时间应用在处理请求，而不是删除key,因此没有采用这一策略.
定期删除+惰性删除是如何工作的呢?
定期删除，redis默认每个100ms检查，是否有过期的key,有过期key则删除。需要说明的是，redis不是每个100ms将所有的key检查一次，而是随机抽取进行检查(如果每隔100ms,全部key进行检查，redis岂不是卡死)。因此，如果只采用定期删除策略，会导致很多key到时间没有删除。
于是，惰性删除派上用场。也就是说在你获取某个key的时候，redis会检查一下，这个key如果设置了过期时间那么是否过期了？如果过期了此时就会删除。
采用定期删除+惰性删除就没其他问题了么?
不是的，如果定期删除没删除key。然后你也没及时去请求key，也就是说惰性删除也没生效。这样，redis的内存会越来越高。那么就应该采用内存淘汰机制。

# maxmemory-policy allkeys-lru

1）noeviction：当内存不足以容纳新写入数据时，新写入操作会报错。应该没人用吧。
2）allkeys-lru：当内存不足以容纳新写入数据时，在键空间中，移除最近最少使用的key。推荐使用。
3）allkeys-random：当内存不足以容纳新写入数据时，在键空间中，随机移除某个key。应该也没人用吧，你不删最少使用Key,去随机删。
4）volatile-lru：当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，移除最近最少使用的key。这种情况一般是把redis既当缓存，又做持久化存储的时候才用。不推荐
5）volatile-random：当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，随机移除某个key。依然不推荐
6）volatile-ttl：当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，有更早过期时间的key优先移除。不推荐
ps：如果没有设置 expire 的key, 不满足先决条件(prerequisites); 那么 volatile-lru, volatile-random 和 volatile-ttl 策略的行为, 和 noeviction(不删除) 基本上一致。　

Security

开启protected mode，设置参数protected-mode为yes；修改Redis服务器的端口号，不使用默认的6379；设置client连接密码

port 3378
protected-mode yes
requirepass demo12!@

disable或者重命名一些不安全命令：

rename-command config config-lmso-redis
rename-command shutdown shutdown-lmso-redis
rename-command flushdb ""
rename-command flushall ""
rename-command keys keys-lmso-redis

enable TLS/SSL

Redis从版本6开始支持SSL / TLS，这是一项可选功能，需要在编译时启用。目前Redis对Windows的支持已经停止更新，现有的Windows版本不支持激活TLS，可以在Windows 10机器上安装Docker，然后下来Redis镜像来运行最新版本的Redis.

#指定端口0以完全禁用非TLS端口，仅启用TLS
port 0
#tls-port配置可在指定端口上接受SSL / TLS连接。这是侦听TCP连接的额外端口，因此可以同时使用TLS和非TLS连接访问不同端口上的Redis实例。
tls-port 3378
#为了支持TLS，必须为Redis配置X.509证书和私钥
tls-cert-file /path/to/redis.crt
tls-key-file /path/to/redis.key
#在验证证书时，必须指定可信任的根CA证书文件或路径
tls-ca-cert-file /path/to/ca.crt
#为了支持基于DH的密码，还可以配置DH params文件
tls-dh-params-file /path/to/redis.dh
#对于从服务器，我们需要设置tls-replication为yes来将Redis的TLS connection转发到主服务器。
tls-replication yes
#如果使用集群，需要设置tls-cluster为yes来激活cluster之间的TLS通信。
tls-cluster yes
#默认情况下，Redis使用双向TLS并要求客户端使用有效证书进行身份验证，为了方便，可以禁用客户端身份验证
tls-auth-clients no

 

posted on 2024-07-17 10:32  GwanMin  阅读(149)  评论(0)    收藏  举报