keepalived实战案例
keepalived实战案例
例1：简单keepalived集群（主备模式）
| 节点 | ip |
|---|---|
| node1 | 1.1.1.10 |
| node2 | 1.1.1.20 |
1）先改/etc/hosts文件，添加ip和主机名
echo -e '1.1.1.10 node1\n1.1.1.20 node2' >> /etc/hosts
2）node1、node2安装keepalived
yum install -y keepalived
cp /etc/keepalived/keepalived.conf{,_bak}
3）改node1的配置文件
需要删除virtual server以下的配置段
cd /etc/keepalived
vim keepalived.conf
global_defs {
notification_email {
root@localhost
}
notification_email_from keep@localhost
smtp_server 127.0.0.1 #邮件服务器改本地
smtp_connect_timeout 30
router_id node1 #虚拟路由器的名称
vrrp_skip_check_adv_addr
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.1 #组播地址,同一组内的虚拟路由器要统一
}
vrrp_instance VI_1 {
state BACKUP #状态要改
interface eth1 #设置的网卡
virtual_router_id 1 #虚拟路由器的标识符要统一
priority 90 #优先级要一个node高一个node低
advert_int 1
authentication {
auth_type PASS
auth_pass qwe123 #认证密码要改
}
virtual_ipaddress {
1.1.1.100 dev eth1 label eth1:0 #设置VIP
}
}
systemctl start keepalived #启动服务即可
scp keep@localhost.conf root@1.1.1.20:/etc/keepalived/
4）改node2的配置文件：
#只需改唯一性的参数即可
vim /etc/keepalived/keepalived.conf
router_id node2
state MASTER
priority 100
systemctl start keepalived 启动服务即可
tcpdump -i ens160 -nn host 224.0.0.1 #抓包查看，可看到本地ip网卡一直向组播ip通告，且带有优先级
例2：简单集群（主/主模式）
| 节点 | ip |
|---|---|
| node1 | 1.1.1.10，vip：1.1.1.100、1.1.1.200 |
| node2 | 1.1.1.20 ，vip：1.1.1.300、1.1.1.400 |
1）node1复制keepalived.conf内容中的
在第一个案例的配置文件上改
vim keepalived.conf
vrrp_instance web1 {
state MASTER
virtual_router_id 2
priority 100
virtual_ipaddress {
1.1.1.100 dev ens160 label ens160:1
1.1.1.200 dev ens160 label ens160:2
}
}
vrrp_instance web2 {
state BACKUP
virtual_router_id 3
priority 80
virtual_ipaddress {
1.1.1.300 dev ens160 label ens160:3
1.1.1.400 dev ens160 label ens160:4
}
}
systemctl restart keepalived
2）node2复制相同的内容，做修改
vim keepalived.conf
vrrp_instance web1 {
state BACKUP
virtual_router_id 2
priority 100
virtual_ipaddress {
1.1.1.100 dev ens160 label ens160:1
1.1.1.200 dev ens160 label ens160:2
}
}
vrrp_instance web2 {
state MASTER
virtual_router_id 3
priority 80
virtual_ipaddress {
1.1.1.300 dev ens160 label ens160:3
1.1.1.400 dev ens160 label ens160:4
}
}
systemctl restart keepalived
例3：抢占模式和非抢占模式
默认为抢占模式preempt，即当高优先级的主机恢复在线后，会抢占低先级的主机的master角色，造成网络抖动
建议设置为非抢占模式nopreempt，即高优级主机恢复后，并不会抢占低优先级主机的master角色。生产模式建议使用
抢占模式：
注意：
- 尽量把各keepalived服务器state设为BACKUP
- 必须禁用vrrp_strict或者strict_mode
1）node1主机配置
global_defs {
notification_email {
root@localhost
}
notification_email_from keep@localhost
smtp_connect_timeout 30
router_id node1
vrrp_skip_check_adv_addr
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.1
}
vrrp_instance VI_1 {
state BACKUP
interface eth1
virtual_router_id 1
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass qwe123
}
virtual_ipaddress {
1.1.1.100 dev eth1 label eth1:0
}
}
2）node2主机配置
global_defs {
notification_email {
root@localhost
}
notification_email_from keep@localhost
smtp_connect_timeout 30
router_id node2
vrrp_skip_check_adv_addr
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.1
}
vrrp_instance VI_1 {
state MASTER
interface eth1
virtual_router_id 1
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass qwe123
}
virtual_ipaddress {
1.1.1.100 dev eth1 label eth1:0
}
}
非抢占模式：
注意：必须将各keepalived 服务器state配置为BACKUP
node1配置
#其他配置不变
vrrp_instance web1 {
state BACKUP
priority 100
nopreempt
...
}
node2配置
#其他配置不变
vrrp_instance web1 {
state BACKUP
priority 100
nopreempt
...
}
例4：单播通信
默认keepalived主机之间利用多播相互通告消息，会造成网络拥塞，可以替换成单播，减少网络流量
在云环境中，默认云平台是不允许使用广播的，必须腾讯云、阿里云需要单独申请使用广播
注意：
- 启用 vrrp_strict 时，不能启用单播
- 在所有节点vrrp_instance语句块中设置对方主机的IP
- 建议设置为专用于对应心跳线网络的地址，而非使用业务网络
node1设置
vim keepalived.conf
vrrp_instance VI_1 {
...
unicast_src_ip 1.1.1.25
unicast_peer {
1.1.1.10
1.1.1.20
}
}
node2配置
vrrp_instance VI_1 {
unicast_src_ip 1.1.1.10
unicast_peer {
1.1.1.25
1.1.1.20
}
}
例5：邮件通知脚本
当keepalived的状态变化时，可以自动触发脚本的执行，比如：发邮件通知用户
默认以用户keepalived_script身份执行脚本，如果此用户不存在，以root执行脚本
可以用指令指定脚本执行用户的身份:
script_user user
1）改keepalived配置文件
global_defs {
enable_script_security #启用脚本执行安全，必须开启
script_user root #脚本执行用户，默认keepalived_script用户，可不写但要创建user
}
vrrp_instance web1 {
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
2）写通知脚本，给所有node
vim notify.sh
#!/bin/bash
rpm -q mailx && rpm -q postfix
if [ $? != 0 ] ;then
yum install mailx postfix -y &>/dev/null
systemctl start postfix
fi
#contact='root@localhost'
contact='1137127273@qq.com'
notify() {
mailsubject="$(hostname) to be $1, vip floating"
mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
scp notify.sh root@1.1.1.20:/etc/keepalived/
3）邮箱设置
vim /mail.rc
set bsdcompat
set from=1137127273@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=1137127273@qq.com
set smtp-auth-password=mjebocrqrmgmbadf
set smtp-auth=login
例6：高可用lvs-dr模型
| 节点 | ip |
|---|---|
| VS1 | 1.1.1.25，VIP：1.1.1.100 |
| VS2 | 1.1.1.35，VIP：1.1.1.100 |
| RS1 | 1.1.1.10 |
| RS2 | 1.1.1.20 |
1）RS主机设置arp、ip。使用之前写的lvs脚本
参考文章中lvs-dr：lvs集群案例
2）VS1主机安装keepalived
yum install -y keepalived
cd /etc/keepalived/
vim keepalived.conf
virtual_server 1.1.1.100 80 {
delay_loop 1
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 1.1.1.10 80 {
weight 1
HTTP_GET {
url {
path /index.html
status_code 200
#使用digest 校验码，生成方法为：
#1. curl -s 1.1.1.10 |md5sum
#2. genhash -s 1.1.1.10 -p 80 -u /index.html
}
retry 3
delay_before_retry 2
connect_timeout 3
}
}
real_server 1.1.1.20 80 {
weight 1
HTTP_GET {
url {
path /index.html
status_code 200
}
retry 3
delay_before_retry 2
connect_timeout 3
}
}
}
scp keepalived.conf root@1.1.1.35:/etc/keepalived/
3）RS2主机安装、设置keepalived
RS1和RS2的notify.sh脚本用前面的案例，再修改优先级、主机名、状态就可以了
yum install -y keepalived
bash -x notify.sh start #先测试运行一遍，脚本是否可用，并安装邮件服务
4）所有VS主机都安装nginx，默认的页面做sorry页面，再启动keepalived
yum install -y nginx
systemctl start keepalived
例7：脚本实现主从切换
配合nginx使用
1）所有keepalived主机添加脚本
vim keepalived.conf
vrrp_script check_down {
script "/usr/bin/test ! -e /etc/keepalived/down"
interval 1
weight -30
fall 3
rise 2
timeout 2
}
vrrp_script check_down {
script "pidof nginx || touch /etc/keepalived/ngx && [ ! -e /etc/keepalived/ngx ]"
interval 1
weight -30
fall 3
rise 2
timeout 2
}
vrrp_instance web1 {
...
track_script {
chk_down
}
}
例8：高可用NGINX
| 节点 | ip |
|---|---|
| nginx1 | 2.2.2.25（外网） 1.1.1.25（内网） |
| nginx2 | 2.2.2.15（外网） 1.1.1.35（内网） |
| 后端服务 | 1.1.1.10（内网） |
1）两个nginx主机，配置keepalived
vim keepalived.conf
global_defs {
notification_email {
root@localhost
}
notification_email_from keep@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id gw
vrrp_skip_check_adv_addr
vrrp_garp_interval 1
vrrp_gna_interval 1
vrrp_mcast_group4 224.0.0.1
script_user root
}
vrrp_script chk_dw {
script "/etc/keepalived/chk_dw.sh"
weight -10
interval 1
fail 1
rise 1
user root
}
vrrp_script chk_ngx {
script "/etc/keepalived/chk_ngx.sh"
weight -10 #脚本退出码1时，降权
interval 2
fail 3
rise 3
user root
}
vrrp_instance VI_1 {
state MASTER
interface ens160
virtual_router_id 1
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass qwe123
}
virtual_ipaddress {
1.1.1.200 dev ens160 label ens160:0
}
track_script {
chk_dw
}
#生产环境可用，监听网卡是否可用
track_interface {
ens160
ens224
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
cd /etc/keepalived/
cat > /etc/keepalived/chk_dw.sh <<eof
#!/bin/bash
[[ -f /etc/keepalived/down ]] && exit 1||exit 0 '
eof
chmod +x chk_dw.sh
cat > /etc/keepalived/chk_ngx.sh <<eof
#!/bin/bash
pidof nginx &> /dev/null ||systemctl restart nginx
eof
chmod +x *.sh
systemctl restart keepalived
systemctl is-active nginx
2）编写notify.sh脚本,backup节点的
vim notify.sh
#!/bin/bash
rpm -q mailx && rpm -q postfix
if [ $? != 0 ] ;then
yum install mailx postfix -y &>/dev/null
systemctl start postfix
fi
#contact='root@localhost'
#contact='1137127273@qq.com'
notify() {
mailsubject="$(hostname) to be $1, vip floating"
mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
systemctl start nginx
notify master
;;
backup)
systemctl stop nginx
#systemctl start nginx backup节点要启用起来，否则主节点挂了，备也没有，权限比较后还是在主节点
notify backup
;;
fault)
systemctl stop nginx
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
例9：云环境搭建keepalived
腾讯云文档：https://cloud.tencent.com/document/product/215/20186
阿里云文档：https://help.aliyun.com/zh/vpc/user-guide/use-highly-available-virtual-ip#233a9d0442yvt
借鉴腾讯云的监听脚本：
#!/bin/bash
#/etc/keepalived/notify_action.sh
log_file=/var/log/keepalived.log
log_write()
{
echo "[`date '+%Y-%m-%d %T'`] $1" >> $log_file
}
[ ! -d /var/keepalived/ ] && mkdir -p /var/keepalived/
case "$1" in
"MASTER" )
echo -n "$1" > /var/keepalived/state
log_write " notify_master"
echo -n "0" /var/keepalived/vip_check_failed_count
;;
"BACKUP" )
echo -n "$1" > /var/keepalived/state
log_write " notify_backup"
;;
"FAULT" )
echo -n "$1" > /var/keepalived/state
log_write " notify_fault"
;;
"STOP" )
echo -n "$1" > /var/keepalived/state
log_write " notify_stop"
;;
*)
log_write "notify_action.sh: STATE ERROR!!!"
;;
esac
浙公网安备 33010602011771号