通过keepalived解决zerofs 文件服务ha 问题

默认zerofs 只能启动一个实例（早期版本），多实例是有问题的，但是日常中我们可能需要ha 的场景，keepalived是一个很不错的选择，以下是一个集成玩法

参考玩法

简单说明: 通过keepalived 的vrrp 实vip 的处理，同时利用在节点的lb 组件实现s3服务的负载均衡，对于默认zerofs 在master 中运行，backup 不运行，当master服务异常，或者停止了，backup 会启动zerofs，同时当keepalived 服务停止的时候部署的zerofs 也应该停止，上边的提到的问题，可以通过keepalived 的notify hook 能力解决，同时注意notify_stop 可能不生效（比较靠谱的解决方法是通过ssytemd 的ExecStopPost 确保当keepalived 服务停止了，zerofs服务也应该正常停止）

参考配置

• keepalived

master

 global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   enable_script_security
   script_user root
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL_dalong
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 83
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.31.200
    }
    notify_master /opt/notify/app_master.sh
    notify_backup /opt/notify/app_backup.sh
    notify_stop /opt/notify/app_stop.sh
}

backup

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   enable_script_security
   script_user root
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL_dalong
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 83
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
      192.168.31.200
    }
    notify_master /opt/notify/app_master.sh
    notify_backup /opt/notify/app_backup.sh
    notify_stop /opt/notify/app_stop.sh
}

app_master.sh

#!/bin/bash
sleep 2
echo "master" > /opt/notify/state.txt
systemctl  restart zerofs.service

app_backup.sh

#!/bin/bash
echo "backup" > /opt/notify/state.txt
systemctl  stop zerofs.service

app_stop.sh

#!/bin/bash
echo "stop" > /opt/notify/state.txt
systemctl  stop zerofs.service

systemd 修改，添加了post 命令，方便可以停止依赖服务

[Unit]
Description=Keepalive Daemon (LVS and VRRP)
After=network-online.target
Wants=network-online.target
# Only start if there is a configuration file
ConditionFileNotEmpty=/etc/keepalived/keepalived.conf

[Service]
Type=simple
# Read configuration variable file if it is present
EnvironmentFile=-/etc/default/keepalived
ExecStart=/usr/sbin/keepalived --dont-fork $DAEMON_ARGS
ExecReload=/bin/kill -HUP $MAINPID
ExecStopPost=/opt/notify/app_stop.sh
[Install]
WantedBy=multi-user.target

说明

以上更多是一个实际的实践，通过一个中间件解决zerofs 单实例ha 问题，机制上并不难，但是通过工具集成起来可以方便的解决实际问题

参考资料

https://www.keepalived.org/manpage.html

https://www.zerofs.net/nfs-access