|NO.Z.00029|——————————|^^ 部署 ^^|——|KuberNetes&二进制部署.V07|5台Server|——|Keepalived|haproxy|
一、Keepalived高可用配置
### --- Keepalived部署说明
~~~ 高可用配置（注意：如果不是高可用集群，haproxy和keepalived无需安装）
~~~ 如果在云上安装也无需执行此章节的步骤，可以直接使用云上的lb，比如阿里云slb，腾讯云elb等
~~~ 公有云要用公有云自带的负载均衡，
~~~ 比如阿里云的SLB，腾讯云的ELB，用来替代haproxy和keepalived，
~~~ 因为公有云大部分都是不支持keepalived的，
~~~ 另外如果用阿里云的话，kubectl控制端不能放在master节点，
~~~ 推荐使用腾讯云，因为阿里云的slb有回环的问题，也就是slb代理的服务器不能反向访问SLB，
~~~ 但是腾讯云修复了这个问题。### --- Keepalived连接流程
~~~ slb -> haproxy -> apiserver二、部署keepalived+haproxy
### --- 所有节点部署keepalived和haproxy
[root@k8s-master01 ~]# yum install keepalived haproxy -y### --- 所有节点配置haproxy，配置一样
[root@k8s-master01 ~]# vim /etc/haproxy/haproxy.cfg
global
maxconn 2000
ulimit-n 16384
log 127.0.0.1 local0 err
stats timeout 30s
defaults
log global
mode http
option httplog
timeout connect 5000
timeout client 50000
timeout server 50000
timeout http-request 15s
timeout http-keep-alive 15s
frontend k8s-master
bind 0.0.0.0:8443
bind 127.0.0.1:8443
mode tcp
option tcplog
tcp-request inspect-delay 5s
default_backend k8s-master
backend k8s-master
mode tcp
option tcplog
option tcp-check
balance roundrobin
default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
server k8s-master01 192.168.1.11:6443 check
server k8s-master02 192.168.1.12:6443 check
server k8s-master03 192.168.1.13:6443 check三、配置keepalived
### --- 所有Master节点配置keepalived，
~~~ 所有Master节点配置KeepAlived，配置不一样，注意区分
~~~ 注意每个节点的IP和网卡（interface参数）
[root@k8s-master01 ~]# vim /etc/keepalived/keepalived.conf### --- 配置k8s-master01.keepalived.conf配置参数
[root@k8s-master01 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state MASTER
interface ens33
mcast_src_ip 192.168.1.11
virtual_router_id 51
priority 101
nopreempt
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {
192.168.1.20
}
track_script {
chk_apiserver
} }### --- 配置k8s-master02.keepalived.conf配置参数
[root@k8s-master02 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
mcast_src_ip 192.168.1.12
virtual_router_id 51
priority 100
nopreempt
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {
192.168.1.20
}
track_script {
chk_apiserver
} }### --- 配置k8s-master03.keepalived.conf配置参数
[root@k8s-master03 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
mcast_src_ip 192.168.1.13
virtual_router_id 51
priority 100
nopreempt
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {
192.168.1.20
}
track_script {
chk_apiserver
} }四、健康检查配置
### --- 所有节点配置keepalived健康检查
[root@k8s-master01 ~]# vim /etc/keepalived/check_apiserver.sh
#!/bin/bash
err=0
for k in $(seq 1 3)
do
check_code=$(pgrep haproxy)
if [[ $check_code == "" ]]; then
err=$(expr $err + 1)
sleep 1
continue
else
err=0
break
fi
done
if [[ $err != "0" ]]; then
echo "systemctl stop keepalived"
/usr/bin/systemctl stop keepalived
exit 1
else
exit 0
fi
#2、授予执行权限
[root@k8s-master01 ~]# chmod +x /etc/keepalived/check_apiserver.sh五、启动haproxy和keepalived
### --- 重新加载配置文件
[root@k8s-master01 ~]# systemctl daemon-reload### --- 启动haproxy
[root@k8s-master01 ~]# systemctl enable --now haproxy
Created symlink from /etc/systemd/system/multi-user.target.wants/haproxy.service to /usr/lib/systemd/system/haproxy.service.### --- 启动keepalived
[root@k8s-master01 ~]# systemctl enable --now keepalived
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.六、验证测试
### --- VIP测试
~~~ ping.vip是否可以ping通
[root@k8s-master01 ~]# ping 192.168.1.20 -t 4
PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
64 bytes from 192.168.1.20: icmp_seq=1 ttl=64 time=0.064 ms
64 bytes from 192.168.1.20: icmp_seq=2 ttl=64 time=0.064 ms
64 bytes from 192.168.1.20: icmp_seq=3 ttl=64 time=0.050 ms
64 bytes from 192.168.1.20: icmp_seq=4 ttl=64 time=0.040 ms
--- 192.168.1.20 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3062ms
rtt min/avg/max/mdev = 0.040/0.054/0.064/0.012 ms### --- 测试keepalived端口是否正常
~~~ 重要：如果安装了keepalived和haproxy，需要测试keepalived是否是正常的
~~~ 如果ping不通且telnet没有出现 ]，则认为VIP不可以，不可在继续往下执行，
~~~ 需要排查keepalived的问题，比如防火墙和selinux，haproxy和keepalived的状态，监听端口等
[root@k8s-master01 ~]# telnet 192.168.1.20 8443
Trying 192.168.1.20...
Connected to 192.168.1.20.
Escape character is '^]'.
Connection closed by foreign host.### --- 排查端口不通解决方案
~~~ # 所有节点查看防火墙状态必须为disable和inactive
[root@k8s-master01 ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)~~~ # 所有节点查看selinux状态，必须为disable
[root@k8s-master01 ~]# getenforce
Disabled~~~ # master节点查看haproxy和keepalived状态
[root@k8s-master01 ~]# systemctl status keepalived haproxy~~~ # master节点查看监听端口：
[root@k8s-master01 ~]# netstat -lntp |grep 8443
Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart
——W.S.Landor
浙公网安备 33010602011771号
