|NO.Z.00077|——————————|^^ 部署 ^^|——|KuberNetes&kubeadm.V06|5台Server|——|keepalived|haproxy|
一、高可用组件部署
### --- 高可用组件部署说明
~~~ （注意：如果不是高可用集群，haproxy和keepalived无需安装）
~~~ 公有云要用公有云自带的负载均衡，
~~~ 比如阿里云的SLB，腾讯云的ELB，用来替代haproxy和keepalived，
~~~ 因为公有云大部分都是不支持keepalived的，
~~~ 另外如果用阿里云的话，kubectl控制端不能放在master节点，推荐使用腾讯云，
~~~ 因为阿里云的slb有回环的问题，也就是slb代理的服务器不能反向访问SLB，但是腾讯云修复了这个问题二、高可用组件安装HAProxy和KeepAlived——master节点
### --- 安装HAProxy和KeepAlived
~~~ 安装HAProxy和KeepAlived
[root@k8s-master01 ~]# yum install keepalived haproxy -y### --- 配置HAProxy配置文件
~~~ 所有Master节点配置HAProxy（详细配置参考HAProxy文档，所有Master节点的HAProxy配置相同）：
~~~ # 创建haproxy配置目录
[root@k8s-master01 ~]# mkdir /etc/haproxy~~~ # 创建haproxy配置文件
[root@k8s-master01 ~]# vim /etc/haproxy/haproxy.cfg
global
maxconn 2000
ulimit-n 16384
log 127.0.0.1 local0 err
stats timeout 30s
defaults
log global
mode http
option httplog
timeout connect 5000
timeout client 50000
timeout server 50000
timeout http-request 15s
timeout http-keep-alive 15s
frontend monitor-in
bind *:33305
mode http
option httplog
monitor-uri /monitor
frontend k8s-master
bind 0.0.0.0:16443
bind 127.0.0.1:16443
mode tcp
option tcplog
tcp-request inspect-delay 5s
default_backend k8s-master
backend k8s-master
mode tcp
option tcplog
option tcp-check
balance roundrobin
default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
server k8s-master01 192.168.1.11:6443 check
server k8s-master02 192.168.1.12:6443 check
server k8s-master03 192.168.1.13:6443 check三、配置keepalived配置文件：keepalived.conf
### --- keepalived配置文件概述
~~~ 所有Master节点配置KeepAlived，配置不一样，
~~~ 注意区分 [root@k8s-master01 pki]# vim /etc/keepalived/keepalived.conf ，
~~~ 注意每个节点的IP和网卡（interface参数）### --- k8s-master01节点keepalived配置文件详解
~~~ 创建k8s-master01.keepalived配置文件
[root@k8s-master01 ~]# mkdir /etc/keepalived
[root@k8s-master01 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
script_user root
enable_script_security
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state MASTER
interface ens33
mcast_src_ip 192.168.1.11
virtual_router_id 51
priority 101
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {
192.168.1.20
}
track_script {
chk_apiserver
}
}### --- k8s-master02节点keepalived配置文件详解
~~~ 创建k8s-master02.keepalived配置文件
[root@k8s-master01 ~]# mkdir /etc/keepalived
[root@k8s-master02 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
script_user root
enable_script_security
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
mcast_src_ip 192.168.1.12
virtual_router_id 51
priority 100
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {
192.168.1.20
}
track_script {
chk_apiserver
}
}### --- k8s-master03节点keepalived配置文件详解
~~~ 创建k8s-master03.keepalived配置文件
[root@k8s-master01 ~]# mkdir /etc/keepalived
[root@k8s-master03 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
script_user root
enable_script_security
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
mcast_src_ip 192.168.1.13
virtual_router_id 51
priority 100
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {
192.168.1.20
}
track_script {
chk_apiserver
}
}四、keepalived监控检查配置文件
### --- 配置keepalived监控检查配置文件——所有master节点——配置文件都是一样的
~~~ # 配置keepalived监控检查配置文件：所有节点
[root@k8s-master01 ~]# cat /etc/keepalived/check_apiserver.sh
#!/bin/bash
err=0
for k in $(seq 1 3)
do
check_code=$(pgrep haproxy)
if [[ $check_code == "" ]]; then
err=$(expr $err + 1)
sleep 1
continue
else
err=0
break
fi
done
if [[ $err != "0" ]]; then
echo "systemctl stop keepalived"
/usr/bin/systemctl stop keepalived
exit 1
else
exit 0
fi~~~ # 为配置文件授予可执行权限
[root@k8s-master01 ~]# chmod +x /etc/keepalived/check_apiserver.sh五、启动haproxy和keepalived
### --- 启动haproxy和keepalived
~~~ # 重新加载配置文件
[root@k8s-master01 ~]# systemctl daemon-reload~~~ # 启动haproxy和keepalived
[root@k8s-master01 ~]# systemctl enable --now haproxy
Created symlink from /etc/systemd/system/multi-user.target.wants/haproxy.service to /usr/lib/systemd/system/haproxy.service.
[root@k8s-master01 ~]# systemctl enable --now keepalived
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.六、验证：重要：如果安装了keepalived和haproxy，需要测试keepalived是否是正常的
### --- 验证VIP：192.168.1.11：所有节点
[root@k8s-master01 ~]# ping 192.168.1.20 -c 4
PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
64 bytes from 192.168.1.20: icmp_seq=1 ttl=64 time=0.074 ms
64 bytes from 192.168.1.20: icmp_seq=2 ttl=64 time=0.046 ms
64 bytes from 192.168.1.20: icmp_seq=3 ttl=64 time=0.050 ms
64 bytes from 192.168.1.20: icmp_seq=4 ttl=64 time=0.053 ms
--- 192.168.1.20 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3061ms
rtt min/avg/max/mdev = 0.046/0.055/0.074/0.014 ms### --- 验证16443端口是否可以访问：所有节点
[root@k8s-master01 ~]# telnet 192.168.1.20 16443
Trying 192.168.1.20...
Connected to 192.168.1.20.
Escape character is '^]'.
Connection closed by foreign host.附录一：keepalived验证失败解决方案：
### --- keepalived验证失败解决方案：
~~~ 如果ping不通且telnet没有出现 ] ，则认为VIP不可以，不可在继续往下执行，
~~~ 需要排查keepalived的问题，比如防火墙和selinux，haproxy和keepalived的状态，监听端口等### --- 所有节点查看防火墙状态必须为disable和inactive：
[root@k8s-master01 ~]# systemctl status firewalld
### --- 所有节点查看selinux状态，必须为disable：
[root@k8s-master01 ~]# getenforce### --- master节点查看haproxy和keepalived状态：
[root@k8s-master01 ~]# systemctl status keepalived haproxy
### --- master节点查看监听端口：
[root@k8s-master01 ~]# netstat -lntp
Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart
——W.S.Landor
浙公网安备 33010602011771号
