K8S集群搭建(部署proxy)

k8s二进制安装:部署proxy,验证集群

---

1. 部署4层反代去代理apiserver

部署主机:(11/12)

1.1 为何要部署

master节点上的3套服务部署完成后,需要使用反向代理去统一两个apiservser的对外端口
这里使用nginx+keepalived的高可用架构部署在7.11和7.12两台机器上

1.2 部署nginx四层反代

1.2.1 端口设定

使用7443端口代理apiserver的6443端口,使用keepalived管理VIP10.4.7.10

1.2.2 yum安装程序

yum install keepalived -y

#新版nginx直接yum安装不支持stream模块，可以直接rpm安装指定版本
rpm -ivh http://nginx.org/packages/centos/7/x86_64/RPMS/nginx-1.14.2-1.el7_4.ngx.x86_64.rpm

1.2.3 配置NGINX

四层代理不能写在默认的conf.d目录下,因为这个目录默认是数据http模块的include
所以要么把四层代理写到主配置文件最下面,要么模仿七层代理创建一个四层代理文件夹

# 1. 在nginx配置文件中增加四层代理配置文件夹
mkdir /etc/nginx/tcp.d/
echo 'include /etc/nginx/tcp.d/*.conf;' >>/etc/nginx/nginx.conf

# 写入代理配置
cat >/etc/nginx/tcp.d/apiserver.conf <<EOF
stream {
upstream kube-apiserver {
server 10.4.7.21:6443 max_fails=3 fail_timeout=30s;
server 10.4.7.22:6443 max_fails=3 fail_timeout=30s;
}
server {
listen 7443;
proxy_connect_timeout 2s;
proxy_timeout 900s;
proxy_pass kube-apiserver;
}
}
EOF

1.2.4 启动nginx

nginx -t
systemctl start nginx
systemctl enable nginx

1.3 配置keepalived

1.3.1创建端口监测脚本

# 创建脚本
cat >/etc/keepalived/check_port.sh <<'EOF'
#!/bin/bash
#keepalived 监控端口脚本
#使用方法：等待keepalived传入端口参数,检查改端口是否存在并返回结果
CHK_PORT=$1
if [ -n "$CHK_PORT" ];then
PORT_PROCESS=`ss -lnt|grep $CHK_PORT|wc -l`
if [ $PORT_PROCESS -eq 0 ];then
echo "Port $CHK_PORT Is Not Used,End."
exit 1
fi
else
echo "Check Port Cant Be Empty!"
fi
EOF

# 给与脚本执行权限
chmod +x /etc/keepalived/check_port.sh

1.3.2 创建keepalived主配置文件

部署主机:(11)

主机定义为10.4.7.11,从机定义为10.4.7.12
注意:主配置文件添加了nopreempt参数,非抢占式,意味着VIP发生漂移后,主重新启动后也不会夺回VIP,目的是为了稳定性

cat >/etc/keepalived/keepalived.conf <<'EOF'
! Configuration File for keepalived
global_defs {
router_id 10.4.7.11
}
vrrp_script chk_nginx {
script "/etc/keepalived/check_port.sh 7443"
interval 2
weight -20
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 251
priority 100
advert_int 1
mcast_src_ip 10.4.7.11
nopreempt
authentication {
auth_type PASS
auth_pass 11111111
}
track_script {
chk_nginx
}
virtual_ipaddress {
10.4.7.10
}
}
EOF

1.3.3 创建keepalived从配置文件

部署主机:(12)

cat >/etc/keepalived/keepalived.conf <<'EOF'
! Configuration File for keepalived
global_defs {
router_id 10.4.7.12
}
vrrp_script chk_nginx {
script "/etc/keepalived/check_port.sh 7443"
interval 2
weight -20
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 251
mcast_src_ip 10.4.7.12
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 11111111
}
track_script {
chk_nginx
}
virtual_ipaddress {
10.4.7.10
}
}
EOF

1.3.4 启动keepalived并验证

systemctl start keepalived
systemctl enable keepalived
ip addr|grep '10.4.7.10'

2. 验证kubernetes集群

2.1 在任意一个节点上创建一个资源配置清单

21或者22都可

cat >/root/nginx-ds.yaml <<'EOF'
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: nginx-ds
spec:
template:
metadata:
labels:
app: nginx-ds
spec:
containers:
- name: my-nginx
image: harbor.zq.com/public/nginx:v1.17.9
ports:
- containerPort: 80
EOF

2.2 应用资源配置，并检查

2.2.1 应用资源配置

kubectl create -f /root/nginx-ds.yaml
[root@hdss7-22 conf]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-ds-j777c 1/1 Running 0 8s
nginx-ds-nwsd6 1/1 Running 0 8s

2.2.2在另一台node节点上检查

kubectl get pods
kubectl get pods -o wide
curl 172.7.22.2

2.2.3 查看kubernetes是否搭建好

[root@hdss7-22 conf]# kubectl get cs
NAME STATUS MESSAGE ERROR
etcd-0 Healthy {"health": "true"}
etcd-2 Healthy {"health": "true"}
etcd-1 Healthy {"health": "true"}
controller-manager Healthy ok
scheduler Healthy ok
[root@hdss7-21 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
hdss7-21.host.com Ready master,node 6d1h v1.15.5
hdss7-22.host.com Ready <none> 6d1h v1.15.5
[root@hdss7-22 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-ds-j777c 1/1 Running 0 6m45s
nginx-ds-nwsd6 1/1 Running 0 6m45s