kubernetes-搭建之二进制部署

二进制部署（HA）

一、规划

环境初始化

☑️配置静态IP ☑️配置主机名

☑️Hosts ☑️免密登录

☑️Firewalld ☑️Selinux ☑️Swap

☑️修改内核参数 ☑️Repo

☑️配置时间同步

☑️Iptables

☑️Docker ☑️修改Docker配置

k8s集群角色	IP	主机名	组件
master1	192.168.56.129	anyu967master1	apiserver、controller-manager、scheduler、etcd、docker、keepalived、nginx
master2	192.168.56.130	anyu967master2	apiserver、controller-manager、scheduler、etcd、docker、keepalived、nginx
master3	192.168.56.131	anyu967master3	apiserver、controller-manager、scheduler、etcd、docker
node1	192.168.56.132	anyu967node1	kubelet、kube-proxy、docker、calico、coredns
VIP	192.168.56.134	-	-

二、搭建

2.1. 搭建etcd集群

# master1,2,3
mkdir -p /etc/etcd/ssl
chmod +x cfssl-certinfo_linux-amd64 cfssljson_linux-amd64 cfssl_linux-amd64
mv cfssl-certinfo_linux-amd64 /usr/local/cfssl
# CA证书
vim ca-csr.json
cfssl gencert -initca ca-csr.json |cfssljson -bare ca
vim ca-config.json
vim etcd-csr.json  # 改IP(控制节点IP、VIP)
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes etcd-csr.json |cfssljson -bare etcd
# 部署etcd
vim etcd.conf   # master1,2,3's IP:2380
vim etcd.service

cp -a ca*.pem /etc/etcd/ssl/
cp -a etcd*.pem /etc/etvd/ssl/
cp -a etcd.conf /etc/etcd/
cp -a etcd.service /usr/lib/systemd/system/

for i in master2 master3;do rsync -vaz etcd.conf $i:/etc/etcd/;done
for i in master2 master3;do rsync -vaz etcd*.pem ca*.pem $i:/etc/etcd/ssl/;done
for i in master2 master3;do rsync -vaz etcd.service $i:/usr/lib/systemd/system/;done

mkdir -p /var/lib/etcd/default.etcd   # 更新etcd.conf配置
systemctl daemon-reload
systemctl enable etcd.service
systemctl start etcd.service

/usr/local/bin/etcdctl --write-out=table --cacert=/etc/etcd/ssl/ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem --endpoints=https://192.168.56.129:2379,https://192.168.56.130:2379,https://192.168.56.131:2379  endpoint health

2.2. 安装k8s组件
【金山文档】 二进制安装多master节点的k8s集群-1.20+稳定版本-更新版本

kubernetes/CHANGELOG at master · kubernetes/kubernetes

google gcr.io、k8s.gcr.io 国内镜像

cat > token.csv <<EOF
$(head -c 16 /dev/random |od -An -t x |tr -d ' '),kubelet-bootstrap,10001,"system:kubelet-bootstrap"
EOF

三、测试验证

docker load -i busybox-1.28.tar.gz
kubectl run busybox --image busybox:1.28 --restart=Never --rm -it busybox --sh

ping www.baidu.com
ping podip

nslookup kubernetes.default.svc.cluster.local

四、keepalive和nginx实现apiserver高可用

见2.2

# nginx
# 四层负载均衡，为两台Master apiserver组件提供负载均衡
stream {
     log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
     access_log /var/log/nginx/k8s-access.log main;
     upstream k8s-apiserver {
         server 192.168.40.180:6443; # xianchaomaster1 APISERVER IP:PORT
         server 192.168.40.181:6443; # xianchaomaster2 APISERVER IP:PORT
         server 192.168.40.182:6443; # xianchaomaster3 APISERVER IP:PORT
     }
     server {
         listen 16443; # 由于nginx与master节点复用，这个监听端口不能是6443，否则会冲突
         proxy_pass k8s-apiserver;
     }
}