Kubespray部署k8s v1.24.x集群
Kubespray是一个官方推荐的，用于在生产环境部署k8s集群的工具。基于ansible批量下发任务的方式完成一键部署、升级、增加删除节点等等。
本文使用kubespray v2.20.0 版本，用于部署 k8s v1.24.6 版本。
1、主机准备
| 主机名 | IP | 配置 | 操作系统 |
| console | 192.168.0.32 | 4C8G | Centos7.9 |
| master | 192.168.0.7 | 4C16G | Centos7.9 |
| node | 192.168.0.151 | 4C16G | Centos7.9 |
2、主机环境初始化
修改主机名称
hostnamectl set-hostname k8s-master # 列出其中一台，其他类似修改
同步3台机器时间
ntpdate ntp.aliyun.com
# 如果没有该命令，按下命令安装
yum install -u ntpdate
关闭swap虚拟内存
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab && sudo swapoff -a
关闭防护墙和selinux
systemctl stop firewalld && sudo systemctl disable firewalld setenforce 0 && sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config
内核优化参数
# cat <<EOF > /etc/sysctl.d/k8s.conf net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-iptables = 1 fs.may_detach_mounts = 1 vm.overcommit_memory=1 vm.panic_on_oom=0 fs.inotify.max_user_watches=89100 fs.file-max=52706963 fs.nr_open=52706963 net.netfilter.nf_conntrack_max=2310720 net.ipv4.tcp_keepalive_time = 600 net.ipv4.tcp_keepalive_probes = 3 net.ipv4.tcp_keepalive_intvl =15 net.ipv4.tcp_max_tw_buckets = 36000 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_max_orphans = 327680 net.ipv4.tcp_orphan_retries = 3 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.ip_conntrack_max = 65536 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.tcp_timestamps = 0 net.core.somaxconn = 16384 net.ipv6.conf.all.disable_ipv6 = 0 net.ipv6.conf.default.disable_ipv6 = 0 net.ipv6.conf.lo.disable_ipv6 = 0 net.ipv6.conf.all.forwarding = 1 EOF # sysctl --system 生效
配置控制机与两台k8s主机SSH免密
# ssh-keygen # console控制机 # ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.19.130 # copy公钥 # ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.19.131 # copy公钥
console主机上拉取kubespary镜像（docker环境提前安装）
# docker pull quay.m.daocloud.io/kubespray/kubespray:v2.20.0
注意： 这个镜像配置了国内加速，能访问外网就可以拉到。但是该镜像比较大3.6G，需要耐心等待
3、下载kubespray源码包（下面都在console节点操作）
下载对应版本：kubespray v2.20.0
# wget https://github.com/kubernetes-sigs/kubespray/archive/refs/tags/v2.20.0.tar.gz
解压压缩包
# tar -xvf v2.20.0.tar.gz # mv kubespray-v2.20.0 kubespray
修改kubespray配置信息
# cd kubespray
# cp -rfp inventory/sample inventory/mycluster
# vim inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml
kube_network_plugin: cilium # 选择网络插件，支持 cilium, calico, weave 和 flannel
kube_service_addresses: 10.233.0.0/18 # 设置 Service 网段
kube_pods_subnet: 10.233.64.0/18 # 设置 Pod 网段
# 支持 docker, crio 和 containerd，推荐 containerd.
container_manager: containerd
# 是否开启 kata containers
kata_containers_enabled: false
# 是否开启自动更新证书，推荐开启。
auto_renew_certificates: true
# 容器数据目录
# vim ./inventory/mycluster/group_vars/all/containerd.yml
containerd_storage_dir: "/data/containerd"
...
containerd_registries:
"docker.io":
- "http://hub-mirror.c.163.com"
- "https://mirror.aliyuncs.com"
containerd_snapshotter: "native"
修改etcd数据目录
# vim inventory/mycluster/group_vars/all/etcd.yml etcd_data_dir: /data/etcd
打开排错日志
# vim inventory/mycluster/group_vars/all/all.yml unsafe_show_logs: true
配置主机列表
# vim inventory/mycluster/inventory.ini [all] master ansible_host=192.168.0.7 node1 ansible_host=192.168.0.151 [kube_control_plane] master [etcd] master [kube_node] node1 [k8s_cluster:children] kube_control_plane kube_node
修改在线资源下载路径为daocloud源，这样在国内也成功下载集群所需的资源
# 备份
cp inventory/mycluster/group_vars/all/offline.yml{,.bak}
# 修改files_repo
sed -i 's@^# files_repo: .*@files_repo: "https://files.m.daocloud.io"@g' inventory/mycluster/group_vars/all/offline.yml
# 修改registry_repo
sed -i 's@^# kube_image_repo: .*@kube_image_repo: "k8s.m.daocloud.io"@g' inventory/mycluster/group_vars/all/offline.yml
sed -i 's@^# gcr_image_repo: .*@gcr_image_repo: "gcr.m.daocloud.io"@g' inventory/mycluster/group_vars/all/offline.yml
sed -i 's@^# github_image_repo: .*@github_image_repo: "ghcr.m.daocloud.io"@g' inventory/mycluster/group_vars/all/offline.yml
sed -i 's@^# docker_image_repo: .*@docker_image_repo: "docker.m.daocloud.io"@g' inventory/mycluster/group_vars/all/offline.yml
sed -i 's@^# quay_image_repo: .*@quay_image_repo: "quay.m.daocloud.io"@g' inventory/mycluster/group_vars/all/offline.yml
# 取消注释 启用files_repo和registry_host
sed -i -E '/# .*\{\{ files_repo/s/^# //g' inventory/mycluster/group_vars/all/offline.yml
sed -i -E '/# .*\{\{ registry_host/s/^# //g' inventory/mycluster/group_vars/all/offline.yml
4、一键部署
启动kubespray容器
docker run --rm -it \ -v $(pwd):/kubespray \ -v /root/.ssh:/root/.ssh \ quay.m.daocloud.io/kubespray/kubespray:v2.20.0 \ bash
注意：该命令要在kubespray解压缩后的目录中执行才可以
一键部署
# ansible-playbook -i inventory/mycluster/inventory.ini --user=root -b -v cluster.yml
接下来不出意外就要报红了，哈哈。
它有一个下载的检验，需要改一下method类型
vim roles/download/tasks/download_file.yml
再次执行一键部署脚本就可以了。 这个问题如果是用的默认的国外源应该是不会遇到的。
参考链接：https://www.magiccloudnet.com/kubespray/
浙公网安备 33010602011771号