ubuntu26.04，离线升级k8s，根据kubeadm的deb离线包 + 离线镜像，从现有k8s-v1.32.13升级到k8s-v1.33.9（NLB + 3 master + 2 worker节点）
ubuntu 26.04。
集群规划（5 节点）
master节点01，主机名：k8s-master-01，ip：172.21.64.36
master节点02，主机名：k8s-master-02，ip：172.21.64.37
master节点03，主机名：k8s-master-03，ip：172.21.64.38
工作节点01：主机名：k8s-worker-01，ip：172.21.64.39
工作节点02：主机名：k8s-worker-02，ip：172.21.64.40
| 机器名 | 内网 ip | 角色 |
| k8s-master-01 | 172.21.64.36 | master |
| k8s-master-02 | 172.21.64.37 | master |
| k8s-master-03 | 172.21.64.38 | master |
| k8s-worker-01 | 172.21.64.39 | worker |
| k8s-worker-02 | 172.21.64.40 | worker |
一，升级前准备离线包
1. 下载kubeadm -v1.33.9离线包
离线资源清单及下载流程参考：ubunbu 26.04，用bash脚本自动下载kubeadm的deb离线安装包
离线包目录结构：
/data/k8s/k8s-v1.33.9 ├── calico-images-v3.31.6.list ├── calico-images-v3.31.6.tar ├── calico-v3.31.6.yaml ├── ingress-nginx-images-v1.14.5.list ├── ingress-nginx-images-v1.14.5.tar ├── ingress-nginx-v1.14.5.yaml ├── k8s-debs │ ├── containerd.io_2.3.3-1~ubuntu.26.04~resolute_amd64.deb │ ├── cri-tools_1.33.0-1.1_amd64.deb │ ├── kubeadm_1.33.9-1.1_amd64.deb │ ├── kubectl_1.33.9-1.1_amd64.deb │ ├── kubelet_1.33.9-1.1_amd64.deb │ └── kubernetes-cni_1.6.0-1.1_amd64.deb ├── k8s-images-v1.33.9.list ├── k8s-images-v1.33.9.tar └── k8s-offline.conf
2. 把离线包上传到所有节点
#!/bin/bash set -euo pipefail # 节点列表 nodes=( 172.21.64.36 172.21.64.37 172.21.64.38 172.21.64.39 172.21.64.40 ) src_dir="/data/k8s/k8s-v1.33.9" dst_dir="/data/k8s/k8s-v1.33.9" for ip in "${nodes[@]}"; do echo "=====================================" echo "处理节点 $ip" echo "=====================================" # 远程创建目录 ssh root@"$ip" "mkdir -p ${dst_dir}" # 同步全部离线资源 rsync -avz --progress ${src_dir}/ root@${ip}:${dst_dir}/ done echo "✅ 所有节点离线资源分发完成"
二，备份etcd（仅master-01节点执行）
1. 安装etcdctl
mkdir -p /data/etcd
cd /data/etcd wget https://github.com/etcd-io/etcd/releases/download/v3.6.5/etcd-v3.6.5-linux-amd64.tar.gz tar -zxvf etcd-v3.6.5-linux-amd64.tar.gz cp etcd-v3.6.5-linux-amd64/etcdctl /usr/local/bin/ chmod +x /usr/local/bin/etcdctl #校验版本，必须输出3.6.5 etcdctl version
2.导出etcd数据库
# 备份etcd快照，保存到本地，故障回滚使用
mkdir -p /data/k8s
ETCDCTL_API=3 etcdctl snapshot save /data/k8s/etcd-snapshot-pre-upgrade-v1.34.11.db.$(date +%Y%m%d) \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \d
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key
三，升级
升级分三种角色：
| 节点 | 升级命令 |
| 第一个 master |
kubeadm upgrade plan kubeadm upgrade apply v1.33.9 |
| 其余 master | kubeadm upgrade node |
| worker | kubeadm upgrade node |
1. 【第一阶段，所有节点导入镜像】
1.1 所有安装nerdctl（已经安装就跳过）
下载：https://github.com/containerd/nerdctl/releases/download/v2.3.4/nerdctl-2.3.4-linux-amd64.tar.gz
执行部署
curl https://github.com/containerd/nerdctl/releases/download/v2.3.4/nerdctl-2.3.4-linux-amd64.tar.gz tar -zxvf nerdctl-2.3.4-linux-amd64.tar.gz mv nerdctl /usr/local/bin/ chmod +x /usr/local/bin/nerdctl # 同样配置默认命名空间 mkdir -p /etc/nerdctl cat > /etc/nerdctl/nerdctl.toml <<EOF namespace = "k8s.io" address = "/run/containerd/containerd.sock" cgroup_manager = "systemd" EOF
1.2 批量分发镜像到其它节点
#!/bin/bash set -euo pipefail # 源&目标目录，所有节点保持相同路径 work_dir="/data/k8s/k8s-v1.33.9" node_ips=( "172.21.64.37" "172.21.64.38" "172.21.64.39" "172.21.64.40" ) for ip in "${node_ips[@]}"; do echo "==================== 分发到节点 ${ip} ====================" # 先在远端确保父目录存在 ssh root@"${ip}" "mkdir -p $(dirname ${work_dir})" # -r 递归，--delete 可选：远端目录删除本地不存在的旧文件，按需开启 rsync -av --progress "${work_dir}/" root@"${ip}":"${work_dir}/" done echo "====全部节点目录分发完成===="
1.3 批量导入镜像
#!/bin/bash set -euo pipefail # 定义目录变量 tar_dir="/data/k8s/k8s-v1.33.9" node_ips=( "172.21.64.37" "172.21.64.38" "172.21.64.39" "172.21.64.40" ) for ip in "${node_ips[@]}"; do echo "==================== 处理节点 ${ip} ====================" ssh root@${ip} " set -euo pipefail tar_dir=\"${tar_dir}\" cd \"\${tar_dir}\" ctr -n k8s.io images import k8s-images-*.tar ctr -n k8s.io images import calico-images-*.tar ctr -n k8s.io images import ingress-nginx-images-*.tar ctr -n k8s.io images list | wc -l " done echo "====全部节点镜像导入完成===="
2. 第一个 mastert升级（k8s-master-01）
重要：只有第一个 master 执行 kubeadm upgrade apply，其余 master 执行kubeadm upgrade node
#!/bin/bash set -euo pipefail # ========== 版本变量，只需要修改此处 ========== k8s_version="v1.33.9" deb_base_dir="/data/k8s/k8s-${k8s_version}/k8s-debs" # ============================================ #解除版本锁定 apt-mark unhold cloud-init containerd.io cri-tools intel-microcode kubeadm kubectl kubelet kubernetes-cni # 1. 确认升级计划（仅在master-01节点执行） kubeadm upgrade plan # 2. 升级kubeadm cd "${deb_base_dir}" dpkg -i kubeadm_*.deb # 确认版本 kubeadm version -o short # 执行控制平面升级（更新 apiserver/controller-manager/scheduler/etcd/coredns/kube-proxy 的 manifest） kubeadm upgrade apply "${k8s_version}" -y # 3. 升级containerd # 备份配置（防止 dpkg 覆盖） cp /etc/containerd/config.toml /etc/containerd/config.toml.bak.$(date +%Y%m%d) dpkg -i containerd.io_*.deb systemctl restart containerd # 4. 升级 kubelet + kubectl dpkg -i kubelet_*.deb kubectl_*.deb cri-tools_*.deb kubernetes-cni_*.deb systemctl daemon-reload systemctl restart kubelet # 锁定版本自动更新 apt-mark hold cloud-init containerd.io cri-tools intel-microcode kubeadm kubectl kubelet kubernetes-cni # 5. 等待节点 Ready echo "等待节点变为Ready..." kubectl wait --for=condition=Ready node/$(hostname) --timeout=15s kubectl get nodes containerd --version kubelet --version
2.2 其它 master节点
k8s_version="v1.33.9" # 1. 升级kubeadm cd /data/k8s/k8s-${k8s_version}/k8s-debs #解除版本锁定kubectl kubelet kubernetes-cni apt-mark unhold cloud-init containerd.io cri-tools intel-microcode kubeadm # 升级kubeadm dpkg -i kubeadm_*.deb kubeadm upgrade node # 3. 升级 containerd # 备份配置 cp /etc/containerd/config.toml /etc/containerd/config.toml.bak.$(date +%Y%m%d) dpkg -i containerd.io_*.deb systemctl restart containerd # 4. 升级 kubelet + kubectl dpkg -i kubelet_*.deb kubectl_*.deb cri-tools_*.deb kubernetes-cni_*.deb systemctl daemon-reload systemctl restart kubelet # 锁定版本自动更新 apt-mark hold cloud-init containerd.io cri-tools intel-microcode kubeadm kubectl kubelet kubernetes-cni # 等待节点 Ready echo "等待节点变为Ready..." kubectl wait --for=condition=Ready node/$(hostname) --timeout=15s kubectl get nodes containerd --version kubelet --version
3. 【第三阶段（一），第一个worker节点升级 】
worker 升级需要在 master-01 和 worker 本身 之间切换操作，穿插执行。以 k8s-worker-01 为例：
第 1 步：在 master-01 上先驱逐worker-01节点
kubectl drain k8s-worker-01 --ignore-daemonsets --delete-emptydir-data
第 2 步：在 k8s-worker-01 上执行升级
k8s_version="v1.33.9" cd /data/k8s/k8s-${k8s_version}/k8s-debs #解除版本锁定 apt-mark unhold cloud-init containerd.io cri-tools intel-microcode kubeadm kubectl kubelet kubernetes-cni # 升级kubeadm dpkg -i kubeadm_*.deb kubeadm upgrade node # 升级 containerd # 先备份配置（防止 dpkg 覆盖） cp /etc/containerd/config.toml /etc/containerd/config.toml.bak.$(date +%Y%m%d) dpkg -i containerd.io_*.deb # 升级 kubelet + kubectl dpkg -i kubelet_*.deb kubectl_*.deb cri-tools_*.deb kubernetes-cni_*.deb #重启 systemctl daemon-reload systemctl restart containerd systemctl restart kubelet # 锁定版本自动更新 apt-mark hold cloud-init containerd.io cri-tools intel-microcode kubeadm kubectl kubelet kubernetes-cni
第 3 步：回到 master-01 上 取消驱逐 worker-01
kubectl uncordon k8s-worker-01
第 4 步：确认 Ready 后，对 k8s-worker-02 重复同样操作
4. 【第三阶段（二），第二个worker节点升级 】
worker 升级需要在 master-01 和 worker 本身 之间切换操作，穿插执行。以 k8s-worker-01 为例：
第 1 步：在任意master节点上先驱逐worker-02节点
kubectl drain k8s-worker-02 --ignore-daemonsets --delete-emptydir-data
第 2 步：在 k8s-worker-01 上执行升级
k8s_version="v1.33.9"
cd /data/k8s/k8s-${k8s_version}/k8s-debs
#解除版本锁定
apt-mark unhold cloud-init containerd.io cri-tools intel-microcode kubeadm kubectl kubelet kubernetes-cni
# 升级kubeadm
dpkg -i kubeadm_*.deb
kubeadm upgrade node
# 升级 containerd
# 先备份配置（防止 dpkg 覆盖）
cp /etc/containerd/config.toml /etc/containerd/config.toml.bak.$(date +%Y%m%d)
dpkg -i containerd.io_*.deb
# 升级 kubelet + kubectl
dpkg -i kubelet_*.deb kubectl_*.deb cri-tools_*.deb kubernetes-cni_*.deb
#重启
systemctl daemon-reload
systemctl restart containerd
systemctl restart kubelet
# 锁定版本自动更新
apt-mark hold cloud-init containerd.io cri-tools intel-microcode kubeadm kubectl kubelet kubernetes-cni
第 3 步：回到任意master节点， 取消驱逐 worker-02
kubectl uncordon k8s-worker-02
第 4 步：确认 Ready 后，对 k8s-worker-03 / k8s-worker-04 / k8s-worker-05 等重复同样的操作
5. 【第四阶段，在master-01节点升级calico 与ingress】
caliico与ingress 可以兼容三个大版本k8s，如果这个版本需要更新calico与 ingress。
把所有worker节点升级完后再升级calico 与 ingress
kubectl apply -f calico-v3.31.6.yaml kubectl apply -f ingress-nginx-v1.14.5.yaml
四，最终验证
所有节点升级完后：
kubectl get nodes -owide # 5 个节点 VERSION 全部 v1.33.9，STATUS Ready kubectl get pods -A -owide # 所有 Pod Running，没有 CrashLoopBackOff kubectl get pods -A | grep -v Running | grep -v Completed # 应该没有输出（除了正常的 Completed Job）
输出：
root@k8s-master-01:/data/k8s/k8s-v1.33.9/k8s-debs# kubectl get nodes -owide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-master-01 Ready control-plane 17h v1.33.9 172.21.64.36 <none> Ubuntu 26.04 LTS 7.0.0-28-generic containerd://2.3.3 k8s-master-02 Ready control-plane 15h v1.33.9 172.21.64.37 <none> Ubuntu 26.04 LTS 7.0.0-28-generic containerd://2.3.3 k8s-master-03 Ready control-plane 15h v1.33.9 172.21.64.38 <none> Ubuntu 26.04 LTS 7.0.0-28-generic containerd://2.3.3 k8s-worker-01 Ready <none> 14h v1.33.9 172.21.64.39 <none> Ubuntu 26.04 LTS 7.0.0-28-generic containerd://2.3.3 k8s-worker-02 Ready <none> 14h v1.33.9 172.21.64.40 <none> Ubuntu 26.04 LTS 7.0.0-28-generic containerd://2.3.3 root@k8s-master-01:/data/k8s/k8s-v1.33.9/k8s-debs# kubectl get pod -A -owide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kube-system calico-kube-controllers-678b6c5f64-8lt4p 1/1 Running 2 (134m ago) 17h 192.168.151.134 k8s-master-01 <none> <none> kube-system calico-node-74l7c 1/1 Running 1 (40m ago) 14h 172.21.64.40 k8s-worker-02 <none> <none> kube-system calico-node-ccskt 1/1 Running 1 (40m ago) 14h 172.21.64.39 k8s-worker-01 <none> <none> kube-system calico-node-mvqx6 1/1 Running 1 (141m ago) 15h 172.21.64.37 k8s-master-02 <none> <none> kube-system calico-node-slmvj 1/1 Running 1 (155m ago) 17h 172.21.64.36 k8s-master-01 <none> <none> kube-system calico-node-xv5qh 1/1 Running 1 (139m ago) 15h 172.21.64.38 k8s-master-03 <none> <none> kube-system coredns-674b8bbfcf-9k624 1/1 Running 0 4m2s 192.168.183.129 k8s-master-03 <none> <none> kube-system coredns-674b8bbfcf-dqw49 1/1 Running 0 2m7s 192.168.36.194 k8s-worker-01 <none> <none> kube-system etcd-k8s-master-01 1/1 Running 2 (21m ago) 21m 172.21.64.36 k8s-master-01 <none> <none> kube-system etcd-k8s-master-02 1/1 Running 2 (9m11s ago) 9m11s 172.21.64.37 k8s-master-02 <none> <none> kube-system etcd-k8s-master-03 1/1 Running 2 (7m12s ago) 7m13s 172.21.64.38 k8s-master-03 <none> <none> kube-system kube-apiserver-k8s-master-01 1/1 Running 2 (21m ago) 21m 172.21.64.36 k8s-master-01 <none> <none> kube-system kube-apiserver-k8s-master-02 1/1 Running 2 (9m11s ago) 9m11s 172.21.64.37 k8s-master-02 <none> <none> kube-system kube-apiserver-k8s-master-03 1/1 Running 2 (7m12s ago) 7m13s 172.21.64.38 k8s-master-03 <none> <none> kube-system kube-controller-manager-k8s-master-01 1/1 Running 1 (21m ago) 21m 172.21.64.36 k8s-master-01 <none> <none> kube-system kube-controller-manager-k8s-master-02 1/1 Running 1 (9m11s ago) 9m11s 172.21.64.37 k8s-master-02 <none> <none> kube-system kube-controller-manager-k8s-master-03 1/1 Running 1 (7m12s ago) 7m13s 172.21.64.38 k8s-master-03 <none> <none> kube-system kube-proxy-6tfgz 1/1 Running 0 6m46s 172.21.64.36 k8s-master-01 <none> <none> kube-system kube-proxy-7dkxq 1/1 Running 0 7m12s 172.21.64.38 k8s-master-03 <none> <none> kube-system kube-proxy-k6s2r 1/1 Running 0 7m20s 172.21.64.37 k8s-master-02 <none> <none> kube-system kube-proxy-vk6p7 1/1 Running 0 7m18s 172.21.64.40 k8s-worker-02 <none> <none> kube-system kube-proxy-wqbvp 1/1 Running 0 7m19s 172.21.64.39 k8s-worker-01 <none> <none> kube-system kube-scheduler-k8s-master-01 1/1 Running 1 (21m ago) 21m 172.21.64.36 k8s-master-01 <none> <none> kube-system kube-scheduler-k8s-master-02 1/1 Running 1 (9m11s ago) 9m11s 172.21.64.37 k8s-master-02 <none> <none> kube-system kube-scheduler-k8s-master-03 1/1 Running 1 (7m12s ago) 7m13s 172.21.64.38 k8s-master-03 <none> <none>
浙公网安备 33010602011771号