狂自私

导航

由于swap导致节点为NotReady

现象

我的节点状态如下：

bzkj@master-1:/etc/kubernetes$ kubectl get nodes
NAME       STATUS     ROLES                  AGE    VERSION
master-1   Ready      control-plane,master   387d   v1.23.4
worker-1   Ready      worker                 272d   v1.23.4
worker-2   NotReady   <none>                 204d   v1.23.4

worker-2节点的状态不对，登录该节点，查看kubelet服务的状态:

bzkj@worker-2:~$ systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
     Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/kubelet.service.d
             └─10-kubeadm.conf
     Active: activating (auto-restart) (Result: exit-code) since Thu 2025-05-22 12:41:01 CST; 8s ago
       Docs: https://kubernetes.io/docs/home/
    Process: 2660932 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=1/FAILURE)
   Main PID: 2660932 (code=exited, status=1/FAILURE)

好像没看出来啥，使用其他方式看看：

bzkj@worker-2:~$ journalctl -u kubelet -f
-- Logs begin at Fri 2025-04-18 10:29:13 CST. --
May 22 12:42:27 worker-2 systemd[1]: Started kubelet: The Kubernetes Node Agent.
May 22 12:42:27 worker-2 kubelet[2661199]: I0522 12:42:27.457628 2661199 server.go:198] "Warning: For remote container runtime, --pod-infra-container-image is ignored in kubelet, which should be set in that remote runtime instead"
May 22 12:42:27 worker-2 kubelet[2661199]: I0522 12:42:27.472431 2661199 server.go:446] "Kubelet version" kubeletVersion="v1.23.4"
May 22 12:42:27 worker-2 kubelet[2661199]: I0522 12:42:27.472903 2661199 server.go:874] "Client rotation is on, will bootstrap in background"
May 22 12:42:27 worker-2 kubelet[2661199]: I0522 12:42:27.475017 2661199 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
May 22 12:42:27 worker-2 kubelet[2661199]: I0522 12:42:27.476609 2661199 dynamic_cafile_content.go:156] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt"
May 22 12:42:32 worker-2 kubelet[2661199]: I0522 12:42:32.492631 2661199 server.go:693] "--cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /"
May 22 12:42:32 worker-2 kubelet[2661199]: E0522 12:42:32.492789 2661199 server.go:302] "Failed to run kubelet" err="failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename\t\t\t\tType\t\tSize\tUsed\tPriority /swap.img                               file\t\t8388604\t0\t-2]"
May 22 12:42:32 worker-2 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
May 22 12:42:32 worker-2 systemd[1]: kubelet.service: Failed with result 'exit-code'.
May 22 12:42:42 worker-2 systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 57482.
May 22 12:42:42 worker-2 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
May 22 12:42:42 worker-2 systemd[1]: Started kubelet: The Kubernetes Node Agent.
May 22 12:42:42 worker-2 kubelet[2661243]: I0522 12:42:42.708566 2661243 server.go:198] "Warning: For remote container runtime, --pod-infra-container-image is ignored in kubelet, which should be set in that remote runtime instead"
May 22 12:42:42 worker-2 kubelet[2661243]: I0522 12:42:42.723376 2661243 server.go:446] "Kubelet version" kubeletVersion="v1.23.4"
May 22 12:42:42 worker-2 kubelet[2661243]: I0522 12:42:42.723650 2661243 server.go:874] "Client rotation is on, will bootstrap in background"
May 22 12:42:42 worker-2 kubelet[2661243]: I0522 12:42:42.725000 2661243 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
May 22 12:42:42 worker-2 kubelet[2661243]: I0522 12:42:42.726215 2661243 dynamic_cafile_content.go:156] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt"
May 22 12:42:47 worker-2 kubelet[2661243]: I0522 12:42:47.746014 2661243 server.go:693] "--cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /"
May 22 12:42:47 worker-2 kubelet[2661243]: E0522 12:42:47.746187 2661243 server.go:302] "Failed to run kubelet" err="failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename\t\t\t\tType\t\tSize\tUsed\tPriority /swap.img                               file\t\t8388604\t0\t-2]"
May 22 12:42:47 worker-2 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
May 22 12:42:47 worker-2 systemd[1]: kubelet.service: Failed with result 'exit-code'.
May 22 12:42:57 worker-2 systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 57483.
May 22 12:42:57 worker-2 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
May 22 12:42:57 worker-2 systemd[1]: Started kubelet: The Kubernetes Node Agent.
May 22 12:42:57 worker-2 kubelet[2661284]: I0522 12:42:57.968713 2661284 server.go:198] "Warning: For remote container runtime, --pod-infra-container-image is ignored in kubelet, which should be set in that remote runtime instead"
May 22 12:42:57 worker-2 kubelet[2661284]: I0522 12:42:57.989649 2661284 server.go:446] "Kubelet version" kubeletVersion="v1.23.4"
May 22 12:42:57 worker-2 kubelet[2661284]: I0522 12:42:57.990020 2661284 server.go:874] "Client rotation is on, will bootstrap in background"
May 22 12:42:57 worker-2 kubelet[2661284]: I0522 12:42:57.991746 2661284 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
May 22 12:42:57 worker-2 kubelet[2661284]: I0522 12:42:57.992591 2661284 dynamic_cafile_content.go:156] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt"
q^C
bzkj@worker-2:~$

可以看到这句报错信息：

May 22 12:42:32 worker-2 kubelet[2661199]: E0522 12:42:32.492789 2661199 server.go:302] "Failed to run kubelet" err="failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename\t\t\t\tType\t\tSize\tUsed\tPriority /swap.img                               file\t\t8388604\t0\t-2]"

这告诉我们，swap分区存在，需要关闭或者做其他配置操作；

从 kubelet 的日志中可以看到，kubelet 启动失败的原因是由于 启用了 swap 分区。Kubernetes 默认不支持在启用了 swap 的节点上运行，因此你需要禁用 swap，或者在 kubelet 配置中明确设置 --fail-swap-on=false 来允许它继续运行。

解决步骤：

1. 禁用 Swap

最推荐的做法是禁用 swap。可以使用以下步骤禁用 swap：

  1. 临时禁用 swap（直到下次重启）：

    sudo swapoff -a

  2. 永久禁用 swap，编辑 /etc/fstab 文件，注释掉关于 swap 的行：

    sudo vim /etc/fstab

    找到类似以下的行，并在前面加上 # 来注释掉：

    /swap.img swap swap defaults 0 0

  3. 禁用 swap 后，重新启动 kubelet：

    sudo systemctl restart kubelet

2. 如果不想禁用 Swap（不推荐）

如果你因为某些原因不想禁用 swap，可以修改 kubelet 的启动参数，在启动时允许使用 swap。

  1. 编辑 kubelet 的配置文件：

    sudo vim /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

  2. 在 KUBELET_KUBEADM_ARGS 变量中添加 --fail-swap-on=false，例如：

    KUBELET_KUBEADM_ARGS=--fail-swap-on=false

  3. 重新加载并重启 kubelet：

    sudo systemctl daemon-reload
sudo systemctl restart kubelet

3. 确认 kubelet 状态

禁用 swap 或修改配置后，确保 kubelet 启动成功，查看状态：

systemctl status kubelet

4. 检查节点状态

再次检查集群节点的状态：

kubectl get nodes

如果一切正常，worker-2 节点应该会变为 Ready 状态。

总结：

Kubernetes 不允许启用 swap，你需要禁用 swap 或者在 kubelet 配置中明确允许使用 swap。禁用 swap 是推荐的做法，通常能带来更好的性能和稳定性。

posted on 2025-05-22 13:25  狂自私  阅读(32)  评论(0)    收藏  举报