rancher 添加主节点后，一直显示Waiting for probes: calico, etcd, kube-apiserver, kube-controller-manager, kube-scheduler, kubelet

1. 问题描述

rancher集群 注册主节点后，一直显示如下状态

Waiting for probes: calico, etcd, kube-apiserver, kube-controller-manager, kube-scheduler, kubelet

 查看rancher-system-agent.service日志，显示什么证书问题

[root@k8snode1 ~]# journalctl -u rancher-system-agent.service -f
-- Logs begin at Thu 2026-04-16 13:35:00 CST. --
Apr 17 11:38:22 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:22+08:00" level=error msg="error loading CA cert for probe (kube-scheduler) /var/lib/rancher/rke2/server/tls/kube-scheduler/kube-scheduler.crt: open /var/lib/rancher/rke2/server/tls/kube-scheduler/kube-scheduler.crt: no such file or directory"
Apr 17 11:38:22 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:22+08:00" level=error msg="error while appending ca cert to pool for probe kube-scheduler"
Apr 17 11:38:27 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:27+08:00" level=error msg="error loading CA cert for probe (kube-scheduler) /var/lib/rancher/rke2/server/tls/kube-scheduler/kube-scheduler.crt: open /var/lib/rancher/rke2/server/tls/kube-scheduler/kube-scheduler.crt: no such file or directory"
Apr 17 11:38:27 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:27+08:00" level=error msg="error while appending ca cert to pool for probe kube-scheduler"
Apr 17 11:38:27 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:27+08:00" level=error msg="error loading CA cert for probe (kube-controller-manager) /var/lib/rancher/rke2/server/tls/kube-controller-manager/kube-controller-manager.crt: open /var/lib/rancher/rke2/server/tls/kube-controller-manager/kube-controller-manager.crt: no such file or directory"
Apr 17 11:38:27 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:27+08:00" level=error msg="error while appending ca cert to pool for probe kube-controller-manager"
Apr 17 11:38:32 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:32+08:00" level=error msg="error loading CA cert for probe (kube-scheduler) /var/lib/rancher/rke2/server/tls/kube-scheduler/kube-scheduler.crt: open /var/lib/rancher/rke2/server/tls/kube-scheduler/kube-scheduler.crt: no such file or directory"
Apr 17 11:38:32 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:32+08:00" level=error msg="error while appending ca cert to pool for probe kube-scheduler"
Apr 17 11:38:32 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:32+08:00" level=error msg="error loading CA cert for probe (kube-controller-manager) /var/lib/rancher/rke2/server/tls/kube-controller-manager/kube-controller-manager.crt: open /var/lib/rancher/rke2/server/tls/kube-controller-manager/kube-controller-manager.crt: no such file or directory"
Apr 17 11:38:32 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:32+08:00" level=error msg="error while appending ca cert to pool for probe kube-controller-manager"
Apr 17 11:38:37 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:37+08:00" level=error msg="error loading CA cert for probe (kube-controller-manager) /var/lib/rancher/rke2/server/tls/kube-controller-manager/kube-controller-manager.crt: open /var/lib/rancher/rke2/server/tls/kube-controller-manager/kube-controller-manager.crt: no such file or directory"
Apr 17 11:38:37 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:37+08:00" level=error msg="error while appending ca cert to pool for probe kube-controller-manager"
Apr 17 11:38:37 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:37+08:00" level=error msg="error loading CA cert for probe (kube-scheduler) /var/lib/rancher/rke2/server/tls/kube-scheduler/kube-scheduler.crt: open /var/lib/rancher/rke2/server/tls/kube-scheduler/kube-scheduler.crt: no such file or directory"
Apr 17 11:38:37 k8snode1 rancher-system-agent[187392]: time="2026-04-17T11:38:37+08:00" level=error msg="error while appending ca cert to pool for probe kube-scheduler"

2. 排查

这种问题一般来说大概率是镜像还没下载下来，可以查看日志/var/lib/rancher/rke2/agent/containerd/containerd.log，如果 显示在PullImage，大概率是在下载镜像

[root@k8snode1 ~]# tail -f /var/lib/rancher/rke2/agent/containerd/containerd.log
time="2026-04-17T11:31:04.261615283+08:00" level=info msg="ImageUpdate event name:\"registry.cn-hangzhou.aliyuncs.com/rancher/rke2-cloud-provider:v1.28.15-0.20241016053552-63bfb1936862-build20241016\"  labels:{key:\"io.cattle.rke2.pinned\"  value:\"pinned\"}  labels:{key:\"io.cri-containerd.image\"  value:\"managed\"}  labels:{key:\"io.cri-containerd.pinned\"  value:\"pinned\"}"
time="2026-04-17T11:31:04.263173833+08:00" level=info msg="PullImage \"registry.cn-hangzhou.aliyuncs.com/rancher/hardened-etcd:v3.5.13-k3s1-build20240910\""
time="2026-04-17T11:33:19.398822852+08:00" level=info msg="ImageCreate event name:\"registry.cn-hangzhou.aliyuncs.com/rancher/hardened-etcd:v3.5.13-k3s1-build20240910\"  labels:{key:\"io.cri-containerd.image\"  value:\"managed\"}"
time="2026-04-17T11:33:19.421460454+08:00" level=info msg="stop pulling image registry.cn-hangzhou.aliyuncs.com/rancher/hardened-etcd:v3.5.13-k3s1-build20240910: active requests=0, bytes read=17369514"
time="2026-04-17T11:33:19.444416276+08:00" level=info msg="ImageCreate event name:\"sha256:19f8e656ed901d1063ee554fa072f1618aa6ce975ae58400d031eab6be7856d6\"  labels:{key:\"io.cri-containerd.image\"  value:\"managed\"}"
time="2026-04-17T11:33:19.472945713+08:00" level=info msg="ImageCreate event name:\"registry.cn-hangzhou.aliyuncs.com/rancher/hardened-etcd@sha256:aab0615152a890ae971b2211afb2a5f377ea2048431aae0e9ff0776b276c3c84\"  labels:{key:\"io.cri-containerd.image\"  value:\"managed\"}"
time="2026-04-17T11:33:19.474226041+08:00" level=info msg="Pulled image \"registry.cn-hangzhou.aliyuncs.com/rancher/hardened-etcd:v3.5.13-k3s1-build20240910\" with image id \"sha256:19f8e656ed901d1063ee554fa072f1618aa6ce975ae58400d031eab6be7856d6\", repo tag \"registry.cn-hangzhou.aliyuncs.com/rancher/hardened-etcd:v3.5.13-k3s1-build20240910\", repo digest \"registry.cn-hangzhou.aliyuncs.com/rancher/hardened-etcd@sha256:aab0615152a890ae971b2211afb2a5f377ea2048431aae0e9ff0776b276c3c84\", size \"17366118\" in 2m15.210994349s"
time="2026-04-17T11:33:19.474276027+08:00" level=info msg="PullImage \"registry.cn-hangzhou.aliyuncs.com/rancher/hardened-etcd:v3.5.13-k3s1-build20240910\" returns image reference \"sha256:19f8e656ed901d1063ee554fa072f1618aa6ce975ae58400d031eab6be7856d6\""
time="2026-04-17T11:33:19.495745856+08:00" level=info msg="ImageUpdate event name:\"registry.cn-hangzhou.aliyuncs.com/rancher/hardened-etcd:v3.5.13-k3s1-build20240910\"  labels:{key:\"io.cattle.rke2.pinned\"  value:\"pinned\"}  labels:{key:\"io.cri-containerd.image\"  value:\"managed\"}  labels:{key:\"io.cri-containerd.pinned\"  value:\"pinned\"}"
time="2026-04-17T11:33:19.497306350+08:00" level=info msg="PullImage \"registry.cn-hangzhou.aliyuncs.com/rancher/hardened-kubernetes:v1.28.15-rke2r1-build20241023\""

 

可以执行命令crictl image ls(命令在/var/lib/rancher/rke2/bin/ 这个目录中)，应该能看到日志里在下载的镜像本地还不存在，结合sar -n DEV 1查看网络流量，看看是否有在干活。大概率是镜像下载太慢了，耐心等待吧.....

[root@k8snode1 ~]# crictl image ls
IMAGE                                                           TAG                                                    IMAGE ID            SIZE
registry.cn-hangzhou.aliyuncs.com/rancher/hardened-etcd         v3.5.13-k3s1-build20240910                             19f8e656ed901       17.4MB
registry.cn-hangzhou.aliyuncs.com/rancher/rke2-cloud-provider   v1.28.15-0.20241016053552-63bfb1936862-build20241016   675a63d01e14d       20.8MB