Leo Zhang
我是一块砖,哪里需要哪里搬!

Kubernetes 需要 PKI 证书才能进行基于 TLS 的身份验证,如果你是使用kubeadm安装的 Kubernetes,则会自动生成集群所需的证书。

♦  API 服务器端点的证书

♦  Kubelet 的客户端证书,用于 API 服务器身份验证

♦  集群管理员的客户端证书,用于 API 服务器身份认证

♦  API 服务器的客户端证书,用于和 Kubelet 的会话

♦  API 服务器的客户端证书,用于和 etcd 的会话

♦  控制管理器的客户端证书/kubeconfig,用于和 API server 的会话

♦  调度器的客户端证书/kubeconfig,用于和 API server 的会话

♦  前端代理的客户端及服务端证书

详情参考官方说明https://kubernetes.io/zh/docs/setup/best-practices/certificates/ 

 


查看证书

注意:默认根证书有效期为10年,其他所有证书有效期为1年。

[root@ymt108 ~]# cd /etc/kubernetes/pki
[root@ymt108 pki]# tree
.
├── apiserver.crt
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
├── apiserver.key
├── apiserver-kubelet-client.crt
├── apiserver-kubelet-client.key
├── ca.crt
├── ca.key
├── etcd
│   ├── ca.crt
│   ├── ca.key
│   ├── healthcheck-client.crt
│   ├── healthcheck-client.key
│   ├── peer.crt
│   ├── peer.key
│   ├── server.crt
│   └── server.key
├── front-proxy-ca.crt
├── front-proxy-ca.key
├── front-proxy-client.crt
├── front-proxy-client.key
├── sa.key
└── sa.pub

1 directory, 22 files
[root@ymt108 pki]# kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Jul 03, 2021 01:02 UTC   322d                                    no
apiserver                  Jul 03, 2021 01:02 UTC   322d            ca                      no
apiserver-etcd-client      Jul 03, 2021 01:02 UTC   322d            etcd-ca                 no
apiserver-kubelet-client   Jul 03, 2021 01:02 UTC   322d            ca                      no
controller-manager.conf    Jul 03, 2021 01:02 UTC   322d                                    no
etcd-healthcheck-client    Jul 03, 2021 01:02 UTC   322d            etcd-ca                 no
etcd-peer                  Jul 03, 2021 01:02 UTC   322d            etcd-ca                 no
etcd-server                Jul 03, 2021 01:02 UTC   322d            etcd-ca                 no
front-proxy-client         Jul 03, 2021 01:02 UTC   322d            front-proxy-ca          no
scheduler.conf             Jul 03, 2021 01:02 UTC   322d                                    no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Jul 01, 2030 01:02 UTC   9y              no
etcd-ca                 Jul 01, 2030 01:02 UTC   9y              no
front-proxy-ca          Jul 01, 2030 01:02 UTC   9y              no

 


更新证书

1、手动生成证书自定义时长

    我们可以通过 easyrsaopenssl 或 cfssl 手动地为集群生成证书,然后自定义证书时长。

    详情参考官方说明:https://kubernetes.io/zh/docs/concepts/cluster-administration/certificates/

  

2、定期升级集群来升级证书时长

    kubeadm 会在控制面板升级的时候更新所有证书,这个功能旨在解决最简单的用例。

    详情参考官方说明:https://kubernetes.cn/zh/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/

  • 如果你对此类证书的更新没有特殊要求,并且定期执行 Kubernetes 版本升级(每次升级之间的间隔时间少于 1 年),则 kubeadm 将确保你的集群保持最新状态并保持合理的安全性。
  • 如果你对证书更新有更复杂的需求,则可通过将 --certificate-renewal=false 传递给 kubeadm upgrade apply 或者 kubeadm upgrade node ,从而选择不采用默认行为。

 

3、通过kubeadm命令升级证书时长

你可以随时通过 kubeadm alpha certs renew 命令手动更新证书,也可以选择更新单个证书或者全部证书。

流程如下:备份配置 -> 获取集群配置  ->  升级所有证书  ->  删除已有配置 -> 重新生成配置  -> 拷贝config ->  重启k8s容器 -> 重启kubelet

 注意: 如果你运行了一个 HA 集群,这个命令需要在所有控制面板节点上执行。

[root@k8s-master ~]# kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 May 17, 2022 02:52 UTC   364d                                    no
apiserver                  May 17, 2022 02:52 UTC   364d            ca                      no
apiserver-etcd-client      May 17, 2022 02:52 UTC   364d            etcd-ca                 no
apiserver-kubelet-client   May 17, 2022 02:52 UTC   364d            ca                      no
controller-manager.conf    May 17, 2022 02:52 UTC   364d                                    no
etcd-healthcheck-client    May 17, 2022 02:52 UTC   364d            etcd-ca                 no
etcd-peer                  May 17, 2022 02:52 UTC   364d            etcd-ca                 no
etcd-server                May 17, 2022 02:52 UTC   364d            etcd-ca                 no
front-proxy-client         May 17, 2022 02:52 UTC   364d            front-proxy-ca          no
scheduler.conf             May 17, 2022 02:52 UTC   364d                                    no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      May 15, 2031 02:52 UTC   9y              no
etcd-ca                 May 15, 2031 02:52 UTC   9y              no
front-proxy-ca          May 15, 2031 02:52 UTC   9y              no
[root@k8s-master ~]#
[root@k8s-master ~]# date -s "2022-2-2"
Wed Feb  2 00:00:00 PST 2022
[root@k8s-master ~]# cp -r /etc/kubernetes  /etc/kubernetes_old
[root@k8s-master ~]# kubeadm config view > kubeadm-upgrade.yaml
[root@k8s-master ~]# cat kubeadm-upgrade.yaml
apiServer:
  extraArgs:
    authorization-mode: Node,RBAC
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.17.5
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}
[root@k8s-master ~]# kubeadm alpha certs renew all --config kubeadm-upgrade.yaml
W0202 00:00:39.255964   43273 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0202 00:00:39.256023   43273 validation.go:28] Cannot validate kubelet config - no validator is available
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed
[root@k8s-master ~]#
[root@k8s-master ~]# kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Feb 02, 2023 08:00 UTC   364d                                    no
apiserver                  Feb 02, 2023 08:00 UTC   364d            ca                      no
apiserver-etcd-client      Feb 02, 2023 08:00 UTC   364d            etcd-ca                 no
apiserver-kubelet-client   Feb 02, 2023 08:00 UTC   364d            ca                      no
controller-manager.conf    Feb 02, 2023 08:00 UTC   364d                                    no
etcd-healthcheck-client    Feb 02, 2023 08:00 UTC   364d            etcd-ca                 no
etcd-peer                  Feb 02, 2023 08:00 UTC   364d            etcd-ca                 no
etcd-server                Feb 02, 2023 08:00 UTC   364d            etcd-ca                 no
front-proxy-client         Feb 02, 2023 08:00 UTC   364d            front-proxy-ca          no
scheduler.conf             Feb 02, 2023 08:00 UTC   364d                                    no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      May 15, 2031 02:52 UTC   9y              no
etcd-ca                 May 15, 2031 02:52 UTC   9y              no
front-proxy-ca          May 15, 2031 02:52 UTC   9y              no
[root@k8s-master ~]#
[root@k8s-master ~]# rm -rf /etc/kubernetes/*.conf
[root@k8s-master ~]# kubeadm init phase kubeconfig all --config kubeadm-upgrade.yaml
W0202 00:01:00.936473   43453 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0202 00:01:00.936517   43453 validation.go:28] Cannot validate kubelet config - no validator is available
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[root@k8s-master ~]#
[root@k8s-master ~]# \cp /etc/kubernetes/admin.conf $HOME/.kube/config
[root@k8s-master ~]# docker ps |grep -E 'k8s_kube-apiserver|k8s_kube-controller-manager|k8s_kube-scheduler|k8s_etcd_etcd' | awk -F ' ' '{print $1}' |xargs docker restart
912e36b808af
57e3c34bf9eb
2b53c4267a3b
a19d0ad23539
[root@k8s-master ~]# date -s "2023-1-1" 
Sun Jan 1 00:00:00 PST 2023
[root@k8s-master ~]# systemctl restart kubelet
[root@k8s-master ~]# kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Feb 02, 2023 08:00 UTC   31d                                     no
apiserver                  Feb 02, 2023 08:00 UTC   31d             ca                      no
apiserver-etcd-client      Feb 02, 2023 08:00 UTC   31d             etcd-ca                 no
apiserver-kubelet-client   Feb 02, 2023 08:00 UTC   31d             ca                      no
controller-manager.conf    Feb 02, 2023 08:00 UTC   31d                                     no
etcd-healthcheck-client    Feb 02, 2023 08:00 UTC   31d             etcd-ca                 no
etcd-peer                  Feb 02, 2023 08:00 UTC   31d             etcd-ca                 no
etcd-server                Feb 02, 2023 08:00 UTC   31d             etcd-ca                 no
front-proxy-client         Feb 02, 2023 08:00 UTC   31d             front-proxy-ca          no
scheduler.conf             Feb 02, 2023 08:00 UTC   31d                                     no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      May 15, 2031 05:57 UTC   8y              no
etcd-ca                 May 15, 2031 05:57 UTC   8y              no
front-proxy-ca          May 15, 2031 05:57 UTC   8y              no
[root@k8s-master ~]# kubectl get nodes -w
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   77s   v1.17.5
k8s-master   Ready    master   261d   v1.17.5
[root@k8s-master ~]# kubectl get nodes -w
NAME         STATUS   ROLES    AGE    VERSION
k8s-master   Ready    master   261d   v1.17.5
k8s-master   Ready    master   261d   v1.17.5
k8s-master   NotReady   master   594d   v1.17.5
k8s-master   NotReady   master   594d   v1.17.5
k8s-master   NotReady   master   594d   v1.17.5
k8s-master   Ready      master   594d   v1.17.5
k8s-master   Ready      master   594d   v1.17.5
[root@k8s-master ~]# kubectl get pod --all-namespaces -w
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE
kube-system   coredns-9d85f5447-cm47b              1/1     Running   0          117s
kube-system   coredns-9d85f5447-mmhrk              1/1     Running   0          117s
kube-system   etcd-k8s-master                      1/1     Running   0          114s
kube-system   kube-apiserver-k8s-master            1/1     Running   0          114s
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          114s
kube-system   kube-flannel-ds-amd64-v87r4          1/1     Running   0          37s
kube-system   kube-proxy-k288j                     1/1     Running   0          117s
kube-system   kube-scheduler-k8s-master            1/1     Running   0          114s
kube-system   coredns-9d85f5447-cm47b              1/1     Running   0          261d
kube-system   coredns-9d85f5447-mmhrk              1/1     Running   0          261d
kube-system   etcd-k8s-master                      1/1     Running   0          261d
kube-system   kube-apiserver-k8s-master            1/1     Running   0          261d
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          261d
kube-system   kube-flannel-ds-amd64-v87r4          1/1     Running   0          261d
kube-system   kube-proxy-k288j                     1/1     Running   0          261d
kube-system   kube-scheduler-k8s-master            1/1     Running   0          261d
[root@k8s-master ~]# kubectl get pod --all-namespaces -w
kube-system   etcd-k8s-master                      1/1     Running   0          261d
kube-system   kube-apiserver-k8s-master            1/1     Running   0          261d
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          261d
kube-system   coredns-9d85f5447-mmhrk              0/1     Running   0          261d
kube-system   coredns-9d85f5447-cm47b              0/1     Running   0          261d
kube-system   kube-scheduler-k8s-master            1/1     Running   0          261d
kube-system   coredns-9d85f5447-cm47b              1/1     Running   0          261d
kube-system   coredns-9d85f5447-mmhrk              1/1     Running   0          261d
kube-system   etcd-k8s-master                      1/1     Running   0          594d
kube-system   kube-apiserver-k8s-master            1/1     Running   0          594d
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          594d
kube-system   kube-flannel-ds-amd64-v87r4          1/1     Running   0          594d
kube-system   kube-proxy-k288j                     1/1     Running   0          594d
kube-system   kube-scheduler-k8s-master            1/1     Running   0          594d
kube-system   coredns-9d85f5447-cm47b              1/1     Running   0          594d
kube-system   coredns-9d85f5447-mmhrk              1/1     Running   0          594d
[root@localhost k8s]# kubeadm init phase kubeconfig all --help
Generates all kubeconfig files

Usage:
  kubeadm init phase kubeconfig all [flags]

Flags:
      --apiserver-advertise-address string   The IP address the API Server will advertise it's listening on. Specify '0.0.0.0' to use the address of the default network interface.
      --apiserver-bind-port int32            Port for the API Server to bind to. (default 6443)
      --cert-dir string                      The path where to save and store the certificates. (default "/etc/kubernetes/pki")
      --config string                        Path to kubeadm config file. WARNING: Usage of a configuration file is experimental.
  -h, --help                                 help for all
      --kubeconfig-dir string                The path where to save the kubeconfig file. (default "/etc/kubernetes")
      --node-name string                     Specify the node name.

Global Flags:
      --log-file string   If non-empty, use this log file
      --rootfs string     [EXPERIMENTAL] The path to the 'real' host root filesystem.
      --skip-headers      If true, avoid header prefixes in the log messages
  -v, --v Level           log level for V logs

 

 

    个人想法: 手动生成证书和编译kubeadm都有点繁琐,定期升级k8s版本也得要项目情况,最简单就是每年进行一次 kubeadm alpha certs renew 即可。

参考博文: 使用 kubeadm 进行证书管理          附025.kubeadm部署Kubernetes更新证书     Kubernetes v1.13.0 证书升级详解

作者:Leozhanggg

出处:https://www.cnblogs.com/leozhanggg/p/13401877.html

本文版权归作者和博客园共有,欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接,否则保留追究法律责任的权利。

 

posted on 2020-08-14 16:30  LeoZhanggg  阅读(729)  评论(1编辑  收藏  举报