|NO.Z.00027|——————————|^^ 部署 ^^|——|KuberNetes&二进制部署.V05|5台Server|——|证书生成|
一、生成证书：
### --- 生成证书：Master01下载生成证书工具（下载不成功可以去百度网盘）及创建资源目录
~~~ etcd及kubernetes证书生成
~~~ 二进制安装最关键步骤，一步错误全盘皆输，一定要注意每个步骤都要是正确的### --- 下载证书生成工具
[root@k8s-master01 ~]# wget "https://pkg.cfssl.org/R1.2/cfssl_linux-amd64" -O /usr/local/bin/cfssl
[root@k8s-master01 ~]# wget "https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64" -O /usr/local/bin/cfssljson
[root@k8s-master01 ~]# chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson~~~ # etcd证书：所有安装etcd节点创建etcd证书目录
[root@k8s-master01 ~]# mkdir /etc/etcd/ssl -p
~~~ # 所有节点创建kubernetes相关目录
[root@k8s-master01 ~]# mkdir -p /etc/kubernetes/pki二、生成etcd证书
### --- Master01节点生成etcd证书；生成证书的CSR文件：证书签名请求文件，配置了一些域名、公司、单位
~~~ # 生成etcd CA证书和CA证书的key
[root@k8s-master01 ~]# cd /root/k8s-ha-install/pki
[root@k8s-master01 pki]# cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare /etc/etcd/ssl/etcd-ca
~~~ 注：输出结果：
2021/04/09 17:52:06 [INFO] generating a new CA key and certificate from CSR
2021/04/09 17:52:06 [INFO] generate received request
2021/04/09 17:52:06 [INFO] received CSR
2021/04/09 17:52:06 [INFO] generating key: rsa-2048
2021/04/09 17:52:06 [INFO] encoded CSR
2021/04/09 17:52:06 [INFO] signed certificate with serial number 423190820026756858541446334719884914519938174735### --- 颁发etcd证书
~~~ # 通过生成的ca证书及key颁发证书
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/etcd/ssl/etcd-ca.pem \
-ca-key=/etc/etcd/ssl/etcd-ca-key.pem \
-config=ca-config.json \
-hostname=127.0.0.1,k8s-master01,k8s-master02,k8s-master03,192.168.1.11,192.168.1.12,192.168.1.13 \
-profile=kubernetes \
etcd-csr.json | cfssljson -bare /etc/etcd/ssl/etcd
~~~ 输出结果：
2021/04/09 17:52:57 [INFO] generate received request
2021/04/09 17:52:57 [INFO] received CSR
2021/04/09 17:52:57 [INFO] generating key: rsa-2048
2021/04/09 17:52:57 [INFO] encoded CSR
2021/04/09 17:52:57 [INFO] signed certificate with serial number 713011014384658330270180341022355700267979852195### --- 将etcd证书发送到其它节点
~~~ # 定义变量
[root@k8s-master01 pki]# MasterNodes='k8s-master02 k8s-master03'
[root@k8s-master01 pki]# WorkNodes='k8s-node01 k8s-node02'~~~ # 发送证书到其它节点
[root@k8s-master01 pki]# for NODE in $MasterNodes; do
ssh $NODE "mkdir -p /etc/etcd/ssl"
for FILE in etcd-ca-key.pem etcd-ca.pem etcd-key.pem etcd.pem; do
scp /etc/etcd/ssl/${FILE} $NODE:/etc/etcd/ssl/${FILE}
done
done
~~~ 注：输出结果：
etcd-ca-key.pem 100% 1675 925.2KB/s 00:00
etcd-ca.pem 100% 1367 904.5KB/s 00:00
etcd-key.pem 100% 1679 715.8KB/s 00:00
etcd.pem 100% 1509 526.8KB/s 00:00
etcd-ca-key.pem 100% 1675 994.7KB/s 00:00
etcd-ca.pem 100% 1367 507.2KB/s 00:00
etcd-key.pem 100% 1679 682.7KB/s 00:00
etcd.pem 三、生成证书：k8s组件证书-kube-apiserver证书
### --- Master01生成kubernetes证书
~~~ # 生成kubernetes ca证书和ca证书的key
[root@k8s-master01 pki]# cd /root/k8s-ha-install/pki
[root@k8s-master01 pki]# cfssl gencert -initca ca-csr.json | cfssljson -bare /etc/kubernetes/pki/ca
~~~ 注：输出结果：
2021/04/09 17:55:35 [INFO] generating a new CA key and certificate from CSR
2021/04/09 17:55:35 [INFO] generate received request
2021/04/09 17:55:35 [INFO] received CSR
2021/04/09 17:55:35 [INFO] generating key: rsa-2048
2021/04/09 17:55:35 [INFO] encoded CSR
2021/04/09 17:55:35 [INFO] signed certificate with serial number 312724731765196138565235611759823205222208149928### --- 为kubernetes颁发证书
~~~ # 颁发证书
~~~ 10.96.0.是k8s service的网段，如果说需要更改k8s service网段，那就需要更改10.96.0.1，
~~~ 如果不是高可用集群，192.168.1.11为Master01的IP
[root@k8s-master01 pki]# cfssl gencert -ca=/etc/kubernetes/pki/ca.pem -ca-key=/etc/kubernetes/pki/ca-key.pem -config=ca-config.json -hostname=10.96.0.1,192.168.1.20,127.0.0.1,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,192.168.1.11,192.168.1.12,192.168.1.13 -profile=kubernetes apiserver-csr.json | cfssljson -bare /etc/kubernetes/pki/apiserver
~~~ 注：输出结果：
2021/04/09 17:56:51 [INFO] generate received request
2021/04/09 17:56:51 [INFO] received CSR
2021/04/09 17:56:51 [INFO] generating key: rsa-2048
2021/04/09 17:56:51 [INFO] encoded CSR
2021/04/09 17:56:51 [INFO] signed certificate with serial number 210976959466905225261553741556880293283669782216四、生成apiserver的聚合证书
### --- 生成apiserver的聚合证书。生成apiserver的ca证书和ca证书的key
~~~ # 生成apiserver的聚合证书。Requestheader-client-xxx requestheader-allowwd-xxx:aggerator
[root@k8s-master01 pki]# cfssl gencert -initca front-proxy-ca-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-ca
~~~ 注：输出结果：
2021/04/09 17:57:34 [INFO] generating a new CA key and certificate from CSR
2021/04/09 17:57:34 [INFO] generate received request
2021/04/09 17:57:34 [INFO] received CSR
2021/04/09 17:57:34 [INFO] generating key: rsa-2048
2021/04/09 17:57:34 [INFO] encoded CSR
2021/04/09 17:57:34 [INFO] signed certificate with serial number 229937635932613642720561308611651745796925115826### --- 为apiserver颁发证书
[root@k8s-master01 pki]# cfssl gencert -ca=/etc/kubernetes/pki/front-proxy-ca.pem -ca-key=/etc/kubernetes/pki/front-proxy-ca-key.pem -config=ca-config.json -profile=kubernetes front-proxy-client-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-client
~~~ 注：输出结果：
2021/04/09 17:58:01 [INFO] generate received request
2021/04/09 17:58:01 [INFO] received CSR
2021/04/09 17:58:01 [INFO] generating key: rsa-2048
2021/04/09 17:58:01 [INFO] encoded CSR
2021/04/09 17:58:01 [INFO] signed certificate with serial number 341267100255324312793031252506134637751260697587
2021/04/09 17:58:01 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").五、生成证书ControllerManager证书
### --- 为controller-manager颁发证书
~~~ # 生成ControllerManager证书
~~~ 注意，如果不是高可用集群，192.168.0.236:8443改为master01的地址，8443改为apiserver的端口，默认是6443
~~~ # set-cluster：设置一个集群项
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
manager-csr.json | cfssljson -bare /etc/kubernetes/pki/controller-manager
~~~ 注：输出结果：
2021/04/09 17:59:19 [INFO] generate received request
2021/04/09 17:59:19 [INFO] received CSR
2021/04/09 17:59:19 [INFO] generating key: rsa-2048
2021/04/09 17:59:19 [INFO] encoded CSR
2021/04/09 17:59:19 [INFO] signed certificate with serial number 57476090859638169695558073350960312663800570839
2021/04/09 17:59:19 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").### --- 设置set-cluster集群项
~~~ # 设置集群项
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=https://192.168.0.236:8443 \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：输出结果：
Cluster "kubernetes" set.### --- 设置环境项上下文
~~~ # 设置一个环境项，一个上下文
[root@k8s-master01 pki]# kubectl config set-context system:kube-controller-manager@kubernetes \
--cluster=kubernetes \
--user=system:kube-controller-manager \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：输出结果：
Context "system:kube-controller-manager@kubernetes" created.### --- 设置用户项
~~~ # set-credentials 设置一个用户项
[root@k8s-master01 pki]# kubectl config set-credentials system:kube-controller-manager \
--client-certificate=/etc/kubernetes/pki/controller-manager.pem \
--client-key=/etc/kubernetes/pki/controller-manager-key.pem \
--embed-certs=true \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：输出结果：
User "system:kube-controller-manager" set.### --- 使用某个环境当做默认环境
[root@k8s-master01 pki]# kubectl config use-context system:kube-controller-manager@kubernetes \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：输出结果：
Switched to context "system:kube-controller-manager@kubernetes".六、生成kube-scheduler证书
### --- 为kube-scheduler颁发证书
~~~ # 为kube-scheduler颁发证书
~~~ 注意，如果不是高可用集群，192.168.0.236:8443改为master01的地址，8443改为apiserver的端口，默认是6443
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
scheduler-csr.json | cfssljson -bare /etc/kubernetes/pki/scheduler
~~~ 注：输出结果：
2021/04/09 18:05:35 [INFO] generate received request
2021/04/09 18:05:35 [INFO] received CSR
2021/04/09 18:05:35 [INFO] generating key: rsa-2048
2021/04/09 18:05:36 [INFO] encoded CSR
2021/04/09 18:05:36 [INFO] signed certificate with serial number 581707959605151325249703913778192497550173882170
2021/04/09 18:05:36 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").### --- 设置集群项
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=https://192.168.0.236:8443 \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：输出结果：
Cluster "kubernetes" set.### --- 设置环境项上下文
[root@k8s-master01 pki]# kubectl config set-credentials system:kube-scheduler \
--client-certificate=/etc/kubernetes/pki/scheduler.pem \
--client-key=/etc/kubernetes/pki/scheduler-key.pem \
--embed-certs=true \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：输出结果：
User "system:kube-scheduler" set.### --- 设置用户项
[root@k8s-master01 pki]# kubectl config set-context system:kube-scheduler@kubernetes \
--cluster=kubernetes \
--user=system:kube-scheduler \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：输出结果：
Context "system:kube-scheduler@kubernetes" created.### --- 使用某个环境当做默认环境
[root@k8s-master01 pki]# kubectl config use-context system:kube-scheduler@kubernetes \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：输出结果：
Switched to context "system:kube-scheduler@kubernetes".七、生成kubernetes-admin用户证书
### --- 为kubernetes-admin用户颁发证书
~~~ 注意，如果不是高可用集群，192.168.0.236:8443改为master01的地址，8443改为apiserver的端口，默认是6443
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
admin-csr.json | cfssljson -bare /etc/kubernetes/pki/admin
~~~ 注：输出结果：
2021/04/09 18:10:11 [INFO] generate received request
2021/04/09 18:10:11 [INFO] received CSR
2021/04/09 18:10:11 [INFO] generating key: rsa-2048
2021/04/09 18:10:12 [INFO] encoded CSR
2021/04/09 18:10:12 [INFO] signed certificate with serial number 604960830409772440587252668095907626459060809354
2021/04/09 18:10:12 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").### --- 设置集群项
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.1.20:8443 --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：输出结果：
Cluster "kubernetes" set.### --- 设置环境项上下文
[root@k8s-master01 pki]# kubectl config set-credentials kubernetes-admin --client-certificate=/etc/kubernetes/pki/admin.pem --client-key=/etc/kubernetes/pki/admin-key.pem --embed-certs=true --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：输出结果：
User "kubernetes-admin" set.### --- 设置用户项
[root@k8s-master01 pki]# kubectl config set-context kubernetes-admin@kubernetes --cluster=kubernetes --user=kubernetes-admin --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：输出结果：
Context "kubernetes-admin@kubernetes" created.### --- 使用某个环境当做默认环境
[root@k8s-master01 pki]# kubectl config use-context kubernetes-admin@kubernetes --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：输出结果：
Switched to context "kubernetes-admin@kubernetes".八、创建ServiceAccount key——>secret
### --- 创建ServiceAccount key
[root@k8s-master01 pki]# openssl genrsa -out /etc/kubernetes/pki/sa.key 2048
~~~ 注：输出结果：
Generating RSA private key, 2048 bit long modulus
....................+++
.......................................+++
e is 65537 (0x10001)### --- 生成serviceAccount key
[root@k8s-master01 pki]# openssl rsa -in /etc/kubernetes/pki/sa.key -pubout -out /etc/kubernetes/pki/sa.pub
writing RSA key九、将证书发送至其它节点
### --- 将证书发送至其它节点
[root@k8s-master01 pki]# for NODE in k8s-master02 k8s-master03; do
for FILE in $(ls /etc/kubernetes/pki | grep -v etcd); do
scp /etc/kubernetes/pki/${FILE} $NODE:/etc/kubernetes/pki/${FILE};
done;
for FILE in admin.kubeconfig controller-manager.kubeconfig scheduler.kubeconfig; do
scp /etc/kubernetes/${FILE} $NODE:/etc/kubernetes/${FILE};
done;
done
~~~ 输出结果：
admin.csr 100% 1025 20.8KB/s 00:00
admin-key.pem 100% 1679 643.1KB/s 00:00
admin.pem 100% 1444 408.3KB/s 00:00
apiserver.csr 100% 1029 462.9KB/s 00:00
apiserver-key.pem 100% 1679 611.6KB/s 00:00
apiserver.pem 100% 1692 62.5KB/s 00:00
ca.csr 100% 1025 274.1KB/s 00:00
ca-key.pem 100% 1679 629.6KB/s 00:00
ca.pem 100% 1411 577.3KB/s 00:00
controller-manager.csr 100% 1082 368.9KB/s 00:00
controller-manager-key.pem 100% 1679 537.2KB/s 00:00
controller-manager.pem 100% 1501 477.9KB/s 00:00
front-proxy-ca.csr 100% 891 400.5KB/s 00:00
front-proxy-ca-key.pem 100% 1675 473.6KB/s 00:00
front-proxy-ca.pem 100% 1143 296.4KB/s 00:00
front-proxy-client.csr 100% 903 233.5KB/s 00:00
front-proxy-client-key.pem 100% 1675 35.1KB/s 00:00
front-proxy-client.pem 100% 1188 59.8KB/s 00:00
sa.key 100% 1675 764.3KB/s 00:00
sa.pub 100% 451 226.2KB/s 00:00
scheduler.csr 100% 1058 370.8KB/s 00:00
scheduler-key.pem 100% 1675 799.9KB/s 00:00
scheduler.pem 100% 1476 727.9KB/s 00:00
admin.kubeconfig 100% 6452 306.1KB/s 00:00
controller-manager.kubeconfig 100% 6584 1.9MB/s 00:00
scheduler.kubeconfig ### --- 查看生成的证书及证书数量
~~~ # 查看生成的所有证书
[root@k8s-master01 pki]# ls /etc/kubernetes/pki/
admin.csr apiserver.csr ca.csr controller-manager.csr front-proxy-ca.csr front-proxy-client.csr sa.key scheduler-key.pem
admin-key.pem apiserver-key.pem ca-key.pem controller-manager-key.pem front-proxy-ca-key.pem front-proxy-client-key.pem sa.pub scheduler.pem
admin.pem apiserver.pem ca.pem controller-manager.pem front-proxy-ca.pem front-proxy-client.pem scheduler.csr~~~ # 查看生成证书数量
[root@k8s-master01 pki]# ls /etc/kubernetes/pki/ |wc -l
23Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart
——W.S.Landor
浙公网安备 33010602011771号