|NO.Z.00052|——————————|^^ 部署 ^^|——|KuberNetes&二进制部署.V05|3台Server|——|证书生成|
一、生成证书：
### --- Master01下载生成证书工具（下载不成功可以去百度网盘）及创建资源目录
~~~ etcd及kubernetes证书生成
~~~ 二进制安装最关键步骤，一步错误全盘皆输，一定要注意每个步骤都要是正确的### ---下载证书生成工具
[root@k8s-master01 ~]# wget "https://pkg.cfssl.org/R1.2/cfssl_linux-amd64" -O /usr/local/bin/cfssl
[root@k8s-master01 ~]# wget "https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64" -O /usr/local/bin/cfssljson
[root@k8s-master01 ~]# chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson~~~ # etcd证书：所有安装etcd节点创建etcd证书目录
[root@k8s-master01 ~]# mkdir /etc/etcd/ssl -p
~~~ # 所有节点创建kubernetes相关目录
[root@k8s-master01 ~]# mkdir -p /etc/kubernetes/pki二、生成etcd证书
### --- Master01节点生成etcd证书；生成证书的CSR文件：证书签名请求文件，配置了一些域名、公司、单位
~~~ # 生成etcd CA证书和CA证书的key
[root@k8s-master01 ~]# cd /root/k8s-ha-install/pki
[root@k8s-master01 pki]# cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare /etc/etcd/ssl/etcd-ca
~~~ 注：输出结果：
2021/05/12 19:15:31 [INFO] generating a new CA key and certificate from CSR
2021/05/12 19:15:31 [INFO] generate received request
2021/05/12 19:15:31 [INFO] received CSR
2021/05/12 19:15:31 [INFO] generating key: rsa-2048
2021/05/12 19:15:32 [INFO] encoded CSR
2021/05/12 19:15:32 [INFO] signed certificate with serial number 417879652597954519889260948756440442182907581235### --- 颁发etcd证书
~~~ # 通过生成的ca证书及key颁发证书
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/etcd/ssl/etcd-ca.pem \
-ca-key=/etc/etcd/ssl/etcd-ca-key.pem \
-config=ca-config.json \
-hostname=127.0.0.1,k8s-master01,k8s-node01,k8s-node02,192.168.1.11,192.168.1.14,192.168.1.15 \
-profile=kubernetes \
etcd-csr.json | cfssljson -bare /etc/etcd/ssl/etcd
~~~ 注：输出结果：
2021/05/12 19:20:27 [INFO] generate received request
2021/05/12 19:20:27 [INFO] received CSR
2021/05/12 19:20:27 [INFO] generating key: rsa-2048
2021/05/12 19:20:27 [INFO] encoded CSR
2021/05/12 19:20:27 [INFO] signed certificate with serial number 452010686264797775985430527541917102604725591793### --- 将etcd证书发送到其它节点
~~~ # 定义变量
[root@k8s-master01 pki]# MasterNodes='k8s-node01 k8s-node02'
[root@k8s-master01 pki]# WorkNodes='k8s-node01 k8s-node02'~~~ # 发送证书到其它节点
[root@k8s-master01 pki]# for NODE in $MasterNodes; do
ssh $NODE "mkdir -p /etc/etcd/ssl"
for FILE in etcd-ca-key.pem etcd-ca.pem etcd-key.pem etcd.pem; do
scp /etc/etcd/ssl/${FILE} $NODE:/etc/etcd/ssl/${FILE}
done
done
~~~ 注：输出结果：
etcd-ca-key.pem 100% 1679 877.4KB/s 00:00
etcd-ca.pem 100% 1367 648.2KB/s 00:00
etcd-key.pem 100% 1675 634.7KB/s 00:00
etcd.pem 100% 1501 350.7KB/s 00:00
etcd-ca-key.pem 100% 1679 468.6KB/s 00:00
etcd-ca.pem 100% 1367 387.5KB/s 00:00
etcd-key.pem 100% 1675 404.1KB/s 00:00
etcd.pem 三、生成证书：k8s组件证书-kube-apiserver证书
### --- Master01生成kubernetes证书
~~~ # 生成kubernetes ca证书和ca证书的key
[root@k8s-master01 pki]# cd /root/k8s-ha-install/pki
[root@k8s-master01 pki]# cfssl gencert -initca ca-csr.json | cfssljson -bare /etc/kubernetes/pki/ca
~~~ 注：输出结果：
2021/05/12 19:24:28 [INFO] generating a new CA key and certificate from CSR
2021/05/12 19:24:28 [INFO] generate received request
2021/05/12 19:24:28 [INFO] received CSR
2021/05/12 19:24:28 [INFO] generating key: rsa-2048
2021/05/12 19:24:28 [INFO] encoded CSR
2021/05/12 19:24:28 [INFO] signed certificate with serial number 447109712814672408133045353535582932352630134506### --- 为kubernetes颁发证书
~~~ # 颁发证书
~~~ 10.96.0.是k8s service的网段，如果说需要更改k8s service网段，那就需要更改10.96.0.1，
~~~ 如果不是高可用集群，192.168.1.11为Master01的IP
[root@k8s-master01 pki]# cfssl gencert -ca=/etc/kubernetes/pki/ca.pem -ca-key=/etc/kubernetes/pki/ca-key.pem -config=ca-config.json -hostname=10.96.0.1,192.168.1.11,127.0.0.1,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,192.168.1.11,192.168.1.14,192.168.1.15 -profile=kubernetes apiserver-csr.json | cfssljson -bare /etc/kubernetes/pki/apiserver
~~~ 注：输出结果：
2021/05/12 19:27:37 [INFO] generate received request
2021/05/12 19:27:37 [INFO] received CSR
2021/05/12 19:27:37 [INFO] generating key: rsa-2048
2021/05/12 19:27:37 [INFO] encoded CSR
2021/05/12 19:27:37 [INFO] signed certificate with serial number 84788908146128667480104666726419859131741151671四、生成apiserver的聚合证书
### --- 生成apiserver的聚合证书。生成apiserver的ca证书和ca证书的key
~~~ # 生成apiserver的聚合证书。Requestheader-client-xxx requestheader-allowwd-xxx:aggerator
[root@k8s-master01 pki]# cfssl gencert -initca front-proxy-ca-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-ca
~~~ 注：输出结果：
netes front-proxy-client-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-client
2021/05/12 19:28:52 [INFO] generating a new CA key and certificate from CSR
2021/05/12 19:28:52 [INFO] generate received request
2021/05/12 19:28:52 [INFO] received CSR
2021/05/12 19:28:52 [INFO] generating key: rsa-2048
2021/05/12 19:28:52 [INFO] encoded CSR
2021/05/12 19:28:52 [INFO] signed certificate with serial number 516217615073867303946934109541825127048193138588### --- 为apiserver颁发证书
[root@k8s-master01 pki]# cfssl gencert -ca=/etc/kubernetes/pki/front-proxy-ca.pem -ca-key=/etc/kubernetes/pki/front-proxy-ca-key.pem -config=ca-config.json -profile=kubernetes front-proxy-client-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-client
2021/05/12 19:28:52 [INFO] generate received request
2021/05/12 19:28:52 [INFO] received CSR
2021/05/12 19:28:52 [INFO] generating key: rsa-2048
2021/05/12 19:28:53 [INFO] encoded CSR
2021/05/12 19:28:53 [INFO] signed certificate with serial number 696831972351009499497638617028756244195237115056
2021/05/12 19:28:53 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
~~~ 注：输出结果：五、生成证书ControllerManager证书
### --- 为controller-manager颁发证书
~~~ # 生成ControllerManager证书
~~~ 注意：如果不是高可用集群，192.168.1.11:8443改为master01的地址，8443改为apiserver的端口，默认是6443
~~~ #set-cluster：设置一个集群项
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
manager-csr.json | cfssljson -bare /etc/kubernetes/pki/controller-manager
~~~ 注：输出结果：
2021/05/12 19:31:16 [INFO] generate received request
2021/05/12 19:31:16 [INFO] received CSR
2021/05/12 19:31:16 [INFO] generating key: rsa-2048
2021/05/12 19:31:16 [INFO] encoded CSR
2021/05/12 19:31:16 [INFO] signed certificate with serial number 384930359684285771093349021380105968424292629747
2021/05/12 19:31:16 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").### --- 设置set-cluster集群项
~~~ # 设置集群项
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=https://192.168.1.11:6443 \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：输出结果：
Cluster "kubernetes" set.### --- 设置环境项上下文
~~~ # 设置一个环境项，一个上下文
[root@k8s-master01 pki]# kubectl config set-context system:kube-controller-manager@kubernetes \
--cluster=kubernetes \
--user=system:kube-controller-manager \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：输出结果：
Context "system:kube-controller-manager@kubernetes" created.### --- 设置用户项
~~~ # set-credentials 设置一个用户项
[root@k8s-master01 pki]# kubectl config set-credentials system:kube-controller-manager \
--client-certificate=/etc/kubernetes/pki/controller-manager.pem \
--client-key=/etc/kubernetes/pki/controller-manager-key.pem \
--embed-certs=true \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：输出结果：
User "system:kube-controller-manager" set.### --- 使用某个环境当做默认环境
~~~ # 使用某个环境当做默认环境
[root@k8s-master01 pki]# kubectl config use-context system:kube-controller-manager@kubernetes \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：输出结果：
Switched to context "system:kube-controller-manager@kubernetes".六、生成kube-scheduler证书
### --- 为kube-scheduler颁发证书
~~~ # 注意，如果不是高可用集群，192.168.1.11:8443改为master01的地址，8443改为apiserver的端口，默认是6443
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
scheduler-csr.json | cfssljson -bare /etc/kubernetes/pki/scheduler
~~~ 注：输出结果：
2021/05/12 19:36:56 [INFO] generate received request
2021/05/12 19:36:56 [INFO] received CSR
2021/05/12 19:36:56 [INFO] generating key: rsa-2048
2021/05/12 19:36:56 [INFO] encoded CSR
2021/05/12 19:36:56 [INFO] signed certificate with serial number 331467505877315472816673290342178535942330545986
2021/05/12 19:36:56 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").### --- 设置集群项
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=https://192.168.1.11:6443 \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：输出结果：
Cluster "kubernetes" set.### --- 设置环境项上下文
[root@k8s-master01 pki]# kubectl config set-credentials system:kube-scheduler \
--client-certificate=/etc/kubernetes/pki/scheduler.pem \
--client-key=/etc/kubernetes/pki/scheduler-key.pem \
--embed-certs=true \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：输出结果：
User "system:kube-scheduler" set.### --- 设置用户项
[root@k8s-master01 pki]# kubectl config set-context system:kube-scheduler@kubernetes \
--cluster=kubernetes \
--user=system:kube-scheduler \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：输出结果：
Context "system:kube-scheduler@kubernetes" created.### --- 使用某个环境当做默认环境
[root@k8s-master01 pki]# kubectl config use-context system:kube-scheduler@kubernetes \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：输出结果：
Switched to context "system:kube-scheduler@kubernetes".七、生成kubernetes-admin用户证书
### --- 为kubernetes-admin用户颁发证书
~~~ # 注意，如果不是高可用集群，192.168.1.11:8443改为master01的地址，8443改为apiserver的端口，默认是6443
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
admin-csr.json | cfssljson -bare /etc/kubernetes/pki/admin
~~~ 注：输出结果：
2021/05/12 19:39:12 [INFO] generate received request
2021/05/12 19:39:12 [INFO] received CSR
2021/05/12 19:39:12 [INFO] generating key: rsa-2048
2021/05/12 19:39:12 [INFO] encoded CSR
2021/05/12 19:39:12 [INFO] signed certificate with serial number 267138241499797848649434576652196091163365718803
2021/05/12 19:39:12 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").### --- 设置集群项
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.1.11:6443 --kubeconfig=/etc/kubernetes/admin.kubeconfig
--client-certificate=/etc/kubernetes/pki/admin.pem --client-key=/etc/kubernetes/pki/admin-key.pem --embed-certs=true --kubeconfig=/etc/kubernetes/admin.kubeconfig
kubectl config set-context kubernetes-admin@kubernetes --cluster=kubernetes --user=kubernetes-admin --kubeconfig=/etc/kubernetes/admin.kubeconfig
kubectl config use-context kubernetes-admin@kubernetes --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：输出结果：
Cluster "kubernetes" set.### --- 设置环境项上下文
[root@k8s-master01 pki]# kubectl config set-credentials kubernetes-admin --client-certificate=/etc/kubernetes/pki/admin.pem --client-key=/etc/kubernetes/pki/admin-key.pem --embed-certs=true --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：输出结果：
User "kubernetes-admin" set.### --- 设置用户项
[root@k8s-master01 pki]# kubectl config set-context kubernetes-admin@kubernetes --cluster=kubernetes --user=kubernetes-admin --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：输出结果：
Context "kubernetes-admin@kubernetes" created.### --- 使用某个环境当做默认环境
[root@k8s-master01 pki]# kubectl config use-context kubernetes-admin@kubernetes --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：输出结果：
Switched to context "kubernetes-admin@kubernetes".八、创建ServiceAccount key——>secret
### --- 创建ServiceAccount key
[root@k8s-master01 pki]# openssl genrsa -out /etc/kubernetes/pki/sa.key 2048
~~~ 输出结果：
Generating RSA private key, 2048 bit long modulus
.......................................+++
..........+++
e is 65537 (0x10001)### --- 生成serviceAccount key
[root@k8s-master01 pki]# openssl rsa -in /etc/kubernetes/pki/sa.key -pubout -out /etc/kubernetes/pki/sa.pub
writing RSA key九、将证书发送至其它节点
### --- 将证书发送至其它节点
[root@k8s-master01 pki]# for NODE in k8s-node01 k8s-node02; do
for FILE in $(ls /etc/kubernetes/pki | grep -v etcd); do
scp /etc/kubernetes/pki/${FILE} $NODE:/etc/kubernetes/pki/${FILE};
done;
for FILE in admin.kubeconfig controller-manager.kubeconfig scheduler.kubeconfig; do
scp /etc/kubernetes/${FILE} $NODE:/etc/kubernetes/${FILE};
done;
done
~~~ 输出结果：
admin.csr 100% 1025 357.0KB/s 00:00
admin-key.pem 100% 1679 734.2KB/s 00:00
admin.pem 100% 1444 552.1KB/s 00:00
apiserver.csr 100% 1029 246.7KB/s 00:00
apiserver-key.pem 100% 1675 583.3KB/s 00:00
apiserver.pem 100% 1692 631.7KB/s 00:00
ca.csr 100% 1025 248.2KB/s 00:00
ca-key.pem 100% 1675 593.3KB/s 00:00
ca.pem 100% 1411 707.0KB/s 00:00
controller-manager.csr 100% 1082 292.6KB/s 00:00
controller-manager-key.pem 100% 1675 316.4KB/s 00:00
controller-manager.pem 100% 1501 540.6KB/s 00:00
front-proxy-ca.csr 100% 891 453.1KB/s 00:00
front-proxy-ca-key.pem 100% 1679 64.9KB/s 00:00
front-proxy-ca.pem 100% 1143 575.4KB/s 00:00
front-proxy-client.csr 100% 903 248.0KB/s 00:00
front-proxy-client-key.pem 100% 1679 408.5KB/s 00:00
front-proxy-client.pem 100% 1188 296.9KB/s 00:00
sa.key 100% 1679 387.5KB/s 00:00
sa.pub 100% 451 156.3KB/s 00:00
scheduler.csr 100% 1058 465.2KB/s 00:00
scheduler-key.pem 100% 1679 238.3KB/s 00:00
scheduler.pem 100% 1476 372.8KB/s 00:00
admin.kubeconfig 100% 6452 143.4KB/s 00:00
controller-manager.kubeconfig 100% 6580 1.5MB/s 00:00
scheduler.kubeconfig 100% 6512 1.5MB/s 00:00
admin.csr 100% 1025 116.5KB/s 00:00
admin-key.pem 100% 1679 78.4KB/s 00:00
admin.pem 100% 1444 365.7KB/s 00:00
apiserver.csr 100% 1029 274.5KB/s 00:00
apiserver-key.pem 100% 1675 196.8KB/s 00:00
apiserver.pem 100% 1692 338.6KB/s 00:00
ca.csr 100% 1025 115.5KB/s 00:00
ca-key.pem 100% 1675 393.6KB/s 00:00
ca.pem 100% 1411 143.1KB/s 00:00
controller-manager.csr 100% 1082 139.4KB/s 00:00
controller-manager-key.pem 100% 1675 157.9KB/s 00:00
controller-manager.pem 100% 1501 277.9KB/s 00:00
front-proxy-ca.csr 100% 891 201.4KB/s 00:00
front-proxy-ca-key.pem 100% 1679 214.3KB/s 00:00
front-proxy-ca.pem 100% 1143 167.7KB/s 00:00
front-proxy-client.csr 100% 903 169.0KB/s 00:00
front-proxy-client-key.pem 100% 1679 393.0KB/s 00:00
front-proxy-client.pem 100% 1188 235.9KB/s 00:00
sa.key 100% 1679 94.5KB/s 00:00
sa.pub 100% 451 55.0KB/s 00:00
scheduler.csr 100% 1058 371.9KB/s 00:00
scheduler-key.pem 100% 1679 325.1KB/s 00:00
scheduler.pem 100% 1476 201.7KB/s 00:00
admin.kubeconfig 100% 6452 1.1MB/s 00:00
controller-manager.kubeconfig 100% 6580 687.2KB/s 00:00
scheduler.kubeconfig ### --- 查看生成的证书及证书数量
~~~ # 查看生成的所有证书
[root@k8s-master01 pki]# ls /etc/kubernetes/pki/
admin.csr apiserver.csr ca.csr controller-manager.csr front-proxy-ca.csr front-proxy-client.csr sa.key scheduler-key.pem
admin-key.pem apiserver-key.pem ca-key.pem controller-manager-key.pem front-proxy-ca-key.pem front-proxy-client-key.pem sa.pub scheduler.pem
admin.pem apiserver.pem ca.pem controller-manager.pem front-proxy-ca.pem front-proxy-client.pem scheduler.csr~~~ # 查看生成证书数量
[root@k8s-master01 pki]# ls /etc/kubernetes/pki/ |wc -l
23
Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart
——W.S.Landor
浙公网安备 33010602011771号
