使用二进制搭建一个k8s环境并使用Bootstrap Token方式增加一台Node

环境准备：

两台虚拟机一台master节点，一台node节点；

master节点为 2h2g，ip为1.1.1.10；

node节点为1h1g，ip为1.1.1.11；

搭建环境：

cd ~

yum install -y epel-release
yum -y install lrzsz ansible

下载软件包到本地
链接：https://pan.baidu.com/s/1EWnJoJjAD3GNqghOwgodWQ 提取码：tlvz

通过rz上传离线包：binary_pkg.tar.gz
mkdir /root/binary_pkg
tar -zxvf binary_pkg.tar.gz -C /root/binary_pkg

git clone https://github.com/lizhenliang/ansible-install-k8s

cd ansible-install-k8s/
vim hosts

[master]
# 如果部署单Master，只保留一个Master节点
# 默认Naster节点也部署Node组件
1.1.1.10 node_name=k8s-master1

[node]


[etcd]
1.1.1.10 etcd_name=etcd-1

[lb]

[k8s:children]
master
node

vim group_vars/all.yml

# 安装目录
software_dir: '/root/binary_pkg'
k8s_work_dir: '/opt/kubernetes'
etcd_work_dir: '/opt/etcd'
tmp_dir: '/tmp/k8s'

# 集群网络
service_cidr: '10.0.0.0/24'
cluster_dns: '10.0.0.2'   # 与roles/addons/files/coredns.yaml中IP一致，并且是service_cidr中的IP
pod_cidr: '10.244.0.0/16' # 与roles/addons/files/kube-flannel.yaml中网段一致
service_nodeport_range: '30000-32767'
cluster_domain: 'cluster.local'

# 高可用，如果部署单Master，该项忽略


# 自签证书可信任IP列表，为方便扩展，可添加多个预留IP
cert_hosts:
  # 包含所有LB、VIP、Master IP和service_cidr的第一个IP
  k8s:
    - 10.0.0.1
    - 1.1.1.10
    - 1.1.1.11
    - 1.1.1.12
  # 包含所有etcd节点IP
  etcd:
    - 1.1.1.10

ansible-playbook -i hosts single-master-deploy.yml -uroot -k

kubectl get node
NAME          STATUS   ROLES    AGE   VERSION
k8s-master1   Ready    <none>   14m   v1.18.6

 

使用Bootstrap Token方式增加一台Node

安装docker
master节点上面的包scp到node节点
scp -r ansible-install-k8s/roles/docker/* 1.1.1.11:/root/
scp docker-19.03.9.tgz 1.1.1.11:/root/

tar -zxvf docker-19.03.9.tgz

cp -rf docker/* /usr/bin/

cd ansible-install-k8s/roles/docker/files/

cp docker.service /usr/lib/systemd/system/

mkdir /etc/docker
cp daemon.json /etc/docker/

systemctl enable docker
systemctl restart docker

添加node节点

从1.1.1.10 master节点上面分发文件
scp -r /opt/kubernetes/ 1.1.1.11:/opt/
scp /usr/lib/systemd/system/{kubelet,kube-proxy}.service 1.1.1.11:/usr/lib/systemd/system/

分发cni网络
scp -r cni/ 1.1.1.11:/opt/


在新添加的1.1.1.11 node节点上面操作
删掉master节点的证书，后面会重新生成
cd /opt/kubernetes/ssl
rm -rf kubelet* ca-key.pem etcd server*

ls
ca.pem  kube-proxy-key.pem  kube-proxy.pem

cd /opt/kubernetes/cfg

删掉之前节点的“bootstrap.kubeconfig”和“kubelet.kubeconfig”

rm -f bootstrap.kubeconfig kubelet.kubeconfig

修改主机名(修改为新添加的主机名)
vim kubelet.conf
vim kube-proxy-config.yml

在1.1.1.10 master节点操作
确认已经开启bootstrap-token
cat /opt/kubernetes/cfg/kube-apiserver.conf|grep boot
--enable-bootstrap-token-auth=true \

参考链接（中文版有坑，使用英文版靠谱）：https://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens/
创建secret

apiVersion: v1
kind: Secret
metadata:
  name: bootstrap-token-07401b
  namespace: kube-system
type: bootstrap.kubernetes.io/token
stringData:
  description: "The default bootstrap token generated by 'kubeadm init'."
  token-id: 07401b
  # stringData 类型数据不用64位编码
  token-secret: f395accd246ae52d
  # 时间改晚几天即可
  expiration: "2020-12-30T03:22:11Z"
  usage-bootstrap-authentication: "true"
  usage-bootstrap-signing: "true"

kubectl create -f secret.yaml

查询
kubectl get secret

参考链接（使用英文版靠谱）：https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/
创建ClusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: create-csrs-for-bootstrapping
subjects:
- kind: Group
  name: system:bootstrappers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: system:node-bootstrapper
  apiGroup: rbac.authorization.k8s.io

kubectl create -f ClusterRoleBinding.yaml

新添加的节点上面
vim bootstrap.kubeconfig

apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority: /opt/kubernetes/ssl/ca.pem
    server: https://1.1.1.10:6443
  name: bootstrap
contexts:
- context:
    cluster: bootstrap
    user: kubelet-bootstrap
  name: bootstrap
current-context: bootstrap
preferences: {}
users:
- name: kubelet-bootstrap
  user:
    token: 07401b.f395accd246ae52d

systemctl daemon-reload
systemctl enable kubelet.service
systemctl start kubelet

在master节点上面

kubectl get csr
NAME                                                   AGE     SIGNERNAME                                    REQUESTOR                 CONDITION
node-csr-Jz1rAdZyWJ0APz-Gkhh9w8HGDZOw_y3ptKHrZNmxzpk   2m47s   kubernetes.io/kube-apiserver-client-kubelet   system:bootstrap:07401b   Pending
node-csr-k666nwRKVcRuzYIICbwlCCAsCwmP81FGMV9zKfsh-88   40m     kubernetes.io/kube-apiserver-client-kubelet   kubelet-bootstrap         Approved,Issued

查看详细信息
kubectl describe csr node-csr-Jz1rAdZyWJ0APz-Gkhh9w8HGDZOw_y3ptKHrZNmxzpk

批准加入集群

kubectl certificate approve node-csr-Jz1rAdZyWJ0APz-Gkhh9w8HGDZOw_y3ptKHrZNmxzpk

kubectl get node
NAME          STATUS   ROLES    AGE   VERSION
k8s-master1   Ready    <none>   42m   v1.18.6
k8s-node1     Ready    <none>   61s   v1.18.6