Kubernets V1.25.0 集群部署

• 生产环境部署K8S 的2种方式
• 服务器硬件配置推荐
• 使用kubeadm快速部署一个k8s集群
• 部署的网络组件起什么作用
• Kubernets将弃用Docker！
• kubeconfig配置文件
• kubectl命令行管理工具
• 牛刀小试，快速部署一个网站
• 基本资源概念

| 生产环境部署K8S的2种方式

*  kubeadm
kubeadm是一个工具, 提供kubeadm init 和kubeadm join，用于快速部署kubernets集群。
部署地址：
     
*  二进制
从官方下载发行版的二进制包, 手工部署每个组件, 组成kubernets集群.
下载地址

| 服务器硬件配置推荐

实验环境	K8S master / node	2C2G+
测试环境	k8s-master	CPU	2核
		内存	4G
		硬盘	20G
	k8s-node	cpu	4核
		内存	8G
		硬盘	20G
生产环境	k8s-master	CPU	8核
		内存	16G
		硬盘	100G
	k8s-node	cpu	16核心
		内存	64G
		硬盘	500G

| 使用kubeadm 快速搭建K8S集群

1 安装Docker

2  创建一个master节点
kubeadm init

3  将一个Node节点加入到当前集群中
kubeadm join <master 节点的IP和端口>

4  部署容器网络(CNI)
kubectl apply -f calico.yaml

5  部署Web UI(Dashboard)

规划

服务器角色	IP地址
k8s-master-1	192.168.3.101
k8s-node-1	192.168.3.104
k8s-node-2	192.168.3.105

操作系统初始化配置(所有节点)

# 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld

# 关闭selinux
sed -i 's#enforcing#disabled#g' /etc/selinux/config

# 关闭swap
swapoff -a  # 临时
sed -ri 's/.*swap.*/#&/' /etc/fstab  # 永久

# 根据规划设置主机名
hostnamectl set-hostname <hostname>

# 在master添加hosts
cat >> /etc/hosts << EOF
192.168.3.101 k8s-master-1
192.168.3.102 k8s-master-2
192.168.3.103 k8s-master-3

192.168.3.104 k8s-node-1
192.168.3.105 k8s-node-2
192.168.3.106 k8s-node-3

#192.168.3.107 k8s-node-5
#192.168.3.108 k8s-node-6
EOF

#
cat >> /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sysctl --system

# 时间同步
yum install ntpdate -y
ntpdate time.windows.com

2 安装docker
2.1 安装docker

yum install -y yum-utils
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum -y install docker-ce
systemctl enable docker
systemctl start docker  

配置镜像下载加速器:

cat > /etc/docker/daemon.json <<EOF
{
"registry-mirrors":["https://b9pmyelo.mirror.aliyuncs.com"],
"exec-opts":["native.cgroupdriver=systemd"],
"insecure-registries":["192.168.3.200"]
}
EOF

systemctl restart docker
docker info

2.2 安装cni-dockerd （使cri兼容docker命令）
Kubernets v1.24移除docker-shim的支持，而docker engine 默认又不支持cri标准

wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.2.5/cri-dockerd-0.2.5-3.el7.x86_64.rpm
rpm -ivh cri-dockerd-0.2.5-3.el7.x86_64.rpm

vim /usr/lib/systemd/system/cri-docker.service 
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
Requires=cri-docker.socket

[Service]
Type=notify
## 这行需要添加一个pod镜像fd://的后面
ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.7
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always

StartLimitBurst=3

StartLimitInterval=60s

LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

TasksMax=infinity
Delegate=yes
KillMode=process

[Install]
WantedBy=multi-user.target

# 启动服务
systemctl daemon-reload
systemctl enable cri-docker && systemctl start cri-docker

#验证
ps -ef | grep cri-docker

#注意:
这里/usr/bin/cri-dockerd一定要加上参数:
  –pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.7
用来指定所用的pause镜像是哪个，否则默认拉取k8s.gcr.io/pause:3.6，会导致安装失败。

2.3 添加阿里云yum软件源

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0       
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

2.4 安装kubeadm, kubelet 和 kubectl
由于版本更新频繁, 这里指定版本号部署:

# 此处仅仅开机启动kubelet
# master节点
yum install -y kubeadm-1.25.0 kubectl-1.25.0 kubelet-1.25.0
systemctl enable kubelet

# Node节点
yum install -y kubelet-1.25.0 kubeadm-1.25.0
systemctl enable kubelet

# master节点
kubectl 

# node节点安装
kubelet 
kubeadm

• 3 部署Kubernets Master

在 192.168.3.101 （k8s-master-1）上执行.

### 此处注意初始化的IP地址,
kubeadm init \
  --apiserver-advertise-address=192.168.3.101 \
  --image-repository registry.aliyuncs.com/google_containers \
  --kubernetes-version v1.25.0 \
  --service-cidr=10.96.0.0/12 \
  --pod-network-cidr=10.244.0.0/16 \
  --cri-socket=unix:///var/run/cri-dockerd.sock \
  --ignore-preflight-errors=all

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.3.102:6443 --token 7g8yft.huf43apns5oxljw3 \
	--discovery-token-ca-cert-hash sha256:2a3d84d0387ecf8822c3cb2652cb592965da1f9ecd5675c04287b21853cda6ea


[root@k8s-master-2 ~]# kubectl get nodes 
NAME           STATUS     ROLES           AGE   VERSION
k8s-master-2   NotReady   control-plane   16h   v1.25.0

# 初始化错误挽救方式：
kubeadm reset --cri-socket=unix:///var/run/cri-dockerd.sock
cd /etc/kubernetes/ && rm -rf ./*

注：由于网络插件还没有部署, 还没有准备就绪 NotReady， 先加入node节点，后面在安装网络插件.
参考资料：

• 4 Node 加入Kubernetes 集群

在任意Node上执行，k8s-node-2 , 192.168.3.104
向集群添加新节点，执行在kubeadm init输出的kubeadm join命令并手动加上 --cri-socket=unix:///var/run/cri-dockerd.sock：

## 执行添加客户端命令
kubeadm join 192.168.3.102:6443 --token 7g8yft.huf43apns5oxljw3 \
	--discovery-token-ca-cert-hash sha256:2a3d84d0387ecf8822c3cb2652cb592965da1f9ecd5675c04287b21853cda6ea --cri-socket=unix:///var/run/cri-dockerd.sock

## 运行结果
[root@k8s-node-2 ~]# kubeadm join 192.168.3.102:6443 --token 7g8yft.huf43apns5oxljw3 \
> --discovery-token-ca-cert-hash sha256:2a3d84d0387ecf8822c3cb2652cb592965da1f9ecd5675c04287b21853cda6ea --cri-socket=unix:///var/run/cri-dockerd.sock
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.


# 错误 "Running pre-flight checks"
[root@k8s-node-3 ~]# kubeadm join 192.168.3.101:6443 --token t87af5.w4gssl8b2is6ctdm         --discovery-token-ca-cert-hash sha256:...............
[preflight] Running pre-flight checks

 查看master 和 node 节点服务器的时间,是时间不同步造成的无法加入.

默认 Token 有效期为24小时,当过期之后，该token就不可用了，这时就需要重新创建token, 可以直接使用命令快捷生成：

# 查看token 列表
kubeadm token list

# 重新生成 token
kubeadm token create --print-join-command

参考资料：https://kubernetes.io/zh-cn/docs/reference/setup-tools/kubeadm/kubeadm-join/

之后所有的操作只在master节点上进行

• 5 部署容器网络（CNI）

Calico 是一个纯三层的数据中心网络方案, 是目前kubernetes主流的网络方案
下载 YAML：

## master
wget https://docs.projectcalico.org/manifests/calico.yaml --no-check-certificate

## 修改calico.yaml文件：
......
- name: CALICO_IPV4POOL_CIDR
  value: "10.244.0.0/16"
......
只需要改这一个地方即可,打开注释，注意格式一定要对其。value这个值需要写kubeadm init 时的
"--pod-network-cidr=10.244.0.0/16" 改成和这个一样的网络地址即可.

## 执行更新calico.yaml
kubectl apply -f calico.yaml

[root@k8s-master-2 ~]# kubectl apply -f calico.yaml 
poddisruptionbudget.policy/calico-kube-controllers created
serviceaccount/calico-kube-controllers created
serviceaccount/calico-node created
configmap/calico-config created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
.......

## 执行命令查看结果
kubectl get pod -n kube-system
kubectl get pod -A

[root@k8s-master-2 ~]# kubectl get pod -A
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-f79f7749d-d5w4b   1/1     Running   0          67s
kube-system   calico-node-2m5km                         1/1     Running   0          67s
kube-system   calico-node-877tr                         1/1     Running   0          67s
kube-system   calico-node-sm6w7                         1/1     Running   0          67s

## 加入进来的节点也已经Ready状态:
[root@k8s-master-2 ~]# kubectl get node 
NAME           STATUS   ROLES           AGE     VERSION
k8s-master-2   Ready    control-plane   2d19h   v1.25.0
k8s-node-2     Ready    <none>          2d3h    v1.25.0
k8s-node-3     Ready    <none>          2d3h    v1.25.0

• 6 部署 Dashboard

  Dashboard 是官方提供的一个UI，可用于基本管理K8S资源.

## wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.4.0/aio/deploy/recommended.yaml

为了方便认识组件，recommanded.yaml改成 kubernetes-dashboard.yaml
默认Dashboard只能集群内部访问，修改service为NodePort类型，暴露到外部.

vim kubernetes-dashboard.yaml
......
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 30001              # 通过Nodeport方式暴露30001端口
  selector:
    k8s-app: kubernetes-dashboard
  type: NodePort                   # NodePort类型
......

## 创建dashboard
kubectl apply -f kubernetes-dashboard.yaml
kubectl get pods -n kubernetes-dashboard

访问地址: https://NodeIP:30001
错误提示：
chrome 出现不能访问的错误连接,在当前页面输入"thisisunsafe"即可访问dashboard页面，或者换个浏览器.
创建service account 并绑定默认cluster-admin管理员集群角色：

# 创建用户：
kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard

# 用户授权：
kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:dashboard-admin

# 获取用户Token
kubectl create token dashboard-admin -n kubernetes-dashboard

使用输出的token登录dashboard.（任意节点IP+端口均可访问）