遇一山,过一山,处处有风景;只要勇敢向前,一路尽是繁花盛开。 | (点击查看→)【测试干货】python/java自动化、持续集成、性能、测开、简历、笔试面试等

快速搭建一套k8s集群环境

 

k8s系列汇总,请查看这里https://www.cnblogs.com/uncleyong/p/15499743.html

持续集成系列汇总,请查看这里https://www.cnblogs.com/uncleyong/p/16721826.html

参考官网

kubeadm是官方提供的快速搭建k8s集群的开源工具,对于非运维人员学习k8s,kubeadm方式安装相对更简单。

kubeadm创建一个集群:https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/

kubeadm创建一个高可用集群:https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/

 

安装前提

One or more machines running a deb/rpm-compatible Linux OS; for example: Ubuntu or CentOS.
2 GiB or more of RAM per machine--any less leaves little room for your apps.
At least 2 CPUs on the machine that you use as a control-plane node.
Full network connectivity among all machines in the cluster. You can use either a public or a private network.

  

网段规划

虚拟机(宿主机):192.168.117.x

虚拟机nat网络使用的是vmnet8 

service:10.96.0.0/12

pod:172.16.0.0/12

参考:https://www.cnblogs.com/uncleyong/p/6959568.html

 

集群规划

高可用集群

 

由于资源有限,这里暂时不安装高可用(haproxy+lvs)集群

对于学习而言,我们搭建一个非高可用集群即可,1个master,2个node

master:192.168.117.171,主机名为k8s-master01

node1:192.168.117.172,主机名为k8s-node01

node2:192.168.117.173,主机名为k8s-node02

 

说明:ip请根据你vmware网络实际情况设置

 

下面是ip地址范围

 

创建虚拟机

通过vof模板文件创建虚拟机(如需要vof模板文件,请微信联系获取)

vof已完成相关配置:https://www.cnblogs.com/uncleyong/p/15471002.html

 

创建master

vmware直接打开ovf文件

 

输入新虚拟机名称和存储路径

 

调整内存和cpu

 

调整为4c8g

 

ip a

 

修改ip:vim /etc/sysconfig/network-scripts/ifcfg-ens33

192.168.117.171

重启网卡:systemctl restart network

 

创建node1

 

调整为3c8g

 

修改ip:vim /etc/sysconfig/network-scripts/ifcfg-ens33

192.168.117.172

重启网卡:systemctl restart network

 

创建node2

 

调整为3c8g

 

修改ip:vim /etc/sysconfig/network-scripts/ifcfg-ens33

192.168.117.173

重启网卡:systemctl restart network

 

xshell分别连接master、node1、node2

 

虚拟机存放目录

 

修改主机名

分别在3个虚拟机上执行:

171:hostnamectl set-hostname k8s-master01

172:hostnamectl set-hostname k8s-node01

173:hostnamectl set-hostname k8s-node02

 

断开xshell后重新连接,即可看到新的主机名

 

 

 

或者不断开xshell,执行命令:bash

确认一下:

 

 

当然,也可以一步到位:hostnamectl set-hostname k8s-master01 && bash

配置hosts

所有节点:vim /etc/hosts 

 

192.168.117.171 k8s-master01
192.168.117.172 k8s-node01
192.168.117.173 k8s-node02

 

说明:下面需要保留

 

 

配置k8s需要的yum源

所有节点:

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

 

删除包含内容的行

sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo

 

清除yum缓存:yum clean all

生成yum缓存:yum makecache fast

更新yum源:yum -y update

 

master01节点免密钥登录其他节点

在master01上操作:ssh-keygen -t rsa

一直回车

 

公钥发到其它节点

for i in k8s-master01 k8s-node01 k8s-node02;do ssh-copy-id -i .ssh/id_rsa.pub $i;done

 

验证

ssh k8s-node02,未输入密码就连接到k8s-node02节点了

 

exit,返回到master01节点

 

安装k8s组件

官网:https://kubernetes.io/

最新版本是1.22

 

查看小版本:yum list kubeadm.x86_64 --showduplicates | sort -r | grep 1.22

 

我们安装最新版:1.22.3

所有节点执行:yum install kubeadm-1.22.3 kubelet-1.22.3 kubectl-1.22.3 -y

 

所有节点配置kubelet使用阿里云的pause镜像

cat >/etc/sysconfig/kubelet<<EOF
KUBELET_EXTRA_ARGS="--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5"
EOF

 

所有节点设置kubelet开机自启动:systemctl enable kubelet

查看是否开机启动:systemctl is-enabled kubelet

 

集群初始化

参考:https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/

master节点创建kubeadm-config.yaml配置文件(里面ip改为你自己的,我的是:192.168.117.171;文件从网盘下载)

master节点拉取镜像:kubeadm config images pull --config /root/kubeadm-config.yaml

 

master01节点初始化:kubeadm init --config /root/kubeadm-config.yaml  --upload-certs

 

 

 

初始化成功,生成token,其它节点加入时使用

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 192.168.117.171:6443 --token 7t2weq.bjbawausm0jaxury \
	--discovery-token-ca-cert-hash sha256:af8e08cec375af6b26a33ae55b5093c1c8c2361eb93dcfd415ce30c72a863f66 \
	--control-plane --certificate-key 3ee5335688b1d714274f826f05ec73443aa926edf03d1aa07e51bb4390ee0dd3

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.117.171:6443 --token 7t2weq.bjbawausm0jaxury \
	--discovery-token-ca-cert-hash sha256:af8e08cec375af6b26a33ae55b5093c1c8c2361eb93dcfd415ce30c72a863f66

 

生成的配置文件和证书文件 

ls /etc/kubernetes/

ls /etc/kubernetes/manifests/

ls /etc/kubernetes/pki/

 

 

master01配置访问k8s集群的环境变量

cat <<EOF >> /root/.bashrc
export KUBECONFIG=/etc/kubernetes/admin.conf
EOF

让配置生效

source /root/.bashrc

  

节点状态:kubectl get node

 

pod状态(所有的系统组件都是以容器的方式运行)

kubectl get po -n kube-system

 

因为calico还没安装,还没法调度到节点,所以上面coredns是pending状态

 

集群加入node节点

node节点执行

kubeadm join 192.168.117.171:6443 --token 7t2weq.bjbawausm0jaxury \
	--discovery-token-ca-cert-hash sha256:af8e08cec375af6b26a33ae55b5093c1c8c2361eb93dcfd415ce30c72a863f66

 

node01执行

 

 

node02执行

 

查看集群状态:kubectl get node

 

此时执行:kubectl get po -A -owide

ip都是宿主机的ip,因为这些pod和宿主机共用网络,而coredns不和宿主机共用网络,ip一列是<none>,需要网络插件来分配ip

 

安装calico

https://docs.projectcalico.org/getting-started/kubernetes/self-managed-onprem/onpremises

配置文件获取地址:https://docs.projectcalico.org/manifests/calico.yaml

curl https://docs.projectcalico.org/manifests/calico.yaml -O

修改

            - name: CALICO_IPV4POOL_CIDR
              value: "172.16.0.0/12"

 

kubectl apply -f calico.yaml(calico.yam文件和涉及的镜像,都从网盘下载)

kubectl get po -n kube-system -owide

说明:下面有一个Pending,是因为master有污点

 

kubectl get node

 

将master01节点的front-proxy-ca.crt复制到所有node节点

scp /etc/kubernetes/pki/front-proxy-ca.crt k8s-node01:/etc/kubernetes/pki/front-proxy-ca.crt

scp /etc/kubernetes/pki/front-proxy-ca.crt k8s-node02:/etc/kubernetes/pki/front-proxy-ca.crt

 

安装metrics server和dashboard

https://www.cnblogs.com/uncleyong/p/15701535.html

 

其它配置

去掉污点允许master节点部署pod

kubeadm安装的k8s集群,master节点默认不允许部署pod

kubectl  describe node |grep NoSchedule  -C 5

 

去掉:kubectl  taint node  -l node-role.kubernetes.io/master node-role.kubernetes.io/master:NoSchedule-

 

kubectl get po -A -owide,pending的消失了,可以看到,是调度到了master节点

kube-proxy改为性能更好的ipvs

master执行:kubectl edit cm kube-proxy -n kube-system

搜索mode,改为:

 

ks8集群验证

1、查看node状态

2、查看容器状态

3、验证calico
  pod可以ping本节点同namespace的pod
  pod可以ping本节点跨namespace的pod
  pod可以ping跨节点相同namespace的pod
  pod可以ping跨节点不同namespace的pod
  所有节点可以ping一个pod
  pod可以ping外网

4、验证kube-proxy

5、k8s和coredns的svc能telnet通

6、验证coredns
  解析不同namespace
  解析相同namespace

【bak,待更新】

 

原文:https://www.cnblogs.com/uncleyong/p/15499732.html

 

posted @ 2021-11-11 22:08  全栈测试笔记  阅读(3074)  评论(0编辑  收藏  举报
浏览器标题切换
浏览器标题切换end