Kubernetes集群安装

Kubernetes的安装过程还是比较繁琐的，网上也提供了不少的教程，总结下来安装方式大概可以分为三类。

二进制安装（上手难度高，不推荐）
kubeadm安装（官方推荐，基于命令行形式，安装过程相对简化）
基于第三方工具（以容器安装容器，安装简单，但是也屏蔽了安装过程，快速体验可以采用这种方式安装）

这里使用腾讯云服务器来安装，准备了以下四台节点。其中三台服务器作为k8s集群节点，一台服务器辅助安装使用，在使用kubeadm安装时非必要，使用第三方辅助工具时候需要用到。

服务器配置	IP地址	节点名称	备注
2核心4G	10.0.4.17	master	集群主节点
2核心4G	10.0.4.5	node1	集群工作节点
2核心4G	10.0.4.8	node2	集群工作节点
2核心2G	10.0.16.15	support	辅助节点（非集群必须）

注：以下安装操作使用节点名称来代替具体的服务器

本篇安装教程主要包含两部分内容，k8s集群的安装方式以及web管理界面的安装方式。

k8s集群安装

KuboardSpray

KuboardSpray是一个图形化的 K8S 集群离线安装、维护工具，可以通过Docker启动，并且提供图形化配置功能，实现一键安装k8s集群。
首先support节点需要提供Docker支持，在support节点上使用Docker安装KuboardSpray，执行以下命令：

docker run -d \
  --privileged \
  --restart=unless-stopped \
  --name=kuboard-spray \
  -p 80:80/tcp \
  -e TZ=Asia/Shanghai \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v ~/kuboard-spray-data:/data \
  eipwork/kuboard-spray:latest-amd64
  # 如果您是 arm64 环境，请将标签里的 amd64 修改为 arm64，例如 eipwork/kuboard-spray:latest-arm64
  # 如果抓不到这个镜像，可以尝试一下这个备用地址：
  # swr.cn-east-2.myhuaweicloud.com/kuboard/kuboard-spray:latest-amd64

安装启动，默认登录信息admin/Kuboard123，登录后界面如下所示，可以修改密码

选择系统设置-->资源包管理，列出了一些列所支持的k8s版本，此处选择v1.23.1。

等待资源安装过程，安装完成如下所示

返回主界面，选择集群管理，配置集群信息

安装前注意事项：

此处可以参考Kuboard官方提供的：https://www.kuboard.cn/install/history-k8s/install-k8s-1.21.x.html
master节点防火墙需要开启6443端口。

安装后的k8s集群软件版本为：

Kubernetes v1.21.x
- calico 3.17.1
- nginx-ingress 1.9.1
Containerd.io 1.4.3

确保没有问题后，开始准备安装。等待安装过程，取决网络速度，安装containerd/kubelet/kubeadm/kubectl。

当出现如下提示信息时，k8s集群安装完成。安装内容主要包含

master节点
- kubeadm、kubectl、kubelet
node1/node2节点
- kubectl、kubelet
- kubectl、kubelet

Kubeadm

kubeadm 是 Kubernetes 官方支持的安装方式，“二进制” 不是。关于使用kubeadm安装的方式就不再赘述，这里推荐kuboard官方提供的安装教程：

https://www.kuboard.cn/install/history-k8s/install-k8s-1.21.x.html

关于安装版本的说明：

Kubernetes v1.21 开始，默认移除 docker 的依赖，如果宿主机上安装了 docker 和 containerd，将优先使用 docker 作为容器运行引擎，如果宿主机上未安装 docker 只安装了 containerd，将使用 containerd 作为容器运行引擎
本文使用 containerd 作为容器运行引擎

Dashboard安装

k8s提供了强大的集群管理功能，但是对于使用kubectl操作的方式过于专业，因此官方和第三方都提供了web图形化界面支持的工具，下面介绍两种安装方式。

Kubernetes Dashboard

Kubernetes Dashboard 是 Kubernetes 的官方 Web UI。使用 Kubernetes Dashboard，您可以：

向 Kubernetes 集群部署容器化应用
诊断容器化应用的问题
管理集群的资源
查看集群上所运行的应用程序
创建、修改Kubernetes 上的资源（例如 Deployment、Job、DaemonSet等）
展示集群上发生的错误

下面介绍如何安装，使用下面命令获取安装模板yaml文件，这里使用v2.5.0版本的dashboard。

wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.5.0/aio/deploy/recommended.yaml

下载yaml文件后，注意修改service kubernetes-dashboard的类型为NodePort，可以在集群外部访问。

# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: Namespace
metadata:
  name: kubernetes-dashboard

---

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-csrf
  namespace: kubernetes-dashboard
type: Opaque
data:
  csrf: ""

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-key-holder
  namespace: kubernetes-dashboard
type: Opaque

---

kind: ConfigMap
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-settings
  namespace: kubernetes-dashboard

---

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
rules:
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
    verbs: ["get", "update", "delete"]
    # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["kubernetes-dashboard-settings"]
    verbs: ["get", "update"]
    # Allow Dashboard to get metrics.
  - apiGroups: [""]
    resources: ["services"]
    resourceNames: ["heapster", "dashboard-metrics-scraper"]
    verbs: ["proxy"]
  - apiGroups: [""]
    resources: ["services/proxy"]
    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
    verbs: ["get"]

---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
rules:
  # Allow Metrics Scraper to get metrics from the Metrics server
  - apiGroups: ["metrics.k8s.io"]
    resources: ["pods", "nodes"]
    verbs: ["get", "list", "watch"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.5.0
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --auto-generate-certificates
            - --namespace=kubernetes-dashboard
            # Uncomment the following line to manually specify Kubernetes API server Host
            # If not specified, Dashboard will attempt to auto discover the API server and connect
            # to it. Uncomment only if the default does not work.
            # - --apiserver-host=http://my-address:port
          volumeMounts:
            - name: kubernetes-dashboard-certs
              mountPath: /certs
              # Create on-disk volume to store exec logs
            - mountPath: /tmp
              name: tmp-volume
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            initialDelaySeconds: 30
            timeoutSeconds: 30
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      volumes:
        - name: kubernetes-dashboard-certs
          secret:
            secretName: kubernetes-dashboard-certs
        - name: tmp-volume
          emptyDir: {}
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 8000
      targetPort: 8000
  selector:
    k8s-app: dashboard-metrics-scraper

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: dashboard-metrics-scraper
  template:
    metadata:
      labels:
        k8s-app: dashboard-metrics-scraper
    spec:
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: dashboard-metrics-scraper
          image: kubernetesui/metrics-scraper:v1.0.7
          ports:
            - containerPort: 8000
              protocol: TCP
          livenessProbe:
            httpGet:
              scheme: HTTP
              path: /
              port: 8000
            initialDelaySeconds: 30
            timeoutSeconds: 30
          volumeMounts:
          - mountPath: /tmp
            name: tmp-volume
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      volumes:
        - name: tmp-volume
          emptyDir: {}

执行命令，开始应用yaml文件

kubectl apply -f recommended.yaml

创建完成后，查看deployments或pods对应状态。

[root@master ~]# kubectl get deployments -n kubernetes-dashboard
NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
dashboard-metrics-scraper   1/1     1            1           6m7s
kubernetes-dashboard        1/1     1            1           6m7s

[root@master ~]# kubectl get pods -n kubernetes-dashboard
NAME                                         READY   STATUS    RESTARTS   AGE
dashboard-metrics-scraper-799d786dbf-p5h8c   1/1     Running   0          6m48s
kubernetes-dashboard-546cbc58cd-p7j7d        1/1     Running   0          6m48s

创建ServiceAccount以及ClusterRoleBinding，新建k8s-auth.yaml文件

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

应用RBAC yaml文件

[root@master ~]# kubectl apply -f k8s-auth.yaml
serviceaccount/admin-user created
clusterrolebinding.rbac.authorization.k8s.io/admin-user unchanged

然后通过浏览器访问Kubernetes Dashboard，如果出现不能访问的情况：可能跟集群使用的证书有关。查看名称空间所应用的证书，kubernetes-dashboard-certs对应的DATA为0，因此这种场景下就需要自签发证书了。

[root@master ~]# kubectl get secrets -n kubernetes-dashboard
NAME                               TYPE                                  DATA   AGE
admin-user-token-zqfhp             kubernetes.io/service-account-token   3      101s
default-token-p2szn                kubernetes.io/service-account-token   3      11m
kubernetes-dashboard-certs         Opaque                                0      11m
kubernetes-dashboard-csrf          Opaque                                1      11m
kubernetes-dashboard-key-holder    Opaque                                2      11m
kubernetes-dashboard-token-dqhdt   kubernetes.io/service-account-token   3      11m

删除集群中kubernetes-dashboard名称空间下的证书

kubectl delete secret kubernetes-dashboard-certs -n kubernetes-dashboard

生成证书

openssl genrsa -out dashboard.key 2048 
openssl req -new -out dashboard.csr -key dashboard.key -subj '/CN=your ip address'
openssl x509 -req -in dashboard.csr -signkey dashboard.key -out dashboard.crt

生成新的secret

kubectl create secret generic kubernetes-dashboard-certs --from-file=dashboard.key --from-file=dashboard.crt -n kubernetes-dashboard

删除原有pod，由于使用development创建资源，会自动创建新Pod资源。

[root@master ~]# kubectl delete pod kubernetes-dashboard-546cbc58cd-p7j7d -n kubernetes-dashboard
pod "kubernetes-dashboard-546cbc58cd-p7j7d" deleted

查看新Pod创建状况，kubernetes-dashboard-546cbc58cd-pgs8s已经处于运行状态。

[root@master ~]# kubectl get pods -n kubernetes-dashboard
NAME                                         READY   STATUS    RESTARTS   AGE
dashboard-metrics-scraper-799d786dbf-p5h8c   1/1     Running   0          22m
kubernetes-dashboard-546cbc58cd-pgs8s        1/1     Running   0          72s

然后通过https://ip:端口访问

通过命令获取登录token

kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin-user | awk '{print $1}')

# 输出结果
Name:         admin-user-token-zqfhp
Namespace:    kubernetes-dashboard
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: admin-user
              kubernetes.io/service-account.uid: 7b0b0752-cda6-4931-8762-28d1683aab7d

Type:  kubernetes.io/service-account-token

Data
====
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IjY5akgzTk1kTWVQWjR1TUdwUXBGdE53a1kxcWF1TDJTLTRDVU1jSTB1ZkUifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLXpxZmhwIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI3YjBiMDc1Mi1jZGE2LTQ5MzEtODc2Mi0yOGQxNjgzYWFiN2QiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZXJuZXRlcy1kYXNoYm9hcmQ6YWRtaW4tdXNlciJ9.QtJ0dY4K97w77cBpS3dJ_7X3s_eu_O6uEIT5aQ6Pi0dq4jIuNtsQEx6dcRcN2rdQH6loDpsaZjA3AFiDkE1tdmgab-A1nhyPFp8TV7aQ4Ga7bMstx4azUo1E71-9ynh43O69JgmP2995p10eRjA8ZA74uwSEjREBKVki5g9daWUfZ6d7vupbjkmqgOcIbb4UCclgBDKBL4UM0CXi7v2eWGJOOOcaTv2UpibifLhHSMqJ-Tl3U96iXGyKkEk41D_fDvtUT7h7PSqwZyvP6amOXRxE4kuI2kwlzv9SNol8oXLYjlE3q5m2OSuZ6YHOX9bH4gFy_H9_tCOcxwKky4hKDA
ca.crt:     1099 bytes
namespace:  20 bytes

在登录页面输入token，进入首页看到集群的资源信息

配置用户名/密码登录
写入鉴权文件

echo 'admin,admin,1' > /etc/kubernetes/pki/basic_auth_file

修改配置文件

vim /etc/kubernetes/manifests/kube-apiserver.yaml
# 增加如下参数
- --basic-auth-file=/etc/kubernetes/pki/basic_auth_file

修改Kubernetes dashboard配置文件yaml，在args添加配置

  - --authentication-mode=basic

绑定用户权限

kubectl create clusterrolebinding  login-on-dashboard-with-cluster-admin  --clusterrole=cluster-admin --user=admin

重新加载Dashboard

kubectl apply -f kubernetes-dashboard.yaml

Kuboard Dashboard

Kuboard是一款开源的k8s图形化管理软件，提供了一键安装的方式。

kubectl apply -f https://addons.kuboard.cn/kuboard/kuboard-v3.yaml
# 您也可以使用下面的指令，唯一的区别是，该指令使用华为云的镜像仓库替代 docker hub 分发 Kuboard 所需要的镜像
# kubectl apply -f https://addons.kuboard.cn/kuboard/kuboard-v3-swr.yaml

执行命令监控安装进度

watch kubectl get pods -n kuboard

Every 2.0s: kubectl get pods -n kuboard                                                                       master: Wed Mar 30 22:11:22 2022
NAME                               READY   STATUS    RESTARTS        AGE
kuboard-agent-2-584456bfcb-rrwwf   1/1     Running   1 (2m23s ago)   2m43s
kuboard-agent-64dc57c9b-2d84d      1/1     Running   1 (2m26s ago)   2m43s
kuboard-etcd-69bcb                 1/1     Running   0               4m6s
kuboard-questdb-7bcf54c98c-s825d   1/1     Running   0               2m42s
kuboard-v3-56b4b954c9-745zt        1/1     Running   0               4m6s

容器全部进入running状态后，通过浏览器访问

在浏览器中打开链接 http://your-node-ip-address:30080
输入初始用户名和密码，并登录admin/Kuboard123

具体安装细节参考官方网站：https://www.kuboard.cn/install/v3/install-in-k8s.html

摘自Kuboard官网：Kuboard 的定位和 Dashboard 是相似的，主要的区别在于：

Kuboard 关注微服务参考架构的视角对界面进行组织，参考 Kuboard 简介

Kuboard 中，不需要手工编写 YAML 文件，进一步降低 K8S 使用难度，提高便捷性

Kuboard 可以导出整个微服务架构的部署信息，并在新的名称空间/集群导入配置信息

Kuboard 的一个发展方向是，提供内建的监控套件（目前的全局监控套件成熟度比较高）

参考文档：

posted @ 2022-05-06 10:43 星光Starsray 阅读(224) 评论(0) 编辑收藏举报

会员力量，点亮园子希望

刷新页面返回顶部

星光✨

※星光不问赶路人，时光不负有心人※