Kubernetes(k8s)控制器(一):deployment

一.系统环境

服务器版本 docker软件版本 Kubernetes(k8s)集群版本 CPU架构
CentOS Linux release 7.4.1708 (Core) Docker version 20.10.12 v1.21.9 x86_64

Kubernetes集群架构:k8scloude1作为master节点,k8scloude2,k8scloude3作为worker节点

服务器 操作系统版本 CPU架构 进程 功能描述
k8scloude1/192.168.110.130 CentOS Linux release 7.4.1708 (Core) x86_64 docker,kube-apiserver,etcd,kube-scheduler,kube-controller-manager,kubelet,kube-proxy,coredns,calico k8s master节点
k8scloude2/192.168.110.129 CentOS Linux release 7.4.1708 (Core) x86_64 docker,kubelet,kube-proxy,calico k8s worker节点
k8scloude3/192.168.110.128 CentOS Linux release 7.4.1708 (Core) x86_64 docker,kubelet,kube-proxy,calico k8s worker节点

二.前言

使用deployment的前提是已经有一套可以正常运行的Kubernetes集群,关于Kubernetes(k8s)集群的安装部署,可以查看博客《Centos7 安装部署Kubernetes(k8s)集群》https://www.cnblogs.com/renshengdezheli/p/16686769.html。

三.Kubernetes 控制器

在 Kubernetes 中,控制器通过监控集群 的公共状态,并致力于将当前状态转变为期望的状态。

一个控制器至少追踪一种类型的 Kubernetes 资源。这些 对象 有一个代表期望状态的 spec 字段该资源的控制器负责确保其当前状态接近期望状态。

控制器可能会自行执行操作;在 Kubernetes 中更常见的是一个控制器会发送信息给 API 服务器。

Kubernetes 内置一组控制器,运行在 kube-controller-manager 内。 这些内置的控制器提供了重要的核心功能。

Deployment 控制器和 Job 控制器是 Kubernetes 内置控制器的典型例子。 Kubernetes 允许你运行一个稳定的控制平面,这样即使某些内置控制器失败了, 控制平面的其他部分会接替它们的工作。

四.Deployment概览

一个 Deployment 为 Pod 和 ReplicaSet 提供声明式的更新能力

你负责描述 Deployment 中的 目标状态,而 Deployment 控制器(Controller) 以受控速率更改实际状态, 使其变为期望状态。你可以定义 Deployment 以创建新的 ReplicaSet,或删除现有 Deployment, 并通过新的 Deployment 收养其资源。

注意:不要管理 Deployment 所拥有的 ReplicaSet 。

以下是 Deployments 的典型用例:

  • 创建 Deployment 以将 ReplicaSet 上线。ReplicaSet 在后台创建 Pod。 检查 ReplicaSet 的上线状态,查看其是否成功。
  • 通过更新 Deployment 的 PodTemplateSpec,声明 Pod 的新状态 。 新的 ReplicaSet 会被创建,Deployment 以受控速率将 Pod 从旧 ReplicaSet 迁移到新 ReplicaSet。 每个新的 ReplicaSet 都会更新 Deployment 的修订版本。
  • 如果 Deployment 的当前状态不稳定,回滚到较早的 Deployment 版本。 每次回滚都会更新 Deployment 的修订版本。
  • 扩大 Deployment 规模以承担更多负载。
  • 暂停 Deployment 的上线 以应用对 PodTemplateSpec 所作的多项修改, 然后恢复其执行以启动新的上线版本。
  • 使用 Deployment 状态来判定上线过程是否出现停滞。
  • 清理较旧的不再需要的 ReplicaSet 。

五.创建deployment

创建存放deployment配置文件的目录和namespace

[root@k8scloude1 ~]# mkdir deployment

[root@k8scloude1 ~]# cd deployment/

[root@k8scloude1 deployment]# pwd
/root/deployment

#创建namespace
[root@k8scloude1 deployment]# kubectl create ns deployment
namespace/deployment created

#切换命名空间为deployment
[root@k8scloude1 deployment]# kubens deployment
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "deployment".

[root@k8scloude1 deployment]# kubectl get pod 
No resources found in deployment namespace.

Kubernetes里有很多简写,比如deployment简称deploy,pod简称po

查看pod

[root@k8scloude1 deployment]# kubectl get po
No resources found in deployment namespace.

查看对资源的操作有哪些

[root@k8scloude1 deployment]# kubectl --help | grep resource
  create        Create a resource from a file or from stdin.
  get           显示一个或更多 resources
  delete        Delete resources by filenames, stdin, resources and names, or by resources and label selector
  rollout       Manage the rollout of a resource
  describe      显示一个指定 resource 或者 group 的 resources 详情
  patch         Update field(s) of a resource
  wait          Experimental: Wait for a specific condition on one or many resources.
  api-resources Print the supported API resources on the server

可以查看k8s支持的简写

[root@k8scloude1 deployment]# kubectl api-resources
NAME                              SHORTNAMES   APIVERSION                             NAMESPACED   KIND
bindings                                       v1                                     true         Binding
componentstatuses                 cs           v1                                     false        ComponentStatus
configmaps                        cm           v1                                     true         ConfigMap
endpoints                         ep           v1                                     true         Endpoints
events                            ev           v1                                     true         Event
limitranges                       limits       v1                                     true         LimitRange
namespaces                        ns           v1                                     false        Namespace
nodes                             no           v1                                     false        Node
persistentvolumeclaims            pvc          v1                                     true         PersistentVolumeClaim
persistentvolumes                 pv           v1                                     false        PersistentVolume
pods                              po           v1                                     true         Pod
podtemplates                                   v1                                     true         PodTemplate
replicationcontrollers            rc           v1                                     true         ReplicationController
resourcequotas                    quota        v1                                     true         ResourceQuota
secrets                                        v1                                     true         Secret
serviceaccounts                   sa           v1                                     true         ServiceAccount
services                          svc          v1                                     true         Service
mutatingwebhookconfigurations                  admissionregistration.k8s.io/v1        false        MutatingWebhookConfiguration
validatingwebhookconfigurations                admissionregistration.k8s.io/v1        false        ValidatingWebhookConfiguration
customresourcedefinitions         crd,crds     apiextensions.k8s.io/v1                false        CustomResourceDefinition
apiservices                                    apiregistration.k8s.io/v1              false        APIService
controllerrevisions                            apps/v1                                true         ControllerRevision
daemonsets                        ds           apps/v1                                true         DaemonSet
deployments                       deploy       apps/v1                                true         Deployment
replicasets                       rs           apps/v1                                true         ReplicaSet
statefulsets                      sts          apps/v1                                true         StatefulSet
tokenreviews                                   authentication.k8s.io/v1               false        TokenReview
localsubjectaccessreviews                      authorization.k8s.io/v1                true         LocalSubjectAccessReview
selfsubjectaccessreviews                       authorization.k8s.io/v1                false        SelfSubjectAccessReview
selfsubjectrulesreviews                        authorization.k8s.io/v1                false        SelfSubjectRulesReview
subjectaccessreviews                           authorization.k8s.io/v1                false        SubjectAccessReview
horizontalpodautoscalers          hpa          autoscaling/v1                         true         HorizontalPodAutoscaler
cronjobs                          cj           batch/v1                               true         CronJob
jobs                                           batch/v1                               true         Job
certificatesigningrequests        csr          certificates.k8s.io/v1                 false        CertificateSigningRequest
leases                                         coordination.k8s.io/v1                 true         Lease
bgpconfigurations                              crd.projectcalico.org/v1               false        BGPConfiguration
bgppeers                                       crd.projectcalico.org/v1               false        BGPPeer
blockaffinities                                crd.projectcalico.org/v1               false        BlockAffinity
caliconodestatuses                             crd.projectcalico.org/v1               false        CalicoNodeStatus
clusterinformations                            crd.projectcalico.org/v1               false        ClusterInformation
felixconfigurations                            crd.projectcalico.org/v1               false        FelixConfiguration
globalnetworkpolicies                          crd.projectcalico.org/v1               false        GlobalNetworkPolicy
globalnetworksets                              crd.projectcalico.org/v1               false        GlobalNetworkSet
hostendpoints                                  crd.projectcalico.org/v1               false        HostEndpoint
ipamblocks                                     crd.projectcalico.org/v1               false        IPAMBlock
ipamconfigs                                    crd.projectcalico.org/v1               false        IPAMConfig
ipamhandles                                    crd.projectcalico.org/v1               false        IPAMHandle
ippools                                        crd.projectcalico.org/v1               false        IPPool
ipreservations                                 crd.projectcalico.org/v1               false        IPReservation
kubecontrollersconfigurations                  crd.projectcalico.org/v1               false        KubeControllersConfiguration
networkpolicies                                crd.projectcalico.org/v1               true         NetworkPolicy
networksets                                    crd.projectcalico.org/v1               true         NetworkSet
endpointslices                                 discovery.k8s.io/v1                    true         EndpointSlice
events                            ev           events.k8s.io/v1                       true         Event
ingresses                         ing          extensions/v1beta1                     true         Ingress
flowschemas                                    flowcontrol.apiserver.k8s.io/v1beta1   false        FlowSchema
prioritylevelconfigurations                    flowcontrol.apiserver.k8s.io/v1beta1   false        PriorityLevelConfiguration
nodes                                          metrics.k8s.io/v1beta1                 false        NodeMetrics
pods                                           metrics.k8s.io/v1beta1                 true         PodMetrics
ingressclasses                                 networking.k8s.io/v1                   false        IngressClass
ingresses                         ing          networking.k8s.io/v1                   true         Ingress
networkpolicies                   netpol       networking.k8s.io/v1                   true         NetworkPolicy
runtimeclasses                                 node.k8s.io/v1                         false        RuntimeClass
poddisruptionbudgets              pdb          policy/v1                              true         PodDisruptionBudget
podsecuritypolicies               psp          policy/v1beta1                         false        PodSecurityPolicy
clusterrolebindings                            rbac.authorization.k8s.io/v1           false        ClusterRoleBinding
clusterroles                                   rbac.authorization.k8s.io/v1           false        ClusterRole
rolebindings                                   rbac.authorization.k8s.io/v1           true         RoleBinding
roles                                          rbac.authorization.k8s.io/v1           true         Role
priorityclasses                   pc           scheduling.k8s.io/v1                   false        PriorityClass
csidrivers                                     storage.k8s.io/v1                      false        CSIDriver
csinodes                                       storage.k8s.io/v1                      false        CSINode
csistoragecapacities                           storage.k8s.io/v1beta1                 true         CSIStorageCapacity
storageclasses                    sc           storage.k8s.io/v1                      false        StorageClass
volumeattachments                              storage.k8s.io/v1                      false        VolumeAttachment

查看deployment控制器

[root@k8scloude1 deployment]# kubectl get deploy
No resources found in deployment namespace.

建议使用yaml文件创建deploy,命令行的方式,支持的选项比较少

生成deployment的yaml文件

#nginx为deploy的名字    --image=nginx使用Nginx镜像
[root@k8scloude1 deployment]# kubectl create deploy nginx --image=nginx --dry-run=client -o yaml >nginx.yaml

[root@k8scloude1 deployment]# ll -h nginx.yaml 
-rw-r--r-- 1 root root 384 1月  25 15:39 nginx.yaml

查看生成的deploy yaml配置文件

[root@k8scloude1 deployment]# cat nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        resources: {}
status: {}

pod模板可以有多个标签,只要其中一个标签和matchLabels里的标签一致即可

[root@k8scloude1 deployment]# vim nginx.yaml 

[root@k8scloude1 deployment]# cat nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  #这个labels是deployment的标签
  labels:
    app: nginx
  #deployment名字
  name: nginx
spec:
  #replicas: 1 表示副本数为1,只生成一个pod
  replicas: 1
  #selector标签选择器
  selector:
    matchLabels:
      appx: xyz
  strategy: {}
  #template下面是pod的模板
  template:
    metadata:
      creationTimestamp: null
      #这个labels表示pod标签
      labels:
         #标签
        app: nginx
        appx: xyz
    spec:
      containers:
      - image: nginx
        name: nginx
        resources: {}
status: {}

创建deploy

[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml 
deployment.apps/nginx created

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   0/1     1            0           11s

直接下载nginx镜像太慢,重新修改下yaml文件

[root@k8scloude1 deployment]# kubectl delete deploy nginx 
deployment.apps "nginx" deleted


[root@k8scloude1 deployment]# vim nginx.yaml 

[root@k8scloude1 deployment]# cat nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      appx: xyz
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
        appx: xyz
    spec:
      #优雅删除pod的期限时间
      terminationGracePeriodSeconds: 0
      containers:
      - image: nginx
        name: nginx
        #修改镜像下载策略:IfNotPresent表示本地有镜像就不下载
        imagePullPolicy: IfNotPresent
        resources: {}
status: {}

创建deploy

[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml 
deployment.apps/nginx created

查看deployment

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR
nginx   1/1     1            1           11s   nginx        nginx    appx=xyz

#--show-labels 显示标签
[root@k8scloude1 deployment]# kubectl get deploy -o wide --show-labels
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR   LABELS
nginx   1/1     1            1           60s   nginx        nginx    appx=xyz   app=nginx

可以看到pod也被创建出来了

[root@k8scloude1 deployment]# kubectl get pod -o wide --show-labels
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES   LABELS
nginx-6fcb7995f5-c8lpc   1/1     Running   0          2m25s   10.244.112.146   k8scloude2   <none>           <none>            app=nginx,appx=xyz,pod-template-hash=6fcb7995f5

注意:删除pod之后,deploy会根据副本数replicas,创建缺失的pod,所以要想真正删除pod,需要删除deployment控制器

[root@k8scloude1 deployment]# kubectl delete pod nginx-6fcb7995f5-c8lpc --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "nginx-6fcb7995f5-c8lpc" force deleted

#pod又重新创建了
[root@k8scloude1 deployment]# kubectl get pod -o wide --show-labels
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES   LABELS
nginx-6fcb7995f5-s6smg   1/1     Running   0          5s    10.244.112.147   k8scloude2   <none>           <none>            app=nginx,appx=xyz,pod-template-hash=6fcb7995f5

六.修改deploy副本数

修改deployment副本数有三种方法:

  1. kubectl edit deploy deployname ,直接修改replicas的数目即可,实时生效,注意:可以在线修改deploy但是不能在线修改pod
  2. 可以直接修改yaml文件: 比如:replicas: 6
  3. kubectl scale deployment deployname 在线伸缩副本数

6.1 kubectl edit deploy 修改副本数

查看deploy

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   1/1     1            1           10m

第一种:kubectl edit deploy deployname ,直接修改replicas的数目即可,实时生效,注意可以在线修改deploy但是不能在线修改pod

[root@k8scloude1 deployment]# kubectl edit deploy nginx 
deployment.apps/nginx edited

查看deploy,可以发现副本数变了,READY为3/3

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   3/3     3            3           11m

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR
nginx   3/3     3            3           11m   nginx        nginx    appx=xyz

6.2 修改yaml文件更改副本数

第二种:可以直接修改yaml文件: replicas: 6,修改为6个副本

[root@k8scloude1 deployment]# vim nginx.yaml 

[root@k8scloude1 deployment]# cat nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 6
  selector:
    matchLabels:
      appx: xyz
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
        appx: xyz
    spec:
      terminationGracePeriodSeconds: 0
      containers:
      - image: nginx
        name: nginx
        imagePullPolicy: IfNotPresent
        resources: {}
status: {}

kubectl edit可以立即生效,但是修改yaml文件需要apply才能生效

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   3/3     3            3           14m

[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml 
deployment.apps/nginx configured

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   6/6     6            6           15m

可以看到deploy生成了6个pod

[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
nginx-6fcb7995f5-64gm2   1/1     Running   0          22s     10.244.251.204   k8scloude3   <none>           <none>
nginx-6fcb7995f5-75jfd   1/1     Running   0          22s     10.244.251.206   k8scloude3   <none>           <none>
nginx-6fcb7995f5-8c75n   1/1     Running   0          4m23s   10.244.251.205   k8scloude3   <none>           <none>
nginx-6fcb7995f5-cs6w4   1/1     Running   0          4m23s   10.244.112.143   k8scloude2   <none>           <none>
nginx-6fcb7995f5-s6smg   1/1     Running   0          11m     10.244.112.147   k8scloude2   <none>           <none>
nginx-6fcb7995f5-tpwr4   1/1     Running   0          22s     10.244.112.148   k8scloude2   <none>           <none>

6.3 kubectl scale修改副本数

kubectl scale在线伸缩deploy,--replicas=10修改副本数为10个

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=10
deployment.apps/nginx scaled

[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
nginx-6fcb7995f5-5sv29   1/1     Running   0          4s      10.244.112.152   k8scloude2   <none>           <none>
nginx-6fcb7995f5-64gm2   1/1     Running   0          2m12s   10.244.251.204   k8scloude3   <none>           <none>
nginx-6fcb7995f5-75jfd   1/1     Running   0          2m12s   10.244.251.206   k8scloude3   <none>           <none>
nginx-6fcb7995f5-8c75n   1/1     Running   0          6m13s   10.244.251.205   k8scloude3   <none>           <none>
nginx-6fcb7995f5-cs6w4   1/1     Running   0          6m13s   10.244.112.143   k8scloude2   <none>           <none>
nginx-6fcb7995f5-f6nz9   1/1     Running   0          4s      10.244.112.150   k8scloude2   <none>           <none>
nginx-6fcb7995f5-hx224   1/1     Running   0          4s      10.244.112.149   k8scloude2   <none>           <none>
nginx-6fcb7995f5-s6smg   1/1     Running   0          13m     10.244.112.147   k8scloude2   <none>           <none>
nginx-6fcb7995f5-tpwr4   1/1     Running   0          2m12s   10.244.112.148   k8scloude2   <none>           <none>
nginx-6fcb7995f5-zmsfx   1/1     Running   0          4s      10.244.251.208   k8scloude3   <none>           <none>

修改pod副本为1

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=1
deployment.apps/nginx scaled

[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
nginx-6fcb7995f5-8c75n   1/1     Running   0          6m33s   10.244.251.205   k8scloude3   <none>           <none>

七.HorizontalPodAutoscaler(HPA)

7.1 Pod 水平自动扩缩HorizontalPodAutoscaler概览

在 Kubernetes 中,HorizontalPodAutoscaler 自动更新工作负载资源 (例如 Deployment 或者 StatefulSet), 目的是自动扩缩工作负载以满足需求

水平扩缩意味着对增加的负载的响应是部署更多的 Pod。 这与 “垂直(Vertical)” 扩缩不同,对于 Kubernetes, 垂直扩缩意味着将更多资源(例如:内存或 CPU)分配给已经为工作负载运行的 Pod

如果负载减少,并且 Pod 的数量高于配置的最小值, HorizontalPodAutoscaler 会指示工作负载资源(Deployment、StatefulSet 或其他类似资源)缩减。

水平 Pod 自动扩缩不适用于无法扩缩的对象(例如:DaemonSet。)

HorizontalPodAutoscaler 被实现为 Kubernetes API 资源和控制器。

资源决定了控制器的行为。 在 Kubernetes 控制平面内运行的水平 Pod 自动扩缩控制器会定期调整其目标(例如:Deployment)的所需规模,以匹配观察到的指标, 例如,平均 CPU 利用率、平均内存利用率或你指定的任何其他自定义指标。

image-20230203164833817

7.2 HPA自动伸缩pod副本数实战

HPA(horizontal pod autoscalers)水平自动伸缩 通过检测pod CPU的负载,解决deployment里某pod负 载太重,动态伸缩pod的数量来负载均衡。

现在deploy只生成了一个pod

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-8c75n   1/1     Running   0          45m

查看HPA

[root@k8scloude1 deployment]# kubectl get hpa
No resources found in deployment namespace.

设置HPA:--min=2 最小pod副本数为2,--max=5 最大pod副本数为5

[root@k8scloude1 deployment]# kubectl autoscale deployment nginx --max=5 --min=2
horizontalpodautoscaler.autoscaling/nginx autoscaled

观察pod副本数,发现pod副本数自动变为2了,因为HPA设置了最小pod副本数为2

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-8c75n   1/1     Running   0          49m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-8c75n   1/1     Running   0          49m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-8c75n   1/1     Running   0          49m
nginx-6fcb7995f5-zrdpj   1/1     Running   0          22s

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-8c75n   1/1     Running   0          49m
nginx-6fcb7995f5-zrdpj   1/1     Running   0          39s

使用kubectl scale 设置pod副本数为1

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=1
deployment.apps/nginx scaled

因为autoscale是实时扩展的,所以就算scale修改为1,autoscale也会自动把副本数修改为2

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS              RESTARTS   AGE
nginx-6fcb7995f5-5jl6h   0/1     ContainerCreating   0          0s
nginx-6fcb7995f5-8c75n   1/1     Running             0          50m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-5jl6h   1/1     Running   0          4s
nginx-6fcb7995f5-8c75n   1/1     Running   0          50m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-5jl6h   1/1     Running   0          16s
nginx-6fcb7995f5-8c75n   1/1     Running   0          50m

查看hpa

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   <unknown>/80%   2         5         2          4m18s

删除HPA

[root@k8scloude1 deployment]# kubectl delete hpa nginx 
horizontalpodautoscaler.autoscaling "nginx" deleted

[root@k8scloude1 deployment]# kubectl get hpa
No resources found in deployment namespace.

删除nginx deployment

[root@k8scloude1 deployment]# kubectl delete -f nginx.yaml 
deployment.apps "nginx" deleted

[root@k8scloude1 deployment]# kubectl get deploy
No resources found in deployment namespace.

接下来给pod设置资源限制,对pod的资源限制可以通过pod里的resource字段来限制,resources里的requests字段表示容器所在节点资源的最小值,最低要求,满足不了这个要求pod创建不成功。

requests:cpu: 400m 表示pod需要400个微核心才能创建成功注意:m代表微核心,1个核等于1000个微核心。

[root@k8scloude1 deployment]# vim nginxcpulimit.yaml 

[root@k8scloude1 deployment]# cat nginxcpulimit.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      appx: xyz
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
        appx: xyz
    spec:
      terminationGracePeriodSeconds: 0
      containers:
      - image: nginx
        name: nginx
        imagePullPolicy: IfNotPresent
        resources: 
          #设置资源限制:CPU:400m    #m代表微核心,1个核等于1000个微核心
          requests:
            cpu: 400m
status: {}

创建deployment

[root@k8scloude1 deployment]# kubectl apply -f nginxcpulimit.yaml 
deployment.apps/nginx created

[root@k8scloude1 deployment]# kubectl get deployments -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR
nginx   1/1     1            1           13s   nginx        nginx    appx=xyz

查看node节点的负载

[root@k8scloude1 deployment]# kubectl top node 
W0125 17:06:20.377474   15650 top_node.go:119] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
k8scloude1   230m         11%    1375Mi          42%       
k8scloude2   100m         5%     669Mi           34%       
k8scloude3   113m         5%     737Mi           38%   

设置HPA:最小pod副本数为1,最大pod副本数为15,--cpu-percent=80表示当deployment资源对象的CPU使用率高达80%时,就会进行扩容,最多扩容到15个

[root@k8scloude1 deployment]# kubectl autoscale deployment nginx --min=1 --max=15 --cpu-percent=80
horizontalpodautoscaler.autoscaling/nginx autoscaled

查看hpa

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/80%    1         15        1          69s

安装一个压力测试的包:ab工具用于压力测试

[root@k8scloude1 deployment]# yum -y install httpd-tools

创建一个service服务,服务端口为80,service类型为NodePort

[root@k8scloude1 deployment]# kubectl expose --name=nginxsvc deployment nginx --port=80 --type=NodePort
service/nginxsvc exposed

[root@k8scloude1 deployment]# kubectl get svc -o wide
NAME       TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE   SELECTOR
nginxsvc   NodePort   10.96.231.54   <none>        80:31085/TCP   10s   appx=xyz

查看pod

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-79b8d956bd-wbcq5   1/1     Running   0          12m

使用ab进行压力测试,一直给nginx服务发请求,Nginx服务的网址为:http://k8s节点ip:31085/,也就是http://192.168.110.130:31085/index.html。

ab压力测试命令参数解释:

  • -t:测试所进行的最大秒数。其内部隐含值是-n 50000,它可以使对服务器的测试限制在一个固定的总时间以内。默认时,没有时间限制。
  • -n:在测试会话中所执行的请求个数。默认时,仅执行一个请求。
  • -c:一次产生的请求个数。默认是一次一个。

下面一直给Nginx服务发请求

[root@k8scloude1 ~]# ab -t 600 -n 1000000 -c 1000 http://192.168.110.130:31085/index.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 %sSourceCode%gt;
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.110.130 (be patient)
Completed 100000 requests
^C

Server Software:        nginx/1.21.5
Server Hostname:        192.168.110.130
Server Port:            31085

Document Path:          /index.html
Document Length:        615 bytes

Concurrency Level:      1000
Time taken for tests:   27.096 seconds
Complete requests:      112145
Failed requests:        2029
   (Connect: 0, Receive: 0, Length: 0, Exceptions: 2029)
Write errors:           0
Total transferred:      95814989 bytes
HTML transferred:       69487620 bytes
Requests per second:    4138.87 [#/sec] (mean)
Time per request:       241.612 [ms] (mean)
Time per request:       0.242 [ms] (mean, across all concurrent requests)
Transfer rate:          3453.31 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0  121 219.7     73    2693
Processing:     1  113  84.3     96    5111
Waiting:        0   95  78.9     80    5063
Total:          2  235 241.9    178    7568

Percentage of the requests served within a certain time (ms)
  50%    178
  66%    208
  75%    230
  80%    247
  90%    319
  95%    469
  98%   1188
  99%   1233
 100%   7568 (longest request)

ab压力测试之后,查看pod的变化,pod数自动变为了3个

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-79b8d956bd-622fw   1/1     Running   0          14s
nginx-79b8d956bd-8z5h7   1/1     Running   0          14s
nginx-79b8d956bd-wbcq5   1/1     Running   0          14m

查看hpa的变化

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   31%/80%   1         15        3          11m

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   61%/80%   1         15        3          12m

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   61%/80%   1         15        3          12m
[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   61%/80%   1         15        3          12m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-79b8d956bd-622fw   1/1     Running   0          101s
nginx-79b8d956bd-8z5h7   1/1     Running   0          101s
nginx-79b8d956bd-wbcq5   1/1     Running   0          15m

删除服务并重新创建一个service服务,这次的service服务类型为ClusterIP。

[root@k8scloude1 deployment]# kubectl delete svc nginxsvc

[root@k8scloude1 deployment]# kubectl get svc
No resources found in deployment namespace.

[root@k8scloude1 deployment]# kubectl expose --name=nginxsvc deployment nginx --port=80
service/nginxsvc exposed

[root@k8scloude1 deployment]# kubectl get svc
NAME       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
nginxsvc   ClusterIP   10.109.240.56   <none>        80/TCP    3s

Nginx服务的网址为:http://10.109.240.56:80/index.html,继续进行ab压力测试,给Nginx服务发请求。

#继续ab压力测试
[root@k8scloude1 ~]# ab -t 600 -n 1000000 -c 1000 http://10.109.240.56:80/index.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 %sSourceCode%gt;
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.109.240.56 (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 985106 requests completed

执行ab压力测试的同时观察pod的变化

查看pod的负载

[root@k8scloude1 deployment]# kubectl top pods
W0125 17:25:03.753728   30784 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME                     CPU(cores)   MEMORY(bytes)   
nginx-79b8d956bd-5hq5j   179m         4Mi             
nginx-79b8d956bd-622fw   366m         5Mi             
nginx-79b8d956bd-6wjsl   248m         5Mi             
nginx-79b8d956bd-8z5h7   360m         5Mi             
nginx-79b8d956bd-wbcq5   254m         5Mi      

查看hpa,现在已经有5个pod了

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   71%/80%   1         15        5          16m

继续观察pod

[root@k8scloude1 deployment]# kubectl top pods
W0125 17:25:49.908634   31380 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME                     CPU(cores)   MEMORY(bytes)   
nginx-79b8d956bd-5hq5j   254m         5Mi             
nginx-79b8d956bd-622fw   326m         5Mi             
nginx-79b8d956bd-6wjsl   322m         5Mi             
nginx-79b8d956bd-8z5h7   255m         5Mi             
nginx-79b8d956bd-wbcq5   264m         5Mi             

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   70%/80%   1         15        5          18m

[root@k8scloude1 deployment]# kubectl top pods
W0125 17:27:46.476364   32773 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME                     CPU(cores)   MEMORY(bytes)   
nginx-79b8d956bd-5hq5j   285m         5Mi             
nginx-79b8d956bd-622fw   331m         5Mi             
nginx-79b8d956bd-6wjsl   351m         5Mi             
nginx-79b8d956bd-8z5h7   277m         5Mi             
nginx-79b8d956bd-wbcq5   264m         6Mi    
         
[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   75%/80%   1         15        5          19m

停止给nginx发送请求之后hpa的TARGETS降下来了

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/80%    1         15        5          20m

但是pod数没有降下来

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/80%    1         15        5          20m

[root@k8scloude1 deployment]# kubectl top pods
W0125 17:29:32.004502   34116 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME                     CPU(cores)   MEMORY(bytes)   
nginx-79b8d956bd-5hq5j   0m           5Mi             
nginx-79b8d956bd-622fw   0m           5Mi             
nginx-79b8d956bd-6wjsl   0m           5Mi             
nginx-79b8d956bd-8z5h7   0m           5Mi             
nginx-79b8d956bd-wbcq5   0m           6Mi             

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/80%    1         15        5          21m

注意:pod负载降下来之后,pod副本数不会立即减少,过一段时间pod副本数才会减少,默认时间间隔为5分钟,这样的目的是为了防止pod副本数的抖动。

10分钟之后观察pod,pod副本降下来了。

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/80%    1         15        1          46m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-79b8d956bd-wbcq5   1/1     Running   0          50m

[root@k8scloude1 deployment]# kubectl delete hpa nginx 
horizontalpodautoscaler.autoscaling "nginx" deleted

八.k8s升级镜像

8.1 kubectl edit deployment升级镜像

查看deploy,可以看到镜像为nginx

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   1/1     1            1           179m

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES   SELECTOR
nginx   1/1     1            1           179m   nginx        nginx    appx=xyz

第一种方法:使用kubectl edit deployment deployname升级镜像。把deploy的镜像由nginx升级为hub.c.163.com/library/nginx:1.12.0,修改 - image: nginx 为 - image: hub.c.163.com/library/nginx:1.12.0

[root@k8scloude1 deployment]# kubectl edit deployment nginx 
deployment.apps/nginx edited

修改之后查看deploy,IMAGES变了。

修改deploy的镜像的时候,本质上是删除现有的pod,然后用新镜像创建新pod

[root@k8scloude1 deployment]# kubectl get deployments -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           3h3m   nginx        hub.c.163.com/library/nginx:1.12.0   appx=xyz

[root@k8scloude1 deployment]# kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
nginx-66bcf4489b-ztvnc   1/1     Running   0          77s   10.244.251.211   k8scloude3   <none>           <none>

8.2 kubectl set image deploy升级镜像

查看deploy

[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           6h41m   nginx        hub.c.163.com/library/nginx:1.12.0   appx=xyz

第二种方式:使用kubectl set image deploy deployname 容器名=镜像名 升级镜像。

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0
deployment.apps/nginx image updated

查看deploy,可以发现镜像升级为hub.c.163.com/library/nginx:1.13.0了

[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           6h42m   nginx        hub.c.163.com/library/nginx:1.13.0   appx=xyz

继续升级镜像

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest
deployment.apps/nginx image updated

[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           6h43m   nginx        hub.c.163.com/library/nginx:latest   appx=xyz

[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
nginx-75698d6db5-bzv9d   1/1     Running   0          17s   10.244.251.212   k8scloude3   <none>           <none>

8.3 镜像升级历史记录

查看镜像替换的历史记录 kubectl rollout history deployment nginx,现在镜像替换的历史记录是不清楚的,是因为升级镜像的时候没有record记录。

[root@k8scloude1 deployment]# kubectl rollout history deployment nginx 
deployment.apps/nginx 
REVISION  CHANGE-CAUSE
1         <none>
2         <none>
3         <none>
4         <none>

替换镜像并记录

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.12.0 --record
deployment.apps/nginx image updated

可以看到镜像替换成功

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           6h47m   nginx        hub.c.163.com/library/nginx:1.12.0   appx=xyz

多进行几次镜像替换并记录

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record
deployment.apps/nginx image updated

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest --record
deployment.apps/nginx image updated

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=nginx --record
deployment.apps/nginx image updated

查看镜像替换的历史记录

[root@k8scloude1 deployment]# kubectl rollout history deployment nginx
deployment.apps/nginx 
REVISION  CHANGE-CAUSE
5         kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.12.0 --record=true
6         kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record=true
7         kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest --record=true
8         kubectl set image deploy nginx nginx=nginx --record=true

查看deploy和pod

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES   SELECTOR
nginx   1/1     1            1           6h48m   nginx        nginx    appx=xyz

[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
nginx-79b8d956bd-xblj9   1/1     Running   0          61s   10.244.251.214   k8scloude3   <none>           <none>

kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record=true是第6条修改记录,我们可以进行回滚,回滚到镜像是hub.c.163.com/library/nginx:1.13.0的时候。

[root@k8scloude1 deployment]# kubectl rollout undo deployment nginx --to-revision=6
deployment.apps/nginx rolled back

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           6h51m   nginx        hub.c.163.com/library/nginx:1.13.0   appx=xyz

变更镜像的本质是删除旧版本的pod,然后用新镜像创建pod。

修改deploy副本数为10个

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=10
deployment.apps/nginx scaled

编辑deploy,修改镜像

[root@k8scloude1 deployment]# kubectl edit deployment nginx 
deployment.apps/nginx edited

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES                               SELECTOR
nginx   5/5     5            5           7h1m   nginx        hub.c.163.com/library/nginx:1.13.0   appx=xyz

 #滚动更新配置说明:
strategy:
    rollingUpdate:
          #maxSurge:在升级过程中一次升级几个,可以是数字也可以是百分比
      maxSurge: 25%
          #maxUnavailable 在升级过程中,只能有几个不可用, 一次性删除多少个pod,可以是数字也可以是百分比
      maxUnavailable: 25%
    type: RollingUpdate
posted @ 2023-02-08 02:26  人生的哲理  阅读(1063)  评论(1编辑  收藏  举报