Pod容器自动伸缩(HPA) 测试

HPA（Horizontal Pod Autoscaler）在k8s集群中用于POD水平自动伸缩，它是基于CPU和内存利用率对Deployment和Replicaset控制器中的pod数量进行自动扩缩容（除了CPU和内存利用率之外，也可以基于其他应程序提供的度量指标custom metrics进行自动扩缩容）。pod自动缩放不适用于无法缩放的对象，比如DaemonSets。HPA由Kubernetes API资源和控制器实现。资源决定了控制器的行为，控制器会周期性的获取CPU和内存利用率，并与目标值相比较后来调整replication controller或deployment中的副本数量。

HPA使用前提条件：

集群中部署了Metrics Server插件, 可以获取到Pod的CPU和内存利用率。
Pod部署的Yaml文件中必须设置资源限制和资源请求。

以下对K8S集群使用HPA进行Pod自动伸缩做个测试记录

1. 编译一个测试容器

[root@k8s-master01 work]# cat kevin-t.yaml
apiVersion: v1
kind: Service
metadata:
  name: kevin-t
  namespace: default
  labels:
    app: kevin-t
spec:
  type: NodePort
  selector:
    app: kevin-t
  ports:
  - name: http
    port: 8088
    targetPort: 80
    nodePort: 30888
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kevin-t
  namespace: default
spec:
  replicas: 1
  minReadySeconds: 10
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  selector:
    matchLabels:
      app: kevin-t
  template:
    metadata:
      labels:
        app: kevin-t
    spec:
      containers:
      - name: kevin-t
        image: nginx:1.7.9
        imagePullPolicy: IfNotPresent
        ports:
        - name: kport
          containerPort: 80
        resources:
          requests:
            cpu: 100m
            memory: 50Mi
          limits:
            cpu: 100m
            memory: 50Mi
        lifecycle:
          postStart:
            exec:
              command: ["/bin/sh","-c","touch /tmp/health"]
        livenessProbe:
          exec:
            command: ["test","-e","/tmp/health"]
          initialDelaySeconds: 5
          timeoutSeconds: 3
          periodSeconds: 5
        readinessProbe:
          tcpSocket:
            port: kport
          initialDelaySeconds: 5
          timeoutSeconds: 3
          periodSeconds: 5
 
执行创建
[root@k8s-master01 work]# kubectl apply -f kevin-t.yaml
 
查看pod
[root@k8s-master01 work]# kubectl get pods|grep kevin-t
kevin-t-66cdc9cdbf-jwvnv    1/1     Running   0          119s
 
[root@k8s-master01 work]# kubectl get svc|grep kevin-t
kevin-t       NodePort    10.254.120.80    <none>        8088:30888/TCP   4m20s
 
访问
[root@k8s-master01 work]# curl http://172.16.60.235:30888
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
 
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
 
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
 
查看metrcis-service采集到的该pod的cpu和内存数据
[root@k8s-master01 work]# kubectl top pods
NAME                        CPU(cores)   MEMORY(bytes)
dnsutils-ds-22q87           0m           0Mi
dnsutils-ds-jcvng           0m           0Mi
dnsutils-ds-zjg5l           0m           0Mi
kevin-t-66cdc9cdbf-jwvnv    1m           3Mi
my-nginx-765d6dcff7-gs97l   0m           3Mi
my-nginx-765d6dcff7-lbjm6   0m           2Mi
nginx-ds-98rm2              0m           2Mi
nginx-ds-bbx68              0m           2Mi
nginx-ds-ngqcm              0m           2Mi
 
通过上面可知，使用kevin-t的deployment控制的kevin-t的pod容器资源请求和资源限制都是100m的CPU（即0.1核）和50M的内存。当前该pod所用CPU是1m，内存是3M

2. 创建HPA限制
设置kevin-t的 deployment的最大最小副本数，HPA对应pod的CPU和内存指标做限制。

参数解释：
scaleTargetRef: 要缩放的目标资源
metrics: 计算所需Pod副本数量的指标列表，每个指标单独计算，取所有计算结果的最大值作为最终副本数量
- external 引用非附属于任何对象的全局指标，可以是集群之外的组件的指标数据，如消息队列长度
- object 引用描述集群中某单一对象的特定指标，如Ingress对象上的hits-per-second等
- pods 引用当前被弹性伸缩的Pod对象的特定指标
- resource 引用资源指标，即当前被弹性伸缩的Pod对象中容器的requests和limits中定义的指标
- type 指标源的类型，可为Objects、Pods、Resource

另外注意两个指标阈值参数：

targetAverageUtilization 表示的是百分比
targetAverageValue 表示的是数值，比如100m的CPU、100Mi的内存

[root@k8s-master01 work]# cat kevin-t-hap.yml
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: kevin-t-hap
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: kevin-t
  minReplicas: 1
  maxReplicas: 3
  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 80
  - type: Resource
    resource:
      name: memory
      targetAverageValue: 30Mi
 
  如上，设置了kevin-t的deployment控制的pod的HPA限制，当cpu使用超过设置的80%，内存使用超过30Mi时就触发自动扩容，副本数最小为1，最大为3。
 
  查看创建的HPA
[root@k8s-master01 work]# kubectl get hpa
NAME          REFERENCE            TARGETS                MINPODS   MAXPODS   REPLICAS   AGE
kevin-t-hap   Deployment/kevin-t   3194880/30Mi, 1%/80%   1         3         3          2m15s
 
[root@k8s-master01 work]# kubectl describe hpa kevin-t-hap
Name:                                                  kevin-t-hap
Namespace:                                             default
Labels:                                                <none>
Annotations:                                           CreationTimestamp:  Fri, 14 Aug 2020 17:00:01 +0800
Reference:                                             Deployment/kevin-t
Metrics:                                               ( current / target )
  resource memory on pods:                             3291818666m / 30Mi
  resource cpu on pods  (as a percentage of request):  1% (1m) / 80%
Min replicas:                                          1
Max replicas:                                          3
Deployment pods:                                       3 current / 3 desired
Conditions:
  Type            Status  Reason               Message
  ----            ------  ------               -------
  AbleToScale     True    ScaleDownStabilized  recent recommendations were higher than current one, applying the highest recent recommendation
  ScalingActive   True    ValidMetricFound     the HPA was able to successfully calculate a replica count from memory resource
  ScalingLimited  True    TooManyReplicas      the desired replica count is more than the maximum replica count
Events:
  Type    Reason             Age   From                       Message
  ----    ------             ----  ----                       -------
  Normal  SuccessfulRescale  4m3s  horizontal-pod-autoscaler  New size: 3; reaso

3. 压力测试，验证HPA的扩缩容
使用ab命令进行压力测试，主要有两个参数：
-n ：总共的请求执行数，缺省是1；
-c：并发数，缺省是1；

测试之前kevin-t的pod是一个副本数
[root@k8s-master01 work]# kubectl get pods|grep kevin-t
kevin-t-66cdc9cdbf-jwvnv    1/1     Running   0          119s
 
接下来进行压力测试
[root@k8s-master01 work]# ab -c 1000 -n 500000000000 http://172.16.60.235:30888/
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
 
Benchmarking 172.16.60.235 (be patient)
apr_socket_recv: Connection reset by peer (104)
Total of 12623 requests completed
 
发现此时，kevin-t 已经扩到2个副本了
[root@k8s-master02 ~]# kubectl get pods|grep kevin-t
kevin-t-66cdc9cdbf-5h564    1/1     Running   0          38s
kevin-t-66cdc9cdbf-jwvnv    1/1     Running   0          35m
 
再加大压力测试：
[root@k8s-master01 work]# ab -c 100 -n 50000000000000000 http://172.16.60.235:30888/
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
 
Benchmarking 172.16.60.235 (be patient)
 
在上面压力测试中，另外打开一个终端，发现kevin-t已经扩容到最大的3个副本了！
[root@k8s-master02 ~]# kubectl get pods|grep kevin-t
kevin-t-66cdc9cdbf-5h564    1/1     Running   0          4m52s
kevin-t-66cdc9cdbf-jwvnv    1/1     Running   0          39m
kevin-t-66cdc9cdbf-zpbfs    1/1     Running   0          80s
 
查看hpa，实时查看pod的cpu和内存状态
[root@k8s-master01 work]# kubectl get hpa -w
NAME          REFERENCE            TARGETS                    MINPODS   MAXPODS   REPLICAS   AGE
kevin-t-hap   Deployment/kevin-t   4434602666m/30Mi, 1%/80%   1         3         3          19m
kevin-t-hap   Deployment/kevin-t   4405930666m/30Mi, 1%/80%   1         3         3          19m
kevin-t-hap   Deployment/kevin-t   4441429333m/30Mi, 1%/80%   1         3         3          20m
kevin-t-hap   Deployment/kevin-t   4441429333m/30Mi, 1%/80%   1         3         3          20m
kevin-t-hap   Deployment/kevin-t   4945920/30Mi, 1%/80%       1         3         2          20m
kevin-t-hap   Deployment/kevin-t   4945920/30Mi, 1%/80%       1         3         2          21m
...............
 
观察kevin-t的pod情况
[root@k8s-master02 ~]# kubectl get pods|grep kevin-t -w
kevin-t-66cdc9cdbf-5h564    1/1     Running   0          11m
kevin-t-66cdc9cdbf-jwvnv    1/1     Running   0          46m
kevin-t-66cdc9cdbf-zpbfs    1/1     Running   0          7m36s
[root@k8s-master02 ~]# kubectl get pods|grep kevin-t -w
kevin-t-66cdc9cdbf-5h564    1/1     Terminating   0          12m
kevin-t-66cdc9cdbf-jwvnv    1/1     Running       0          47m
 
发现压力测试过一段时间后，随着pod的cpu和内存使用数值下降，副本数也在逐步减少到最小的1个副本
[root@k8s-master02 ~]# kubectl get pods|grep kevin-t
kevin-t-66cdc9cdbf-jwvnv    1/1     Running   0          49m

posted @ 2020-11-13 17:03 散尽浮华阅读(2316) 评论(0) 编辑收藏举报

会员力量，点亮园子希望

刷新页面返回顶部

散尽浮华

重剑无锋，大巧不工！

Pod容器自动伸缩(HPA) 测试

公告