Kubernetes之（十四）StatefulSet控制器

Kubernetes之（十四）StatefulSet控制器
- 简介
- StatefulSet使用

Kubernetes之（十四）StatefulSet控制器

简介

StatefulSet 作为 Controller 为 Pod 提供唯⼀的标识。它可以保证部署和 scale 的顺序。
StatefulSet是为了解决有状态服务的问题（对应Deployments和ReplicaSets是为⽆状态服务⽽设计），其应⽤场景包括：

稳定的持久化存储，即Pod重新调度后还是能访问到相同的持久化数据，基于PVC来实现
稳定的⽹络标志，即Pod重新调度后其PodName和HostName不变，基于Headless Service（即没有Cluster IP的Service）来实现
有序部署，有序扩展，即Pod是有顺序的，在部署或者扩展的时候要依据定义的顺序依次依次进⾏（即从0到N-1，在下⼀个Pod运⾏之前所有之前的Pod必须都是Running和Ready状态），基于init containers来实现有序收缩，有序删除（即从N-1到0）

从上⾯的应⽤场景可以发现， StatefulSet由以下⼏个部分组成：

⽤于定义⽹络标志（DNS domain）的Headless Service
⽤于创建PersistentVolumes的volumeClaimTemplates
定义具体应⽤的StatefulSet

StatefulSet中每个Pod的DNS格式为 statefulSetName-{0..N-1}.serviceName.namespace.svc.cluster.local ，其中

serviceName 为Headless Service的名字
0..N-1 为Pod所在的序号，从0开始到N-1
statefulSetName 为StatefulSet的名字
namespace 为服务所在的namespace， Headless Servic和StatefulSet必须在相同的namespace
.cluster.local 为Cluster Domain

部署和Scale保证:

对于有N个副本的StatefulSet，Pod将按照{0..N-1}的顺序被创建和部署。
当删除Pod的时候，将按照逆序来终结，从
对Pod执⾏scale操作之前，它所有的前任必须处于Running和Ready状态。
在终⽌Pod前，它所有的继任者必须处于完全关闭状态。

Headless Service：
在deployment中，每一个pod是没有名称，是随机字符串，是无序的。而statefulset中是要求有序的，每一个pod的名称必须是固定的。当节点挂了，重建之后的标识符是不变的，每一个节点的节点名称是不能改变的。pod名称是作为pod识别的唯一标识符，必须保证其标识符的稳定并且唯一。
为了实现标识符的稳定，这时候就需要一个headless service 解析直达到pod，还需要给pod配置一个唯一的名称。

volumeClainTemplate：
大部分有状态副本集都会用到持久存储，比如分布式系统来说，由于数据是不一样的，每个节点都需要自己专用的存储节点。而在deployment中pod模板中创建的存储卷是一个共享的存储卷，多个pod使用同一个存储卷，而statefulset定义中的每一个pod都不能使用同一个存储卷，由此基于pod模板创建pod是不适应的，这就需要引入volumeClainTemplate，当在使用statefulset创建pod时，会自动生成一个PVC，从而请求绑定一个PV，从而有自己专用的存储卷。Pod名称、PVC和PV关系图如下：

StatefulSet使用

使用之前配置的NFS服务器，PV

#nfs服务器
[root@nfs ~]# showmount -e 
Export list for nfs:
/data/volumes/v5 10.0.0.0/24
/data/volumes/v4 10.0.0.0/24
/data/volumes/v3 10.0.0.0/24
/data/volumes/v2 10.0.0.0/24
/data/volumes/v1 10.0.0.0/24


#PV
[root@master volume]# kubectl get pv
NAME    CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
pv001   1Gi        RWO,RWX        Retain           Available                                   3s
pv002   2Gi        RWO            Retain           Available                                   3s
pv003   2Gi        RWO,RWX        Retain           Available                                   3s
pv004   4Gi        RWO,RWX        Retain           Available                                   3s
pv005   4Gi        RWO,RWX        Retain           Available                                   3s                         23h

创建statefulSet

[root@master statefulset]# vim statefulset-v1.yaml
#定义Headless服务
apiVersion: v1
kind: Service
metadata:
  name: myapp
  labels:
    app: myapp
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: myapp-pod
---
#配置statefulset
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: myapp
spec:
  serviceName: myapp-svc
  replicas: 2
  selector:
    matchLabels:
      app: myapp-pod
  template:
    metadata:
      labels:
        app: myapp-pod
    spec:
      containers:
      - name: myapp
        image: ikubernetes/myapp:v1
        ports:
        - name: web
          containerPort: 80
        volumeMounts:
        - name: myappdata
          mountPath: /usr/share/nginx/html/
  volumeClaimTemplates:
  - metadata:
      name: myappdata
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1.5Gi
[root@master statefulset]# kubectl apply -f statefulset-v1.yaml 
service/myapp created
statefulset.apps/myapp created

[root@master statefulset]# kubectl get pods
NAME      READY   STATUS    RESTARTS   AGE
myapp-0   1/1     Running   0          5s
myapp-1   1/1     Running   0          4s
myapp-2   1/1     Running   0          3s
#此时Pod名称不是乱码是从0~N-1
[root@master statefulset]# kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   7d
myapp        ClusterIP   None         <none>        80/TCP    31s

[root@master statefulset]# kubectl get sts
NAME    READY   AGE
myapp   3/3     48s

查看pvc pv资源

[root@master statefulset]# kubectl get pvc
NAME                STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
myappdata-myapp-0   Bound    pv002    2Gi        RWO                           3m56s
myappdata-myapp-1   Bound    pv003    2Gi        RWO,RWX                       2m49s
myappdata-myapp-2   Bound    pv004    4Gi        RWO,RWX                       88s
[root@master statefulset]# kubectl get pv
NAME    CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                       STORAGECLASS   REASON   AGE
pv001   1Gi        RWO,RWX        Retain           Available                                                       22m
pv002   2Gi        RWO            Retain           Bound       default/myappdata-myapp-0                           22m
pv003   2Gi        RWO,RWX        Retain           Bound       default/myappdata-myapp-1                           22m
pv004   4Gi        RWO,RWX        Retain           Bound       default/myappdata-myapp-2                           22m
pv005   4Gi        RWO,RWX        Retain           Available                                                       22m

查看Pod使用的存储卷

[root@master statefulset]# kubectl describe pods myapp-2
......
Volumes:
  myappdata:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  myappdata-myapp-2
......

就算删除Pod后重建的Podi依然绑定在myappdata-myapp-2这个PVC上

滚动更新

[root@master statefulset]# vim statefulset-v1.yaml

        image: ikubernetes/myapp:v3

[root@master statefulset]# kubectl apply -f statefulset-v1.yaml 
service/myapp unchanged
statefulset.apps/myapp configured

[root@master ~]# kubectl get pods -w
NAME      READY   STATUS    RESTARTS   AGE
myapp-0   1/1     Running   0          9m18s
myapp-1   1/1     Running   0          9m17s
myapp-2   1/1     Running   0          9m16s
myapp-2   1/1   Terminating   0     9m22s
myapp-2   0/1   Terminating   0     9m23s
myapp-2   0/1   Terminating   0     9m26s
myapp-2   0/1   Terminating   0     9m26s
myapp-2   0/1   Pending   0     0s
myapp-2   0/1   Pending   0     0s
myapp-2   0/1   ContainerCreating   0     0s
myapp-2   1/1   Running   0     2s
myapp-1   1/1   Terminating   0     9m29s
myapp-1   0/1   Terminating   0     9m30s
myapp-1   0/1   Terminating   0     9m31s
myapp-1   0/1   Terminating   0     9m31s
myapp-1   0/1   Pending   0     0s
myapp-1   0/1   Pending   0     0s
myapp-1   0/1   ContainerCreating   0     0s
myapp-1   1/1   Running   0     1s
myapp-0   1/1   Terminating   0     9m33s
myapp-0   0/1   Terminating   0     9m34s
myapp-0   0/1   Terminating   0     9m35s
myapp-0   0/1   Terminating   0     9m35s
myapp-0   0/1   Pending   0     0s
myapp-0   0/1   Pending   0     0s
myapp-0   0/1   ContainerCreating   0     0s
myapp-0   1/1   Running   0     1s

在创建的每一个Pod中，每一个pod自己的名称都是可以被解析的，如下：

[root@master statefulset]# kubectl get pods -o wide
NAME      READY   STATUS    RESTARTS   AGE   IP            NODE     NOMINATED NODE   READINESS GATES
myapp-0   1/1     Running   0          39s   10.244.2.66   node02   <none>           <none>
myapp-1   1/1     Running   0          42s   10.244.1.56   node01   <none>           <none>
myapp-2   1/1     Running   0          46s   10.244.2.65   node02   <none>           <none>

[root@master statefulset]# kubectl exec -it myapp-1 -- /bin/sh
/ # nslookup myapp-1.myapp-svc.default.svc.cluster.local
nslookup: can't resolve '(null)': Name does not resolve

Name:      myapp-1.myapp-svc.default.svc.cluster.local
Address 1: 10.244.1.56 myapp-1.myapp-svc.default.svc.cluster.local

从上面的解析，我们可以看到在容器当中可以通过对Pod的名称进行解析到ip。其解析的域名格式如下：
pod_name.service_name.ns_name.svc.cluster.local
eg: myapp-0.myapp.default.svc.cluster.local

扩展伸缩

#扩展副本到4个
[root@master statefulset]# kubectl scale sts myapp --replicas=4
statefulset.apps/myapp scaled
#查看结果
[root@master ~]# kubectl get pods -w
NAME      READY   STATUS    RESTARTS   AGE
myapp-0   1/1     Running   0          6m17s
myapp-1   1/1     Running   0          6m20s
myapp-2   1/1     Running   0          6m24s
myapp-3   0/1   Pending   0     0s
myapp-3   0/1   Pending   0     0s
myapp-3   0/1   Pending   0     0s
myapp-3   0/1   ContainerCreating   0     0s
myapp-3   1/1   Running   0     2s

#查看PV绑定
[root@master statefulset]# kubectl get pv
NAME    CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                       STORAGECLASS   REASON   AGE
pv001   1Gi        RWO,RWX        Retain           Available                                                       38m
pv002   2Gi        RWO            Retain           Bound       default/myappdata-myapp-0                           38m
pv003   2Gi        RWO,RWX        Retain           Bound       default/myappdata-myapp-1                           38m
pv004   4Gi        RWO,RWX        Retain           Bound       default/myappdata-myapp-2                           38m
pv005   4Gi        RWO,RWX        Retain           Bound       default/myappdata-myapp-3   

 #打补丁方式缩容
 [root@master statefulset]# kubectl patch sts myapp -p '{"spec":{"replicas":2}}'
statefulset.apps/myapp patched

[root@master ~]# kubectl get pods -w
NAME      READY   STATUS    RESTARTS   AGE
myapp-0   1/1     Running   0          9m26s
myapp-1   1/1     Running   0          9m29s
myapp-2   1/1     Running   0          13s
myapp-3   1/1     Running   0          11s
myapp-3   1/1   Terminating   0     16s
myapp-3   0/1   Terminating   0     18s
myapp-3   0/1   Terminating   0     22s
myapp-3   0/1   Terminating   0     22s
myapp-2   1/1   Terminating   0     24s
myapp-2   0/1   Terminating   0     24s
myapp-2   0/1   Terminating   0     30s
myapp-2   0/1   Terminating   0     30s

更新策略和版本升级

修改更新策略，以partition方式进行更新，更新值为2，只有myapp编号大于等于2的才会进行更新。类似于金丝雀部署方式

[root@master statefulset]# kubectl patch sts myapp -p '{"spec":{"updateStrategy":{"rollingUpdate":{"partition":2}}}}'
statefulset.apps/myapp patched

[root@master statefulset]# kubectl describe sts myapp
Name:               myapp
Namespace:          default
CreationTimestamp:  Wed, 03 Apr 2019 16:53:24 +0800
Selector:           app=myapp-pod
Labels:             <none>
Annotations:        kubectl.kubernetes.io/last-applied-configuration:
                      {"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"name":"myapp","namespace":"default"},"spec":{"replicas":3,"sele...

版本升级，将image的版本升级为v5，升级后对比myapp-2和myapp-1的image版本是不同的。这样就实现了金丝雀发布的效果。

[root@master statefulset]# kubectl get sts -o wide
NAME    READY   AGE   CONTAINERS   IMAGES
myapp   4/4     23m   myapp        ikubernetes/myapp:v5

[root@master statefulset]# kubectl get pods myapp-2 -o yaml |grep image
  - image: ikubernetes/myapp:v5
    imagePullPolicy: IfNotPresent
    image: ikubernetes/myapp:v5
    imageID: docker-pullable://ikubernetes/myapp@sha256:85d1005d172aa8b97d7f1aa67519132cd450f59d01a607d4b4eaf5bcf402ce52
	
[root@master statefulset]# kubectl get pods myapp-0 -o yaml |grep image 
  - image: ikubernetes/myapp:v3
    imagePullPolicy: IfNotPresent
    image: ikubernetes/myapp:v3
    imageID: docker-pullable://ikubernetes/myapp@sha256:b8d74db2515d3c1391c78c5768272b9344428035ef6d72158fd9f6c4239b2c69

将剩余的Pod也更新版本，只需要将更新策略的partition值改为0即可，如下：

[root@master statefulset]# kubectl patch sts myapp -p '{"spec":{"updateStrategy":{"rollingUpdate":{"partition":0}}}}'
statefulset.apps/myapp patched

[root@master statefulset]# kubectl get pods myapp-0 -o yaml |grep image                                             - image: ikubernetes/myapp:v5
    imagePullPolicy: IfNotPresent
    image: ikubernetes/myapp:v5
    imageID: docker-pullable://ikubernetes/myapp@sha256:85d1005d172aa8b97d7f1aa67519132cd450f59d01a607d4b4eaf5bcf402ce52

生产中还是不建议把重要应用使用statefulset，如mysql redis 等。

参考资料

https://www.cnblogs.com/linuxk
马永亮. Kubernetes进阶实战 (云计算与虚拟化技术丛书)
Kubernetes-handbook-jimmysong-20181218

posted @ 2019-04-12 10:15 微落不落阅读(1633) 评论(0) 编辑收藏举报

会员力量，点亮园子希望

刷新页面返回顶部

微落不落

不想当厨子的裁缝不是好司机

Kubernetes之（十四）StatefulSet控制器

Kubernetes之（十四）StatefulSet控制器

简介

StatefulSet使用

滚动更新

扩展伸缩

更新策略和版本升级

参考资料

公告