kubernetes高级调度方式

调度方式:

  • 节点选择器: nodeSelector,nodeName
  • 节点亲和调度: nodeAffinity
  • pod亲和调度: podAffinity

♦ kubectl get nodes --show-labels  可以查看节点存在哪些标签

 

一、节点选择器

♦ nodeSelector选择的标签需要在node上存在,node才会被选中为pod创建的节点

下图中pod使用了nodeSelector选项过滤包含了gup=9527标签的node,然后pod就创建到了ks-node2上:

 

二、节点亲和调度

♦ pod.spec.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution   软亲和性,尽量找满足需求的

apiVersion: v1
kind: Pod
metadata:
  name: pod-node-preferaffinity
  #namespace: develop
  labels:
    app: ngx-demo
    rel: stable
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx-node-preferaffinity
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - preference:
          matchExpressions:
          - key: gpu
            operator: In
            values:
            - "952"
        weight: 10

 

♦ pod.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution  硬亲和性,必须满足需求

root@k8s-master:/home/ubuntu/manifests/k8s/scheduler# cat nodeaffinity_require.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-node-requiredaffinity
  #namespace: develop
  labels:
    app: ngx-demo
    rel: stable
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx-node-requiredaffinity
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: gpu
            operator: In
            values:
            - "952"

 

 使用同样的matchExpressions来匹配label,required必须满足标签要求,否则会一直处于pending状态:

 

三、Pod亲和调度

♦ 使用podAffinity我们需要有一个标准来判断哪些节点属于亲和;使用topologyKey判定是否属于同一个位置

♦ pod亲和性与节点亲和性一样有prefered和required两种类型

♦ pod.spec.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution

root@k8s-master:/home/ubuntu/manifests/k8s/scheduler# cat pod_affinity_require.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-first
  #namespace: develop
  labels:
    app: ngx-demo
    rel: stable
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx-1
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-podrequiredaffinity-second    # 该pod会根据第一个部署的pod标签找到亲和的机器来部署pod
  #namespace: develop
  labels:
    app: ngx-demo
    rel: stable
spec:
  containers:
  - image: tomcat
    imagePullPolicy: IfNotPresent
    name: tomcat-podrequiredaffinity
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:  # 这里匹配的是第一个pod的标签,所以key/values应该是想要pod部署的在一起的label
          - key: app
            operator: In
            values:
            - "ngx-demo"
        topologyKey: kubernetes.io/hostname      #使用的hostname来判断是否在同一位置,也可以使用其它自定义的标签

 

♦ pod.spec.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution

root@k8s-master:/home/ubuntu/manifests/k8s/scheduler# cat pod_affinity_prefer.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-first
  #namespace: develop
  labels:
    app: ngx-demo
    rel: stable
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx-1
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-podrequiredaffinity-second
  #namespace: develop
  labels:
    app: ngx-demo
    rel: stable
spec:
  containers:
  - image: tomcat
    imagePullPolicy: IfNotPresent
    name: tomcat-podrequiredaffinity
  affinity:
    podAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: app
              operator: In
              values:
              - "ngx-demo"
          topologyKey: kubernetes.io/hostname      #可以使用自定义label
        weight: 60

♦ pod.spec.affinity.podAntiAffinity  反亲和性,创建的pod不会在一起,反亲和性也有prefered和required两种类型

apiVersion: v1
kind: Pod
metadata:
  name: pod-first
  #namespace: develop
  labels:
    app: ngx-demo
    rel: stable
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx-1
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-podrequired_anti_affinity-second
  #namespace: develop
  labels:
    app: ngx-demo
    rel: stable
spec:
  containers:
  - image: tomcat
    imagePullPolicy: IfNotPresent
    name: tomcat-podrequired_anti_affinity
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - "ngx-demo"
        topologyKey: kubernetes.io/hostname 

 

四、污点调度

taint --- 污点就是定义在节点上的键值属性

taintToleration --- 容忍度是定义在pod上的键值属性,是一个列表

  • 标签   可定义在所有资源
  • 注解   可定义在所有资源
  • 污点   只能定义在节点上

 taint的effect定义对pod的排斥效果:

  • NoSchedule:仅影响调度过程,对现存的pod不影响
  • NoExecute: 既影响调度,也影响现存pod;不满足容忍的pod对象将被驱逐
  • PreferNoSchedule:尽量不调度,实在没地方运行也可以

♦  如果一个节点上定义了污点,一个pod能否调上来,我们先去检查能够匹配的容忍度和污点;剩余不能被容忍的污点就去检查污点的effect,

  如果是PreferNoSchedule,还是能调度;

  如果是NoSchedule,还没调度的就不能被调度,已经调度完的没影响;

  如果是NoExecute,完全不能被调度,如果此前有运行的pod,后来改了污点,pod将被驱逐

♦  kubectl taint --help  打污点标签

  kubectl taint node k8s-node2 node-type=prod:NoSchedule  给k8s-node2打上prod环境标签,effect为NoSchedule

  kubectl taint node k8s-node2 node-type-   移除所有key为node-type的污点

  kubectl taint node k8s-node2 node-type:NoSchedule-  移除所有key为node-type中effect为NoSchedule的污点

 

 示例:我们首先使用反亲和性特点使得nginx和tomcat的pod不能在同一个node上被创建,然后再tomcat中设置容忍度满足k8s-master节点的污点(默认情况下,master节点是不能被创建普通pod的)

root@k8s-master:/home/ubuntu/manifests/k8s/scheduler# cat pod_toleration_master_taint.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-first
  #namespace: develop
  labels:
    app: ngx-demo
    rel: stable
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx-1
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-toleration-master-taint-second
  #namespace: develop
  labels:
    app: ngx-demo
    rel: stable
spec:
  containers:
  - image: tomcat
    imagePullPolicy: IfNotPresent
    name: tomcat-toleration-master-taint
  tolerations:  # 设置容忍度 可以容忍master节点NoSchedule
  - effect: NoSchedule  # 如果effect为空,则表示容忍所有的
    key: node-role.kubernetes.io/master
operator: Exists affinity: podAntiAffinity:  #反亲和性使得tomat的pod不能和nginx在同一个pod上 requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector: matchExpressions: - key: app operator: In values: - "ngx-demo" topologyKey: kubernetes.io/hostname

 

最终结果使得tomcat的pod可以被调度到master节点:

 

posted @ 2019-08-08 11:29  阿拉米苏  阅读(388)  评论(0编辑  收藏  举报