kubernetes高级调度方式
调度方式：
- 节点选择器： nodeSelector，nodeName
- 节点亲和调度： nodeAffinity
- pod亲和调度： podAffinity
♦ kubectl get nodes --show-labels 可以查看节点存在哪些标签
一、节点选择器
♦ nodeSelector选择的标签需要在node上存在，node才会被选中为pod创建的节点
下图中pod使用了nodeSelector选项过滤包含了gup=9527标签的node，然后pod就创建到了ks-node2上：
二、节点亲和调度
♦ pod.spec.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution 软亲和性，尽量找满足需求的
apiVersion: v1 kind: Pod metadata: name: pod-node-preferaffinity #namespace: develop labels: app: ngx-demo rel: stable spec: containers: - image: nginx imagePullPolicy: IfNotPresent name: nginx-node-preferaffinity affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - preference: matchExpressions: - key: gpu operator: In values: - "952" weight: 10
♦ pod.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution 硬亲和性，必须满足需求
root@k8s-master:/home/ubuntu/manifests/k8s/scheduler# cat nodeaffinity_require.yaml apiVersion: v1 kind: Pod metadata: name: pod-node-requiredaffinity #namespace: develop labels: app: ngx-demo rel: stable spec: containers: - image: nginx imagePullPolicy: IfNotPresent name: nginx-node-requiredaffinity affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: gpu operator: In values: - "952"
使用同样的matchExpressions来匹配label，required必须满足标签要求，否则会一直处于pending状态：
三、Pod亲和调度
♦ 使用podAffinity我们需要有一个标准来判断哪些节点属于亲和；使用topologyKey判定是否属于同一个位置
♦ pod亲和性与节点亲和性一样有prefered和required两种类型
♦ pod.spec.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution
root@k8s-master:/home/ubuntu/manifests/k8s/scheduler# cat pod_affinity_require.yaml apiVersion: v1 kind: Pod metadata: name: pod-first #namespace: develop labels: app: ngx-demo rel: stable spec: containers: - image: nginx imagePullPolicy: IfNotPresent name: nginx-1 --- apiVersion: v1 kind: Pod metadata: name: pod-podrequiredaffinity-second # 该pod会根据第一个部署的pod标签找到亲和的机器来部署pod #namespace: develop labels: app: ngx-demo rel: stable spec: containers: - image: tomcat imagePullPolicy: IfNotPresent name: tomcat-podrequiredaffinity affinity: podAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: # 这里匹配的是第一个pod的标签，所以key/values应该是想要pod部署的在一起的label - key: app operator: In values: - "ngx-demo" topologyKey: kubernetes.io/hostname #使用的hostname来判断是否在同一位置，也可以使用其它自定义的标签
♦ pod.spec.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution
root@k8s-master:/home/ubuntu/manifests/k8s/scheduler# cat pod_affinity_prefer.yaml apiVersion: v1 kind: Pod metadata: name: pod-first #namespace: develop labels: app: ngx-demo rel: stable spec: containers: - image: nginx imagePullPolicy: IfNotPresent name: nginx-1 --- apiVersion: v1 kind: Pod metadata: name: pod-podrequiredaffinity-second #namespace: develop labels: app: ngx-demo rel: stable spec: containers: - image: tomcat imagePullPolicy: IfNotPresent name: tomcat-podrequiredaffinity affinity: podAffinity: preferredDuringSchedulingIgnoredDuringExecution: - podAffinityTerm: labelSelector: matchExpressions: - key: app operator: In values: - "ngx-demo" topologyKey: kubernetes.io/hostname #可以使用自定义label weight: 60
♦ pod.spec.affinity.podAntiAffinity 反亲和性，创建的pod不会在一起，反亲和性也有prefered和required两种类型
apiVersion: v1 kind: Pod metadata: name: pod-first #namespace: develop labels: app: ngx-demo rel: stable spec: containers: - image: nginx imagePullPolicy: IfNotPresent name: nginx-1 --- apiVersion: v1 kind: Pod metadata: name: pod-podrequired_anti_affinity-second #namespace: develop labels: app: ngx-demo rel: stable spec: containers: - image: tomcat imagePullPolicy: IfNotPresent name: tomcat-podrequired_anti_affinity affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - "ngx-demo" topologyKey: kubernetes.io/hostname
四、污点调度
taint --- 污点就是定义在节点上的键值属性
taintToleration --- 容忍度是定义在pod上的键值属性，是一个列表
- 标签 可定义在所有资源
- 注解 可定义在所有资源
- 污点 只能定义在节点上
taint的effect定义对pod的排斥效果：
- NoSchedule：仅影响调度过程，对现存的pod不影响
- NoExecute： 既影响调度，也影响现存pod；不满足容忍的pod对象将被驱逐
- PreferNoSchedule：尽量不调度，实在没地方运行也可以
♦ 如果一个节点上定义了污点，一个pod能否调上来，我们先去检查能够匹配的容忍度和污点；剩余不能被容忍的污点就去检查污点的effect，
如果是PreferNoSchedule，还是能调度；
如果是NoSchedule，还没调度的就不能被调度，已经调度完的没影响；
如果是NoExecute，完全不能被调度，如果此前有运行的pod，后来改了污点，pod将被驱逐
♦ kubectl taint --help 打污点标签
kubectl taint node k8s-node2 node-type=prod:NoSchedule 给k8s-node2打上prod环境标签，effect为NoSchedule
kubectl taint node k8s-node2 node-type- 移除所有key为node-type的污点
kubectl taint node k8s-node2 node-type:NoSchedule- 移除所有key为node-type中effect为NoSchedule的污点
示例：我们首先使用反亲和性特点使得nginx和tomcat的pod不能在同一个node上被创建，然后再tomcat中设置容忍度满足k8s-master节点的污点（默认情况下，master节点是不能被创建普通pod的）
root@k8s-master:/home/ubuntu/manifests/k8s/scheduler# cat pod_toleration_master_taint.yaml apiVersion: v1 kind: Pod metadata: name: pod-first #namespace: develop labels: app: ngx-demo rel: stable spec: containers: - image: nginx imagePullPolicy: IfNotPresent name: nginx-1 --- apiVersion: v1 kind: Pod metadata: name: pod-toleration-master-taint-second #namespace: develop labels: app: ngx-demo rel: stable spec: containers: - image: tomcat imagePullPolicy: IfNotPresent name: tomcat-toleration-master-taint tolerations: # 设置容忍度 可以容忍master节点NoSchedule - effect: NoSchedule # 如果effect为空，则表示容忍所有的 key: node-role.kubernetes.io/master
operator: Exists affinity: podAntiAffinity: #反亲和性使得tomat的pod不能和nginx在同一个pod上 requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - "ngx-demo" topologyKey: kubernetes.io/hostname
最终结果使得tomcat的pod可以被调度到master节点：
