基于jenkins+kubernetes的cicd流程实践三：清除镜像定时任务

6.定时清除历史镜像：

万里长征的最后一步收尾工作，换一种更“原生”的玩法，找一找新感觉，手搓走起.....

参考：https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md

（a）与docker情况一致，每个节点都安装containerd和crictl，使用本地数据卷挂载方式

（b）job一次并行运行和最小完成pod数与工作节点相同数量，pod亲和性调度只会考虑生命周期在running状态的pod

（c）使用pod反亲和性与自身相同的标签不在同一节点，通过控制工作节点数，保证工作节点有且只有一个job，job controller中的控制循环负责调谐期望状态和实际状态，并不会有死锁的存在

（d）设置容器重启策略OnFailure，避免pod漂移，backoffLimit针对容器同样生效

模板：/script/template/rm-images-job.yaml

apiVersion: batch/v1
kind: CronJob
metadata:
  name: {{name}}
  namespace: devops-tools
spec:
  schedule: "* * */7 * *"
  concurrencyPolicy: Replace
  jobTemplate:
    spec:
      completions: {{workerCount}}
      parallelism: {{workerCount}}
      activeDeadlineSeconds: 100
      backoffLimit: 2
      template:
        metadata:
          labels:
            app: {{name}}
        spec:
          securityContext:
            fsGroup: 0
            runAsUser: 0
          serviceAccount: jenkins-admin
          containers:
            - name: {{name}}
              image: myhub.com/devops-tools/kubectl:1.15.3
              command: [ "/bin/sh", "-c" ]
              args:
                - source /etc/podinfo/labels 2>/dev/null;echo "name:${app}";               
                  docker ps -a | grep ${app} | awk '{print $1}' | xargs docker rm -f $1 | xargs echo "docker rm ";
                  docker image prune -a --filter "until=$(date +'%Y-%m-%dT%H:%M:%S' --date='-5 days')" --filter "label=service=${app}" -f;
                  echo "docker rmi CMD:docker image prune -a --filter \"until=$(date +'%Y-%m-%dT%H:%M:%S' --date='-14 days')\" --filter \"label=service=${app}\" -f;finished";                
                  kubectl config set-credentials jenkins-admin --token=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token);
                  kubectl config set-cluster kubernetes --insecure-skip-tls-verify=true --server=https://kubernetes.default.svc.cluster.local;
                  kubectl config set-context mycontext --cluster=kubernetes --user=jenkins-admin;
                  kubectl config use-context mycontext;
                  usedimage=$(kubectl get deployment.apps/${app} -n {{branch}} -o go-template --template='{{(index .spec.template.spec.containers 0).image}}');
                  echo "get usedimage CMD:kubectl get deployment.apps/${app} -n master -o go-template --template='{{(index .spec.template.spec.containers 0).image}}'";echo "usedimage:${usedimage}";
                  usedimageid=$(crictl images | grep ${usedimage} | awk -v ORS="," '{print $1":"$2"@"$3}');echo "usedimageid:${usedimageid}";
                  IFS=",";
                  imagesinfo=$(crictl images | grep ${app} | awk -v ORS="," '{print $1":"$2"@"$3}');
                  echo "get imagesinfo CMD:crictl images | grep ${app} | awk -v ORS="," '{print \$1":"\$2"@"\$3}' ";echo "imagesinfo:${imagesinfo}";
                  arr=(${imagesinfo});
                  IFS="@";
                  for(( i=0;i<${#arr[@]};i++)) do
                  tags=(${arr[i]});echo "processing tag:${tags}";
                  if [ "${tags[0]}" != "${usedimages}" -a "${tags[1]}" != "${usedimagesid}" ];then
                  crictl rmi ${arr[i]};echo "crictl rmi:${arr[i]}";
                  fi;
                  done;
              volumeMounts:
                - mountPath: "/var/run/docker.sock"
                  name: "dockersocket"
                  readOnly: false
                - mountPath: "/etc/docker"
                  name: "dockerconfig"
                  readOnly: false
                - mountPath: "/usr/bin/docker"
                  name: "docker"
                  readOnly: false
                - mountPath: "/var/run/containerd/containerd.sock"
                  name: "containerdsocket"
                  readOnly: false
                - mountPath: "/etc/crictl.yaml"
                  name: "crictlconfig"
                  readOnly: false
                - mountPath: "/usr/local/bin/crictl"
                  name: "crictl"
                  readOnly: false
                - name: podinfo
                  mountPath: /etc/podinfo
                  readOnly: false
                - name: localtime
                  mountPath: /etc/localtime
                  readOnly: true
          restartPolicy: OnFailure
          affinity:
            podAntiAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                - labelSelector:
                    matchExpressions:
                      - key: app
                        operator: In
                        values:
                          - {{name}}
                  topologyKey: kubernetes.io/hostname
          volumes:
            - hostPath:
                path: "/usr/bin/docker"
              name: "docker"
            - hostPath:
                path: "/etc/docker"
              name: "dockerconfig"
            - hostPath:
                path: "/var/run/docker.sock"
              name: "dockersocket"
            - hostPath:
                path: "/usr/local/bin/crictl"
              name: "crictl"
            - hostPath:
                path: "/etc/crictl.yaml"
              name: "crictlconfig"
            - hostPath:
                path: "/var/run/containerd/containerd.sock"
              name: "containerdsocket"
            - name: podinfo
              projected:
                sources:
                  - downwardAPI:
                      items:
                        - path: "labels"
                          fieldRef:
                            fieldPath: metadata.labels
            - name: localtime
              hostPath:
                path: /etc/localtime

7.容器与主机时间同步配置：

参考：https://zhuanlan.zhihu.com/p/156757418

​ （a）jenkinns master pod 时区不从系统中直接获取，有自己接口，自行配置【用户定义时区】

​ （b）其它用到系统时区的容器，本地数据卷的方式挂载到容器中，jenkinns agent pod 时区，pod模板中配置，插件自动每个容器挂载