k8s Back-off restarting failed container的Warning事件

1.现象： pod启动后一直重启

kubectl get event -n default
5m1s        Normal    Pulled    pod/mongodb-0   Container image "mongo:latest" already present on machine
15m         Normal    Started   pod/mongodb-0   Started container mongodb
11s         Warning   BackOff   pod/mongodb-0   Back-off restarting failed container

原理：Back-off restarting failed container的Warning事件，一般是由于通过指定的镜像启动容器后，容器内部没有常驻进程，导致容器启动成功后即退出，从而进行了持续的重启。

解决方法：找到对应的statefulset command: ["/bin/bash", "-ce", "tail -f /dev/null"]

apiVersion: apps/v1
kind: StatefulSet
#kind: pods
metadata:
  name: mongodb
spec:
  replicas: 1
  serviceName: mongodb
  selector:
    matchLabels:
      name: mongodb
  template:
    metadata:
      labels:
        name: mongodb
    spec:
      containers:
      - name: mongodb
        image: mongo:latest
        command: ["/bin/bash","-ce","tail -f /dev/null"]
        command: ["/bin/bash","-ce","exec mongod -f /mgdb/middleware-data/mongodb/conf/mongodb.conf"]
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 27017
          name: mongodb
          protocol: TCP
        volumeMounts:
        - name: mongodb-config
          mountPath: /mgdb/middleware-data/mongodb/conf/
        - name: data
          mountPath: /mgdb/middleware-data/mongodb/
      volumes:
      - name: mongodb-config
        configMap:
          name: mongodb-conf
      - name: data
        hostPath:
          path: /mgdb/middleware-data/mongodb/

#replicas: 2改为1   业务不需要两个了
##当你更改 StatefulSet 的 replicas 字段，并且控制器执行缩容操作时，通常会终止较高编号的 Pod。这是因为较高编号的 Pod 通常是最新创建的，而较低编号的 Pod 是最早创建的
执行：kubectl apply -f mongodb.yaml

kubectl get pods -w   #实时监控pod调度状态
mongodb-0   0/1     Terminating         3          82s
mongodb-0   0/1     Terminating         3          82s
mongodb-0   0/1     Terminating         3          85s
mongodb-0   0/1     Terminating         3          85s
mongodb-0   0/1     Pending             0          0s
mongodb-0   0/1     Pending             0          0s
mongodb-0   0/1     ContainerCreating   0          0s
mongodb-0   0/1     ContainerCreating   0          1s
mongodb-0   0/1     Error               0          2s
mongodb-0   0/1     Error               1          3s
mongodb-0   0/1     CrashLoopBackOff    1          4s
mongodb-0   0/1     Error               2          18s
mongodb-0   0/1     CrashLoopBackOff    2          30s
执行apply之前↑
执行apply之后↓
mongodb-1   1/1     Terminating         12271      53d
mongodb-1   1/1     Terminating         12271      53d
mongodb-1   0/1     Terminating         12271      53d
mongodb-0   0/1     Error               3          46s
mongodb-1   0/1     Terminating         12271      53d
mongodb-1   0/1     Terminating         12271      53d
mongodb-0   0/1     CrashLoopBackOff    3          57s
mongodb-0   1/1     Running             4          96s

#mongo重要数据备份: kubectl exec mongodb-1  -- mongodump --archive > backup.archive