k8s pod,pvc,pv无法删除问题

注意步骤:

一般删除步骤为：先删pod再删pvc最后删pv

但是遇到pv始终处于“Terminating”状态，而且delete不掉

pod一直删不掉

[root@hadoop01 nacos-k8s]# kubectl get pod 
NAME                                  READY   STATUS        RESTARTS   AGE
busybox-deployment-8588b74c4b-4rc24   1/1     Terminating   0          45h
metrics-app-v1-5cdbf6d7f4-8pqxj       1/1     Running       1          24d
metrics-app-v1-5cdbf6d7f4-fcngn       1/1     Running       1          24d

//加上--force
[root@hadoop01 nacos-k8s]# kubectl delete pod busybox-deployment-8588b74c4b-4rc24 --force
warning: --force is ignored because --grace-period is not 0.
pod "busybox-deployment-8588b74c4b-4rc24" deleted
//依然无法删除
[root@hadoop01 nacos-k8s]# kubectl get pod 
NAME                                  READY   STATUS        RESTARTS   AGE
busybox-deployment-8588b74c4b-4rc24   1/1     Terminating   0          45h
metrics-app-v1-5cdbf6d7f4-8pqxj       1/1     Running       1          24d
metrics-app-v1-5cdbf6d7f4-fcngn       1/1     Running       1          24d
//加上--grace-period=0成功删除
[root@hadoop01 nacos-k8s]# kubectl delete pod busybox-deployment-8588b74c4b-4rc24 --force --grace-period=0
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "busybox-deployment-8588b74c4b-4rc24" force deleted

查看pvc被哪个pod使用

[root@hadoop03 storageclass]# kubectl describe pvc nfs-pvc | grep Mounted
Mounted By:    busybox-deployment-8588b74c4b-vw8jd

 

pv pvc删不掉

解决方法：

直接删除k8s中的记录：

kubectl patch pv xxx -p '{"metadata":{"finalizers":null}}'

 

参考信息：

This happens when persistent volume is protected. You should be able to cross verify this:

Command:

kubectl describe pvc PVC_NAME | grep Finalizers

Output:

Finalizers: [kubernetes.io/pvc-protection]

You can fix this by setting finalizers to null using kubectl patch:

kubectl patch pvc PVC_NAME -p '{"metadata":{"finalizers": []}}' --type=merge

实例操作：

[root@test-dev-master01 ~]# kubectl patch pvc test-es01  -p '{"metadata":{"finalizers":null}}' -n xitu-test
persistentvolumeclaim/test-es01 patched
 
 
[root@test-dev-master01 ~]# kubectl patch pvc datadir-zookeeper-zookeeper-0  -p '{"metadata":{"finalizers":null}}' -n xitu-test
persistentvolumeclaim/datadir-zookeeper-zookeeper-0 patched
 
 
[root@test-dev-master01 ~]# kubectl patch pv d-bp1flc3jh6rcc0le0x7y  -p '{"metadata":{"finalizers":null}}' -n xitu-test 
persistentvolume/d-bp1flc3jh6rcc0le0x7y patched

 

注意:

pv 状态要处于Available才可以重新被同名的pvc使用或者被删除

//处于Released状态一直删除不掉
[root@hadoop03 storageclass]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS     CLAIM             STORAGECLASS       REASON   AGE
pvc-b57b8918-b35f-4ab9-ad1f-56443f1dcabb   10Mi       RWX            Delete           Released   default/nfs-pvc   nfs-storageclass            45h

[root@hadoop03 storageclass]# kubectl edit pv pvc-b57b8918-b35f-4ab9-ad1f-56443f1dcabb
//删除这一段
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: nfs-pvc //关联此pvc因此处于Released
    namespace: default
    resourceVersion: "8080884"
    uid: b57b8918-b35f-4ab9-ad1f-56443f1dcabb
[root@hadoop03 storageclass]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM             STORAGECLASS       REASON   AGE
pvc-b57b8918-b35f-4ab9-ad1f-56443f1dcabb   10Mi       RWX            Delete           Available                     nfs-storageclass            46h
//此时就可以删除了
[root@hadoop03 storageclass]# kubectl delete pv pvc-b57b8918-b35f-4ab9-ad1f-56443f1dcabb
persistentvolume "pvc-b57b8918-b35f-4ab9-ad1f-56443f1dcabb" deleted
[root@hadoop03 storageclass]# kubectl get pv
NAME   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM   STORAGECLASS       REASON   AGE

 

https://kubernetes.io/docs/concepts/storage/persistent-volumes/