一、概述
为了屏蔽底层的技术实现细节，让用户更加方便的使用，Kubernetes 便引入了 PV 和 PVC 两个重要的资源对象来实现对存储的管理。
PV 的全称是：PersistentVolume（持久化卷），是对底层的共享存储的一种抽象，PV 由管理员进行创建和配置，它和具体的底层的共享存储技术的实现方式有关，比如 Ceph、GlusterFS、NFS 等，都是通过插件机制完成与共享存储的对接。
PVC 的全称是：PersistentVolumeClaim（持久化卷声明），PVC 是用户存储的一种声明，PVC 和 Pod 比较类似，Pod 消耗的是节点，PVC 消耗的是 PV 资源，Pod 可以请求 CPU 和内存，而 PVC 可以请求特定的存储空间和访问模式。对于真正使用存储的用户不需要关心底层的存储实现细节，只需要直接使用 PVC 即可。
但是通过 PVC 请求到一定的存储空间也很有可能不足以满足应用对于存储设备的各种需求，而且不同的应用程序对于存储性能的要求可能也不尽相同，比如读写速度、并发性能等，为了解决这一问题，Kubernetes 又为我们引入了一个新的资源对象：StorageClass，通过 StorageClass 的定义，管理员可以将存储资源定义为某种类型的资源，比如快速存储、慢速存储等，用户根据 StorageClass 的描述就可以非常直观的知道各种存储资源的具体特性了，这样就可以根据应用的特性去申请合适的存储资源了。
不同情况下，PV和PVC的状态变化如下：
操作 | PV状态 | PVC状态 |
1.创建 PV | Available | - |
2.创建PVC | Available | Pending |
3.过几秒钟 | Bound | Bound |
4.删除PV | -/Terminating | Lost/Bound |
5.重新创建PV | Bound | Bound |
6.删除PVC | Released | - |
7.后端存储不可用 | Failed | - |
8.删除PV的claimRef | Available | - |
二、具体操作
1.创建PV。
正常情况下PV被创建成功后是Available状态：
1.1、创建PV的yaml语句
[root@k8s-master pv]# cat pv.yml
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv
spec:
storageClassName: manual
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
nfs:
path: /data/nfs
server: 192.168.41.210
1.2、创建PV对象并查看PV的状态，可以看到状态是available，表示可以用于PVC绑定
[root@k8s-master pv]# kubectl apply -f pv.yml
persistentvolume/nfs-pv created
[root@k8s-master pv]# kubectl get pv,pvc -owide
]NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
persistentvolume/nfs-pv 1Gi RWO Retain Available manual 8s Filesystem
2.创建PVC。
刚刚添加的PVC状态是Pending,如有有合适的PV这个Pending状态会立刻变为Bound状态，同时相对应得PVC状态也变为Bound，PVC和PV进行了绑定。
2.1、创建pvc的yaml语句
[root@k8s-master pv]# cat pvc.yml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nfs-pvc
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
2.2、创建PVC资源对象，查看刚完成的状态是Pending
[root@k8s-master pv]# kubectl apply -f pvc.yml
persistentvolumeclaim/nfs-pvc created
[root@k8s-master pv]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
nfs-pvc Pending manual 40s
2.3、当PVC找到合适的PV时进行绑定，此时就会立刻变成Bound状态，PV也从available变成Bound状态。
[root@k8s-master pv]# kubectl get pv,pvc -owide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
persistentvolume/nfs-pv 1Gi RWO Retain Bound default/nfs-pvc manual 12m Filesystem
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE
persistentvolumeclaim/nfs-pvc Bound nfs-pv 1Gi RWO manual 5s Filesystem
3.删除PV
由于PV和PVC处于bound状态，如果对PV进行删除，会出现什么样的情况呢
[root@k8s-master pv]# kubectl delete pv/nfs-pv
persistentvolume "nfs-pv" deleted
^C
[root@k8s-master pv]# kubectl get pv,pvc -owide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
persistentvolume/nfs-pv 1Gi RWO Retain Terminating default/nfs-pvc manual 16m Filesystem
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE
persistentvolumeclaim/nfs-pvc Bound nfs-pv 1Gi RWO manual 4m51s Filesystem
由于删除了PV，被夯住了不能被真正的删除，这个时候PV的状态会变成Terminating，而对应的PVC还是bound状态，也就是说这个时候由于PV和PVC已经绑定到一起了，就不能删除PV，而此刻的PV状态是Terminating，对于PVC还是没有任何影响。而如果要强制删除PV则需要编辑PV并删除PV中的ffinalizers属性来强制删除PV。
编辑保存完成后PV就会被真正删除了，而PVC的状态变成Lost了
[root@k8s-master pv]# kubectl edit pv/nfs-pv
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: PersistentVolume
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"PersistentVolume","metadata":{"annotations":{},"name":"nfs-pv"},"spec":{"accessModes":["ReadWriteOnce"],"capacity":{"storage":"1Gi"},"nfs":{"path":"/data/nfs","server":"192.168.41.210"},"persistentVolumeReclaimPolicy":"Retain","storageClassName":"manual"}}
pv.kubernetes.io/bound-by-controller: "yes"
creationTimestamp: "2021-12-09T06:44:03Z"
deletionGracePeriodSeconds: 0
deletionTimestamp: "2021-12-09T07:00:29Z"
# finalizers:
# - kubernetes.io/pv-protection
name: nfs-pv
resourceVersion: "5932828"
uid: 589721d9-0b45-4ce3-b6de-89fe3d99ba89
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 1Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: nfs-pvc
namespace: default
resourceVersion: "5932445"
uid: af7b20e2-53d7-44ff-bb09-29ed0039475c
nfs:
path: /data/nfs
server: 192.168.41.210
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
volumeMode: Filesystem
status:
phase: Bound
persistentvolume/nfs-pv edited
[root@k8s-master pv]# kubectl get pv,pvc -owide
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE
persistentvolumeclaim/nfs-pvc Lost nfs-pv 0 manual 12m Filesystem
4.重新创建PV
4.1、当看到PVC的状态是lost时，可以查看该PVC状态信息，发现里边还绑定有PV的信息
[root@k8s-master pv]# kubectl get pvc nfs-pvc -o yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"name":"nfs-pvc","namespace":"default"},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"manual"}}
pv.kubernetes.io/bind-completed: "yes"
pv.kubernetes.io/bound-by-controller: "yes"
creationTimestamp: "2021-12-09T06:56:00Z"
finalizers:
- kubernetes.io/pvc-protection
name: nfs-pvc
namespace: default
resourceVersion: "5933450"
uid: af7b20e2-53d7-44ff-bb09-29ed0039475c
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: manual
volumeMode: Filesystem
volumeName: nfs-pv
status:
phase: Lost
4.2、要解决这个问题，只需要重新把之前的PV创建出来即可。当PV创建成功后，PVC和PV的状态都变成Bound了
[root@k8s-master pv]# kubectl apply -f pv.yml
persistentvolume/nfs-pv created
[root@k8s-master pv]# kubectl get pv,pvc -owide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
persistentvolume/nfs-pv 1Gi RWO Retain Available manual 3s Filesystem
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE
persistentvolumeclaim/nfs-pvc Lost nfs-pv 0 manual 22m Filesystem
[root@k8s-master pv]# kubectl get pv,pvc -owide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
persistentvolume/nfs-pv 1Gi RWO Retain Bound default/nfs-pvc manual 9s Filesystem
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE
persistentvolumeclaim/nfs-pvc Bound nfs-pv 1Gi RWO manual 22m Filesystem
5.删除PVC
5.1、删除PVC，查看PV的相关状态
[root@k8s-master pv]# kubectl delete pvc/nfs-pvc
persistentvolumeclaim "nfs-pvc" deleted
[root@k8s-master pv]# kubectl get pv,pvc -owide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
persistentvolume/nfs-pv 1Gi RWO Retain Released default/nfs-pvc manual 4m47s Filesystem
PVC被删除后，PV变成了Released的状态了，通过查看后面的claimRef属性，其中依然保留着PVC的绑定信息。
5.2、导出PV的相关信息
[root@k8s-master pv]# kubectl get pv/nfs-pv -o yaml
apiVersion: v1
kind: PersistentVolume
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"PersistentVolume","metadata":{"annotations":{},"name":"nfs-pv"},"spec":{"accessModes":["ReadWriteOnce"],"capacity":{"storage":"1Gi"},"nfs":{"path":"/data/nfs","server":"192.168.41.210"},"persistentVolumeReclaimPolicy":"Retain","storageClassName":"manual"}}
pv.kubernetes.io/bound-by-controller: "yes"
creationTimestamp: "2021-12-09T07:18:00Z"
finalizers:
- kubernetes.io/pv-protection
name: nfs-pv
resourceVersion: "5934724"
uid: d207a3fc-5688-4c32-b9d5-102a01f70990
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 1Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: nfs-pvc
namespace: default
resourceVersion: "5933450"
uid: af7b20e2-53d7-44ff-bb09-29ed0039475c
nfs:
path: /data/nfs
server: 192.168.41.210
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
volumeMode: Filesystem
status:
phase: Released
由于PVC被删除了，PV的状态变成了Released,那么我们能否重建PVC是不是就可以和之前的PV进行绑定了呢，答案是不可以的，由于PVC只能和Available的状态的PV进行绑定。
所以需要我们手工进行干预，真实生产环境下管理员会把数据备份或迁移出来，然后修改 PV，删除 claimRef 对 PVC 的引用，这个时候 Kubernetes 的 PV 控制器 watch 到 PV 变化后，就会将 PV 修改为 Available 状态，Available 状态的 PV 当然就可以被其他 PVC 绑定了。
5.3、直接编辑 PV 删除 cliamRef 属性中的内容即可：
[root@k8s-master pv]# kubectl edit pv/nfs-pv
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: PersistentVolume
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"PersistentVolume","metadata":{"annotations":{},"name":"nfs-pv"},"spec":{"accessModes":["ReadWriteOnce"],"capacity":{"storage":"1Gi"},"nfs":{"path":"/data/nfs","server":"192.168.41.210"},"persistentVolumeReclaimPolicy":"Retain","storageClassName":"manual"}}
pv.kubernetes.io/bound-by-controller: "yes"
creationTimestamp: "2021-12-09T07:18:00Z"
finalizers:
- kubernetes.io/pv-protection
name: nfs-pv
resourceVersion: "5934724"
uid: d207a3fc-5688-4c32-b9d5-102a01f70990
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 1Gi
nfs:
path: /data/nfs
server: 192.168.41.210
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
volumeMode: Filesystem
status:
phase: Released
persistentvolume/nfs-pv edited
[root@k8s-master pv]# kubectl get pv,pvc -owide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
persistentvolume/nfs-pv 1Gi RWO Retain Available manual 13m Filesystem
删除完成后，这个时候 PV 就会变成正常的 Available 状态了，重新去重建之前的 PVC 当然就可以正常绑定了：
[root@k8s-master pv]# kubectl apply -f pvc.yml
persistentvolumeclaim/nfs-pvc created
[root@k8s-master pv]# kubectl get pv,pvc -owide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
persistentvolume/nfs-pv 1Gi RWO Retain Released default/nfs-pvc manual 15m Filesystem
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE
persistentvolumeclaim/nfs-pvc Pending manual 2s Filesystem
