Kerbernetes的volume应用进阶
Kerbernetes的volume应用进阶
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.PV及PVC概述
1>.什么是persistentVolume(持久卷，简称PV)
[root@test110.yinzhengjie.org.cn ~]# cat /etc/exports /yinzhengjie/data/volume1 172.200.0.0/21(rw,no_root_squash) /yinzhengjie/data/volume2 172.200.0.0/21(rw,no_root_squash) /yinzhengjie/data/volume3 172.200.0.0/21(rw,no_root_squash) [root@test110.yinzhengjie.org.cn ~]#
所谓的PV其实就是存储卷的意思,它可并不是和存储系统一一对应的关系哟，它对应的是存储系统的一个存储空间对应一个pv。
我们以NFS的配置文件为例(参考上面的"/etc/exports"文件),"/yinzhengjie/data/volume1"其实对应的就是一个pv,"/yinzhengjie/data/volume2"和"/yinzhengjie/data/volume3"均各自对应一个pv。
2>.什么是persistentVolumeClaim(简称PVC)
我们知道Kubernetes集群有两个级别，一种是名称空间，一种是集群资源。Pod属于名称空间的资源，而pv则是集群资源，如果名称空间的Pod想要使用pv,则需要将pv注册(说白了就是请求占用pv)到和该Pod相同的名称空间中，我们称之为persistentVolumeClaim(简称PVC)。 所谓的PVC我们可以理解为PV的一个客户端,pvc和pv之间没有直接联系,一个PVC可以绑定一个pv，此时pvc才会该pv建立关系，该pv就处于binding状态。与此同时,一个名称空间的pvc占用了某个pv，则该pv不能被其它名称的空间占用。 如果一个pv没有被任何pvc所绑定，那么该pv就处于available。
温馨提示:
我们在Pod上使用persistentVolumeClaim类型的存储卷来使用pvc,我们在Pod上定义了pvc的存储卷容器是不能直接使用它的，Pod中的容器需要使用"volumeMounts"字段去挂载存储卷。
[root@master200.yinzhengjie.org.cn ~]# kubectl explain pods.spec.volumes.persistentVolumeClaim KIND: Pod VERSION: v1 RESOURCE: persistentVolumeClaim <Object> DESCRIPTION: PersistentVolumeClaimVolumeSource represents a reference to a PersistentVolumeClaim in the same namespace. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims PersistentVolumeClaimVolumeSource references the user's PVC in the same namespace. This volume finds the bound PV and mounts that volume for the pod. A PersistentVolumeClaimVolumeSource is, essentially, a wrapper around another type of volume that is owned by someone else (the system). FIELDS: claimName <string> -required- ClaimName is the name of a PersistentVolumeClaim in the same namespace as the pod using this volume. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims readOnly <boolean> Will force the ReadOnly setting in VolumeMounts. Default false. [root@master200.yinzhengjie.org.cn ~]#
3>.pvc的访问模式(accessModes)取决于pv
一般存储系统的访问模型大致分为以下三种: 单路读写(ReadWriteOnce,简称为RWO) 多路只读(ReadOnlyMany,简称为ROM) 多路读写(ReadWriteMany,简称为RWM) 同一个pvc是否可以同时被多个Pod同时使用呢？ 这取决于pv是否支持多路读写(比如NFS就支持多路客户端同时读写访问，而Iscsi这种块级别的存储则不支持多路读写)。如果pv支持多路读写，那么与之绑定的pvc也支持多路读写,也就是说支持多个Pod同时绑定到同一个pvc上，反之则反。 博主推荐阅读: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#lifecycle-of-a-volume-and-claim
4>.pv的回收(reClaim)策略
pv除了上面我们所说的binding和available两种状态外还有两种就是回收(reClaim)状态。 pv的回收(reClaim)策略有以下三种: Recycle(deprecated,已弃用): 删除pvc时,只把pv上的数据删除,也就是说并没有把pv资源删除，该策略已被Google废弃,在后续版本可能会被移除，因此不推荐使用。 Delete: 删除PVC时,也会同时删除pv资源。 Retain: 删除pvc时,pv和pv上的数据军均不删除，对于重要的数据生产环境推荐使用这种策略。 博主推荐阅读: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes
5>.PVC的生命周期
pv的供给(Provisioning)方式：
static:
说白了就得咱们人工手动创建pv,比如NFS等
dynamic:
说白了就是无序人工介入，存储系统自动就支持pv的创建啦,比如Cephfs等，它是需要借助于存储类(StorageClass,简称SC),当创建pvc时,pvc会向SC请求，SC已经内置了如何连接pv的方式。
pvc的生命周期分为四种状态: Provisioning(即PV是如何创建的,NFS默认只支持静态创建pv,而cephfs是支持动态创建pv) Binding(绑定了PV) Using(Pod在使用pvc) Reclaiming(回收状态)。 博主推荐阅读: https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/
二.持久卷persistentVolumeClaim(简称PVC)存储实战案例
1>.构建NFS网络文件系统
[root@test110.yinzhengjie.org.cn ~]# yum -y install nfs-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.bit.edu.cn * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 extras | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 updates/7/x86_64/primary_db | 6.7 MB 00:00:01 Resolving Dependencies --> Running transaction check ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Running transaction check ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed ---> Package quota.x86_64 1:4.01-19.el7 will be installed --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64 --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64 ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed --> Running transaction check ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64 --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64 ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: nfs-utils x86_64 1:1.3.0-0.65.el7 base 412 k Installing for dependencies: gssproxy x86_64 0.7.0-26.el7 base 110 k keyutils x86_64 1.5.8-3.el7 base 54 k libbasicobjects x86_64 0.1.1-32.el7 base 26 k libcollection x86_64 0.7.0-32.el7 base 42 k libini_config x86_64 1.3.1-32.el7 base 64 k libnfsidmap x86_64 0.25-19.el7 base 50 k libpath_utils x86_64 0.2.1-32.el7 base 28 k libref_array x86_64 0.1.5-32.el7 base 27 k libtirpc x86_64 0.2.4-0.16.el7 base 89 k libverto-libevent x86_64 0.2.5-4.el7 base 8.9 k quota x86_64 1:4.01-19.el7 base 179 k quota-nls noarch 1:4.01-19.el7 base 90 k rpcbind x86_64 0.2.0-48.el7 base 60 k tcp_wrappers x86_64 7.6-77.el7 base 78 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+14 Dependent packages) Total download size: 1.3 M Installed size: 3.6 M Downloading packages: (1/15): keyutils-1.5.8-3.el7.x86_64.rpm | 54 kB 00:00:00 (2/15): libcollection-0.7.0-32.el7.x86_64.rpm | 42 kB 00:00:00 (3/15): libini_config-1.3.1-32.el7.x86_64.rpm | 64 kB 00:00:00 (4/15): gssproxy-0.7.0-26.el7.x86_64.rpm | 110 kB 00:00:00 (5/15): libnfsidmap-0.25-19.el7.x86_64.rpm | 50 kB 00:00:00 (6/15): libref_array-0.1.5-32.el7.x86_64.rpm | 27 kB 00:00:00 (7/15): libpath_utils-0.2.1-32.el7.x86_64.rpm | 28 kB 00:00:00 (8/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm | 8.9 kB 00:00:00 (9/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00 (10/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm | 26 kB 00:00:00 (11/15): quota-4.01-19.el7.x86_64.rpm | 179 kB 00:00:00 (12/15): quota-nls-4.01-19.el7.noarch.rpm | 90 kB 00:00:00 (13/15): tcp_wrappers-7.6-77.el7.x86_64.rpm | 78 kB 00:00:00 (14/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm | 412 kB 00:00:00 (15/15): rpcbind-0.2.0-48.el7.x86_64.rpm | 60 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 2.9 MB/s | 1.3 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libbasicobjects-0.1.1-32.el7.x86_64 1/15 Installing : libref_array-0.1.5-32.el7.x86_64 2/15 Installing : libcollection-0.7.0-32.el7.x86_64 3/15 Installing : libtirpc-0.2.4-0.16.el7.x86_64 4/15 Installing : rpcbind-0.2.0-48.el7.x86_64 5/15 Installing : 1:quota-nls-4.01-19.el7.noarch 6/15 Installing : tcp_wrappers-7.6-77.el7.x86_64 7/15 Installing : 1:quota-4.01-19.el7.x86_64 8/15 Installing : keyutils-1.5.8-3.el7.x86_64 9/15 Installing : libnfsidmap-0.25-19.el7.x86_64 10/15 Installing : libpath_utils-0.2.1-32.el7.x86_64 11/15 Installing : libini_config-1.3.1-32.el7.x86_64 12/15 Installing : libverto-libevent-0.2.5-4.el7.x86_64 13/15 Installing : gssproxy-0.7.0-26.el7.x86_64 14/15 Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64 15/15 Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/15 Verifying : libverto-libevent-0.2.5-4.el7.x86_64 2/15 Verifying : 1:quota-4.01-19.el7.x86_64 3/15 Verifying : gssproxy-0.7.0-26.el7.x86_64 4/15 Verifying : libpath_utils-0.2.1-32.el7.x86_64 5/15 Verifying : libnfsidmap-0.25-19.el7.x86_64 6/15 Verifying : keyutils-1.5.8-3.el7.x86_64 7/15 Verifying : 1:nfs-utils-1.3.0-0.65.el7.x86_64 8/15 Verifying : tcp_wrappers-7.6-77.el7.x86_64 9/15 Verifying : libcollection-0.7.0-32.el7.x86_64 10/15 Verifying : libref_array-0.1.5-32.el7.x86_64 11/15 Verifying : libbasicobjects-0.1.1-32.el7.x86_64 12/15 Verifying : rpcbind-0.2.0-48.el7.x86_64 13/15 Verifying : libini_config-1.3.1-32.el7.x86_64 14/15 Verifying : 1:quota-nls-4.01-19.el7.noarch 15/15 Installed: nfs-utils.x86_64 1:1.3.0-0.65.el7 Dependency Installed: gssproxy.x86_64 0:0.7.0-26.el7 keyutils.x86_64 0:1.5.8-3.el7 libbasicobjects.x86_64 0:0.1.1-32.el7 libcollection.x86_64 0:0.7.0-32.el7 libini_config.x86_64 0:1.3.1-32.el7 libnfsidmap.x86_64 0:0.25-19.el7 libpath_utils.x86_64 0:0.2.1-32.el7 libref_array.x86_64 0:0.1.5-32.el7 libtirpc.x86_64 0:0.2.4-0.16.el7 libverto-libevent.x86_64 0:0.2.5-4.el7 quota.x86_64 1:4.01-19.el7 quota-nls.noarch 1:4.01-19.el7 rpcbind.x86_64 0:0.2.0-48.el7 tcp_wrappers.x86_64 0:7.6-77.el7 Complete! [root@test110.yinzhengjie.org.cn ~]#
[root@test110.yinzhengjie.org.cn ~]# mkdir -pv /yinzhengjie/data/volume{1,2,3,4,5} mkdir: created directory ‘/yinzhengjie/data’ mkdir: created directory ‘/yinzhengjie/data/volume1’ mkdir: created directory ‘/yinzhengjie/data/volume2’ mkdir: created directory ‘/yinzhengjie/data/volume3’ mkdir: created directory ‘/yinzhengjie/data/volume4’ mkdir: created directory ‘/yinzhengjie/data/volume5’ [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/ -R /yinzhengjie/data/: total 0 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume1 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume2 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume3 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume4 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume5 /yinzhengjie/data/volume1: total 0 /yinzhengjie/data/volume2: total 0 /yinzhengjie/data/volume3: total 0 /yinzhengjie/data/volume4: total 0 /yinzhengjie/data/volume5: total 0 [root@test110.yinzhengjie.org.cn ~]#
[root@test110.yinzhengjie.org.cn ~]# vim /etc/exports [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# cat /etc/exports /yinzhengjie/data/volume1 172.200.0.0/21(rw,no_root_squash) /yinzhengjie/data/volume2 172.200.0.0/21(rw,no_root_squash) /yinzhengjie/data/volume3 172.200.0.0/21(rw,no_root_squash) /yinzhengjie/data/volume4 172.200.0.0/21(rw,no_root_squash) /yinzhengjie/data/volume5 172.200.0.0/21(rw,no_root_squash) [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/ total 0 drwxr-xr-x 2 polkitd root 22 Feb 10 06:40 volume1 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume2 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume3 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume4 drwxr-xr-x 2 root root 6 Feb 10 06:04 volume5 [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# exportfs -rav exporting 172.200.0.0/21:/yinzhengjie/data/volume5 exporting 172.200.0.0/21:/yinzhengjie/data/volume4 exporting 172.200.0.0/21:/yinzhengjie/data/volume3 exporting 172.200.0.0/21:/yinzhengjie/data/volume2 exporting 172.200.0.0/21:/yinzhengjie/data/volume1 [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]#
[root@test110.yinzhengjie.org.cn ~]# systemctl status nfs ● nfs-server.service - NFS server and services Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled; vendor preset: disabled) Active: inactive (dead) [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# systemctl start nfs [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# systemctl enable nfs Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service. [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# systemctl status nfs ● nfs-server.service - NFS server and services Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled) Drop-In: /run/systemd/generator/nfs-server.service.d └─order-with-mounts.conf Active: active (exited) since Mon 2020-02-10 06:08:52 CST; 6s ago Main PID: 6116 (code=exited, status=0/SUCCESS) CGroup: /system.slice/nfs-server.service Feb 10 06:08:52 test110.yinzhengjie.org.cn systemd[1]: Starting NFS server and services... Feb 10 06:08:52 test110.yinzhengjie.org.cn systemd[1]: Started NFS server and services. [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# ss -ntl | grep 2049 LISTEN 0 64 *:2049 *:* LISTEN 0 64 :::2049 :::* [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# iptables -vnL Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# iptables -vnL -t nat Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination [root@test110.yinzhengjie.org.cn ~]#
nfs无法启动: 如果你nfs启动失败很可能是由于rpcbind服务没有启动导致的。因为nfs服务是依赖于rpc服务的，我的报错信息如下所示: Feb 10 18:36:52 test110.yinzhengjie.org.cn systemd[1]: rpcbind.socket failed to listen on sockets: Address family not supported by protocol 解决方案: 经查阅资料是需要禁用ipv6,修改rpc的配置文件后并重启rpcbind服务，之后nfs可以成功启动啦。具体操作如下所示。 [root@test110.yinzhengjie.org.cn ~]# find /etc/ -name '*rpcbind.socket*' /etc/systemd/system/sockets.target.wants/rpcbind.socket [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# vim /etc/systemd/system/sockets.target.wants/rpcbind.socket [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# cat /etc/systemd/system/sockets.target.wants/rpcbind.socket [Unit] Description=RPCbind Server Activation Socket [Socket] ListenStream=/var/run/rpcbind.sock # RPC netconfig can't handle ipv6/ipv4 dual sockets BindIPv6Only=ipv6-only ListenStream=0.0.0.0:111 ListenDatagram=0.0.0.0:111 #ListenStream=[::]:111 #果然监听了ipv6地址，将这一行注释即可 ListenDatagram=[::]:111 [Install] WantedBy=sockets.target [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# systemctl daemon-reload [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# systemctl restart rpcbind.socket [root@test110.yinzhengjie.org.cn ~]# [root@test110.yinzhengjie.org.cn ~]# systemctl start nfs [root@test110.yinzhengjie.org.cn ~]#
2>.在K8S集群的每一个节点安装nfs客户端驱动
[root@master200.yinzhengjie.org.cn ~]# yum -y install nfs-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.bit.edu.cn * extras: mirror.bit.edu.cn * updates: mirror.bit.edu.cn base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 extras | 2.9 kB 00:00:00 kubernetes | 1.4 kB 00:00:00 updates | 2.9 kB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Running transaction check ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed ---> Package quota.x86_64 1:4.01-19.el7 will be installed --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64 --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64 ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed --> Running transaction check ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64 --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64 ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: nfs-utils x86_64 1:1.3.0-0.65.el7 base 412 k Installing for dependencies: gssproxy x86_64 0.7.0-26.el7 base 110 k keyutils x86_64 1.5.8-3.el7 base 54 k libbasicobjects x86_64 0.1.1-32.el7 base 26 k libcollection x86_64 0.7.0-32.el7 base 42 k libini_config x86_64 1.3.1-32.el7 base 64 k libnfsidmap x86_64 0.25-19.el7 base 50 k libpath_utils x86_64 0.2.1-32.el7 base 28 k libref_array x86_64 0.1.5-32.el7 base 27 k libtirpc x86_64 0.2.4-0.16.el7 base 89 k libverto-libevent x86_64 0.2.5-4.el7 base 8.9 k quota x86_64 1:4.01-19.el7 base 179 k quota-nls noarch 1:4.01-19.el7 base 90 k rpcbind x86_64 0.2.0-48.el7 base 60 k tcp_wrappers x86_64 7.6-77.el7 base 78 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+14 Dependent packages) Total download size: 1.3 M Installed size: 3.6 M Downloading packages: (1/15): keyutils-1.5.8-3.el7.x86_64.rpm | 54 kB 00:00:00 (2/15): gssproxy-0.7.0-26.el7.x86_64.rpm | 110 kB 00:00:00 (3/15): libini_config-1.3.1-32.el7.x86_64.rpm | 64 kB 00:00:00 (4/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm | 26 kB 00:00:00 (5/15): libpath_utils-0.2.1-32.el7.x86_64.rpm | 28 kB 00:00:00 (6/15): libcollection-0.7.0-32.el7.x86_64.rpm | 42 kB 00:00:00 (7/15): libref_array-0.1.5-32.el7.x86_64.rpm | 27 kB 00:00:00 (8/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00 (9/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm | 8.9 kB 00:00:00 (10/15): quota-4.01-19.el7.x86_64.rpm | 179 kB 00:00:00 (11/15): libnfsidmap-0.25-19.el7.x86_64.rpm | 50 kB 00:00:00 (12/15): rpcbind-0.2.0-48.el7.x86_64.rpm | 60 kB 00:00:00 (13/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm | 412 kB 00:00:00 (14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm | 78 kB 00:00:00 (15/15): quota-nls-4.01-19.el7.noarch.rpm | 90 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 1.4 MB/s | 1.3 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libbasicobjects-0.1.1-32.el7.x86_64 1/15 Installing : libref_array-0.1.5-32.el7.x86_64 2/15 Installing : libcollection-0.7.0-32.el7.x86_64 3/15 Installing : libtirpc-0.2.4-0.16.el7.x86_64 4/15 Installing : rpcbind-0.2.0-48.el7.x86_64 5/15 Installing : 1:quota-nls-4.01-19.el7.noarch 6/15 Installing : tcp_wrappers-7.6-77.el7.x86_64 7/15 Installing : 1:quota-4.01-19.el7.x86_64 8/15 Installing : keyutils-1.5.8-3.el7.x86_64 9/15 Installing : libnfsidmap-0.25-19.el7.x86_64 10/15 Installing : libpath_utils-0.2.1-32.el7.x86_64 11/15 Installing : libini_config-1.3.1-32.el7.x86_64 12/15 Installing : libverto-libevent-0.2.5-4.el7.x86_64 13/15 Installing : gssproxy-0.7.0-26.el7.x86_64 14/15 Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64 15/15 Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/15 Verifying : libverto-libevent-0.2.5-4.el7.x86_64 2/15 Verifying : 1:quota-4.01-19.el7.x86_64 3/15 Verifying : gssproxy-0.7.0-26.el7.x86_64 4/15 Verifying : libpath_utils-0.2.1-32.el7.x86_64 5/15 Verifying : libnfsidmap-0.25-19.el7.x86_64 6/15 Verifying : keyutils-1.5.8-3.el7.x86_64 7/15 Verifying : 1:nfs-utils-1.3.0-0.65.el7.x86_64 8/15 Verifying : tcp_wrappers-7.6-77.el7.x86_64 9/15 Verifying : libcollection-0.7.0-32.el7.x86_64 10/15 Verifying : libref_array-0.1.5-32.el7.x86_64 11/15 Verifying : libbasicobjects-0.1.1-32.el7.x86_64 12/15 Verifying : rpcbind-0.2.0-48.el7.x86_64 13/15 Verifying : libini_config-1.3.1-32.el7.x86_64 14/15 Verifying : 1:quota-nls-4.01-19.el7.noarch 15/15 Installed: nfs-utils.x86_64 1:1.3.0-0.65.el7 Dependency Installed: gssproxy.x86_64 0:0.7.0-26.el7 keyutils.x86_64 0:1.5.8-3.el7 libbasicobjects.x86_64 0:0.1.1-32.el7 libcollection.x86_64 0:0.7.0-32.el7 libini_config.x86_64 0:1.3.1-32.el7 libnfsidmap.x86_64 0:0.25-19.el7 libpath_utils.x86_64 0:0.2.1-32.el7 libref_array.x86_64 0:0.1.5-32.el7 libtirpc.x86_64 0:0.2.4-0.16.el7 libverto-libevent.x86_64 0:0.2.5-4.el7 quota.x86_64 1:4.01-19.el7 quota-nls.noarch 1:4.01-19.el7 rpcbind.x86_64 0:0.2.0-48.el7 tcp_wrappers.x86_64 0:7.6-77.el7 Complete! [root@master200.yinzhengjie.org.cn ~]#
[root@node201.yinzhengjie.org.cn ~]# yum -y install nfs-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirror.bit.edu.cn base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 extras | 2.9 kB 00:00:00 kubernetes | 1.4 kB 00:00:00 updates | 2.9 kB 00:00:00 updates/7/x86_64/primary_db | 6.7 MB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Running transaction check ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed ---> Package quota.x86_64 1:4.01-19.el7 will be installed --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64 --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64 ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed --> Running transaction check ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64 --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64 ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: nfs-utils x86_64 1:1.3.0-0.65.el7 base 412 k Installing for dependencies: gssproxy x86_64 0.7.0-26.el7 base 110 k keyutils x86_64 1.5.8-3.el7 base 54 k libbasicobjects x86_64 0.1.1-32.el7 base 26 k libcollection x86_64 0.7.0-32.el7 base 42 k libini_config x86_64 1.3.1-32.el7 base 64 k libnfsidmap x86_64 0.25-19.el7 base 50 k libpath_utils x86_64 0.2.1-32.el7 base 28 k libref_array x86_64 0.1.5-32.el7 base 27 k libtirpc x86_64 0.2.4-0.16.el7 base 89 k libverto-libevent x86_64 0.2.5-4.el7 base 8.9 k quota x86_64 1:4.01-19.el7 base 179 k quota-nls noarch 1:4.01-19.el7 base 90 k rpcbind x86_64 0.2.0-48.el7 base 60 k tcp_wrappers x86_64 7.6-77.el7 base 78 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+14 Dependent packages) Total download size: 1.3 M Installed size: 3.6 M Downloading packages: (1/15): gssproxy-0.7.0-26.el7.x86_64.rpm | 110 kB 00:00:00 (2/15): keyutils-1.5.8-3.el7.x86_64.rpm | 54 kB 00:00:00 (3/15): libcollection-0.7.0-32.el7.x86_64.rpm | 42 kB 00:00:00 (4/15): libnfsidmap-0.25-19.el7.x86_64.rpm | 50 kB 00:00:00 (5/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm | 26 kB 00:00:00 (6/15): libini_config-1.3.1-32.el7.x86_64.rpm | 64 kB 00:00:00 (7/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00 (8/15): libref_array-0.1.5-32.el7.x86_64.rpm | 27 kB 00:00:00 (9/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm | 8.9 kB 00:00:00 (10/15): libpath_utils-0.2.1-32.el7.x86_64.rpm | 28 kB 00:00:00 (11/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm | 412 kB 00:00:00 (12/15): quota-4.01-19.el7.x86_64.rpm | 179 kB 00:00:00 (13/15): quota-nls-4.01-19.el7.noarch.rpm | 90 kB 00:00:00 (14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm | 78 kB 00:00:00 (15/15): rpcbind-0.2.0-48.el7.x86_64.rpm | 60 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 2.2 MB/s | 1.3 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libbasicobjects-0.1.1-32.el7.x86_64 1/15 Installing : libref_array-0.1.5-32.el7.x86_64 2/15 Installing : libcollection-0.7.0-32.el7.x86_64 3/15 Installing : libtirpc-0.2.4-0.16.el7.x86_64 4/15 Installing : rpcbind-0.2.0-48.el7.x86_64 5/15 Installing : 1:quota-nls-4.01-19.el7.noarch 6/15 Installing : tcp_wrappers-7.6-77.el7.x86_64 7/15 Installing : 1:quota-4.01-19.el7.x86_64 8/15 Installing : keyutils-1.5.8-3.el7.x86_64 9/15 Installing : libnfsidmap-0.25-19.el7.x86_64 10/15 Installing : libpath_utils-0.2.1-32.el7.x86_64 11/15 Installing : libini_config-1.3.1-32.el7.x86_64 12/15 Installing : libverto-libevent-0.2.5-4.el7.x86_64 13/15 Installing : gssproxy-0.7.0-26.el7.x86_64 14/15 Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64 15/15 Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/15 Verifying : libverto-libevent-0.2.5-4.el7.x86_64 2/15 Verifying : 1:quota-4.01-19.el7.x86_64 3/15 Verifying : gssproxy-0.7.0-26.el7.x86_64 4/15 Verifying : libpath_utils-0.2.1-32.el7.x86_64 5/15 Verifying : libnfsidmap-0.25-19.el7.x86_64 6/15 Verifying : keyutils-1.5.8-3.el7.x86_64 7/15 Verifying : 1:nfs-utils-1.3.0-0.65.el7.x86_64 8/15 Verifying : tcp_wrappers-7.6-77.el7.x86_64 9/15 Verifying : libcollection-0.7.0-32.el7.x86_64 10/15 Verifying : libref_array-0.1.5-32.el7.x86_64 11/15 Verifying : libbasicobjects-0.1.1-32.el7.x86_64 12/15 Verifying : rpcbind-0.2.0-48.el7.x86_64 13/15 Verifying : libini_config-1.3.1-32.el7.x86_64 14/15 Verifying : 1:quota-nls-4.01-19.el7.noarch 15/15 Installed: nfs-utils.x86_64 1:1.3.0-0.65.el7 Dependency Installed: gssproxy.x86_64 0:0.7.0-26.el7 keyutils.x86_64 0:1.5.8-3.el7 libbasicobjects.x86_64 0:0.1.1-32.el7 libcollection.x86_64 0:0.7.0-32.el7 libini_config.x86_64 0:1.3.1-32.el7 libnfsidmap.x86_64 0:0.25-19.el7 libpath_utils.x86_64 0:0.2.1-32.el7 libref_array.x86_64 0:0.1.5-32.el7 libtirpc.x86_64 0:0.2.4-0.16.el7 libverto-libevent.x86_64 0:0.2.5-4.el7 quota.x86_64 1:4.01-19.el7 quota-nls.noarch 1:4.01-19.el7 rpcbind.x86_64 0:0.2.0-48.el7 tcp_wrappers.x86_64 0:7.6-77.el7 Complete! [root@node201.yinzhengjie.org.cn ~]#
[root@node202.yinzhengjie.org.cn ~]# yum -y install nfs-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.bit.edu.cn * extras: mirror.bit.edu.cn * updates: mirror.bit.edu.cn base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 extras | 2.9 kB 00:00:00 kubernetes | 1.4 kB 00:00:00 updates | 2.9 kB 00:00:00 updates/7/x86_64/primary_db | 6.7 MB 00:00:01 Resolving Dependencies --> Running transaction check ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Running transaction check ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed ---> Package quota.x86_64 1:4.01-19.el7 will be installed --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64 --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64 ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed --> Running transaction check ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64 --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64 ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: nfs-utils x86_64 1:1.3.0-0.65.el7 base 412 k Installing for dependencies: gssproxy x86_64 0.7.0-26.el7 base 110 k keyutils x86_64 1.5.8-3.el7 base 54 k libbasicobjects x86_64 0.1.1-32.el7 base 26 k libcollection x86_64 0.7.0-32.el7 base 42 k libini_config x86_64 1.3.1-32.el7 base 64 k libnfsidmap x86_64 0.25-19.el7 base 50 k libpath_utils x86_64 0.2.1-32.el7 base 28 k libref_array x86_64 0.1.5-32.el7 base 27 k libtirpc x86_64 0.2.4-0.16.el7 base 89 k libverto-libevent x86_64 0.2.5-4.el7 base 8.9 k quota x86_64 1:4.01-19.el7 base 179 k quota-nls noarch 1:4.01-19.el7 base 90 k rpcbind x86_64 0.2.0-48.el7 base 60 k tcp_wrappers x86_64 7.6-77.el7 base 78 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+14 Dependent packages) Total download size: 1.3 M Installed size: 3.6 M Downloading packages: (1/15): keyutils-1.5.8-3.el7.x86_64.rpm | 54 kB 00:00:00 (2/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm | 26 kB 00:00:00 (3/15): libnfsidmap-0.25-19.el7.x86_64.rpm | 50 kB 00:00:00 (4/15): libini_config-1.3.1-32.el7.x86_64.rpm | 64 kB 00:00:00 (5/15): libref_array-0.1.5-32.el7.x86_64.rpm | 27 kB 00:00:00 (6/15): libpath_utils-0.2.1-32.el7.x86_64.rpm | 28 kB 00:00:00 (7/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm | 8.9 kB 00:00:00 (8/15): gssproxy-0.7.0-26.el7.x86_64.rpm | 110 kB 00:00:00 (9/15): libcollection-0.7.0-32.el7.x86_64.rpm | 42 kB 00:00:00 (10/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm | 412 kB 00:00:00 (11/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00 (12/15): quota-4.01-19.el7.x86_64.rpm | 179 kB 00:00:00 (13/15): tcp_wrappers-7.6-77.el7.x86_64.rpm | 78 kB 00:00:00 (14/15): quota-nls-4.01-19.el7.noarch.rpm | 90 kB 00:00:00 (15/15): rpcbind-0.2.0-48.el7.x86_64.rpm | 60 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 1.9 MB/s | 1.3 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libbasicobjects-0.1.1-32.el7.x86_64 1/15 Installing : libref_array-0.1.5-32.el7.x86_64 2/15 Installing : libcollection-0.7.0-32.el7.x86_64 3/15 Installing : libtirpc-0.2.4-0.16.el7.x86_64 4/15 Installing : rpcbind-0.2.0-48.el7.x86_64 5/15 Installing : 1:quota-nls-4.01-19.el7.noarch 6/15 Installing : tcp_wrappers-7.6-77.el7.x86_64 7/15 Installing : 1:quota-4.01-19.el7.x86_64 8/15 Installing : keyutils-1.5.8-3.el7.x86_64 9/15 Installing : libnfsidmap-0.25-19.el7.x86_64 10/15 Installing : libpath_utils-0.2.1-32.el7.x86_64 11/15 Installing : libini_config-1.3.1-32.el7.x86_64 12/15 Installing : libverto-libevent-0.2.5-4.el7.x86_64 13/15 Installing : gssproxy-0.7.0-26.el7.x86_64 14/15 Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64 15/15 Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/15 Verifying : libverto-libevent-0.2.5-4.el7.x86_64 2/15 Verifying : 1:quota-4.01-19.el7.x86_64 3/15 Verifying : gssproxy-0.7.0-26.el7.x86_64 4/15 Verifying : libpath_utils-0.2.1-32.el7.x86_64 5/15 Verifying : libnfsidmap-0.25-19.el7.x86_64 6/15 Verifying : keyutils-1.5.8-3.el7.x86_64 7/15 Verifying : 1:nfs-utils-1.3.0-0.65.el7.x86_64 8/15 Verifying : tcp_wrappers-7.6-77.el7.x86_64 9/15 Verifying : libcollection-0.7.0-32.el7.x86_64 10/15 Verifying : libref_array-0.1.5-32.el7.x86_64 11/15 Verifying : libbasicobjects-0.1.1-32.el7.x86_64 12/15 Verifying : rpcbind-0.2.0-48.el7.x86_64 13/15 Verifying : libini_config-1.3.1-32.el7.x86_64 14/15 Verifying : 1:quota-nls-4.01-19.el7.noarch 15/15 Installed: nfs-utils.x86_64 1:1.3.0-0.65.el7 Dependency Installed: gssproxy.x86_64 0:0.7.0-26.el7 keyutils.x86_64 0:1.5.8-3.el7 libbasicobjects.x86_64 0:0.1.1-32.el7 libcollection.x86_64 0:0.7.0-32.el7 libini_config.x86_64 0:1.3.1-32.el7 libnfsidmap.x86_64 0:0.25-19.el7 libpath_utils.x86_64 0:0.2.1-32.el7 libref_array.x86_64 0:0.1.5-32.el7 libtirpc.x86_64 0:0.2.4-0.16.el7 libverto-libevent.x86_64 0:0.2.5-4.el7 quota.x86_64 1:4.01-19.el7 quota-nls.noarch 1:4.01-19.el7 rpcbind.x86_64 0:0.2.0-48.el7 tcp_wrappers.x86_64 0:7.6-77.el7 Complete! [root@node202.yinzhengjie.org.cn ~]#
[root@node203.yinzhengjie.org.cn ~]# yum -y install nfs-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirror.bit.edu.cn base | 3.6 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 extras | 2.9 kB 00:00:00 kubernetes | 1.4 kB 00:00:00 updates | 2.9 kB 00:00:00 updates/7/x86_64/primary_db | 6.7 MB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64 --> Running transaction check ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64 ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed ---> Package quota.x86_64 1:4.01-19.el7 will be installed --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64 --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64 ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed --> Running transaction check ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64 --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64 ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: nfs-utils x86_64 1:1.3.0-0.65.el7 base 412 k Installing for dependencies: gssproxy x86_64 0.7.0-26.el7 base 110 k keyutils x86_64 1.5.8-3.el7 base 54 k libbasicobjects x86_64 0.1.1-32.el7 base 26 k libcollection x86_64 0.7.0-32.el7 base 42 k libini_config x86_64 1.3.1-32.el7 base 64 k libnfsidmap x86_64 0.25-19.el7 base 50 k libpath_utils x86_64 0.2.1-32.el7 base 28 k libref_array x86_64 0.1.5-32.el7 base 27 k libtirpc x86_64 0.2.4-0.16.el7 base 89 k libverto-libevent x86_64 0.2.5-4.el7 base 8.9 k quota x86_64 1:4.01-19.el7 base 179 k quota-nls noarch 1:4.01-19.el7 base 90 k rpcbind x86_64 0.2.0-48.el7 base 60 k tcp_wrappers x86_64 7.6-77.el7 base 78 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+14 Dependent packages) Total download size: 1.3 M Installed size: 3.6 M Downloading packages: (1/15): libcollection-0.7.0-32.el7.x86_64.rpm | 42 kB 00:00:00 (2/15): gssproxy-0.7.0-26.el7.x86_64.rpm | 110 kB 00:00:00 (3/15): libpath_utils-0.2.1-32.el7.x86_64.rpm | 28 kB 00:00:00 (4/15): libref_array-0.1.5-32.el7.x86_64.rpm | 27 kB 00:00:00 (5/15): keyutils-1.5.8-3.el7.x86_64.rpm | 54 kB 00:00:00 (6/15): libnfsidmap-0.25-19.el7.x86_64.rpm | 50 kB 00:00:00 (7/15): libini_config-1.3.1-32.el7.x86_64.rpm | 64 kB 00:00:00 (8/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00 (9/15): quota-nls-4.01-19.el7.noarch.rpm | 90 kB 00:00:00 (10/15): quota-4.01-19.el7.x86_64.rpm | 179 kB 00:00:00 (11/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm | 26 kB 00:00:00 (12/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm | 8.9 kB 00:00:00 (13/15): rpcbind-0.2.0-48.el7.x86_64.rpm | 60 kB 00:00:00 (14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm | 78 kB 00:00:00 (15/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm | 412 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 2.6 MB/s | 1.3 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libbasicobjects-0.1.1-32.el7.x86_64 1/15 Installing : libref_array-0.1.5-32.el7.x86_64 2/15 Installing : libcollection-0.7.0-32.el7.x86_64 3/15 Installing : libtirpc-0.2.4-0.16.el7.x86_64 4/15 Installing : rpcbind-0.2.0-48.el7.x86_64 5/15 Installing : 1:quota-nls-4.01-19.el7.noarch 6/15 Installing : tcp_wrappers-7.6-77.el7.x86_64 7/15 Installing : 1:quota-4.01-19.el7.x86_64 8/15 Installing : keyutils-1.5.8-3.el7.x86_64 9/15 Installing : libnfsidmap-0.25-19.el7.x86_64 10/15 Installing : libpath_utils-0.2.1-32.el7.x86_64 11/15 Installing : libini_config-1.3.1-32.el7.x86_64 12/15 Installing : libverto-libevent-0.2.5-4.el7.x86_64 13/15 Installing : gssproxy-0.7.0-26.el7.x86_64 14/15 Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64 15/15 Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/15 Verifying : libverto-libevent-0.2.5-4.el7.x86_64 2/15 Verifying : 1:quota-4.01-19.el7.x86_64 3/15 Verifying : gssproxy-0.7.0-26.el7.x86_64 4/15 Verifying : libpath_utils-0.2.1-32.el7.x86_64 5/15 Verifying : libnfsidmap-0.25-19.el7.x86_64 6/15 Verifying : keyutils-1.5.8-3.el7.x86_64 7/15 Verifying : 1:nfs-utils-1.3.0-0.65.el7.x86_64 8/15 Verifying : tcp_wrappers-7.6-77.el7.x86_64 9/15 Verifying : libcollection-0.7.0-32.el7.x86_64 10/15 Verifying : libref_array-0.1.5-32.el7.x86_64 11/15 Verifying : libbasicobjects-0.1.1-32.el7.x86_64 12/15 Verifying : rpcbind-0.2.0-48.el7.x86_64 13/15 Verifying : libini_config-1.3.1-32.el7.x86_64 14/15 Verifying : 1:quota-nls-4.01-19.el7.noarch 15/15 Installed: nfs-utils.x86_64 1:1.3.0-0.65.el7 Dependency Installed: gssproxy.x86_64 0:0.7.0-26.el7 keyutils.x86_64 0:1.5.8-3.el7 libbasicobjects.x86_64 0:0.1.1-32.el7 libcollection.x86_64 0:0.7.0-32.el7 libini_config.x86_64 0:1.3.1-32.el7 libnfsidmap.x86_64 0:0.25-19.el7 libpath_utils.x86_64 0:0.2.1-32.el7 libref_array.x86_64 0:0.1.5-32.el7 libtirpc.x86_64 0:0.2.4-0.16.el7 libverto-libevent.x86_64 0:0.2.5-4.el7 quota.x86_64 1:4.01-19.el7 quota-nls.noarch 1:4.01-19.el7 rpcbind.x86_64 0:0.2.0-48.el7 tcp_wrappers.x86_64 0:7.6-77.el7 Complete! [root@node203.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# mount -t nfs 172.200.1.110:/yinzhengjie/data/volume1 /mnt #在K8S node节点测试NFS服务是否可以成功挂载 [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# mount | grep mnt 172.200.1.110:/yinzhengjie/data/volume1 on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=172.200.1.200,local_lock=none,addr=172.200.1.110) [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# df -h | grep mnt 172.200.1.110:/yinzhengjie/data/volume1 1.6T 416M 1.6T 1% /mnt [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
3>.静态供给(Provisioning)创建pv(由于pv是集群资源,并不属于名称空间，因此在定义pv时我们不能指定名称空间哟~)
[root@master200.yinzhengjie.org.cn ~]# kubectl explain pv KIND: PersistentVolume VERSION: v1 DESCRIPTION: PersistentVolume (PV) is a storage resource provisioned by an administrator. It is analogous to a node. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes FIELDS: apiVersion <string> APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources kind <string> Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds metadata <Object> Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata spec <Object> Spec defines a specification of a persistent volume owned by the cluster. Provisioned by an administrator. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistent-volumes status <Object> Status represents the current information/status for the persistent volume. Populated by the system. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistent-volumes [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl explain pv.spec KIND: PersistentVolume VERSION: v1 RESOURCE: spec <Object> DESCRIPTION: Spec defines a specification of a persistent volume owned by the cluster. Provisioned by an administrator. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistent-volumes PersistentVolumeSpec is the specification of a persistent volume. FIELDS: accessModes <[]string> AccessModes contains all ways the volume can be mounted. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes awsElasticBlockStore <Object> AWSElasticBlockStore represents an AWS Disk resource that is attached to a kubelet's host machine and then exposed to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore azureDisk <Object> AzureDisk represents an Azure Data Disk mount on the host and bind mount to the pod. azureFile <Object> AzureFile represents an Azure File Service mount on the host and bind mount to the pod. capacity <map[string]string> A description of the persistent volume's resources and capacity. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#capacity cephfs <Object> CephFS represents a Ceph FS mount on the host that shares a pod's lifetime cinder <Object> Cinder represents a cinder volume attached and mounted on kubelets host machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md claimRef <Object> ClaimRef is part of a bi-directional binding between PersistentVolume and PersistentVolumeClaim. Expected to be non-nil when bound. claim.VolumeName is the authoritative bind between PV and PVC. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#binding csi <Object> CSI represents storage that is handled by an external CSI driver (Beta feature). fc <Object> FC represents a Fibre Channel resource that is attached to a kubelet's host machine and then exposed to the pod. flexVolume <Object> FlexVolume represents a generic volume resource that is provisioned/attached using an exec based plugin. flocker <Object> Flocker represents a Flocker volume attached to a kubelet's host machine and exposed to the pod for its usage. This depends on the Flocker control service being running gcePersistentDisk <Object> GCEPersistentDisk represents a GCE Disk resource that is attached to a kubelet's host machine and then exposed to the pod. Provisioned by an admin. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk glusterfs <Object> Glusterfs represents a Glusterfs volume that is attached to a host and exposed to the pod. Provisioned by an admin. More info: https://examples.k8s.io/volumes/glusterfs/README.md hostPath <Object> HostPath represents a directory on the host. Provisioned by a developer or tester. This is useful for single-node development and testing only! On-host storage is not supported in any way and WILL NOT WORK in a multi-node cluster. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath iscsi <Object> ISCSI represents an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod. Provisioned by an admin. local <Object> Local represents directly-attached storage with node affinity mountOptions <[]string> A list of mount options, e.g. ["ro", "soft"]. Not validated - mount will simply fail if one is invalid. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#mount-options nfs <Object> NFS represents an NFS mount on the host. Provisioned by an admin. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs nodeAffinity <Object> NodeAffinity defines constraints that limit what nodes this volume can be accessed from. This field influences the scheduling of pods that use this volume. persistentVolumeReclaimPolicy <string> What happens to a persistent volume when released from its claim. Valid options are Retain (default for manually created PersistentVolumes), Delete (default for dynamically provisioned PersistentVolumes), and Recycle (deprecated). Recycle must be supported by the volume plugin underlying this PersistentVolume. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#reclaiming photonPersistentDisk <Object> PhotonPersistentDisk represents a PhotonController persistent disk attached and mounted on kubelets host machine portworxVolume <Object> PortworxVolume represents a portworx volume attached and mounted on kubelets host machine quobyte <Object> Quobyte represents a Quobyte mount on the host that shares a pod's lifetime rbd <Object> RBD represents a Rados Block Device mount on the host that shares a pod's lifetime. More info: https://examples.k8s.io/volumes/rbd/README.md scaleIO <Object> ScaleIO represents a ScaleIO persistent volume attached and mounted on Kubernetes nodes. storageClassName <string> Name of StorageClass to which this persistent volume belongs. Empty value means that this volume does not belong to any StorageClass. storageos <Object> StorageOS represents a StorageOS volume that is attached to the kubelet's host machine and mounted into the pod More info: https://examples.k8s.io/volumes/storageos/README.md volumeMode <string> volumeMode defines if a volume is intended to be used with a formatted filesystem or to remain in raw block state. Value of Filesystem is implied when not included in spec. This is a beta feature. vsphereVolume <Object> VsphereVolume represents a vSphere volume attached and mounted on kubelets host machine [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/volumes/pv-demo.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/volumes/pv-demo.yaml apiVersion: v1 kind: PersistentVolume metadata: name: pv-nfs-v2 labels: storsys: nfs spec: accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"] capacity: storage: 5Gi volumeMode: Filesystem persistentVolumeReclaimPolicy: Retain nfs: server: 172.200.1.110 path: /yinzhengjie/data/volume2 [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv No resources found in default namespace. [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/volumes/pv-demo.yaml persistentvolume/pv-nfs-v2 created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-nfs-v2 5Gi RWO,ROX,RWX Retain Available 87s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE pv-nfs-v2 5Gi RWO,ROX,RWX Retain Available 2m44s Filesystem [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE pv-nfs-v2 5Gi RWO,ROX,RWX Retain Available 3m4s Filesystem [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl describe pv Name: pv-nfs-v2 Labels: storsys=nfs Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"PersistentVolume","metadata":{"annotations":{},"labels":{"storsys":"nfs"},"name":"pv-nfs-v2"},"spec":{"accessMo... Finalizers: [kubernetes.io/pv-protection] StorageClass: Status: Available Claim: Reclaim Policy: Retain Access Modes: RWO,ROX,RWX VolumeMode: Filesystem Capacity: 5Gi Node Affinity: <none> Message: Source: Type: NFS (an NFS mount that lasts the lifetime of a pod) Server: 172.200.1.110 Path: /yinzhengjie/data/volume2 ReadOnly: false Events: <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
4>.定义pvc(pvc属于名称空间资源，因此我们在定义pvc时可以指定名称空间哟~)
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE pv-nfs-v2 5Gi RWO,ROX,RWX Retain Available 3m4s Filesystem [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/volumes/pvc-demo.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/volumes/pvc-demo.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: redis-data namespace: yinzhengjie-volume spec: accessModes: - ReadWriteOnce volumeMode: Filesystem resources: requests: storage: 3Gi [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/volumes/pvc-demo.yaml persistentvolumeclaim/redis-data created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pvc -n yinzhengjie-volume NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE redis-data Bound pv-nfs-v2 5Gi RWO,ROX,RWX 66s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE pv-nfs-v2 5Gi RWO,ROX,RWX Retain Available 3m4s Filesystem [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pvc -n yinzhengjie-volume -o wide NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE redis-data Bound pv-nfs-v2 5Gi RWO,ROX,RWX 101s Filesystem [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl describe pvc -n yinzhengjie-volume Name: redis-data Namespace: yinzhengjie-volume StorageClass: Status: Bound Volume: pv-nfs-v2 Labels: <none> Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"name":"redis-data","namespace":"yinzhengjie-volume"},"spec... pv.kubernetes.io/bind-completed: yes pv.kubernetes.io/bound-by-controller: yes Finalizers: [kubernetes.io/pvc-protection] Capacity: 5Gi Access Modes: RWO,ROX,RWX VolumeMode: Filesystem Mounted By: <none> Events: <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pv -n yinzhengjie-volume NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-nfs-v2 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-volume/redis-data 14m [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE pv-nfs-v2 5Gi RWO,ROX,RWX Retain Bound yinzhengjie-volume/redis-data 18m Filesystem [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
5>.创建Pod并应用pvc
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/pod-demo-redis.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/pod-demo-redis.yaml apiVersion: v1 kind: Pod metadata: name: redis-demo namespace: yinzhengjie-volume labels: app: redis-demo spec: containers: - name: redis image: redis:alpine ports: - containerPort: 6379 name: redis volumeMounts: - mountPath: /data name: redisdata volumes: - name: redisdata persistentVolumeClaim: claimName: redis-data [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp 1/1 Running 1 20h 10.244.3.14 node203.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/pod-demo-redis.yaml pod/redis-demo created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp 1/1 Running 1 20h 10.244.3.14 node203.yinzhengjie.org.cn <none> <none> redis-demo 1/1 Running 0 7s 10.244.1.25 node201.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp 1/1 Running 1 20h 10.244.3.14 node203.yinzhengjie.org.cn <none> <none> redis-demo 1/1 Running 0 96s 10.244.1.25 node201.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl describe pods redis-demo -n yinzhengjie-volume Name: redis-demo Namespace: yinzhengjie-volume Priority: 0 Node: node201.yinzhengjie.org.cn/172.200.1.201 Start Time: Mon, 10 Feb 2020 03:26:02 +0800 Labels: app=redis-demo Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"app":"redis-demo"},"name":"redis-demo","namespace":"yinzhengjie-vo... Status: Running IP: 10.244.1.25 IPs: IP: 10.244.1.25 Containers: redis: Container ID: docker://395ba89580fa126ee755358f2afe8b9922b2084304b3b61a88ded14526e3979c Image: redis:alpine Image ID: docker-pullable://redis@sha256:cb9783b1c39bb34f8d6572406139ab325c4fac0b28aaa25d5350495637bb2f76 Port: 6379/TCP Host Port: 0/TCP State: Running Started: Mon, 10 Feb 2020 03:26:03 +0800 Ready: True Restart Count: 0 Environment: <none> Mounts: /data from redisdata (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-bxflv (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: redisdata: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: redis-data ReadOnly: false default-token-bxflv: Type: Secret (a volume populated by a Secret) SecretName: default-token-bxflv Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled <unknown> default-scheduler Successfully assigned yinzhengjie-volume/redis-demo to node201.yinzhengjie.org.cn Normal Pulled 108s kubelet, node201.yinzhengjie.org.cn Container image "redis:alpine" already present on machine Normal Created 108s kubelet, node201.yinzhengjie.org.cn Created container redis Normal Started 108s kubelet, node201.yinzhengjie.org.cn Started container redis [root@master200.yinzhengjie.org.cn ~]#
6>.连接pod并创建测试数据
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp 1/1 Running 1 20h 10.244.3.14 node203.yinzhengjie.org.cn <none> <none> redis-demo 1/1 Running 0 3m33s 10.244.1.25 node201.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl exec -it redis-demo -n yinzhengjie-volume -- /bin/sh /data # /data # redis-cli 127.0.0.1:6379> 127.0.0.1:6379> KEYS * (empty list or set) 127.0.0.1:6379> 127.0.0.1:6379> SET myblog "https://www.cnblogs.com/yinzhengjie/" OK 127.0.0.1:6379> 127.0.0.1:6379> KEYS * 1) "myblog" 127.0.0.1:6379> 127.0.0.1:6379> BGSAVE Background saving started 127.0.0.1:6379> 127.0.0.1:6379> exit /data # /data # ls -l total 4 -rw-r--r-- 1 redis redis 142 Feb 10 2020 dump.rdb /data # /data # exit [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# mount -t nfs 172.200.1.110:/yinzhengjie/data/volume2 /mnt [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# mount | grep mnt 172.200.1.110:/yinzhengjie/data/volume2 on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=172.200.1.200,local_lock=none,addr=172.200.1.110) [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# df -h | grep mnt 172.200.1.110:/yinzhengjie/data/volume2 1.6T 416M 1.6T 1% /mnt [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# ll /mnt/ total 4 -rw-r--r-- 1 polkitd 1000 142 Feb 10 2020 dump.rdb [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
7>.删除Pod并重新创建Pod验证数据是否丢失
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp 1/1 Running 1 20h 10.244.3.14 node203.yinzhengjie.org.cn <none> <none> redis-demo 1/1 Running 0 11m 10.244.1.25 node201.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl delete -f /yinzhengjie/data/k8s/manifests/basic/pod-demo-redis.yaml pod "redis-demo" deleted [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp 1/1 Running 1 20h 10.244.3.14 node203.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/pod-demo-redis.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/pod-demo-redis.yaml apiVersion: v1 kind: Pod metadata: name: redis-demo namespace: yinzhengjie-volume labels: app: redis-demo spec: nodeName: node203.yinzhengjie.org.cn containers: - name: redis image: redis:alpine ports: - containerPort: 6379 name: redis volumeMounts: - mountPath: /data name: redisdata volumes: - name: redisdata persistentVolumeClaim: claimName: redis-data [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp 1/1 Running 1 20h 10.244.3.14 node203.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/pod-demo-redis.yaml pod/redis-demo created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp 1/1 Running 1 20h 10.244.3.14 node203.yinzhengjie.org.cn <none> <none> redis-demo 1/1 Running 0 24s 10.244.3.19 node203.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp 1/1 Running 1 20h 10.244.3.14 node203.yinzhengjie.org.cn <none> <none> redis-demo 1/1 Running 0 2m49s 10.244.3.19 node203.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl exec -it redis-demo -n yinzhengjie-volume -- /bin/sh /data # /data # redis-cli 127.0.0.1:6379> 127.0.0.1:6379> KEYS * 1) "myblog" 127.0.0.1:6379> 127.0.0.1:6379> get myblog "https://www.cnblogs.com/yinzhengjie/" 127.0.0.1:6379> 127.0.0.1:6379>
8>.删除pvc注意事项
如下图所示，当有Pods在使用pvc时，我们无法是无法手动删除pvc的，这是官方的提出的一种存储保护机制。 博主推荐阅读: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#storage-object-in-use-protection
想要删除pvc得确保没有Pods使用pvc资源，如果把正在使用pvc的所有Pods资源删除掉，此时就可以理解删除pvc了，如下图所示。 删除pvc后，如果pvc的删除策略是"Retain"，虽然删除pvc是成功了，但并不会删除pv资源及其所对应的存储系统中的数据，我们需要手动删除。
三.使用存储类案例
[root@master200.yinzhengjie.org.cn ~]# kubectl explain sc KIND: StorageClass VERSION: storage.k8s.io/v1 DESCRIPTION: StorageClass describes the parameters for a class of storage for which PersistentVolumes can be dynamically provisioned. StorageClasses are non-namespaced; the name of the storage class according to etcd is in ObjectMeta.Name. FIELDS: allowVolumeExpansion <boolean> AllowVolumeExpansion shows whether the storage class allow volume expand allowedTopologies <[]Object> Restrict the node topologies where volumes can be dynamically provisioned. Each volume plugin defines its own supported topology specifications. An empty TopologySelectorTerm list means there is no topology restriction. This field is only honored by servers that enable the VolumeScheduling feature. apiVersion <string> APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources kind <string> Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds metadata <Object> Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata mountOptions <[]string> Dynamically provisioned PersistentVolumes of this storage class are created with these mountOptions, e.g. ["ro", "soft"]. Not validated - mount of the PVs will simply fail if one is invalid. parameters <map[string]string> Parameters holds the parameters for the provisioner that should create volumes of this storage class. provisioner <string> -required- Provisioner indicates the type of the provisioner. reclaimPolicy <string> Dynamically provisioned PersistentVolumes of this storage class are created with this reclaimPolicy. Defaults to Delete. volumeBindingMode <string> VolumeBindingMode indicates how PersistentVolumeClaims should be provisioned and bound. When unset, VolumeBindingImmediate is used. This field is only honored by servers that enable the VolumeScheduling feature. [root@master200.yinzhengjie.org.cn ~]#
存储类是为动态供给pv的，默认情况下，nfs是不支持动态供给的，但是可以通过插件实现让nfs也支持动态供给pv哟，参考连接: https://github.com/kubernetes-incubator/external-storage/tree/master/nfs/deploy/kubernetes
博主推荐阅读: https://kubernetes.io/docs/concepts/storage/storage-classes/
当你的才华还撑不起你的野心的时候，你就应该静下心来学习。当你的能力还驾驭不了你的目标的时候，你就应该沉下心来历练。问问自己，想要怎样的人生。
欢迎加入基础架构自动化运维：598432640，大数据SRE进阶之路：959042252，DevOps进阶之路：526991186
