k8s对接Samba&CIFS文件共享存储
简介
kubernetes提供CSI驱动将外部存储提供给pod中使用,例如我们可以通过CSI驱动对接Samba/CIFS共享文件存储
前提
- Kubernetes 1.16+
安装
安装SMB CSI驱动(二选一)
通过kubectl安装
curl -skSL https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/v0.6.0/deploy/install-driver.sh | bash -s v0.6.0 --
通过helm安装
helm repo add csi-driver-smb https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts
helm install csi-driver-smb csi-driver-smb/csi-driver-smb --namespace kube-system
查看pod状态是否正常
kubectl -n kube-system get pod |grep csi-smb
在kubernetes集群中安装Samba服务(可选)
首先创建一个secret保存用户和密码
kubectl create secret generic smbcreds --from-literal username=rancher --from-literal password="rancher"
接着创建Samba服务
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/smb-provisioner/smb-server.yaml
查看Samba服务状态是否正常
kubectl get pod |grep smb
创建storage class
新建storage-class.yaml
文件
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: smb
provisioner: smb.csi.k8s.io
parameters:
source: "//smb-server.default.svc.cluster.local/share"
csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
csi.storage.k8s.io/node-stage-secret-namespace: "default"
createSubDir: "false" # optional: create a sub dir for new volume
reclaimPolicy: Retain # only retain is supported
volumeBindingMode: Immediate
mountOptions:
- dir_mode=0777
- file_mode=0777
- uid=1001
- gid=1001
其中csi.storage.k8s.io/node-stage-secret-name
对应的是上一步创建的secret
创建storage class
kubectl create -f storage-class.yaml
部署应用
创建一个statefulset类型的应用,并挂载Samba卷
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: statefulset-smb
labels:
app: nginx
spec:
serviceName: statefulset-smb
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
nodeSelector:
"kubernetes.io/os": linux
containers:
- name: statefulset-smb
image: mcr.microsoft.com/oss/nginx/nginx:1.19.5
command:
- "/bin/bash"
- "-c"
- set -euo pipefail; while true; do echo $(date) >> /mnt/smb/outfile; sleep 1; done
volumeMounts:
- name: persistent-storage
mountPath: /mnt/smb
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
app: nginx
volumeClaimTemplates:
- metadata:
name: persistent-storage
annotations:
volume.beta.kubernetes.io/storage-class: smb
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
创建该应用
kubectl apply -f statefulset-smb.yaml
查看应用状态
kubectl get pod
在pod中执行df -h
查看Samba挂载情况
# kubectl exec -it statefulset-smb-0 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
# df -h
Filesystem Size Used Avail Use% Mounted on
overlay 40G 19G 20G 50% /
tmpfs 64M 0 64M 0% /dev
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/mapper/ubuntu--vg-ubuntu--lv 40G 19G 20G 50% /etc/hosts
//smb-server.default.svc.cluster.local/share 40G 21G 20G 52% /mnt/smb
shm 64M 0 64M 0% /dev/shm
tmpfs 3.9G 12K 3.9G 1% /run/secrets/kubernetes.io/serviceaccount
tmpfs 3.9G 0 3.9G 0% /proc/acpi
tmpfs 3.9G 0 3.9G 0% /proc/scsi
tmpfs 3.9G 0 3.9G 0% /sys/firmware
可以看到,/mnt/smb
目录挂载了//smb-server.default.svc.cluster.local/share
Samba文件存储
对接 Windows 共享文件夹
创建新的storage class yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: smb-windows
provisioner: smb.csi.k8s.io
parameters:
source: "//172.16.30.8/share"
csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
csi.storage.k8s.io/node-stage-secret-namespace: "default"
createSubDir: "false" # optional: create a sub dir for new volume
reclaimPolicy: Retain # only retain is supported
volumeBindingMode: Immediate
mountOptions:
- dir_mode=0777
- file_mode=0777
- uid=1001
- gid=1001
其中source
填写对应的Windows主机的IP地址和共享文件路径
创建storage class
kubectl create -f storage-class-win.yaml
其他的操作都一样,正常创建工作负载对接PVC即可