aws ec2 配置k8s集群nginx和mysql挂载nfs卷

nfs-kernel-serveris an alias fornfs-server

启动nfs服务

amazon linux 2上已经安装了nfs-utils和rpcbind

# systemctl start rpcbind
# systemctl enable rpcbind
# systemctl start nfs-server    
# systemctl enable nfs-server 

配置

# mkdir mysql
# mkdir data
# vim /etc/exports
/home/ec2-user/data 10.0.0.0/16(rw)
/home/ec2-user/mysql 10.0.0.0/16(rw)
# systemctl reload nfs

在另一台实例上

$ showmount -e 192.168.66.53
Export list for 192.168.66.53:
/home/ec2-user/data  192.168.64.0/20
/home/ec2-user/mysql 192.168.64.0/20

挂载，测试创建文件没有问题

mount -t nfs <ip>:/home/ec2-user/data  ~/data

或者

# vim /etc/fstab 
<ip>:/data  /mnt/public       nfs    defaults 0 0
# mount -a  

nginx挂载

直接在pod中挂载

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-dep
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        volumeMounts:
        - name: wwwroot
          mountPath: /usr/share/nginx/html
        ports:
        - containerPort: 80
      volumes:
        - name: wwwroot
          nfs:
            server: 10.0.3.113
            path: /home/ec2-user/data

创建pv

apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteMany
  nfs:
    server: <ip>
    path: /home/ec2-user/data

pvc

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-dep
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        volumeMounts:
        - name: wwwroot
          mountPath: /usr/share/nginx/html
        ports:
        - containerPort: 80
      volumes:
      - name: wwwroot
        persistentVolumeClaim:
          claimName: my-pvc

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi

mysql挂载

创建configmap

apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql-config
  labels:
    app: mysql
data:
  my.cnf: |-
    [client]
    default-character-set=utf8mb4
    [mysql]
    default-character-set=utf8mb4
    [mysqld] 
    max_connections = 2000
    secure_file_priv=/var/lib/mysql
    sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION

创建pv和pvc

apiVersion: v1
kind: PersistentVolume
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  capacity:          
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  nfs: 
    server: <ip>
    path: /home/ec2-user/mysql
  persistentVolumeReclaimPolicy: Retain  
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: mysql
spec:
  resources:
    requests:
      storage: 5Gi
  accessModes:
  - ReadWriteOnce
  selector:
    matchLabels:
      app: mysql

部署mysql

root 密码通过环境变量 MYSQL_ROOT_PASSWORD 注入

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  replicas: 2
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:     
      containers:
      - name: mysql
        image: mysql:latest
        ports:
        - containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "zhaojiew"
        resources:
          limits:
            cpu: 1000m
            memory: 512Mi
          requests:
            cpu: 1000m
            memory: 512Mi
        livenessProbe:
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 3
          exec:
            command: ["mysqladmin", "-uroot", "-p${MYSQL_ROOT_PASSWORD}", "ping"]
        readinessProbe:  
          initialDelaySeconds: 10
          periodSeconds: 10
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 3
          exec:
            command: ["mysqladmin", "-uroot", "-p${MYSQL_ROOT_PASSWORD}", "ping"]
        volumeMounts:
        - name: data
          mountPath: /var/lib/mysql
        - name: config
          mountPath: /etc/mysql/conf.d/my.cnf
          subPath: my.cnf
        - name: localtime
          readOnly: true
          mountPath: /etc/localtime
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: mysql
      - name: config      
        configMap:
          name: mysql-config
      - name: localtime
        hostPath:
          type: File
          path: /etc/localtime

创建service

apiVersion: v1
kind: Service
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  type: NodePort
  ports:
  - name: mysql
    port: 3306
    targetPort: 3306
    nodePort: 30336
  selector:
    app: mysql

注意，挂载nfs之后启动mysql报错，使用NFS做持久化的时候要注意添加no_root_squash权限否则出现以下错误

Entrypoint script for MySQL Server 5.7.34-1debian10 started.
chown: changing ownership of '/var/lib/mysql/': Operation not permitted

解决，为nfs添加 no_root_squash

/home/ec2-user/mysql 192.168.64.0/20(rw,no_root_squash)
/home/ec2-user/data 192.168.64.0/20(rw,no_root_squash)