k8s搭建redis集群(可在k8s以外环境访问)

前言

从网上找的K8S教程要么是单机版redis支持集群外访问，要redis集群版不支持k8s以外环境访问。k8s环境外不能访问的原因在于：Redis Cluster 的节点 IP通告问题，当一个外部客户端通过 NodeIP:NodePort 连接到其中一个 Redis 节点，并且需要访问存储在 另一个 节点上的数据时，第一个节点会向客户端发送一个 MOVED 重定向指令。这个指令包含的是目标节点的 内部 Pod IP 和 内部端口 (6379)。本教程的意义在于可解决k8s集群外的客户端访问。

一、集群模式

3台节点搭建的集群，可在k8s集群外进行访问。由于redis本来是基于内存，对数据进行的缓存，因此没有对数据做持久化操作。

本教程只讲操作，不讲原理，如有疑问可私信。

二、步骤部署

提前：准备好redis镜像，并上传到镜像仓库！！！

1)修改redisyaml文件

　　针对3个redis编写了3个yaml文件（见附件一、二、三），并放到同一目录。

　　修改一： --cluster-announce-ip，ip应为k8s的任一节点即可。

　　修改二：namespace，为实际的命名空间；

　　修改三：image为实际的redis镜像名。

2）启动redis

　　在配置文件当前目录执行 kubectl apply -f ./

3）初始化集群

执行kubectl get pods -A -o wide|grep redis 找到redis的pod ip,如下图所示。

然后在其中任意一个pod中执行始初化命令

kubectl exec -it -n dz-business-common redis-cluster-0-0 /bin/sh

进入容器后，再执行：

redis-cli --cluster create 172.27.74.132:6379 172.27.74.79:6379 172.27.75.64:6379

然后在出现的提示中输入“yes”即可，完成后退出容器即可。

三、访问

在本示例中redis集群访问地址：172.31.2.41:32379,172.31.2.41:32380,172.31.2.41:32381

附件一、redis-cluster-0.yaml

---
apiVersion: v1
kind: ConfigMap
metadata:
 name: redis-cluster-0
 namespace: dz-business-common
data:
 update-node.sh: |
   #!/bin/sh
   REDIS_NODES="/data/nodes.conf"

   if [ -f "${REDIS_NODES}" ]; then
       echo "Updating ${REDIS_NODES} with POD_IP: ${POD_IP}"
       sed -i -e "/myself/ s@[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}@${POD_IP}@" ${REDIS_NODES}
   else
       echo "${REDIS_NODES} not found, skipping update."
       # mkdir -p /data
   fi

   echo "Starting redis-server with announce IP: ${NODE_IP}, Port: ${NODE_PORT_CLIENT}, Bus Port: ${NODE_PORT_GOSSIP}"
   mkdir -p /data
   exec redis-server /conf/redis.conf \
       --cluster-announce-ip "172.31.2.41" \
       --cluster-announce-port "${NODE_PORT_CLIENT}" \
       --cluster-announce-bus-port "${NODE_PORT_GOSSIP}"
 redis.conf: |+
   cluster-enabled yes
   cluster-require-full-coverage no
   cluster-node-timeout 15000
   cluster-config-file /data/nodes.conf 
   cluster-migration-barrier 1
   # Maxmemory 80GB 
   port 6379
   appendonly yes
   protected-mode no
   dir /data 
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
 name: redis-cluster-0
 namespace: dz-business-common
spec:
 serviceName: redis-cluster-0
 replicas: 1
 selector: 
   matchLabels:
     app: redis-cluster-0 
 template: 
   metadata:
     labels:
       app: redis-cluster-0 
   spec:
     terminationGracePeriodSeconds: 20 
     containers:
     - name: redis
       image: 172.30.4.32:1180/alg/redis:6.2.6
       ports:
       - containerPort: 6379
         name: client
       - containerPort: 16379
         name: gossip
       command: ["/conf/update-node.sh"]
       env:
       - name: POD_IP
         valueFrom:
           fieldRef:
             fieldPath: status.podIP
       - name: NODE_IP 
         valueFrom:
           fieldRef:
             fieldPath: status.hostIP
       - name: NODE_PORT_CLIENT 
         value: "32379" 
       - name: NODE_PORT_GOSSIP 
         value: "32479" 
       volumeMounts:
       - name: conf
         mountPath: /conf
         readOnly: false
       - name: data 
         mountPath: /data
     volumes:
     - name: conf
       configMap:
         name: redis-cluster-0
         defaultMode: 0755
     - name: data 
       hostPath:
         path: /pie/data/redis-0 
         type: DirectoryOrCreate 
---
apiVersion: v1
kind: Service
metadata:
 name: redis-cluster-0
 namespace: dz-business-common
spec:
 type: NodePort
 ports:
 - port: 6379
   targetPort: 6379
   name: client
   protocol: TCP
   nodePort: 32379
 - port: 16379
   targetPort: 16379
   name: gossip
   protocol: TCP 
   nodePort: 32479
 selector:
   app: redis-cluster-0

附件二、redis-cluster-1.yaml

---
apiVersion: v1
kind: ConfigMap
metadata:
 name: redis-cluster-1
 namespace: dz-business-common
data:
 update-node.sh: |
   #!/bin/sh
   REDIS_NODES="/data/nodes.conf"

   if [ -f "${REDIS_NODES}" ]; then
       echo "Updating ${REDIS_NODES} with POD_IP: ${POD_IP}"
       sed -i -e "/myself/ s@[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}@${POD_IP}@" ${REDIS_NODES}
   else
       echo "${REDIS_NODES} not found, skipping update."
       # mkdir -p /data
   fi

   echo "Starting redis-server with announce IP: ${NODE_IP}, Port: ${NODE_PORT_CLIENT}, Bus Port: ${NODE_PORT_GOSSIP}"
   mkdir -p /data
   exec redis-server /conf/redis.conf \
       --cluster-announce-ip "172.31.2.41" \
       --cluster-announce-port "${NODE_PORT_CLIENT}" \
       --cluster-announce-bus-port "${NODE_PORT_GOSSIP}"
 redis.conf: |+
   cluster-enabled yes
   cluster-require-full-coverage no
   cluster-node-timeout 15000
   cluster-config-file /data/nodes.conf 
   cluster-migration-barrier 1
   # Maxmemory 80GB 
   port 6379
   appendonly yes
   protected-mode no
   dir /data 
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
 name: redis-cluster-1
 namespace: dz-business-common
spec:
 serviceName: redis-cluster-1
 replicas: 1
 selector: 
   matchLabels:
     app: redis-cluster-1 
 template: 
   metadata:
     labels:
       app: redis-cluster-1 
   spec:
     terminationGracePeriodSeconds: 20
     containers:
     - name: redis
       image: 172.30.4.32:1180/alg/redis:6.2.6
       ports:
       - containerPort: 6379
         name: client
       - containerPort: 16379
         name: gossip
       command: ["/conf/update-node.sh"]
       env:
       - name: POD_IP
         valueFrom:
           fieldRef:
             fieldPath: status.podIP
       - name: NODE_IP 
         valueFrom:
           fieldRef:
             fieldPath: status.hostIP
       - name: NODE_PORT_CLIENT 
         value: "32380" 
       - name: NODE_PORT_GOSSIP 
         value: "32480" 
       volumeMounts:
       - name: conf
         mountPath: /conf
         readOnly: false
       - name: data
         mountPath: /data
     volumes:
     - name: conf
       configMap:
         name: redis-cluster-1 
         defaultMode: 0755
     - name: data
       hostPath:
         path: /pie/data/redis-1 
         type: DirectoryOrCreate
---
apiVersion: v1
kind: Service
metadata:
 name: redis-cluster-1
 namespace: dz-business-common
spec:
 type: NodePort
 ports:
 - port: 6379
   targetPort: 6379
   protocol: TCP
   name: client
   nodePort: 32380
 - port: 16379
   targetPort: 16379
   name: gossip
   protocol: TCP 
   nodePort: 32480
 selector:
   app: redis-cluster-1

附件三、redis-cluster-2.yaml

---
apiVersion: v1
kind: ConfigMap
metadata:
 name: redis-cluster-2
 namespace: dz-business-common
data:
 update-node.sh: |
   #!/bin/sh
   REDIS_NODES="/data/nodes.conf"

   if [ -f "${REDIS_NODES}" ]; then
       echo "Updating ${REDIS_NODES} with POD_IP: ${POD_IP}"
       sed -i -e "/myself/ s@[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}@${POD_IP}@" ${REDIS_NODES}
   else
       echo "${REDIS_NODES} not found, skipping update."
       # mkdir -p /data
   fi

   echo "Starting redis-server with announce IP: ${NODE_IP}, Port: ${NODE_PORT_CLIENT}, Bus Port: ${NODE_PORT_GOSSIP}"
   mkdir -p /data
   exec redis-server /conf/redis.conf \
       --cluster-announce-ip "172.31.2.41" \
       --cluster-announce-port "${NODE_PORT_CLIENT}" \
       --cluster-announce-bus-port "${NODE_PORT_GOSSIP}"
 redis.conf: |+
   cluster-enabled yes
   cluster-require-full-coverage no
   cluster-node-timeout 15000
   cluster-config-file /data/nodes.conf 
   cluster-migration-barrier 1
   # Maxmemory 80GB 
   port 6379
   appendonly yes
   protected-mode no
   dir /data 
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
 name: redis-cluster-2
 namespace: dz-business-common
spec:
 serviceName: redis-cluster-2
 replicas: 1
 selector: 
   matchLabels:
     app: redis-cluster-2 
 template: 
   metadata:
     labels:
       app: redis-cluster-2 
   spec:
     terminationGracePeriodSeconds: 20
     containers:
     - name: redis
       image: 172.30.4.32:1180/alg/redis:6.2.6
       ports:
       - containerPort: 6379
         name: client
       - containerPort: 16379
         name: gossip
       command: ["/conf/update-node.sh"]
       env:
       - name: POD_IP
         valueFrom:
           fieldRef:
             fieldPath: status.podIP
       - name: NODE_IP 
         valueFrom:
           fieldRef:
             fieldPath: status.hostIP
       - name: NODE_PORT_CLIENT 
         value: "32381" 
       - name: NODE_PORT_GOSSIP 
         value: "32481" 
       volumeMounts:
       - name: conf
         mountPath: /conf
         readOnly: false
       - name: data
         mountPath: /data
     volumes:
     - name: conf
       configMap:
         name: redis-cluster-2 
         defaultMode: 0755
     - name: data
       hostPath:
         path: /pie/data/redis-2 
         type: DirectoryOrCreate
---
apiVersion: v1
kind: Service
metadata:
 name: redis-cluster-2
 namespace: dz-business-common
spec:
 type: NodePort
 ports:
 - port: 6379
   targetPort: 6379
   protocol: TCP
   name: client
   nodePort: 32381
 - port: 16379
   targetPort: 16379
   name: gossip
   protocol: TCP 
   nodePort: 32481
 selector:
   app: redis-cluster-2

posted @ 2025-04-01 12:40 Runner_Jack 阅读(809) 评论(0) 收藏举报

刷新页面返回顶部

Runner_Jack

k8s搭建redis集群(可在k8s以外环境访问)

公告