K8s Ingress-nginx 部署和实例演示

这里直接是安装和演示的过程，概念理论部分自行到官网学习，就不再赘述了。

Ingress 介绍：https://kubernetes.io/zh/docs/concepts/services-networking/ingress/

一、k8s 安装 ingress-nginx

下载 Ingress-nginx yaml文件

[root@k8s-master01 ingress-nginx]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
[root@k8s-master01 ingress-nginx]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/baremetal/service-nodeport.yaml
[root@k8s-master01 ingress-nginx]# ls
mandatory.yaml  service-nodeport.yaml

mandatory.yaml下载后需要添加hostNetwork: true，否则无法通过 K8s 节点 IP 地址绑定域名外部访问。

ingress-controller 会直接使用 K8s 物理机的 DNS 来解析域名，而不再使用 K8s 内部的 DNS 来解析域名。

mandatory.yaml配置修改如下：

...
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      # wait up to five minutes for the drain of connections
      # 添加 hostNetwork: true
      hostNetwork: true
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        kubernetes.io/os: linux

service-nodeport.yaml配置

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

修改好后，开始安装 Ingress-nginx

[root@k8s-master01 ingress-nginx]# kubectl apply -f mandatory.yaml
[root@k8s-master01 ingress-nginx]# kubectl apply -f service-nodeport.yaml

查看 Ingress Pod 是否 Running 状态

[root@k8s-master01 ingress-nginx]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-6979d75b9d-fqf8v   1/1     Running   0          73s

二、创建实例测试 Ingress

my-nginx.yml 配置文件：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      run: my-nginx
  replicas: 1
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx:latest
        resources:
          limits:
            memory: "128Mi"
            cpu: "500m"
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    run: my-nginx
  type: ClusterIP
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
    - host: test.ingress.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx-service
                port:
                  number: 8080

创建 Deployment、service、ingress

[root@k8s-master01 ingress-nginx]# kubectl apply -f my-nginx.yml 
deployment.apps/my-nginx created
service/nginx-service created
ingress.networking.k8s.io/example-ingress created

检查是否创建成功

[root@k8s-master01 ingress-nginx]# kubectl get ingress -o wide
NAME              CLASS    HOSTS              ADDRESS        PORTS   AGE
example-ingress   <none>   test.ingress.com   10.233.31.38   80      40s
[root@k8s-master01 ingress-nginx]# kubectl get pod 
NAME                        READY   STATUS    RESTARTS   AGE
my-nginx-6d97ddfff5-7jkp6   1/1     Running   0          44s
[root@k8s-master01 ingress-nginx]# kubectl get svc
NAME            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
kubernetes      ClusterIP   10.233.0.1     <none>        443/TCP    30d
nginx-service   ClusterIP   10.233.39.79   <none>        8080/TCP   48s

nginx-ingress-controller所在的节点与域名做好hosts绑定，就可以访问后端 Pod 服务了

[root@k8s-master01 ingress-nginx]# kubectl get pod -n ingress-nginx -o wide
NAME                                        READY   STATUS    RESTARTS   AGE     IP              NODE         NOMINATED NODE   READINESS GATES
nginx-ingress-controller-6979d75b9d-fqf8v   1/1     Running   0          9m49s   192.168.3.248   k8s-node01   <none>           <none>
[root@k8s-master01 ingress-nginx]# vim /etc/hosts
[root@k8s-master01 ingress-nginx]# tail -1 /etc/hosts
192.168.3.248 test.ingress.com

我这里nginx-ingress-controller所在的节点是 k8s-node01，它的IP地址是192.168.3.248，所以把刚刚自定义的域名进行绑定解析。

访问成功test.ingress.com域名

三、优化 Ingress-nginx

通过上述的实验，我们可以发现默认是 Deployment 类型，将 Ingress 改为 DaemonSet 类似，让每个工作节点运行一个 ingress-controller，这样可以达到高可用效果，某个节点宕机了，不会影响 Ingress 工作。

其实生成环境的做法是在 K8s 工作节点上安装 keepalive ，通过 VIP 作为对外 IP地址与域名解析。

• 将 Deployment 改为 DaemonSet；并删掉 replicas；
• 添加字段 hostNetwork: true（刚刚在上面已经添加了）
• nodeSelector：给工作节点打上标签，master 节点不会部署。

把之前的 ingress-controller 删除掉，修改配置文件后，重新创建：

[root@k8s-master01 ingress-nginx]# kubectl delete -f mandatory.yaml

修改 mandatory.yaml 配置文件：

[root@k8s-master01 ingress-nginx]# vim mandatory.yaml

对 master 设置污点，其他工作节点都部署

[root@k8s-master01 ingress-nginx]# kubectl get nodes
NAME           STATUS   ROLES                  AGE   VERSION
k8s-master01   Ready    control-plane,master   30d   v1.21.5
k8s-node01     Ready    worker                 30d   v1.21.5
k8s-node02     Ready    worker                 30d   v1.21.5
[root@k8s-master01 ingress-nginx]# kubectl taint nodes k8s-master01 node-role.kubernetes.io/master=true:NoSchedule
node/k8s-master01 tainted

重新创建 Ingress-nginx，以及查看状态

[root@k8s-master01 ingress-nginx]# kubectl apply -f mandatory.yaml
[root@k8s-master01 ingress-nginx]# kubectl get pod -n ingress-nginx -o wide

无论域名绑定那个工作节点IP地址，都可以解析到后端的 Pod 服务了，一个节点宕机了不会影响 Ingress 工作。

最后，以上属于测试环境实验，想要真正的 Ingress 高可用，还是得结合 Keepalived 或者 SLB 一起实现。