Kubernetes Gateway API

Kubernetes Gateway API

Gateway API 是 Kubernetes 1.19 版本引入的一种新的 API 规范，会成为 Ingress 的下一代替代方案。主要原因是 Ingress 资源对象不能很好的满足网络需求，很多场景下 Ingress 控制器都需要通过定义 annotations 或者 crd 来进行功能扩展，这对于使用标准和支持是非常不利的，新推出的 Gateway API 旨在通过可扩展的面向角色的接口来增强服务网络。

2022 年 5 月份Kubernetes Gateway API才发布了 Beta 版本，当前大多数组织应该还在使用稳定的 Ingress API。

官方地址：https://gateway-api.sigs.k8s.io/

官方仓库：https://github.com/kubernetes-sigs/gateway-api

安装 Gateway API CRDS

k8s 版本：v1.30.1

Gateway API 版本：v1.2.1

$ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created

$ kubectl api-resources | grep gateway
gatewayclasses                      gc           gateway.networking.k8s.io/v1        false        GatewayClass
gateways                            gtw          gateway.networking.k8s.io/v1        true         Gateway
grpcroutes                                       gateway.networking.k8s.io/v1        true         GRPCRoute
httproutes                                       gateway.networking.k8s.io/v1        true         HTTPRoute
referencegrants                     refgrant     gateway.networking.k8s.io/v1beta1   true         ReferenceGrant

安装 Gateway API Controller

Gateway API 只是安装了一些 crd，实现了上层 api，真正的 Gateway API 下游实现（包含 GatewayClass 资源），即基础设备供应商，类似 ingress-controller，支持列表参考 Gateway Controller - Kubernetes Gateway API

这里安装 Envoy Gateway API Controller 为例。

安装metallb LoadBalancer

首先安装metallb LoadBalancer，参考地址：https://metallb.universe.tf/installation/ ：

# 首先 kube-proxy 如果使用 ipvs 模式，则启用严格 ARP：
$ cat /etc/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
WorkingDirectory=/var/lib/kube-proxy
ExecStart=/opt/kube/bin/kube-proxy \
  --config=/var/lib/kube-proxy/kube-proxy-config.yaml
Restart=always
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

# strictARP: False 改为 strictARP: True
vi /var/lib/kube-proxy/kube-proxy-config.yaml
kind: KubeProxyConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
clientConnection:
  kubeconfig: "/etc/kubernetes/kube-proxy.kubeconfig"
# 根据clusterCIDR 判断集群内部和外部流量，配置clusterCIDR选项后，kube-proxy 会对访问 Service IP 的请求做 SNAT
clusterCIDR: "172.20.0.0/16"
conntrack:
  maxPerCore: 32768
  min: 131072
  tcpCloseWaitTimeout: 1h0m0s
  tcpEstablishedTimeout: 24h0m0s
healthzBindAddress: 0.0.0.0:10256
# hostnameOverride 值必须与 kubelet 的对应一致，否则 kube-proxy 启动后会找不到该 Node，从而不会创建任何 iptables 规则
hostnameOverride: "k8s-10-10-10-147"
metricsBindAddress: 0.0.0.0:10249
mode: "ipvs"
ipvs:
  excludeCIDRs: null
  minSyncPeriod: 0s
  scheduler: ""
  strictARP: True
  syncPeriod: 30s
  tcpFinTimeout: 0s
  tcpTimeout: 0s
  udpTimeout: 0s

$ systemctl daemon-reload
$ systemctl restart kube-proxy.service

$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.8/config/manifests/metallb-native.yaml
namespace/metallb-system created
customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created
customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created
customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/servicel2statuses.metallb.io created
serviceaccount/controller created
serviceaccount/speaker created
role.rbac.authorization.k8s.io/controller created
role.rbac.authorization.k8s.io/pod-lister created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/controller created
rolebinding.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
configmap/metallb-excludel2 created
secret/metallb-webhook-cert created
service/metallb-webhook-service created
deployment.apps/controller created
daemonset.apps/speaker created
validatingwebhookconfiguration.admissionregistration.k8s.io/metallb-webhook-configuration created

$ kubectl -n metallb-system get pod
NAME                          READY   STATUS    RESTARTS   AGE
controller-6dd967fdc7-9vzk7   1/1     Running   0          4m9s
speaker-tfn99                 1/1     Running   0          4m9s
  • Controler：Deployment，用于监听 Service 的变更，分配/回收 IP 地址。
  • Speaker：DaemonSet，对外广播 Service 的 IP 地址。

首先创建一个地址池：

# vi ip-pool.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: ip-pool
  namespace: metallb-system
spec:
  addresses:
    - 10.10.10.200-10.10.10.245
  • 注意：提供的 IP 段必须跟 k8s 集群主机所在同一个网段中，且尚未使用的 IP 段

创建一个广播声明，关联上面的 IP 池对象：

# vi advertise.yaml
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: l2adver
  namespace: metallb-system
spec:
  ipAddressPools:
    - ip-pool
  • 如果不设置关联到 IPAdressPool，L2Advertisement 默认会关联所有可用的 IPAdressPool

应用：

kubectl apply -f ip-pool.yaml
kubectl apply -f advertise.yaml

安装 Envoy Gateway API Controller

官方地址：https://gateway.envoyproxy.io/docs/tasks/quickstart/

helm install eg --create-namespace oci://docker.io/envoyproxy/gateway-helm --version v0.0.0-latest -n envoy-gateway-system --skip-crds

  • --version v0.0.0-latest 最新版本，也可以指定版本 --version v1.2.4

  • --skip-crds 跳过 Gateway API CRDS ，前面已经安装

$ kubectl -n envoy-gateway-system get all
NAME                                 READY   STATUS    RESTARTS   AGE
pod/envoy-gateway-7774bd75b7-ksk4z   1/1     Running   0          2m27s

NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                   AGE
service/envoy-gateway   ClusterIP   10.68.158.175   <none>        18000/TCP,18001/TCP,18002/TCP,19001/TCP   2m27s

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/envoy-gateway   1/1     1            1           2m27s

NAME                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/envoy-gateway-7774bd75b7   1         1         1       2m27s

创建 GatewayClass：

# vi GatewayClass.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller

创建 deployment 示例：

# vi backend.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: backend
---
apiVersion: v1
kind: Service
metadata:
  name: backend
  labels:
    app: backend
    service: backend
spec:
  ports:
    - name: http
      port: 80
      targetPort: 80
  selector:
    app: backend
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: backend
      version: v1
  template:
    metadata:
      labels:
        app: backend
        version: v1
    spec:
      serviceAccountName: backend
      containers:
        - image: nginx:latest
          imagePullPolicy: IfNotPresent
          name: backend
          ports:
            - containerPort: 80
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace

创建 Gateway 与 HTTPRoute

# vi frontend.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: eg
spec:
  gatewayClassName: eg
  listeners:
    - name: http
      protocol: HTTP
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: backend
spec:
  parentRefs:
    - name: eg
  hostnames:
    - "www.example.com"
  rules:
    - backendRefs:
        - group: ""
          kind: Service
          name: backend
          port: 80
          weight: 1
      matches:
        - path:
            type: PathPrefix
            value: /

应用：

$ kubectl apply -f GatewayClass.yaml 
gatewayclass.gateway.networking.k8s.io/eg created

$ kubectl get Gatewayclass
NAME   CONTROLLER                                      ACCEPTED   AGE
eg     gateway.envoyproxy.io/gatewayclass-controller   True       11s




$ kubectl apply -f backend.yaml 
serviceaccount/backend created
service/backend created
deployment.apps/backend created

$ kubectl get all
NAME                           READY   STATUS    RESTARTS   AGE
pod/backend-59c5fd8d48-qgd4k   1/1     Running   0          25s

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/backend      ClusterIP   10.68.184.237   <none>        80/TCP    25s
service/kubernetes   ClusterIP   10.68.0.1       <none>        443/TCP   6h24m

NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/backend   1/1     1            1           25s

NAME                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/backend-59c5fd8d48   1         1         1       25s



$ kubectl apply -f frontend.yaml 
gateway.gateway.networking.k8s.io/eg created
httproute.gateway.networking.k8s.io/backend created

# 可以看到配置完成后，由 GatewayClass 绑定的 Controller 为我们提供一个具体存在 Pod 作为流量入口，
# 需要注意的是，各家实现在此处还是略有不同，比如说 Envoy 当你创建 Gateway 资源后，Envoy Controller 
# 会创建一个 Deployment 资源为你提供入口流量 Pod ，然而 Nginx 则是自己本身就是流量入口 Pod 不会创建新的。
$ kubectl -n envoy-gateway-system get pod
NAME                                         READY   STATUS    RESTARTS   AGE
envoy-default-eg-e41e7b31-5dbf9db8bf-bh5zd   2/2     Running   0          116s
envoy-gateway-7774bd75b7-ksk4z               1/1     Running   0          20m

$ kubectl -n envoy-gateway-system get deployment
NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
envoy-default-eg-e41e7b31   1/1     1            1           2m
envoy-gateway               1/1     1            1           20m

$ kubectl -n envoy-gateway-system get svc
NAME                        TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                                   AGE
envoy-default-eg-e41e7b31   LoadBalancer   10.68.238.117   10.10.10.200   80:31762/TCP                              2m4s
envoy-gateway               ClusterIP      10.68.158.175   <none>         18000/TCP,18001/TCP,18002/TCP,19001/TCP   20m

$ kubectl get gateway
NAME   CLASS   ADDRESS        PROGRAMMED   AGE
eg     eg      10.10.10.200   True         4m36s

$ kubectl get httproute
NAME      HOSTNAMES             AGE
backend   ["www.example.com"]   4m38s

访问：

export GATEWAY_HOST=$(kubectl get gateway/eg -o jsonpath='{.status.addresses[0].value}')
curl --verbose --header "Host: www.example.com" http://$GATEWAY_HOST/
posted @ 2024-12-23 11:09  leffss  阅读(137)  评论(0)    收藏  举报