【K8S】Kubernetes 中的 Ingress 详解

一、Ingress 概述与核心概念

1.1 什么是 Ingress？

Ingress 是 Kubernetes 中管理外部访问集群内部服务的 API 对象，它提供了 HTTP 和 HTTPS 路由的七层负载均衡能力。与 Service 的四层负载均衡不同，Ingress 能够基于主机名、路径等应用层信息进行更精细的流量路由。

1.2 为什么需要 Ingress？

在 Kubernetes 中，服务暴露的传统方式存在局限性：

1. NodePort：需要在每个节点开放端口，难以管理且存在端口冲突风险
2. LoadBalancer：每个服务都需要独立的云负载均衡器，成本高昂
3. 缺乏应用层路由：无法基于域名、路径等七层信息进行路由

Ingress 解决了这些问题，提供了统一的应用层入口网关。

1.3 Ingress 核心组件

Ingress 资源：声明式的路由规则配置
Ingress Controller：实际的流量处理组件

二、Ingress 资源规范详解

2.1 API 版本演进

# networking.k8s.io/v1 (Kubernetes 1.19+)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: example.com
    http:
      paths:
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 80

# extensions/v1beta1 (已废弃)
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: deprecated-ingress

2.2 核心字段解析

2.2.1 rules 字段：路由规则定义

rules:
- host: "api.example.com"  # 主机名匹配
  http:
    paths:
    - path: "/v1/users"     # 路径匹配
      pathType: Prefix      # 路径匹配类型
      backend:
        service:
          name: user-service
          port:
            number: 8080
    - path: "/v1/orders"
      pathType: Exact       # 精确匹配
      backend:
        service:
          name: order-service
          port:
            number: 8080

pathType 类型详解：

• Prefix：前缀匹配（默认），按路径分段匹配
• Exact：精确匹配，路径必须完全一致
• ImplementationSpecific：由 Ingress Controller 决定匹配方式

2.2.2 TLS 配置：HTTPS 终止

tls:
- hosts:
  - "secure.example.com"
  secretName: example-tls-secret  # 包含 TLS 证书的 Secret

TLS Secret 创建：

kubectl create secret tls example-tls-secret \
  --cert=path/to/cert.pem \
  --key=path/to/key.pem

2.2.3 默认后端：兜底路由

spec:
  defaultBackend:  # 无匹配规则时的默认路由
    service:
      name: default-service
      port:
        number: 80

三、Ingress Controller 深度解析

3.1 常见 Ingress Controller 对比

Controller	底层技术	特点	适用场景
Nginx Ingress	Nginx	性能优秀，功能丰富	通用场景
Traefik	Go	动态配置，自动服务发现	微服务，容器化
HAProxy Ingress	HAProxy	高性能，稳定	高并发要求
Istio Gateway	Envoy	服务网格集成	微服务治理
ALB Ingress	AWS ALB	云原生集成	AWS 环境

3.2 Nginx Ingress Controller 架构

3.2.1 控制器工作流程

// 简化的控制器逻辑
type NginxIngressController struct {
    store store.Storer
    nginx *ngx.Config
}

func (c *NginxIngressController) Run(stopCh chan struct{}) {
    // 监听资源变化
    ingHandler := handlers.NewIngressHandler(c.store, c.nginx)
    epHandler := handlers.NewEndpointHandler(c.store, c.nginx)
    
    // 启动事件处理循环
    for {
        select {
        case event := <-c.ingressEvents:
            c.handleIngressChange(event)
        case event := <-c.endpointEvents:
            c.handleEndpointChange(event)
        case <-stopCh:
            return
        }
    }
}

func (c *NginxIngressController) handleIngressChange(event watch.Event) {
    ingress := event.Object.(*networkingv1.Ingress)
    
    // 生成 NGINX 配置
    config := c.generateNginxConfig(ingress)
    
    // 更新配置并重载
    c.nginx.UpdateConfig(config)
    c.nginx.Reload()
}

3.3 配置生成机制

3.3.1 NGINX 配置模板

# nginx.conf 模板示例
http {
    upstream upstream_default-backend {
        # 动态生成的上游配置
        {{ range $endpoint := .Upstreams }}
        server {{ $endpoint.Address }}:{{ $endpoint.Port }};
        {{ end }}
    }
    
    server {
        server_name example.com;
        
        location /api/ {
            proxy_pass http://upstream_api-service;
            # 注解生成的配置
            {{ if .ProxyBufferSize }}
            proxy_buffers {{ .ProxyBufferSize }};
            {{ end }}
        }
    }
}

3.3.2 动态配置更新

// 配置渲染逻辑
func (c *NginxIngressController) generateNginxConfig(ingresses []*networkingv1.Ingress) *ngx.Config {
    config := &ngx.Config{}
    
    for _, ingress := range ingresses {
        // 处理注解
        annotations := parseAnnotations(ingress.Annotations)
        
        // 生成 server 块
        server := c.generateServerBlock(ingress, annotations)
        config.Servers = append(config.Servers, server)
        
        // 生成 upstream 块
        for _, rule := range ingress.Spec.Rules {
            for _, path := range rule.HTTP.Paths {
                upstream := c.generateUpstream(path.Backend.Service)
                config.Upstreams = append(config.Upstreams, upstream)
            }
        }
    }
    
    return config
}

四、高级配置与注解系统

4.1 Nginx Ingress 注解详解

4.1.1 负载均衡配置

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: advanced-ingress
  annotations:
    # 负载均衡算法
    nginx.ingress.kubernetes.io/load-balance: "ip_hash"
    # 会话保持时间
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "route"
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
    
    # 上游配置
    nginx.ingress.kubernetes.io/upstream-hash-by: "$binary_remote_addr"
    nginx.ingress.kubernetes.io/upstream-max-fails: "3"
    nginx.ingress.kubernetes.io/upstream-fail-timeout: "10s"
spec:
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-service
            port:
              number: 8080

4.1.2 代理与缓冲配置

metadata:
  annotations:
    # 连接超时
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "30s"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "30s"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "30s"
    
    # 缓冲配置
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "8k"
    nginx.ingress.kubernetes.io/proxy-buffering: "on"
    nginx.ingress.kubernetes.io/proxy-buffers-number: "4"
    
    # 重试机制
    nginx.ingress.kubernetes.io/proxy-next-upstream: "error timeout http_502"
    nginx.ingress.kubernetes.io/proxy-next-upstream-tries: "3"

4.2 认证与安全配置

4.2.1 基本认证

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: auth-ingress
  annotations:
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: basic-auth
    nginx.ingress.kubernetes.io/auth-realm: "Authentication Required"
spec:
  rules:
  - host: secure.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

创建认证 Secret：

# 创建认证文件
htpasswd -c auth admin
# 创建 Kubernetes Secret
kubectl create secret generic basic-auth --from-file=auth

4.2.2 OAuth 与 JWT 认证

metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-url: "https://auth.example.com/oauth2/auth"
    nginx.ingress.kubernetes.io/auth-signin: "https://auth.example.com/oauth2/start?rd=$escaped_request_uri"
    nginx.ingress.kubernetes.io/auth-response-headers: "Authorization, X-User-Id"
    
    # JWT 验证
    nginx.ingress.kubernetes.io/auth-token-secret: "jwt-secret"
    nginx.ingress.kubernetes.io/auth-token-header: "Authorization"

4.3 流量控制与限流

4.3.1 速率限制

metadata:
  annotations:
    # 连接数限制
    nginx.ingress.kubernetes.io/limit-connections: "10"
    nginx.ingress.kubernetes.io/limit-rps: "100"
    nginx.ingress.kubernetes.io/limit-rpm: "1000"
    
    # 突发限制
    nginx.ingress.kubernetes.io/limit-burst: "20"
    
    # 基于客户端IP限流
    nginx.ingress.kubernetes.io/limit-whitelist: "10.0.0.0/8"

4.3.2 熔断与健康检查

metadata:
  annotations:
    # 主动健康检查
    nginx.ingress.kubernetes.io/upstream-max-fails: "3"
    nginx.ingress.kubernetes.io/upstream-fail-timeout: "30s"
    nginx.ingress.kubernetes.io/health-check: "true"
    nginx.ingress.kubernetes.io/health-check-path: "/health"
    nginx.ingress.kubernetes.io/health-check-interval: "30s"

五、Ingress 高级特性

5.1 金丝雀发布与流量拆分

5.1.1 基于权重的流量拆分

# 主 Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: main-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "false"
spec:
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: main-service
            port:
              number: 80

# 金丝雀 Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: canary-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"  # 10% 流量
spec:
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: canary-service
            port:
              number: 80

5.1.2 基于请求头的流量拆分

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: header-canary-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "X-Canary"
    nginx.ingress.kubernetes.io/canary-by-header-value: "true"
    nginx.ingress.kubernetes.io/canary-weight: "50"

5.2 路径重写与重定向

5.2.1 路径重写

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rewrite-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  rules:
  - host: rewrite.example.com
    http:
      paths:
      - path: /api/v1(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 8080

5.2.2 SSL 重定向与 HSTS

metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/hsts: "true"
    nginx.ingress.kubernetes.io/hsts-max-age: "31536000"
    nginx.ingress.kubernetes.io/hsts-include-subdomains: "true"

5.3 自定义错误页面

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: custom-error-ingress
  annotations:
    nginx.ingress.kubernetes.io/default-backend: error-service
    nginx.ingress.kubernetes.io/custom-http-errors: "404,500,502,503"
spec:
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-service
            port:
              number: 80
---
apiVersion: v1
kind: Service
metadata:
  name: error-service
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: error-handler

六、Ingress 控制器源码分析

6.1 Nginx Ingress 控制器架构

6.1.1 主控制器循环

// 控制器主逻辑
func (n *NGINXController) Run(stopCh <-chan struct{}) {
    // 启动同步循环
    wait.Until(n.sync, n.syncPeriod, stopCh)
    
    // 启动健康检查
    go n.healthCheck.Run(stopCh)
    
    // 启动状态更新
    go n.updateStatusRunning(stopCh)
}

func (n *NGINXController) sync() {
    // 获取所有相关资源
    ings := n.store.ListIngresses()
    endpoints := n.store.GetServiceEndpoints()
    secrets := n.store.GetSecrets()
    
    // 生成配置
    config := n.generateConfig(ings, endpoints, secrets)
    
    // 检查配置变化
    if !n.runningConfig.Equal(config) {
        // 写入新配置
        if err := n.nginx.UpdateConfig(config); err != nil {
            klog.Errorf("Failed to update NGINX config: %v", err)
            return
        }
        
        // 重载 NGINX
        if err := n.nginx.Reload(); err != nil {
            klog.Errorf("Failed to reload NGINX: %v", err)
            return
        }
        
        n.runningConfig = config
    }
}

6.1.2 配置生成器

// 配置生成核心逻辑
func (n *NGINXController) generateConfig(ingresses []*ingress.Ingress, endpoints map[string][]ingress.Endpoint, secrets map[string]*v1.Secret) *ingress.Configuration {
    cfg := &ingress.Configuration{}
    
    // 处理每个 Ingress 资源
    for _, ing := range ingresses {
        // 解析注解
        anns := n.getAnnotations(ing)
        
        // 生成服务器配置
        server := n.createServer(ing, anns, endpoints)
        if server != nil {
            cfg.Servers = append(cfg.Servers, server)
        }
        
        // 生成上游配置
        for _, rule := range ing.Spec.Rules {
            if rule.HTTP == nil {
                continue
            }
            for _, path := range rule.HTTP.Paths {
                upstream := n.createUpstream(ing, path.Backend.Service, endpoints)
                cfg.Upstreams = append(cfg.Upstreams, upstream)
            }
        }
    }
    
    return cfg
}

6.2 注解解析器

// 注解解析实现
type annotationsParser struct {
    annotations map[string]string
}

func (p *annotationsParser) getString(key string) string {
    if val, exists := p.annotations[key]; exists {
        return val
    }
    return ""
}

func (p *annotationsParser) getInt(key string) int {
    strVal := p.getString(key)
    if strVal == "" {
        return 0
    }
    val, err := strconv.Atoi(strVal)
    if err != nil {
        klog.Warningf("Error parsing annotation %s: %v", key, err)
        return 0
    }
    return val
}

func (p *annotationsParser) getBool(key string) bool {
    strVal := p.getString(key)
    val, err := strconv.ParseBool(strVal)
    if err != nil {
        klog.Warningf("Error parsing annotation %s: %v", key, err)
        return false
    }
    return val
}

七、性能优化与最佳实践

7.1 性能优化策略

7.1.1 连接优化

# 高性能 Ingress 配置
metadata:
  annotations:
    # 连接优化
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "15s"
    nginx.ingress.kubernetes.io/proxy-next-upstream-timeout: "5s"
    nginx.ingress.kubernetes.io/upstream-keepalive-connections: "100"
    nginx.ingress.kubernetes.io/upstream-keepalive-timeout: "60s"
    nginx.ingress.kubernetes.io/upstream-keepalive-requests: "1000"
    
    # 缓冲优化
    nginx.ingress.kubernetes.io/proxy-buffer-size: "16k"
    nginx.ingress.kubernetes.io/proxy-buffers-number: "8"
    nginx.ingress.kubernetes.io/proxy-buffering: "on"
    
    # 压缩优化
    nginx.ingress.kubernetes.io/gzip: "on"
    nginx.ingress.kubernetes.io/gzip-types: "application/json application/javascript text/css"

7.1.2 资源分配优化

# Ingress Controller 资源分配
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
spec:
  template:
    spec:
      containers:
      - name: controller
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1000m"
        # 启用 HPA
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: nginx-ingress-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx-ingress-controller
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

7.2 安全最佳实践

7.2.1 网络安全策略

# 网络策略限制访问
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: ingress-allow-only
spec:
  podSelector:
    matchLabels:
      app: nginx-ingress
  policyTypes:
  - Ingress
  ingress:
  - ports:
    - protocol: TCP
      port: 80
    - protocol: TCP
      port: 443
    from:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 10.0.0.0/8  # 排除内网

7.2.2 TLS 安全配置

metadata:
  annotations:
    # 现代 TLS 配置
    nginx.ingress.kubernetes.io/ssl-ciphers: "ECDHE-RSA-AES128-GCM-SHA256,ECDHE-ECDSA-AES128-GCM-SHA256"
    nginx.ingress.kubernetes.io/ssl-protocols: "TLSv1.2 TLSv1.3"
    nginx.ingress.kubernetes.io/ssl-prefer-server-ciphers: "true"
    nginx.ingress.kubernetes.io/hsts: "true"
    nginx.ingress.kubernetes.io/hsts-max-age: "63072000"

八、监控与故障排查

8.1 监控指标收集

8.1.1 Prometheus 监控

# ServiceMonitor 配置
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: nginx-ingress-monitor
  labels:
    app: nginx-ingress
spec:
  selector:
    matchLabels:
      app: nginx-ingress
  endpoints:
  - port: metrics
    interval: 30s
    path: /metrics

关键监控指标：

• nginx_ingress_controller_requests：请求数量
• nginx_ingress_controller_ingress_upstream_latency_seconds：上游延迟
• nginx_ingress_controller_response_duration_seconds：响应时间
• nginx_ingress_controller_connections：连接数

8.2 日志配置

# 详细日志配置
metadata:
  annotations:
    nginx.ingress.kubernetes.io/enable-access-log: "true"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      access_log /var/log/nginx/access.log main;
      error_log /var/log/nginx/error.log warn;
      
      # 自定义日志格式
      log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for" '
                      'upstream_addr=$upstream_addr '
                      'request_time=$request_time '
                      'upstream_response_time=$upstream_response_time';

8.3 故障排查工具

# 检查 Ingress 状态
kubectl describe ingress my-ingress

# 查看 Ingress Controller 日志
kubectl logs -l app=nginx-ingress -n ingress-nginx

# 检查配置生成
kubectl exec -it nginx-ingress-controller-pod -- cat /etc/nginx/nginx.conf

# 测试路由规则
curl -v -H "Host: app.example.com" http://ingress-ip/api/health

九、总结

Kubernetes Ingress 提供了强大而灵活的应用层流量管理能力。通过深入理解其架构原理、配置机制和最佳实践，可以构建出高性能、高可用的入口网关解决方案。随着云原生技术的发展，Ingress 将继续演进，提供更丰富的功能和更好的性能表现。