Ingress Controller 与 Service Mesh：现代Kubernetes流量管理的双重奏

老马啸西风2025/8/31大约 7 分钟

在现代Kubernetes生态系统中，流量管理已经从简单的Service负载均衡发展为复杂的多层流量控制体系。Ingress Controller和Service Mesh作为两个关键组件，分别在集群边缘和内部服务间通信中发挥着重要作用。本文将深入探讨Ingress Controller与Service Mesh的工作原理、协同关系以及在云原生应用中的最佳实践。

Ingress Controller详解

Ingress Controller是Kubernetes中用于管理外部访问集群服务的组件，它通过Ingress资源定义路由规则，将外部流量路由到集群内部的Service。

Ingress资源定义

1. 基础Ingress配置

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 80
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

2. TLS配置

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tls-example-ingress
spec:
  tls:
  - hosts:
    - example.com
    secretName: example-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

主流Ingress Controller

1. NGINX Ingress Controller

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
    spec:
      containers:
      - name: nginx-ingress-controller
        image: k8s.gcr.io/ingress-nginx/controller:v1.1.0
        args:
        - /nginx-ingress-controller
        - --configmap=$(POD_NAMESPACE)/nginx-configuration
        - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
        - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443

2. Traefik Ingress Controller

apiVersion: apps/v1
kind: Deployment
metadata:
  name: traefik-ingress-controller
spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik-ingress
  template:
    metadata:
      labels:
        app: traefik-ingress
    spec:
      serviceAccountName: traefik-ingress-controller
      containers:
      - name: traefik-ingress
        image: traefik:v2.6
        args:
        - --api.insecure
        - --providers.kubernetesingress
        - --providers.kubernetescrd
        - --entrypoints.web.address=:80
        - --entrypoints.websecure.address=:443
        ports:
        - name: web
          containerPort: 80
        - name: websecure
          containerPort: 443
        - name: admin
          containerPort: 8080

3. HAProxy Ingress Controller

apiVersion: apps/v1
kind: Deployment
metadata:
  name: haproxy-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      run: haproxy-ingress
  template:
    metadata:
      labels:
        run: haproxy-ingress
    spec:
      serviceAccountName: haproxy-ingress
      containers:
      - name: haproxy-ingress
        image: haproxytech/kubernetes-ingress:1.7
        args:
        - --configmap=default/haproxy-configmap
        - --default-backend-service=default/haproxy-default-backend
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        - name: stat
          containerPort: 1024
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace

Ingress高级特性

1. 路由重写

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rewrite-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /api(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 80

2. 速率限制

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rate-limit-ingress
  annotations:
    nginx.ingress.kubernetes.io/rate-limit-connections: "10"
    nginx.ingress.kubernetes.io/rate-limit-rps: "5"
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

3. 负载均衡策略

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: load-balancing-ingress
  annotations:
    nginx.ingress.kubernetes.io/upstream-hash-by: "$request_uri"
    nginx.ingress.kubernetes.io/load-balance: "least_conn"
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Service Mesh概述

Service Mesh是一种专门处理服务间通信的基础设施层，它负责处理服务之间的网络通信、安全、监控和治理等问题。

核心组件

1. 数据平面（Data Plane）

数据平面由一组代理（如Envoy）组成，它们与应用程序部署在一起，处理服务间的所有网络通信。

2. 控制平面（Control Plane）

控制平面负责管理和配置数据平面中的代理，实现流量管理、安全策略、监控等功能。

Istio架构

1. 数据平面组件

apiVersion: apps/v1
kind: Deployment
metadata:
  name: productpage-v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: productpage
      version: v1
  template:
    metadata:
      labels:
        app: productpage
        version: v1
      annotations:
        sidecar.istio.io/inject: "true"
    spec:
      containers:
      - name: productpage
        image: docker.io/istio/examples-bookinfo-productpage-v1:1.16.2
        ports:
        - containerPort: 9080

2. 控制平面组件

apiVersion: apps/v1
kind: Deployment
metadata:
  name: istiod
  namespace: istio-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: istiod
  template:
    metadata:
      labels:
        app: istiod
    spec:
      containers:
      - name: discovery
        image: docker.io/istio/pilot:1.12.0
        args:
        - "discovery"
        - "--monitoringAddr=:15014"
        - "--log_output_level=default:info"
        - "--domain"
        - "cluster.local"
        ports:
        - containerPort: 8080
        - containerPort: 15010
        - containerPort: 15017

Service Mesh核心功能

1. 流量管理

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
spec:
  hosts:
  - bookinfo.com
  http:
  - route:
    - destination:
        host: productpage
        port:
          number: 9080

2. 故障注入

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ratings-fault-delay
spec:
  hosts:
  - ratings
  http:
  - fault:
      delay:
        percent: 100
        fixedDelay: 2s
    route:
    - destination:
        host: ratings
        port:
          number: 9080

3. 金丝雀发布

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 90
    - destination:
        host: reviews
        subset: v2
      weight: 10

Ingress Controller与Service Mesh的协同

架构整合

1. 边缘到内部的完整流量链路

外部客户端 -> Ingress Controller -> Service Mesh Gateway -> 应用Pod (Sidecar) -> 目标服务

2. 配置示例

# Ingress配置
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mesh-ingress
  annotations:
    nginx.ingress.kubernetes.io/service-upstream: "true"
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80

---
# Istio Gateway配置
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: bookinfo-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "example.com"

---
# VirtualService配置
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
spec:
  hosts:
  - "example.com"
  gateways:
  - bookinfo-gateway
  http:
  - match:
    - uri:
        prefix: /api
    route:
    - destination:
        host: api-service
        port:
          number: 80
  - route:
    - destination:
        host: web-service
        port:
          number: 80

流量管理策略

1. 分层流量控制

# Ingress层的速率限制
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rate-limited-ingress
  annotations:
    nginx.ingress.kubernetes.io/rate-limit-connections: "100"
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80

---
# Service Mesh层的细粒度控制
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: fine-grained-control
spec:
  hosts:
  - api-service
  http:
  - match:
    - headers:
        user-type:
          exact: "premium"
    route:
    - destination:
        host: premium-api-service
        port:
          number: 80
  - route:
    - destination:
        host: standard-api-service
        port:
          number: 80
    retries:
      attempts: 3
      perTryTimeout: 2s

2. 安全策略整合

# Ingress层的TLS终止
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tls-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - example.com
    secretName: example-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80

---
# Service Mesh层的mTLS
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
spec:
  mtls:
    mode: STRICT

高级特性与最佳实践

1. 多集群服务发现

# 跨集群ServiceEntry
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-svc
spec:
  hosts:
  - external-service.example.com
  location: MESH_EXTERNAL
  ports:
  - number: 443
    name: https
    protocol: TLS
  resolution: DNS

2. 可观测性集成

# 分布式追踪配置
apiVersion: telemetry.istio.io/v1alpha1
kind: Telemetry
metadata:
  name: mesh-default
  namespace: istio-system
spec:
  tracing:
  - providers:
    - name: "otel"
    randomSamplingPercentage: 100.0

3. 性能优化

# Envoy代理资源配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: istio-sidecar-injector
  namespace: istio-system
data:
  values: |-
    global:
      proxy:
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 2000m
            memory: 1024Mi

监控与故障排除

关键监控指标

1. Ingress Controller指标

# 请求速率
nginx_ingress_controller_requests_total

# 响应时间
nginx_ingress_controller_request_duration_seconds

# 连接数
nginx_ingress_controller_connections

# SSL指标
nginx_ingress_controller_ssl_expire_time_seconds

2. Service Mesh指标

# 请求指标
istio_requests_total
istio_request_duration_milliseconds

# TCP指标
istio_tcp_sent_bytes_total
istio_tcp_received_bytes_total

# 代理指标
envoy_cluster_upstream_cx_active
envoy_server_live

故障排除策略

1. 流量路径诊断

# 检查Ingress状态
kubectl get ingress

# 检查Ingress Controller日志
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx

# 检查Service Mesh配置
istioctl proxy-config routes <pod-name>.<namespace>

2. 网络连通性测试

# 测试外部访问
curl -H "Host: example.com" http://<ingress-ip>/

# 测试内部服务连通性
kubectl exec -it <source-pod> -c istio-proxy -- curl http://<target-service>.<namespace>:80/

# 检查Sidecar状态
istioctl proxy-status

最佳实践

1. 安全配置

# Ingress安全配置
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: secure-ingress
  annotations:
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-origin: "https://example.com"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
  - hosts:
    - example.com
    secretName: example-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: istio-ingressgateway
            port:
              number: 80

2. 高可用部署

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - nginx-ingress
              topologyKey: kubernetes.io/hostname

3. 资源管理

# 资源限制配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
data:
  worker-processes: "4"
  worker-connections: "65536"
  keep-alive: "60"
  keep-alive-requests: "10000"

总结

Ingress Controller和Service Mesh作为现代Kubernetes流量管理的两个重要组成部分，各自承担着不同的职责但又相互补充。Ingress Controller主要负责集群边缘的流量管理，而Service Mesh则专注于集群内部的服务间通信。

通过合理的架构设计和配置优化，可以构建出高性能、高可用、安全可靠的云原生应用架构。随着云原生技术的不断发展，这两项技术也在持续演进，为构建复杂的分布式系统提供了更好的支撑。

在实际应用中，需要根据具体的业务需求、技术栈和运维能力来选择合适的方案，并建立完善的监控和告警机制，确保系统的稳定运行。未来，随着边缘计算、多云部署等技术的发展，Ingress Controller和Service Mesh将在更广泛的场景中发挥重要作用。