Allow Envoy Gateway to use cloud LoadBalancer providers and be deployed in arbitrary clusters

[wtripp180901]

In Kubernetes clusters containing cloud-specific providers for provisioning LoadBalancers instead of the default MetalLB provider, modifications to the LoadBalancer service deployed by EnvoyProxy are often required. To accommodate these requirements this change

• Exposes kuberentes.envoyService.annotations to allow injecting provider specific annotations
• For load balancer providers which still rely on the deprecated Service spec.loadBalancerIP field (e.g Openstack CCM), exposes EnvoyProxy kubernetes.envoyService.patch to make arbitrary patches to the underlying Service manifest

Also makes RKE2 and cluster-bloom specific config optional to allow envoy-gateway-config to be deployed in existing clusters where routing config for loadbalancing is handled elsewhere

[brownzebra]

Hi, sorry for delay on this, can you show any test evidence and we can take this in?

[wtripp180901]

For the example of Openstack using the Openstack Cloud Controller Manager as a load balancer provider, my config of

apps:
  envoy-gateway-config:
    helmParameters:
      - name: rkeCoreDNS.enabled
        value: false
      - name: domain
        value: "{{ .Values.global.domain }}"
    valuesObject:
      appsGateway:
        nodeSelector: null
        envoyServicePatch:
          type: StrategicMerge
          value:
            spec:
              loadBalancerIP: <my-dev-ip>

gives an EnvoyProxy resource with spec

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  annotations:
    argocd.argoproj.io/tracking-id: envoy-gateway-config:gateway.envoyproxy.io/EnvoyProxy:envoy-gateway-system/access-logging-config
  creationTimestamp: "2026-09-16T08:46:43Z"
  generation: 2
  name: access-logging-config
  namespace: envoy-gateway-system
  resourceVersion: "29079"
  uid: e1faef11-f507-4c80-b166-c216a5a8f281
spec:
  filterOrder:
  - before: envoy.filters.http.ext_authz
    name: envoy.filters.http.lua
  logging:
    level:
      default: warn
  provider:
    kubernetes:
      envoyDeployment:
        pod:
          priorityClassName: system-cluster-critical
      envoyService:
        externalTrafficPolicy: Local
        patch:
          type: StrategicMerge
          value:
            spec:
              loadBalancerIP: <my-dev-ip>
        type: LoadBalancer
    type: Kubernetes
  telemetry:
    accessLog:
      settings:
      - format:
          json:
            aim_service_id: '%REQ(x-aim-service-id)%'
            api_key_id: '%REQ(x-api-key-id)%'
            authority: '%REQ(:AUTHORITY)%'
            backendCluster: '%UPSTREAM_CLUSTER%'
            backendHost: '%UPSTREAM_HOST%'
            bytes_received: '%BYTES_RECEIVED%'
            bytes_sent: '%BYTES_SENT%'
            llm_input_token: '%DYNAMIC_METADATA(io.envoy.ai_gateway:llm_input_token)%'
            llm_output_token: '%DYNAMIC_METADATA(io.envoy.ai_gateway:llm_output_token)%'
            llm_total_token: '%DYNAMIC_METADATA(io.envoy.ai_gateway:llm_total_token)%'
            method: '%REQ(X-ENVOY-ORIGINAL-METHOD?:METHOD)%'
            model: '%REQ(x-ai-eg-model)%'
            path: '%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%'
            protocol: '%PROTOCOL%'
            req_x_forwarded_for: '%REQ(X-FORWARDED-FOR)%'
            request_id: '%REQ(X-REQUEST-ID)%'
            resp_backend_service_time: '%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%'
            response_code: '%RESPONSE_CODE%'
            response_flags: '%RESPONSE_FLAGS%'
            start_time: '%START_TIME%'
            total_duration: '%DURATION%'
            user_agent: '%REQ(USER-AGENT)%'
          type: JSON
        sinks:
        - file:
            path: /dev/stdout
          type: File

which creates a loadbalancer service with spec

apiVersion: v1
kind: Service
metadata:
  annotations:
    loadbalancer.openstack.org/load-balancer-address: <my-dev-ip>
    loadbalancer.openstack.org/load-balancer-id: ed4d101b-b2a6-4d42-9274-c87c1e7946db
  creationTimestamp: "2026-09-16T08:47:35Z"
  finalizers:
  - service.kubernetes.io/load-balancer-cleanup
  labels:
    app.kubernetes.io/component: proxy
    app.kubernetes.io/managed-by: envoy-gateway
    app.kubernetes.io/name: envoy
    gateway.envoyproxy.io/owning-gateway-name: https
    gateway.envoyproxy.io/owning-gateway-namespace: envoy-gateway-system
  name: envoy-envoy-gateway-system-https-5a118663
  namespace: envoy-gateway-system
  ownerReferences:
  - apiVersion: gateway.networking.k8s.io/v1
    kind: GatewayClass
    name: envoy-gateway
    uid: 7f04f3ae-4d3a-44ce-b860-163c6fe0ac74
  resourceVersion: "29130"
  uid: 528eaea8-e3f9-4093-ab33-c7c738a0883f
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 172.29.237.56
  clusterIPs:
  - 172.29.237.56
  externalTrafficPolicy: Local
  healthCheckNodePort: 32688
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  loadBalancerIP: <my-dev-ip>
  ports:
  - name: https-443
    nodePort: 32069
    port: 443
    protocol: TCP
    targetPort: 10443
  selector:
    app.kubernetes.io/component: proxy
    app.kubernetes.io/managed-by: envoy-gateway
    app.kubernetes.io/name: envoy
    gateway.envoyproxy.io/owning-gateway-name: https
    gateway.envoyproxy.io/owning-gateway-namespace: envoy-gateway-system
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: <my-dev-ip>
      ipMode: VIP

and is recognised by OCCM which provisions an Octavia loadbalancer in the Openstack project

I0916 09:08:44.827584      13 controller.go:402] Ensuring load balancer for service envoy-gateway-system/envoy-envoy-gateway-system-https-5a118663
I0916 09:08:44.831127      13 loadbalancer.go:1863] "EnsureLoadBalancer" cluster="aiwbtest" service="envoy-gateway-system/envoy-envoy-gateway-system-https-5a118663"
I0916 09:08:44.831533      13 event.go:389] "Event occurred" object="envoy-gateway-system/envoy-envoy-gateway-system-https-5a118663" fieldPath="" kind="Service" apiVersion="v1" type="Normal" reason="EnsuringLoadBalancer" message="Ensuring load balancer"
I0916 09:08:45.112702      13 event.go:389] "Event occurred" object="envoy-gateway-system/envoy-envoy-gateway-system-https-5a118663" fieldPath="" kind="Service" apiVersion="v1" type="Normal" reason="EnsuredLoadBalancer" message="Ensured load balancer"

[brownzebra]

@rodrodsilo

[rodrodsilo]

Tested successfully. Thanks for contribution, this will help to provide a more agnostic approach on components like envoy-gateway. Thus end users can deploy easily on cloud loadbalancers too.

[rodrodsilo]

@wtripp180901 Please merge main into your branch and resolve sources/envoy-gateway-config/values.yaml by keeping both bodyAuthMaxRequestBytes (from main) and nodeSelector / rkeCoreDNS (from the PR). Then I can merge.