Skip to content

Services of type: LoadBalancer provisioned by cluster-provider-stackit do not attach required security groups to workload nodes #1185

@voigt

Description

@voigt

cloud-provider-stackit creates a STACKIT Load Balancer for a Kubernetes Service of type LoadBalancer, and the Load Balancer reaches READY, but the service is not reachable from outside the cluster.

The issue appears to be that the required security groups are created, but not attached to the respective workload nodes.

Steps to reproduce

  1. Create a workload cluster on STACKIT VMs.

  2. Run:

    kubectl create deployment nginx --image=nginx --port=80
    kubectl expose deployment nginx \
      --port=80 \
      --target-port=80 \
      --type=LoadBalancer
  3. Wait for an external IP:

    kubectl get svc nginx
  4. Try to access the service:

    curl http://<EXTERNAL-IP>:80

Actual behavior

The Kubernetes Service receives an external IP and the STACKIT Load Balancer is created successfully.

In my observed example:

Kubernetes Service exists:

➜ kubectl get svc nginx
NAME    TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)        AGE
nginx   LoadBalancer   10.130.233.143   188.34.117.139   80:31867/TCP   4m58s
kubectl get svc nginx -oyaml
kubectl get svc nginx -oyaml
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2026-06-07T14:49:06Z"
  finalizers:
  - service.kubernetes.io/load-balancer-cleanup
  labels:
    app: nginx
  name: nginx
  namespace: default
  resourceVersion: "9805"
  uid: ebe5dcfb-f7f3-4c08-9fc4-d037ee1cb968
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 10.130.233.143
  clusterIPs:
  - 10.130.233.143
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - nodePort: 31867
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 188.34.117.139
      ipMode: VIP

Load Balancer exists:

❯ stackit load-balancer list

 NAME                                               │ STATE        │ IP ADDRESS     │ LISTENERS │ TARGET POOLS
────────────────────────────────────────────────────┼──────────────┼────────────────┼───────────┼──────────────
 k8s-svc-ebe5dcfb-f7f3-4c08-9fc4-d037ee1cb968-nginx │ STATUS_READY │ 188.34.117.139 │         1 │            1

Security groups related to the particular Load Balancer exist:

➜ stackit security-group list --label-selector lb.stackit.cloud.lb=k8s-svc-ebe5dcfb-f7f3-4c08-9fc4-d037ee1c

 ID                                   │ NAME                                                                          │ STATEFUL │ DESCRIPTION │ LABELS
──────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────┼──────────┼─────────────┼───────────────────────────────────────────────────────────────────────────────────
 70d75846-2ab1-414b-8f4c-7b5a9176ec67 │ loadbalancer/k8s-svc-ebe5dcfb-f7f3-4c08-9fc4-d037ee1cb968-nginx/backend       │ true     │             │ lb.stackit.cloud: , lb.stackit.cloud.lb: k8s-svc-ebe5dcfb-f7f3-4c08-9fc4-d037ee1c
 e2a27725-4712-4911-9939-6cff7953823e │ loadbalancer/k8s-svc-ebe5dcfb-f7f3-4c08-9fc4-d037ee1cb968-nginx/frontend-port │ false    │             │ lb.stackit.cloud: , lb.stackit.cloud.lb: k8s-svc-ebe5dcfb-f7f3-4c08-9fc4-d037ee1c
 e65bdb87-b1d6-48f4-80b7-4dd7e8dfaa71 │ loadbalancer/k8s-svc-ebe5dcfb-f7f3-4c08-9fc4-d037ee1cb968-nginx/backend-port  │ true     │             │ lb.stackit.cloud.lb: k8s-svc-ebe5dcfb-f7f3-4c08-9fc4-d037ee1c, lb.stackit.cloud:

The nginx pod, Service, endpoints, and in-cluster access worked correctly:

kubectl run curl-test --rm -i --restart=Never \
        --image=curlimages/curl \
        --command -- curl -sv http://nginx.default.svc.cluster.local:80/
* Host nginx.default.svc.cluster.local:80 was resolved.
* IPv6: (none)
* IPv4: 10.130.233.143
*   Trying 10.130.233.143:80...
* Established connection to nginx.default.svc.cluster.local (10.130.233.143 port 80) from 192.168.1.204 port 47958
* using HTTP/1.x
> GET / HTTP/1.1
> Host: nginx.default.svc.cluster.local
> User-Agent: curl/8.20.0
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Server: nginx/1.31.1
< Date: Sun, 07 Jun 2026 15:07:54 GMT
< Content-Type: text/html
< Content-Length: 896
< Last-Modified: Fri, 22 May 2026 12:50:47 GMT
< Connection: keep-alive
< ETag: "6a105127-380"

[payload omitted]

External access failed:

curl http://188.34.117.139:80
[Recv failure: Connection reset by peer](curl: (56) Recv failure: Connection reset by peer)

Reason:

The created security group loadbalancer/k8s-svc-ebe5dcfb-f7f3-4c08-9fc4-d037ee1cb968-nginx/backend is not attached to the relevant worker nodes. The worker nodes only have their default set of security groups attached. (Sorry, I do not know how to verify this via CLI, but investigated via STACKIT Portal).

The worker security group allowed intra-cluster traffic, but did not allow TCP ingress on port 31867.

Expected behavior

For a Kubernetes Service of type LoadBalancer, cloud-provider-stackit should make the service reachable end-to-end. This includes creating and attaching the required security groups.

Environment

  • Kubernetes version: v1.35.3
  • Version of cloud-provider-stackit: ghcr.io/stackitcloud/cloud-provider-stackit/cloud-controller-manager:v1.35.3

Current cloud-provider-stackit deployment args:

args:
  - --cloud-provider=stackit
  - --webhook-secure-port=0
  - --concurrent-service-syncs=3
  - --controllers=cloud-node-controller,cloud-node-lifecycle-controller,service-lb-controller
  - --authorization-always-allow-paths=/metrics
  - --leader-elect=true
  - --leader-elect-resource-name=stackit-cloud-controller-manager
  - --cloud-config=/etc/config/cloud.yaml
  - --cluster-name=stackit-workload

Additional information

As a workaround, manually adding a rule from the Load Balancer backend security group to the worker node security group makes the service reachable.

/kind bug

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions