fix(operator): support custom cluster domain#183
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9906b8f84d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let service_fqdn = | ||
| cluster_dns::service_fqdn(&tenant.headless_service_name(), &namespace, cluster_domain); | ||
| Ok(format!("https://{service_fqdn}:9000")) |
There was a problem hiding this comment.
Keep TLS admin clients on a covered Service DNS name
This switches all operator-side TLS admin calls to the headless Service FQDN, but SAN validation only requires the headless/pod DNS names when enableInternodeHttps=true (san_validation_dns_names falls back to configured names when includeGeneratedDnsNames=false). For a TLS tenant with enableInternodeHttps=false and includeGeneratedDnsNames=false whose certificate explicitly covers the previously used *-io.<ns>.svc endpoint, reconciliation can still mark TLS ready, then STS/provisioning/monitor clients fail hostname verification against *-hl.<ns>.svc.<domain>. Please either require this FQDN in validation whenever this client is used or connect to a Service name that is already validated.
Useful? React with 👍 / 👎.
| if entry.default && config.enable_internode_https { | ||
| names.extend(required_tls_dns_names(tenant, namespace, cluster_domain)); | ||
| } else if include_generated_dns_names(entry) { | ||
| names.extend(generated_dns_names(tenant, namespace, cluster_domain)); |
There was a problem hiding this comment.
Keep IO Service names in default TLS SAN checks
When enableInternodeHttps=true, this branch now validates only the headless Service and pod DNS names, so an external TLS Secret with the default includeGeneratedDnsNames=true can pass requireSanMatch while missing tenant-io.<ns>.svc[.<domain>]. The Tenant is still exposed through the *-io Service, so HTTPS clients using that Service get hostname verification failures even though the operator reports TLS ready; include the generated IO Service names in this validation in addition to the internode names.
Useful? React with 👍 / 👎.
| let generated = generate_sts_tls_material( | ||
| &config.namespace, | ||
| &config.service_name, | ||
| &config.cluster_domain, |
There was a problem hiding this comment.
Regenerate managed STS certs when the cluster domain changes
This applies the configured cluster domain only when a new managed sts-tls Secret is generated; load_or_create_sts_tls_material still accepts an existing managed Secret as long as the keys parse. Upgrading an install that already has the auto-generated cluster.local STS certificate and then setting clusterDomain leaves the served certificate without the new *.svc.<domain> SAN, so clients using the advertised FQDN fail TLS until the Secret is manually deleted. Please validate the existing managed Secret SANs against the current service_dns_names and regenerate when they no longer match.
Useful? React with 👍 / 👎.
Type of Change
Related Issues
Closes #152
Summary of Changes
This PR adds first-class support for Kubernetes clusters whose DNS domain is not
cluster.local.Root cause: the operator rendered Tenant peer URLs, generated TLS SANs, and some operator-side TLS clients with a hardcoded
cluster.localassumption. In TLS-enabled deployments this also made SAN validation require DNS names that did not match the actual headless-service certificate shape used by custom-domain clusters.The fix:
OPERATOR_CLUSTER_DOMAINruntime setting and a HelmclusterDomainvalueRUSTFS_VOLUMESpeer URLs with the configured cluster domain*.tenant-hl.ns.svc.example.clusterChecklist
make pre-commit(fmt-check + clippy + test + console-lint + console-fmt-check)[Unreleased](if user-visible change) - N/A: this repository does not currently contain a CHANGELOG.mdImpact
Verification
Additional Notes
The Helm chart now owns
OPERATOR_CLUSTER_DOMAIN; users should setclusterDomaininstead of addingOPERATOR_CLUSTER_DOMAINthroughoperator.env.