Capability and seccomp rejections do not tell the customer how to fix the manifest

[scotwells]

When a customer gets a capability request wrong, the error usually does not tell them how to fix it. A typo reads as if the platform is withholding a real capability. A lowercase name returns a regex. A single mistake can produce two errors where the second contradicts the first. And an unsupported seccomp profile surfaces as a decoder error that never mentions seccomp. In every one of these cases the customer is one small edit away from a working manifest, and the message does not say which edit.

This is the validation surface introduced in #310.

The model to follow

One message already does this right. Input capabilities: {add: [CAP_NET_ADMIN]}:

Invalid value: "CAP_NET_ADMIN": must omit the CAP_ prefix, for example "NET_ADMIN"

It names the mistake and states the fix. The cases below should read like this.

The cases

1. A typo is reported as a policy denial. Input capabilities: {add: [NOT_A_CAPABILITY]}:

Forbidden: capability NOT_A_CAPABILITY is not granted by the "general-purpose" runtime class, which grants AUDIT_CONTROL, AUDIT_READ, ... WAKE_ALARM

This implies the capability exists and is being withheld — a customer reasonably goes looking for a different runtime class or files a support request. It does not exist. The platform never distinguishes an unknown capability name from a real one, so there is no "unknown capability" message and no "did you mean".

2. A lowercase name leaks the raw validator. Input capabilities: {add: [net_admin]}:

Invalid value: "net_admin": spec.template.spec.runtime.sandbox.containers[0].securityContext.capabilities.add[0] in body should match '^[A-Z_]+$'

A regex is not product copy. The message never says capability names are uppercase, and the customer is one character-case away from correct.

3. One mistake produces two errors, and the second is often wrong or contradictory. Input capabilities: {add: [ALL]}:

* Forbidden: ALL may not be added; list each capability the container needs
* Forbidden: capability ALL is not granted by the "general-purpose" runtime class, which grants AUDIT_CONTROL, ... WAKE_ALARM

The second is noise — ALL is not a capability the class could ever grant. The platform rejects ALL and then goes on to check the same entry against the class anyway.

Worse on a class that supports no capability requests at all. Input capabilities: {add: [ALL]} with class: unikernel:

* Forbidden: ALL may not be added; list each capability the container needs
* Forbidden: container capability requests are not supported by the "unikernel" runtime class

The customer is told to list the capabilities they need, immediately before being told the class accepts none. Following the first instruction still fails.

4. An unsupported seccomp profile surfaces as a decoder error. Input seccompProfile: {type: Localhost, localhostProfile: foo.json}:

Error from server (BadRequest): ... strict decoding error: unknown field "spec.template.spec.runtime.sandbox.containers[0].securityContext.seccompProfile.localhostProfile"

localhostProfile is not part of the published API at all, so the request is rejected before any customer-facing validation runs and the message never says Localhost profiles are unsupported. For contrast, type: Localhost with the field omitted gives a good message: Unsupported value: "Localhost": supported values: "RuntimeDefault", "Unconfined".

5. The grantable-capability list is repeated inline in every rejection. The general-purpose class grants 41 capabilities, and the full list is inlined in each capability error — twice when two errors fire. A customer fixing one typo reads roughly 500 characters of allowlist, twice.

Reproduction

Against a control plane with the RuntimeClasses gate on and the general-purpose class published, kubectl apply --dry-run=server a sandbox Workload, varying only containers[0].securityContext.

apiVersion: compute.datumapis.com/v1alpha
kind: Workload
metadata:
  name: cap-errors
spec:
  template:
    spec:
      runtime:
        class: general-purpose
        sandbox:
          containers:
            - name: app
              image: docker.io/library/nginx:latest
              securityContext:
                capabilities:
                  add: [NET_BIND_SERVICE]
  placements:
    - name: default
      cityCodes: [DFW]
      scaleSettings:
        minReplicas: 1

Inputs tested:

securityContext	Case
capabilities: {add: [NOT_A_CAPABILITY]}	1
capabilities: {add: [net_admin]}	2
capabilities: {add: [ALL]}	3, 5
capabilities: {add: [ALL]} with class: unikernel	3
seccompProfile: {type: Localhost, localhostProfile: foo.json}	4
seccompProfile: {type: Localhost}	good message, contrast
capabilities: {add: [CAP_NET_ADMIN]}	the model to follow

[scotwells]

Outcome

A customer who gets a capability request wrong should see a message that names the mistake and states the edit that fixes it. No regexes, no 41-name allowlists, and no second error that contradicts the first. The bar is the message we already get right:

Invalid value: "CAP_NET_ADMIN": must omit the CAP_ prefix, for example "NET_ADMIN"

The scope is narrower than a full rewrite. Some bad messages come from the published API schema, and the platform's own validation never runs for them. Other copy can't honestly say more than the product does today. The proposal below covers only what can ship without weakening the API contract or promising something the platform doesn't do.

What customers can expect today, by class

• general-purpose grants every Linux capability, all 41. When this class rejects a name in capabilities.add, the name isn't a capability. It's a typo, and the message should say so.
• unikernel accepts no capability requests. The message should say to remove capabilities.add.

No class today accepts capability requests but withholds some of them. The withheld-capability copy below is groundwork for a future narrower class, not something customers see today.

Proposed copy

Input	Today	Proposed
add: [NOT_A_CAPABILITY] on general-purpose	Forbidden: capability NOT_A_CAPABILITY is not granted by the "general-purpose" runtime class, which grants AUDIT_CONTROL, … WAKE_ALARM	Invalid value: "NOT_A_CAPABILITY": not a Linux capability name
add: [NET_BIND_SERVIC], one close match	same as above	Invalid value: "NET_BIND_SERVIC": not a Linux capability name; did you mean "NET_BIND_SERVICE"?
add: [ALL] on general-purpose	two errors: ALL may not be added; list each capability the container needs, then capability ALL is not granted by … plus the full list	only the first: Forbidden: ALL may not be added; list each capability the container needs
add: [ALL] on unikernel	two errors: the ALL error tells the customer to list capabilities, then the class refusal says it accepts none	only the ALL error (see "One mistake, one error" for the trade-off)
drop: [NET_ADMN]	accepted and stored silently	Invalid value: "NET_ADMN": not a Linux capability name; did you mean "NET_ADMIN"?
add: [net_admin]	Invalid value: "net_admin": … in body should match '^[A-Z_]+$'	Invalid value: "net_admin": capability names are uppercase, for example "NET_ADMIN" (see the schema prerequisite below)
add: [SYS_ADMIN] on a future class that withholds it	does not occur today	Forbidden: the "<class>" runtime class does not grant SYS_ADMIN; see the grantable capabilities on the "<class>" runtime class
any capabilities.add on unikernel	Forbidden: container capability requests are not supported by the "unikernel" runtime class	unchanged

Notes on the copy:

• ALL stays a capability value. The API documents drop: [ALL] as a supported example, so no message should say ALL isn't a capability. The current ALL message is accurate. The bug is the duplicate error after it.
• Class refusals don't suggest another class. Validation can't see which classes are served where the customer is deploying. A message saying "select general-purpose" could send them straight into a placement failure. The refusal states what this class does, and the placement message is where location and class get matched.
• Checking drop fixes a silent failure. A typo in drop never reaches a runtime check, so the customer believes a capability was removed when it wasn't. ALL stays valid in drop. Workloads already stored with a bad drop entry must still accept unrelated updates, so the check should apply only to entries that are new or changed.

Did-you-mean: narrow, drawn from the full capability set

Keep the suggestion, but only when it's very likely right.

• Candidate set. Suggestions come from the full list of Linux capabilities, not from what the class grants. If suggestions came only from the class's grantable set, then on a narrower class a near-miss for a capability the class withholds would find no match. The customer would be told their real capability doesn't exist, and the actual reason, class policy, would be hidden. That's the defect this issue is about, running backwards. After a suggestion is found, the class check runs on the suggested name. If the class withholds it, the message says so.
• Close match. At most one suggestion. A suggestion is made only when exactly one capability name is within two edits of the input, where a swap of two adjacent characters counts as one edit. If two names are that close, there's no suggestion.
• Why the uniqueness rule matters. Several real names differ by one or two edits: SETUID/SETGID, SETFCAP/SETPCAP, DAC_OVERRIDE/MAC_OVERRIDE, SYS_NICE/SYS_TIME, and FSETID against both SETUID and SETGID. A wrong suggestion in this group passes at apply and fails at runtime, which is worse than no suggestion. Near these names the customer gets the plain "not a Linux capability name" message.
• What it can't catch. A typo that lands exactly on another real name, like SETGID for SETUID, is valid input and will always be accepted.

If the uniqueness rule turns out to suppress most suggestions in practice, drop the feature. The plain message alone already fixes the misleading policy denial.

Drop the 41-name list, point at the class

Remove the inline allowlist from every rejection. On general-purpose it fires only on typos, where it buries the one useful line in about 500 characters of names. It's repeated whenever two errors fire, and it duplicates information the customer can already read. Runtime classes are projected into the customer's control plane, and ordinary viewer access can read them.

Two caveats:

• Finding the list takes a deliberate lookup, not a glance at the error.
• The customer needs a role that can read runtime classes.

When a future class withholds a real capability, the message names the grantable capabilities on that runtime class. It doesn't say "capabilities.grantableCapabilities", because the error's own field path already contains a different capabilities.

One mistake, one error

The platform returns one error for each unsupported request on purpose, so a customer sees the whole gap in a single apply. That principle stays.

The narrow rule: an entry that already failed a name check isn't also checked against class policy. Name checks include the CAP_ prefix, ALL in add, and unknown names. A class verdict on a name we couldn't accept is a guess presented as policy. Every other error, whether per entry, per container, or a sibling refusal like envFrom or volumes, still reports as it does today.

This fixes both contradictory pairs:

• On general-purpose, add: [ALL] produces only the ALL error.
• On unikernel, the container-wide refusal applies to the well-formed entries in capabilities.add. When ALL is the only entry, only the ALL error appears.

The trade-off: a customer who follows that message and lists a real capability learns on the next apply that unikernel accepts none. That's one extra round trip in a narrow case. In return, the platform never tells them to do something and refuses it in the same response. When the list mixes ALL with real names, both errors appear, and together they're consistent: remove ALL, and this class takes no requests.

Prerequisite: some checks live in the published schema

Two cases in the issue are rejected by the published API schema before platform validation runs. Their copy can't be changed by editing platform messages.

Lowercase names: keep, with a schema change. The uppercase-only rule is currently a bare pattern in the schema. The fix is to keep the rule in the schema but express it as a schema rule with a customer-facing message, which the API already does for other fields. The constraint stays part of the published contract and is enforced the same way. The cost: tools that read the schema's pattern directly, like client-side validators and generated reference docs, would see a rule instead of a machine-readable pattern. That's a modest loss for replacing a regex with a sentence. Moving the check into platform validation isn't recommended, because it removes the constraint from the published contract. If the schema-rule approach is rejected, drop this row rather than move the check.

Seccomp Localhost: no change. Two separate cases:

• type: Localhost alone is rejected by the schema with Unsupported value: "Localhost": supported values: "RuntimeDefault", "Unconfined". That's already accurate and actionable.
• Adding localhostProfile produces a decoding error naming an unknown field. That message is worse, but the behavior is right. If the schema accepted the field just so validation could explain it, the field would be legal with every profile type. A customer could set it, have it stored, and reasonably believe their profile is loaded. Datum's promise is that a stored workload states what actually runs, so the hard rejection stays.

Seccomp copy also shouldn't claim that Datum doesn't load custom profiles. What's deliberately absent is a profile loaded from a file on the host. That file would be a path on a machine the customer can't see, and the value would go to the runtime unchecked. That reasoning doesn't cover a profile delivered through a checked path in the future, and error strings get quoted back to us. Any seccomp copy should describe the accepted values, as the schema message above already does.

Summary of recommendations

1. Distinguish "not a Linux capability name" from a class policy denial. On general-purpose every rejected name is a typo.
2. Add a narrow did-you-mean drawn from the full capability set, with a uniqueness rule for confusable names.
3. Check drop entries against the capability set, keeping ALL valid and leaving stored workloads updatable.
4. Remove the inline grantable-capability list, and point at the runtime class in the rare case it matters.
5. Skip class-policy checks for entries that already failed a name check. Otherwise keep reporting every error in one apply.
6. Replace the lowercase regex message through a schema rule with a message, not by moving the check out of the schema.
7. Leave the seccomp Localhost and localhostProfile rejections as they are.
8. Hold the withheld-capability message for a future narrower class. Don't name alternative classes in refusals.