Allow reading the request body in both security and main server logic

[magdzikk]

Fixes #4442.

The problem

An endpoint that reads the request body in serverSecurityLogic and again in the main logic crashes on streaming backends:

val secureBase = endpoint
  .securityIn(stringBody)                    // read #1
  .serverSecurityLogic { body => ... }

secureBase.in(jsonBody[Person])              // read #2
  .serverLogic(user => person => ...)

java.lang.IllegalStateException: This publisher only supports one subscriber
  at org.playframework.netty.HandlerPublisher.subscribe(HandlerPublisher.java:167)

The failure is backend-dependent, so today the DSL permits something whose success is accidental per interpreter:

• works where the body is held in memory: vertx, armeria, finatra, AWS Lambda, zio-http (by default), the sttp stubs;
• works on akka-http, pekko-http and http4s (Ember) only when the whole body arrives with the request headers;
• fails on Netty (above), Play and Nima;
• on jdkhttp, the second read silently gets an empty body.

The approach

Mark one of the two declarations with asSecondary:

val secureBase = endpoint
  .securityIn(auth.bearer[String]())
  .securityIn(stringBody.asSecondary)   // server-side only
  .serverSecurityLogic { case (token, raw) => verifyHmac(token, raw) }

secureBase.post
  .in("people")
  .in(jsonBody[Person])                 // the documented body
  .serverLogic(user => person => ...)

The secondary body is decoded on the server from a buffered copy of the request body. It isn't part of the API contract: it's left out of the documentation, and clients don't send it (they still take a value for it).

Only bodies which can be re-read from buffered bytes can be secondary: String, Array[Byte], ByteBuffer, InputStream, InputStreamRange. Others are rejected at compile time.

Release notes

• An endpoint with more than one ordinary request body is now rejected when routes (or an sttp stub backend) are created. Previously it failed on each request, or worked only on some backends. Mark one of the bodies with asSecondary; hiding its schema isn't enough.
• EndpointVerifier reports these problems as a new InvalidBodyDefinitionError, so existing EndpointVerifier(...) shouldBe empty checks may start failing.
• All servers now decode string request bodies with the Content-Type charset, falling back to the codec's charset. Before, Netty, jdkhttp, Nima, zio-http, Vert.x and the sttp stubs ignored the request's charset, and Akka/Pekko fell back to UTF-8.
• For server interpreter authors: build routes with PrepareServerEndpoints instead of FilterServerEndpoints, and decode whole string bodies with RequestBody.requestCharset(request).getOrElse(codecCharset).

[adamw]

🤖 Automated review (generated with Claude Code review agents; main claims checked against the code, but please verify before acting).

No high-severity bugs found. In both orders (secondary body in securityIn or in in), the backend body is read once, max content length is enforced on the buffered path, and all server interpreters in the repo run the verification.

Design

1. Maybe a smaller design is enough. securityIn and in are decoded separately, so the server could buffer the body whenever both declare one, without the asSecondary DSL. Clients already set security inputs first and main inputs second, so the main body wins. Docs would only need to pick the last body instead of body.headOption (EndpointToOpenAPIPaths.scala:70), or users could use .schema(_.hidden(true)). This would remove the marker, SecondaryBody, most of EndpointBodyVerifier, the Values change, the stub decoder changes and the isSecondary cases in clients/docs/grpc. Endpoints that work today on vertx/armeria/stubs would keep working. What we lose: the compile-time check for non-bufferable bodies (a check when routes are built could replace it). Worth deciding this first.
2. Several bodies in one input group (.in(stringBody).in(stringBody.asSecondary)) is more than [BUG] Cannot use a body in serverSecurityLogic: causes java.lang.IllegalStateException: This publisher only supports one subscriber #4442 needs. It's the only reason for the index-based setBasicInputValue, decodeSecondaryBodies and the in_string_secondary_string_out_string fixture.
3. Every interpreter must remember to check isSecondary. The marker is an attribute, so each Body match needs an if b.isSecondary guard, now copied into ~10 places (5 clients, 3 docs, 2 grpc). An external interpreter still compiles and will send/document the secondary body. A dedicated type, or traversal helpers that skip secondary bodies, would make this safe by default.
4. Docs and verifier disagree. The verifier rejects secondary oneOfBody variants and secondary outputs, but EndpointToParameters.scala:17 and EndpointToOperationResponse.scala:132 silently drop them.

Correctness / security

5. Custom interpreters skip verification. It only runs via PrepareServerEndpoints or direct EndpointBodyVerifier calls (vertx, finatra, zio-http). FilterServerEndpoints / new ServerInterpreter don't run it, and the verifier is private[tapir]. A custom interpreter with a secondary body next to a file/multipart/stream body reads an already consumed body (the [BUG] Cannot use a body in serverSecurityLogic: causes java.lang.IllegalStateException: This publisher only supports one subscriber #4442 error again). Suggestion: verify inside FilterServerEndpoints.apply and drop PrepareServerEndpoints; this also removes the renames in ~20 interpreters.
6. AWS Lambda: verified bytes and main body can differ (AwsRequestBody.scala:24,38-40). For non-base64 bodies, the secondary bytes come from body.getBytes() (platform charset), but the primary string is the original request.body. With a non-UTF-8 default charset or an unpaired surrogate, an HMAC check can pass on bytes that don't match what the main logic gets. Fix: getBytes(UTF_8), and new String(bytes, UTF_8) in decodeStringBody.
7. Stub matchers may see an empty second body. SttpRequestDecoder (stub and stub4) calls forceBodyAsByteArray once per body input; with an InputStream request body the second read is empty. Read the bytes once.

API compatibility

8. DecodeBasicInputsResult.Values gets a third field: source and binary break for interpreter authors. MiMa only runs on core, so it won't catch it. hasSecondaryBody on the sealed trait is always false for Failure, and is computed per request though it only depends on the endpoint.
9. stub4: protected def handleRequest(req) was replaced by requestHandler(), which breaks subclasses.
10. RequestBody.decodeStringBody must stay in sync with each backend's toRaw(StringBody). akka/pekko/armeria copy the charset rule instead of sharing it, and the default ignores the request charset, so external backends may decode differently. Either share the code with toRaw, or use one rule in CachingRequestBody (Content-Type charset, else codec default).
11. InvalidBodyDefinitionError(message: String, rejectedByServers: Boolean) is stringly typed, unlike the other EndpointVerificationErrors, which have one case class per problem. This becomes public API once released.

Tests

The tests can fail when they should, and the cross-backend tests run on all servers, including Netty/jdkhttp/Nima where the body can be read only once. Gaps:

• no test for a secondary body in in that fails to decode, or a primary that fails next to a secondary;
• no empty body case for the akka/pekko/aws/vertx decodeStringBody overrides;
• the Try in CachingRequestBody isn't tested (IdentityMonad can't tell a throw from a failed effect; TryMonad can);
• no secondary body next to a oneOfBody primary;
• assertDoesNotCompile in SecondaryBodyTest passes on any compile error: prefer assertTypeError, and add positive checks for inputStreamBody.asSecondary / inputStreamRangeBody.asSecondary;
• AWS Lambda tests don't run the new ServerSecurityTests cases.

Minor

• Vert.x handlers/package.scala:36 doesn't skip secondary bodies (safe only thanks to the verifier).
• filterOutHiddenInputs → filterOutUndocumentedInputs is an unrelated rename.
• doc/server/interceptors.md:20 still mentions FilterServerEndpoints.
• The allowed secondary body types are listed in 5 places; only 2 are checked by the compiler.
• rejectedByServers is explained twice (scaladoc and doc/testing.md); security.md could be shorter.

[magdzikk]

6. AWS Lambda: verified bytes and main body can differ

Fixed - bodies are now encoded as UTF-8 when read as bytes.

2. Several bodies in one input group

Fixed - now a second body in the same group is rejected.

8. DecodeBasicInputsResult.Values gets a third field

Fixed by the change for #2.

7. Stub matchers may see an empty second body.

Fixed by the change for #2.

5. Custom interpreters skip verification. (...) Suggestion: verify inside FilterServerEndpoints.apply

We already talked about it in #5491 (comment), and decided to not put verification in the filtering method.

1. Maybe a smaller design is enough. securityIn and in are decoded separately, so the server could buffer the body whenever both declare one, without the asSecondary DSL.

The need for a DSL was discussed in #4442, and I still think the points there are valid. Also I checked that buffering in the suggested way would cause problems with Content-Type.

3. Every interpreter must remember to check isSecondary

A new EndpointIO subtype would touch every input match in the codebase and break binary compatibility. The clients' recursive matches can't use a filtering traversal either.

4. Docs and verifier disagree.

It's true, but intended. Docs never validate endpoints, and the guards prevent the dangling $ref mentioned here: #5491 (comment) .

9. stub4: protected def handleRequest(req) was replaced by requestHandler(), which breaks subclasses.

Yes, but it's a change from your commit - do you want me to fix it?

10. RequestBody.decodeStringBody must stay in sync with each backend's toRaw(StringBody). (...) use one rule in CachingRequestBody

Fixed - shared the rule with toRaw on akka, pekko and armeria.

11. InvalidBodyDefinitionError(message: String, rejectedByServers: Boolean) is stringly typed, unlike the other EndpointVerificationErrors, which have one case class per problem. This becomes public API once released.

A field was one of your suggestions here: #5491 (comment) and I liked it. If you prefer, I can change it to a case class per problem, but that would add ~9 new public types.

rejectedByServers is explained twice (scaladoc and doc/testing.md); security.md could be shorter.

Fixed.