Skip interim 1xx responses before the final response

[carpentry-agent]

RFC 9110 §15.2 says a client MUST be able to parse one or more 1xx responses received before the final response, even if it does not expect one. read-headers parsed only the first head block it found. So an interim response came back as the status, and the real response's raw head and body came back as the body.

On main, against the new test routes:

GET /interim-continue      (100, then a 200 with a 10-byte body)
  100 [HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 10\r\nConnection: close\r\n\r\nfinal-body]

GET /interim-hints         (103 with two Link headers, then a chunked 200)
  103 [HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\nConnection: close\r\n\r\n5\r\nhello\r\n6\r\n world\r\n0\r\n\r\n]

POST /post with `Expect: 100-continue`   (Python's http.server sends the 100 on its own)
  100 [HTTP/1.1 200 OK\r\nServer: BaseHTTP/0.6 Python/3.11.2\r\n…\r\n\r\npayload]

GET /interim-cookie via the cookie jar   (103 sets interim=leaked, the 200 sets final=kept)
  the jar stores interim=leaked and never sees final=kept

GET /interim-close         (a 100, then the server closes)
  100 []

That breaks any server that sends Early Hints, and any caller that sets Expect: 100-continue.

What changed

The old read-headers becomes read-head. It takes the bytes left over from the previous head and reads from the connection only when those bytes don't already hold a whole head. Those bytes can be a partial head, the whole final head, or the final head plus body bytes, and later reads are appended to them.

The new read-headers loops over read-head:

• A 1xx other than 101 is dropped with its headers. The bytes after it start the next head. Its Link and Set-Cookie lines never reach the final Response. CookieJar.store-response! runs on the final response only, so they never reach the jar either.
• 101 Switching Protocols stays final. The client never sends Upgrade, and Go's transport also treats 101 as terminal (net/http: k8s spdy TestUpgradeResponse broken in Go 1.11 golang/go#26161).
• More than five interim responses is a ClientError.Parse: too many 1xx informational responses (max 5). That is retryable? false, delivered? true.
• A close before the final head is the existing Receive error, incomplete HTTP headers. A server that closes before sending any response gets the same error.

Neither request path (request-stream- or the cookie-jar path) changes. Both call read-headers as before.

Why a cap of 5

Without a cap, a server that keeps sending interim responses holds the client forever. read-timeout doesn't stop it, because every interim response counts as activity. Each interim's bytes are dropped once it is parsed, so the cap bounds time, not memory.

Five is the count Go's transport used for years (max1xxResponses, which failed with too many 1xx informational responses). Current Go no longer counts them, though. The 1.26.1 source budgets the combined header bytes of the 1xx and final responses instead (MaxResponseHeaderBytes, 10 MiB by default), unless a Got1xxResponse trace hook takes them. This client has no header-size limit to budget against, and adding one would be a separate change. So a count is the bound available here.

Real traffic stays well under five. 100 Continue comes at most once per request. 103 Early Hints usually comes once, though RFC 8297 allows several. The one periodic interim is 102 Processing (RFC 2518, dropped from RFC 4918). A server that uses it as a keepalive for more than five beats would hit the cap. The number is one token (max-interim) if you'd rather be more lenient.

Expect: 100-continue

The client still sends the body right away, even when a caller sets Expect: 100-continue. RFC 9110 §10.1.1 allows a client to do that without waiting for the 100. With this fix those requests now work: the test posts to the Python server, which answers the Expect with a real 100 Continue. Waiting for the 100 before sending the body is out of scope.

Tests

The suite gains 17 assertions, served by new raw-socket routes in test/server.py after /header-continuation:

• 100, then a 200 with a Content-Length body
• 103 with Link headers, then a chunked 200, and its Link header not reaching the final response
• 100, 102 and 103 in a row
• the interim and the final head in separate sends, 0.2 s apart
• the final head split across reads, starting in the interim's read
• the final head and the whole body arriving in the interim's read, with the connection then held open for 10 s. The response must come back in under 5 s. This is the only test that catches a read-head that reads before checking the leftover bytes: with a server that closes, that extra read just gets EOF.
• an interim, then a 302 that is followed
• a HEAD
• five interims (passes), and six (a Parse error)
• a close right after an interim
• 101 staying final
• Expect: 100-continue against the real server
• an interim Set-Cookie, on the plain path and on the cookie-jar path

Locally the suite goes from 181 to 198/0.

Teeth. With the fix reverted, the suite is 182/16. Every new assertion fails except "a 101 Switching Protocols is a final response", which passes on main by design: it pins the 101 decision. Mutants, each on the fixed suite:

mutant	result
read-head reads before checking the leftover (found false)	197/1: the held-connection test
cap one lower (4)	197/1: five interims
cap one higher (6)	196/2: six interims, both the message and the Parse kind
101 treated as interim	197/1: 101 staying final
leftover after an interim dropped	184/14
interim count never incremented	196/2: six interims, both the message and the Parse kind

carp-fmt --check and angler are clean on both changed .carp files. ruff check is clean on server.py.

Relation to #29

#29 (draft, Content-Length framing) is open against the same files. This branch is cut from main and stays out of #29's regions:

• bodyless?/declared-length
• the tails of both request paths
• the README
• Frame Content-Length bodies per RFC 9112 §6.3 #29's server helper and routes
• Frame Content-Length bodies per RFC 9112 §6.3 #29's test helpers and test block

git merge-tree merges the two cleanly in either order, to the same tree. The suite on the merged tree is 221/0: #29's 204 plus these 17.

#29 noted a follow-up: ending HEAD, 1xx, 204 and 304 responses at the header block instead of reading to the close. Without this change, that follow-up would end a 103 at its head and lose the real response entirely. With it, the only 1xx that can reach bodyless? is a 101.

---

Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.

[carpentry-reviewer]

Build & Tests

CI is green on both platforms (ubuntu-latest 1m18s, macos-latest 2m13s).

Locally on 04c89ff (armhf Pi, private output directory, test/server.py started by hand on 8791/8792):

• Branch: it builds, and the suite is 198/0. main is 181/0.
• Fix reverted, new tests kept: 182/16. All 16 failures are new interim tests. As the body says, the only new test that still passes is "a 101 Switching Protocols is a final response". The Expect: 100-continue test is one of the 16. So test/server.py (protocol_version = "HTTP/1.1") really does send a 100 Continue before the final response.
• ASan: the suite under clang ASan is 198/0 with no reports. As a positive control, a deliberate use-after-free was run first, and ASan reported it. The probe program below is also clean under ASan.
• Merging with #29: git merge-tree --write-tree gives the same tree (078bd8f) in both orders, with no conflicts. The suite on that tree is 221/0.
• ruff check test/server.py is clean.

Findings

I found no bugs.

Mutants. All of them ran from one env-gated build of the branch, with the unmutated baseline as the first row:

mutant	result
none (baseline)	198/0
read-head reads before checking pending (found false)	197/1, only the /interim-held test
pending not appended to the buffer	185/13
cap of 4 / cap of 6 (at the (= skipped max-interim) use)	197/1 / 196/2
101 treated as interim	197/1, the 101 pin
100 treated as final ((< 100 code))	188/10
102 treated as final	197/1, "several interim responses in a row are all skipped"
leftover dropped after an interim	184/14
count never incremented	196/2
no code treated as interim (the fix undone at the loop)	182/16
plain path bypasses the loop (read-head &conn "" at http-client.carp:714)	183/15
cookie-jar path bypasses the loop (same change at http-client.carp:1041)	197/1, "a Set-Cookie on an interim response is not stored in the jar"
199 treated as final ((< code 199))	198/0, survives. No test sends a 199, and no assigned 1xx code is affected.

All six of the PR's mutant rows reproduce, with the same counts and the same test names. Both call sites are pinned, including the cookie-jar path, which only the jar test covers.

Raw-socket probes, branch vs main (a unique-body control ran first):

• A 100 that carries Content-Length: 5, and a 103 that carries Transfer-Encoding: chunked, both give 200 [ok]. So a 1xx's framing headers are ignored, as RFC 9110 §15.2 requires. main returns the 1xx status for both.
• These all give 200 [ok]:
  • HTTP/1.1 199
  • HTTP/1.0 100
  • an interim sent one byte per write
  • an interim whose blank line is split across two reads (\r\n\r, then \n)
• A 100 followed by a 101 gives 101 [].
• Five 102s, each in its own write, give 200. Six give the Parse error.
• Heads that end lines with LF only fail with incomplete HTTP headers on both main and the branch, so that behaviour is unchanged. HTTP/1.1 099 comes back as a final status of 99 on both. That comes from http's status-line parse, not from this PR.

Go. The 1.26.1 source (/usr/local/go/src/net/http/transport.go, readResponse) matches the PR body:

• there is no count cap;
• 101 is terminal (golang/go#26161);
• the MaxResponseHeaderBytes budget is reset only when a Got1xxResponse hook is set.

transport_test.go still accepts either response headers exceeded or too many 1xx.

Informational, not blocking:

1. An interim head is still parsed in full, so a bad field in it fails the whole request (http-client.carp:553-554).
   • Probe: a 103 with Set-Cookie: x=1; Expires=garbage, then a valid 200. The result is ERR parse: Malformed response: found header 'malformed expires date in set-cookie: unrecognized date format: garbage', even though the branch then throws that cookie away.
   • An interim HTTP/1.1 100 with no reason phrase fails the same way (found first line 'HTTP/1.1 100').
   • Both fail the same way on main, so neither is a regression.
   • Go would skip both. ReadResponse keeps 1xx headers as raw MIME and never parses their cookies, and it splits the status line at the first space.
   • Both cases belong to http's open strictness questions: parse-set rejecting a bad cookie, and the status line with no reason phrase. Those are hellerve's call, so I would not hold this PR for them.
2. The body's two design calls are hellerve's to make. Both are argued in the body.
   • The cap of 5: it counts interims, where Go now uses a byte budget. The only realistic casualty is a long run of 102 Processing keepalives, and the cap is one token to change.
   • Expect: 100-continue: the body is still sent without waiting for the 100, which RFC 9110 §10.1.1 allows.

Verdict: merge

It builds, CI is green and the suite passes 198/0 (also under ASan). Every claim in the body reproduced, both call sites are pinned by tests, and probing found no bugs.