Frame Content-Length bodies per RFC 9112 §6.3

[carpentry-agent]

Frames Content-Length bodies the way RFC 9112 §6.3 asks. Until now the client read every body to the close and used the length only to detect truncation. So it passed on bytes past the length and waited out servers that held the connection open. It also accepted negative, empty, junk and conflicting lengths, which switched the truncation check off as well.

• The body stops at the length. poll ends as soon as the last declared byte has been delivered and drops anything after it, including bytes that arrived with the headers. A Content-Length: 0 response reads nothing.
• The value must be 1*DIGIT (RFC 9110 §8.6). A comma list or repeated lines count as one value when every member is valid and they are all equal (§6.3 item 5). A sign, junk, an empty value or differing values fail the request with ClientError.Parse.
• No wrapping. A value over 2147483647 fails with ClientError.Parse instead of wrapping (on 64-bit, 4294967301 became 5) or saturating (on 32-bit).
• Transfer-Encoding overrides Content-Length (§6.3 items 3 and 4), chunked or not, valid or not. HEAD, 1xx, 204 and 304 responses still ignore Content-Length and read to the close, as on main.

ResponseStream's fields are unchanged. Both request paths now build the stream through one private open-stream. The README's paragraph on body framing gains two sentences.

Real clients

Measured on this Pi against a raw-socket server: Chromium 154 headless (--dump-dom), curl 7.88.1, and Python 3.11 http.client. Go is not installed here, so its column comes from reading determineBodyLength and parseContentLength in master's net/http/transfer.go; it was not run. The main column was measured on this 32-bit Pi. EOF means the client treated the length as absent and read to the close.

Content-Length, bytes sent	main	this PR	Chromium	curl	Python	Go (source)
5, helloEXTRA	helloEXTRA	hello	hello	hello	hello	hello
5, hello, held open 6 s	waits 6 s	immediate	immediate	immediate	immediate	immediate
-5, hello	hello	Parse error	EOF hello	error (rc 8)	EOF hello	error
empty, hello	hello	Parse error	EOF hello	error (rc 8)	EOF hello	error
5x, hel	hel	Parse error	EOF hel	reads 5, truncated (rc 18)	EOF hel	error
+5, hel	truncated	Parse error	EOF hel	reads 5, truncated (rc 18)	reads 5, IncompleteRead	error
5, 5, hello	hello	hello	hello	hello	EOF hello	error
5, 6, hello	hello	Parse error	refused	reads 5, hello	EOF hello	error
lines 3 and 5, hello	hello	Parse error	refused	last line, hello	first line, hel	error
lines 5 and 5, hello	hello	hello	hello	hello	hello	hello
005, helloEXTRA	helloEXTRA	hello	hello	hello	hello	hello
9, hello	truncated	truncated	no document	truncated (rc 18)	IncompleteRead	error
4294967301, hello	truncated (hello on 64-bit)	Parse error	no document	truncated (rc 18)	IncompleteRead	error
20 digits, hello	truncated (hello on 64-bit)	Parse error	EOF hello	overflow, EOF hello	IncompleteRead	error
TE chunked + 3	hello	hello	hello	hello	hello	hello
TE chunked + abc	hello	hello	hello	error (rc 8)	hello	error
TE xchunked + 5, helloEXTRA	helloEXTRA	helloEXTRA	hello	error (rc 61)	hello	error
204 + abc	204	204	no document, as for any 204	204	204	error

The two 64-bit notes for main follow from Int.from-string, which narrows strtol's long to an int. In the 9 and 4294967301 rows, Chromium's load had not finished after 60 s.

Invalid values. No invalid form is tolerated by all four clients, because Go refuses every one. Chromium reads to the close. Python does too, except that it reads +5 as 5. curl refuses -5 and an empty value but reads the leading digits of 5x and +5. For 5x, then, two tolerant clients pick two different bodies. So this PR refuses all of them, as §6.3 item 5 requires. curl also errors on every row this PR refuses, except the two conflicting rows and the 20-digit row. The forms the clients agree on, repeated identical lines and leading zeros, are accepted. So is the identical comma list the RFC allows: Chromium and curl read it as 5, Python reads to the close, and Go refuses it.

Too large for an Int. Int is 32-bit on every platform, so the client cannot count past 2 GiB. No real client takes the 4294967301 row as a complete body, and refusing keeps it an error. A wrapped length (main on 64-bit) and reading to the close would both report those 5 bytes as a clean 200. The cost is that request-stream can no longer receive a body over 2 GiB that declares its length. Main managed that only by reading to the close.

Non-chunked Transfer-Encoding. Chromium and Python use the Content-Length here, and curl and Go refuse the response. This PR reads to the close, as §6.3 item 4 says and as main did.

Tests

bash test/run.sh gives 204 passed and 0 failed (main: 181/0). The 23 new assertions use raw routes in test/server.py, placed next to the chunked-framing ones. They cover each change above, and also:

• extra bytes arriving in a later read
• Content-Length: 0 on a held connection
• the Int boundary: 2147483647 is still a length, 2147483648 is refused
• the cookie-jar path
• a read through request-stream and ResponseStream.poll

The held routes keep the connection open for 10 s after the body, and the assertions require a return within 5 s by System.time. A slow runner has about 5 s of slack, and a client that waits fails by about 5 s.

Each part was reverted on its own and the full suite run against it:

Reverted part	Suite	Assertions that fail
cap at the length (take! hands out the whole chunk)	199/5	the three extra-bytes tests, plus the identical list and identical lines, whose responses also carry extra bytes
stop once the length is spent (poll-raw's first clause)	201/3	the three held-connection tests
splitting comma lists	202/2	identical list, differing list
agreement check	201/3	differing list, differing lines, cookie-jar path
digit check	200/4	-5, 5x, +5, and the Parse classification
rejecting an empty value	203/1	empty
overflow check	segfault	2147483648 wraps to a negative length and the slice crashes
overflow check as >= (off by one)	203/1	2147483647 is still a length
Transfer-Encoding override	202/2	chunked + abc, xchunked + 5
bodyless skip	195/9	204 and HEAD with an invalid length, plus seven existing HEAD tests

carp-fmt --check and ruff check test/server.py pass, and angler built from its current main (e8525db) is clean on every file CI lints.

---

Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.

[carpentry-reviewer]

Build & Tests

• CI: test (ubuntu-latest) and test (macos-latest) both pass at c084a69.
• Local (32-bit armhf Pi): I built and ran test/http-client.carp against test/server.py on 8791/8792. It gave 204 passed, 0 failed, the count the PR body reports (main: 181/0).

Findings

I found no bugs. Here is what I checked.

1. The before-values hold on main. I ran main (d3921ad) and this branch side by side against a raw-socket server. On 32-bit, main gave:

• 5 with helloEXTRA: helloEXTRA
• 5 with hello, held open 3 s: returned after 3 s (branch: 0 s)
• -5 and empty: hello
• 5x: hel
• +5: a truncation error
• 5, 6: hello
• 4294967301: a truncation error

All of these match the PR's main column. I could not reproduce the 64-bit wrap here: long is 32 bits on armhf, so strtol saturates to 2147483647. The claim does follow from core's Int_from_MINUS_string_MINUS_internal, which is (int)strtol(...) in carp_string.h, and the branch's length-wraps assertion is green on both 64-bit CI runners.

2. Rows the suite does not cover. Every one behaves as intended:

Response	main	branch
Content-Length: 5 (extra spaces), helloEXTRA	helloEXTRA	hello
HTAB 5 HTAB, helloEXTRA	helloEXTRA	hello
21-digit zero-padded …0005, helloEXTRA	helloEXTRA	hello
0, EXTRA	EXTRA	empty body
lines 5 and 5, 5, helloEXTRA	helloEXTRA	hello
lines 5 and content-length: 5, 6	hello	Parse: conflicting 5 and 6
hel with the headers, lo 0.3 s later, then held 3 s	3 s	0 s (the end clause fires on the read path too)
headers, hel, loEXTRA in three separate writes	helloEXTRA	hello
Transfer-Encoding: identity + 5, helloEXTRA	helloEXTRA	helloEXTRA (TE wins, read to the close)
302 hop carrying Content-Length: -1, then a 200	followed	followed (a hop's length is never read)
HEAD, or 304, with Content-Length: 5, held 3 s	3 s	3 s
204 + Content-Length: 0 + stray STRAY	body STRAY	body STRAY

The last two rows are unchanged, as the PR body says. I also sent seven rows through both request-stream with poll and Client.get: extra, held, late-held, three-write split, zero, short body, and -5. The two paths gave the same result on each.

3. Mutation testing aimed at the call sites. I made one env-gated build and ran the full suite once per mutant:

Mutant	Suite
none (control)	204/0
poll-raw read branch returns the chunk without take! (http-client.carp:350)	170/34
poll-raw leftover branch without take! (:344)	191/13
end clause off (:340), the PR's "stop" row	201/3, as in the PR
String.trim dropped in agree-length (:608)	202/2
only a chunked Transfer-Encoding overrides Content-Length (:640)	203/1
plain path builds a length-less stream instead of calling open-stream (:809)	183/21
the same bypass on the cookie-jar path (:1126)	203/1

All are killed. Only one assertion pins the jar path's wiring ("the same holds on the cookie-jar path", conflicting lines). That is enough, because both paths share open-stream.

4. Minor, optional: empty list members. 5,,5, 5, and ,5 all fail with invalid Content-Length ''. Refusing them is defensible. Content-Length is 1*DIGIT, not a #list field, so the empty-element rule in RFC 9110 §5.6.1 does not apply. Accepting an identical list is §8.6's MAY, and Go refuses every list. But '' reads as "the header was empty". If agree-length's error quoted the whole field value ('5,,5'), it would say what the server actually sent.

5. The 2 GiB trade-off. The body states it honestly. One fact for the decision: tracking remaining as (Maybe Long) would not break llm. llm's only positional ResponseStream.init is test/llm.carp:41-48 on llm main. It passes (Maybe.Nothing) in the remaining slot (line 47), which type-checks against (Maybe Long) just as well. No other carpentry-org package calls ResponseStream.remaining. So keeping >2 GiB downloads through request-stream would cost the field and getter type, plus Long arithmetic in parse-length and take!. It would not cost llm. Either way, refusing is the safe default: a wrapped length must never pass as a clean body.

6. Two wording nits in README.md:96-100.

• "too large for an Int" won't tell a reader on a 64-bit host that the cap is 2147483647 bytes, since Int is 32-bit there too. Stating the number would.
• "fails the request" holds only for responses that carry a body. HEAD, 1xx, 204 and 304 responses skip the check, as the PR body says.

Pre-existing and out of scope. Each of these behaves the same on main:

• A 100 Continue is taken as the final response. The caller gets status 100, and the real response, header block included, becomes the body. read-headers reads only the first header block, so any 1xx (103 Early Hints, for example) goes down the same path.
• Content-Length : 5, with a space before the colon, is not recognised, so that body reads to the close.
• The PR keeps the bodyless read-to-close on purpose. Stray bytes after a 204 become its body, and HEAD and 304 wait for the close. That is a natural follow-up: return Just 0 for bodyless? in declared-length, after the 1xx handling above.

Verdict: merge

It builds, passes 204/0 locally and is green on both CI runners. The before-values check out, and every call-site mutant is killed. The only open item is hellerve's call on the 2 GiB trade-off (point 5).