Repository navigation
Reject impossible fields and trailing input in strptime - #31
Conversation
Numeric fields take ASCII digits only; a two-digit field may still be space-padded in place of its leading zero, as asctime dates are. %m, %d, %H, %I, %M and %S are held to the ranges the doc string lists, and %z minutes to 00-59. An explicit month and day must name a day that exists in the parsed year (year 0, a leap year, when the format has none). Input left over once the format is exhausted is an error. Before, 2100-02-31 parsed as 3 March, month 13 made to-ordinal read past the month tables, signed or out-of-range fields went through, and trailing text was ignored.
There was a problem hiding this comment.
Build & Tests
- On
2931e06, run the way CI runs it (-Werrorthrough a_ci_werror.carpwrapper), with a private output directory andCARP_DIRset to the local core plus carp-lang/Carp#1587: 372 passed, 0 failed. CI is green on both legs, including lint, format and gendocs. - The 32 new assertions are additions only. No existing test changed, so 340 + 32 = 372 checks out.
- Teeth. I built the suite once, with each check behind an environment switch, and ran the control first. Every row matches the PR's table. For
%mI confirmed the abort, but did not re-run with the month-13 test removed:
| check removed | suite | failing tests |
|---|---|---|
| none (control) | 372 / 0 | — |
digits only (master's Int.from-string) |
367 / 5 | negative day, plus sign, tab-padded day, space-padded %Y, space-padded %z |
| space padding | 370 / 2 | space-padded month, asctime day |
%m range |
aborts, exit 134 | unsafe-nth's n < a.len assert, as the PR says |
%d range |
371 / 1 | day 00 |
%H / %I / %M range |
370 / 2 each | hour 99 and 24 / %I 13 and 00 / minute 99 and 60 |
%S range |
371 / 1 | second 60 |
%z minutes |
370 / 2 | 99 and 60 minutes |
| month length | 368 / 4 | 31 Feb, 29 Feb 2100, 31 Apr, Feb 30 via %b %d |
| trailing input | 370 / 2 | trailing text, a lone trailing space |
- Before-values. Every master value the description quotes reproduces on
848bbe2:2100-02-31gives day 31.to-ordinal→from-ordinaland the unix-timestamp round trip both give 2100-03-03.2024-13-01gives month 13, andto-ordinalon it aborts on the assert.2024-03--5gives day -5,99:00hour 99 and13:00 PMhour 13.+0999gives 38340 s, which is 9 h 99 min.2024-03-15 junkparses with the junk dropped.
Findings
No defects found. Details of what I checked:
Differential, master vs branch
I ran 9,753 inputs over 28 formats through both versions:
- Formats: the five
HttpDate.parseformats, ISO date and time,%j, ISO week,%U/%W, the%c %D %r %F %Texpansions,%n,%Z,%z, and partial formats. - Field values: each numeric field was swept with digits, out-of-range values, a leading space, a sign, each of tab, CR, LF, VT and FF, a short token, a long token, and a two-byte non-ASCII digit.
- Whole strings: suffixes such as a space,
junk, CR, LF, tab andGMT, a leading space, and truncation. - Calendar and clock: every day 00–32 of months 00–13 in 2023, 2024, 2000 and 2100, in three formats, plus a 9 × 6 × 6 time-of-day grid.
Results:
- No widening: 0 inputs that master rejects and the branch accepts.
- No changed values: 0 inputs that both accept with a different result.
- Narrowings: 2,744 inputs that master accepts and the branch rejects. All fall in the PR's four classes:
- out-of-range field: 1,693
- sign, whitespace or non-digit in a numeric field: 687
- trailing input: 286
- day past the end of its month: 78
- Calendar truth table: on the 5,715 rows it can check, the branch accepts exactly the real dates and times, with 0 mismatches.
The digits-only rule also fixes a master defect the description doesn't list. -1 is the "not given" sentinel for %H, %M and %S, so master parses Fri Mar 15 -1:30:45 2024 under %c as a date with no hour at all, and 14:30:-1 as having no seconds. The branch rejects both.
Downstream
I loaded this strptime over time 0.7.0 in each suite and checked the override took: the trailing-input message appears in the generated main.c.
- http (
f022bf8, 0.7.0): 516 / 0, the same as without the override. - web at carpentry-org/web#70's head (
cf894f7): 440 / 0, the same as without it.
Measured rows for the two decisions this leaves to hellerve
These are not defects; they are facts for the decisions.
1. Second 60. I loaded this strptime under web cf894f7 and added probe tests. Once http and web bump:
| request | today | with this PR |
|---|---|---|
If-Modified-Since at 23:59:60, after Last-Modified |
304 | 200 (ignored) |
If-Unmodified-Since at 23:59:60, before Last-Modified |
412 | 200 (ignored) |
One more fact for the decision: cookie Expires goes through the same HttpDate.parse. RFC 9110 §5.6.7 allows 23:59:60 in an HTTP-date. RFC 6265 §5.1.1 caps a cookie date's second at 59 ("leap seconds cannot be represented in this syntax"). So widening %S to 60 follows RFC 9110 for HTTP-dates, and lets cookie dates accept one value more than RFC 6265 allows. Both directions are harmless in practice.
2. Deleting web's gate afterwards is not quite a no-op. I removed web-strict-date-headers and loaded this strptime:
- Unchanged: all five of web's gate tests still pass. Those are the 31 Feb IMS and IUS, an IMS with trailing text, a lower-case IMS, and an IMS next to a rejected IUS.
- Changed: a dashed cookie-date
If-Modified-Since: Fri, 31-Dec-2100 23:59:59 GMTis ignored today (200) and honoured without the gate (304).HttpDate.parseaccepts the dashed forms on purpose, for cookies; that is class 4 of carpentry-org/web#70's differential.
So the plan to "drop the strict date parser" still has that one row to decide when it is slated.
Verdict: merge
The change is correct. Its tests catch every removed check exactly as the PR's table says. The differential shows it rejects only what the description says it rejects, and both downstream suites are unchanged. It is a draft waiting on the :60 call, and either answer is a one-number change.
Datetime.strptimeaccepted values its own doc string rules out. On master:2100-02-31parses as day 31, which every derived value then treats as 3 March.2024-13-01parses as month 13.to-ordinalon the result reads past the 13-element month tables and aborts onArray.unsafe-nth'sn < a.lenassert.2024-03--5gives day -5,99:00gives hour 99,13:00 PMgives hour 13, and+0999gives an offset with 99 minutes.2024-03-15 junkparses, and the junk is dropped.What changes:
6), as asctime dates are (Sun Nov 6 08:49:37 1994).%zis a sign followed by exactly four digits.%m,%d,%H,%I,%Mand%Sare held to the ranges the doc string already listed, and the%zminutes to 00–59. The already-ranged fields are unchanged.02-29: the year is then 0, which is a leap year. The doc string says so.Unchanged: partial formats still leave missing fields at 0 or
Nothing, and the weekday is still not checked against an explicit date.New errors name the field and the value, for example
%m out of range: 13,%d out of range for 2100-02: 31,%z out of range: +0560andunexpected trailing input at position 10: " junk". Non-digit input keeps itsfailed to parse %d.Docs. The doc string and the README's Parsing section now state these guarantees.
Tests. No existing test changed. Run the way CI runs them (
-Werrorthrough_ci_werror.carp): 372 passed, 0 failed, which is the 340 from before plus 32 new. The new tests cover each rejected shape above plus these boundaries:%I12 (accepted), 00 and 13 (rejected)+0530(accepted) and+0560(rejected)One thing for you to decide. RFC 9110 allows second
60(a leap second) in an HTTP-date.HttpDate.parseused to accept:60and read it as the next second; web #70 notes that web's own parser does the same. With%Sheld to the documented 00–59,:60is now rejected. If HTTP-dates should keep it, widen%Sto 00–60, a one-number change.Teeth: each check removed on its own
%Y, space-padded%z%mrangeunsafe-nthassert, insidedays-in-month; with that test removed, month 00 fails%drange%Hrange%Irange%I13,%I00%Mrange%Srange%zminutesFeb 30via%b %dOracle: Python 3.11
datetime.strptimeThe inputs were 4484 fixed-width strings:
%H %I %M %S %m %dfrom 00 to 99%H:%M:%Sboundary grid%I:%M %pfor 00–13 with AM and PM%zfor both signs, hours 00, 05, 12, 14, 23, 24 and 99, and minutes 00–99%m-%dfor days 28–314243 agree. That includes all 1848 dates, all 600 single fields, the whole time grid, and every
%zwith an hour of 23 or below. The 241 that diverge:%zwith hour 24 or 99 (240). Accepted here, while Python refuses any offset of 24 h or more. Only the minutes were in scope.02-29under%m-%d(1). Year 0 is a leap year here, while Python's default year, 1900, is not.Python also differs outside the fixed-width table:
2024-3-5,9:05), which stay errors here.%d, where this parser allows it on any two-digit field.Seconds 60 and 61 are not a divergence.
time.strptimetakes them, butdatetime.strptimerejects them, as this parser now does.Downstream: http (read-only)
HttpDate.parseis the only caller ofstrptime. What it receives:Expires: the tail of an attribute thatCookie.parse-set-prophas already trimmed.If-Modified-Since/If-Unmodified-Since: trimmed byRequest.parse. A header map built by hand passes through as it is.If-Range: trimmed inif-range-date-match?.So trailing whitespace from a parsed message never reaches
strptime. Trailing text other than whitespace is now rejected:If-Modified-Since: <date>; length=Nis now ignored, which is what RFC 9110 §13.1.3 asks for.Expires=<date> junknow takes the error path that any unparsable date already takes.I ran http's suite with this
strptimeloaded over time 0.7.0. It gives 516 passed, 0 failed, the same as on 0.7.0.Once this ships, http and web can bump, and web can drop the strict date parser it keeps as a gate in front of
Precondition(#65's parser, kept in #70). I've left all of that to you.🤖 Generated with Claude Code
Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.