Skip to content

Reject impossible fields and trailing input in strptime - #31

Merged
hellerve merged 1 commit into
masterfrom
claude/strptime-strict-fields
Oct 3, 2026
Merged

hellerve merged 1 commit into
masterfrom
claude/strptime-strict-fields

Conversation

@carpentry-agent

Copy link
Copy Markdown

Datetime.strptime accepted values its own doc string rules out. On master:

  • 2100-02-31 parses as day 31, which every derived value then treats as 3 March.
  • 2024-13-01 parses as month 13. to-ordinal on the result reads past the 13-element month tables and aborts on Array.unsafe-nth's n < a.len assert.
  • 2024-03--5 gives day -5, 99:00 gives hour 99, 13:00 PM gives hour 13, and +0999 gives an offset with 99 minutes.
  • 2024-03-15 junk parses, and the junk is dropped.

What changes:

  • Digits only. Numeric fields take ASCII digits, with no sign and no other whitespace. A two-digit field may still be space-padded in place of its leading zero ( 6), as asctime dates are (Sun Nov 6 08:49:37 1994). %z is a sign followed by exactly four digits.
  • Ranges. %m, %d, %H, %I, %M and %S are held to the ranges the doc string already listed, and the %z minutes to 00–59. The already-ranged fields are unchanged.
  • Real dates. When the format gives both a month and a day, the day must exist in that month of the parsed year. A format without a year still parses 02-29: the year is then 0, which is a leap year. The doc string says so.
  • Whole input. Anything left over once the format is exhausted is an error.

Unchanged: partial formats still leave missing fields at 0 or Nothing, and the weekday is still not checked against an explicit date.

New errors name the field and the value, for example %m out of range: 13, %d out of range for 2100-02: 31, %z out of range: +0560 and unexpected trailing input at position 10: " junk". Non-digit input keeps its failed to parse %d.

Docs. The doc string and the README's Parsing section now state these guarantees.

Tests. No existing test changed. Run the way CI runs them (-Werror through _ci_werror.carp): 372 passed, 0 failed, which is the 340 from before plus 32 new. The new tests cover each rejected shape above plus these boundaries:

  • Feb 29 in 2024 and 2000 (accepted) and in 2100 (rejected)
  • Apr 30 (accepted) and Apr 31 (rejected)
  • 23:59:59 (accepted) and 24:00 (rejected)
  • %I 12 (accepted), 00 and 13 (rejected)
  • the space-padded asctime day
  • +0530 (accepted) and +0560 (rejected)
  • trailing text and a lone trailing space (both rejected)

One thing for you to decide. RFC 9110 allows second 60 (a leap second) in an HTTP-date. HttpDate.parse used to accept :60 and read it as the next second; web #70 notes that web's own parser does the same. With %S held to the documented 00–59, :60 is now rejected. If HTTP-dates should keep it, widen %S to 00–60, a one-number change.

Teeth: each check removed on its own
check removed failing tests
digits only negative day, plus sign, tab-padded day, space-padded %Y, space-padded %z
space padding space-padded month, asctime day
%m range the month-13 test aborts the run on the same unsafe-nth assert, inside days-in-month; with that test removed, month 00 fails
%d range day 00
%H range hour 99, hour 24
%I range %I 13, %I 00
%M range minute 99, minute 60
%S range second 60
%z minutes 99 minutes, 60 minutes
month length 31 February, 29 February 2100, 31 April, Feb 30 via %b %d
trailing input trailing text, a lone trailing space
Oracle: Python 3.11 datetime.strptime

The inputs were 4484 fixed-width strings:

  • every day 00–32 of months 00–13 in 2023, 2024, 2000 and 2100
  • each of %H %I %M %S %m %d from 00 to 99
  • a 560-entry %H:%M:%S boundary grid
  • %I:%M %p for 00–13 with AM and PM
  • %z for both signs, hours 00, 05, 12, 14, 23, 24 and 99, and minutes 00–99
  • %m-%d for days 28–31

4243 agree. That includes all 1848 dates, all 600 single fields, the whole time grid, and every %z with an hour of 23 or below. The 241 that diverge:

  • %z with hour 24 or 99 (240). Accepted here, while Python refuses any offset of 24 h or more. Only the minutes were in scope.
  • 02-29 under %m-%d (1). Year 0 is a leap year here, while Python's default year, 1900, is not.

Python also differs outside the fixed-width table:

  • It accepts single-digit fields (2024-3-5, 9:05), which stay errors here.
  • It allows a space pad only for %d, where this parser allows it on any two-digit field.

Seconds 60 and 61 are not a divergence. time.strptime takes them, but datetime.strptime rejects them, as this parser now does.

Downstream: http (read-only)

HttpDate.parse is the only caller of strptime. What it receives:

  • Set-Cookie Expires: the tail of an attribute that Cookie.parse-set-prop has already trimmed.
  • If-Modified-Since / If-Unmodified-Since: trimmed by Request.parse. A header map built by hand passes through as it is.
  • If-Range: trimmed in if-range-date-match?.

So trailing whitespace from a parsed message never reaches strptime. Trailing text other than whitespace is now rejected:

  • Old IE's If-Modified-Since: <date>; length=N is now ignored, which is what RFC 9110 §13.1.3 asks for.
  • Expires=<date> junk now takes the error path that any unparsable date already takes.

I ran http's suite with this strptime loaded over time 0.7.0. It gives 516 passed, 0 failed, the same as on 0.7.0.

Once this ships, http and web can bump, and web can drop the strict date parser it keeps as a gate in front of Precondition (#65's parser, kept in #70). I've left all of that to you.

🤖 Generated with Claude Code


Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.

Numeric fields take ASCII digits only; a two-digit field may still be
space-padded in place of its leading zero, as asctime dates are. %m, %d,
%H, %I, %M and %S are held to the ranges the doc string lists, and %z
minutes to 00-59. An explicit month and day must name a day that exists
in the parsed year (year 0, a leap year, when the format has none). Input
left over once the format is exhausted is an error.

Before, 2100-02-31 parsed as 3 March, month 13 made to-ordinal read past
the month tables, signed or out-of-range fields went through, and
trailing text was ignored.

@carpentry-reviewer carpentry-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build & Tests

  • On 2931e06, run the way CI runs it (-Werror through a _ci_werror.carp wrapper), with a private output directory and CARP_DIR set to the local core plus carp-lang/Carp#1587: 372 passed, 0 failed. CI is green on both legs, including lint, format and gendocs.
  • The 32 new assertions are additions only. No existing test changed, so 340 + 32 = 372 checks out.
  • Teeth. I built the suite once, with each check behind an environment switch, and ran the control first. Every row matches the PR's table. For %m I confirmed the abort, but did not re-run with the month-13 test removed:
check removed suite failing tests
none (control) 372 / 0 —
digits only (master's Int.from-string) 367 / 5 negative day, plus sign, tab-padded day, space-padded %Y, space-padded %z
space padding 370 / 2 space-padded month, asctime day
%m range aborts, exit 134 unsafe-nth's n < a.len assert, as the PR says
%d range 371 / 1 day 00
%H / %I / %M range 370 / 2 each hour 99 and 24 / %I 13 and 00 / minute 99 and 60
%S range 371 / 1 second 60
%z minutes 370 / 2 99 and 60 minutes
month length 368 / 4 31 Feb, 29 Feb 2100, 31 Apr, Feb 30 via %b %d
trailing input 370 / 2 trailing text, a lone trailing space
  • Before-values. Every master value the description quotes reproduces on 848bbe2:
    • 2100-02-31 gives day 31. to-ordinal → from-ordinal and the unix-timestamp round trip both give 2100-03-03.
    • 2024-13-01 gives month 13, and to-ordinal on it aborts on the assert.
    • 2024-03--5 gives day -5, 99:00 hour 99 and 13:00 PM hour 13.
    • +0999 gives 38340 s, which is 9 h 99 min.
    • 2024-03-15 junk parses with the junk dropped.

Findings

No defects found. Details of what I checked:

Differential, master vs branch

I ran 9,753 inputs over 28 formats through both versions:

  • Formats: the five HttpDate.parse formats, ISO date and time, %j, ISO week, %U/%W, the %c %D %r %F %T expansions, %n, %Z, %z, and partial formats.
  • Field values: each numeric field was swept with digits, out-of-range values, a leading space, a sign, each of tab, CR, LF, VT and FF, a short token, a long token, and a two-byte non-ASCII digit.
  • Whole strings: suffixes such as a space, junk, CR, LF, tab and GMT, a leading space, and truncation.
  • Calendar and clock: every day 00–32 of months 00–13 in 2023, 2024, 2000 and 2100, in three formats, plus a 9 × 6 × 6 time-of-day grid.

Results:

  • No widening: 0 inputs that master rejects and the branch accepts.
  • No changed values: 0 inputs that both accept with a different result.
  • Narrowings: 2,744 inputs that master accepts and the branch rejects. All fall in the PR's four classes:
    • out-of-range field: 1,693
    • sign, whitespace or non-digit in a numeric field: 687
    • trailing input: 286
    • day past the end of its month: 78
  • Calendar truth table: on the 5,715 rows it can check, the branch accepts exactly the real dates and times, with 0 mismatches.

The digits-only rule also fixes a master defect the description doesn't list. -1 is the "not given" sentinel for %H, %M and %S, so master parses Fri Mar 15 -1:30:45 2024 under %c as a date with no hour at all, and 14:30:-1 as having no seconds. The branch rejects both.

Downstream

I loaded this strptime over time 0.7.0 in each suite and checked the override took: the trailing-input message appears in the generated main.c.

  • http (f022bf8, 0.7.0): 516 / 0, the same as without the override.
  • web at carpentry-org/web#70's head (cf894f7): 440 / 0, the same as without it.

Measured rows for the two decisions this leaves to hellerve

These are not defects; they are facts for the decisions.

1. Second 60. I loaded this strptime under web cf894f7 and added probe tests. Once http and web bump:

request today with this PR
If-Modified-Since at 23:59:60, after Last-Modified 304 200 (ignored)
If-Unmodified-Since at 23:59:60, before Last-Modified 412 200 (ignored)

One more fact for the decision: cookie Expires goes through the same HttpDate.parse. RFC 9110 §5.6.7 allows 23:59:60 in an HTTP-date. RFC 6265 §5.1.1 caps a cookie date's second at 59 ("leap seconds cannot be represented in this syntax"). So widening %S to 60 follows RFC 9110 for HTTP-dates, and lets cookie dates accept one value more than RFC 6265 allows. Both directions are harmless in practice.

2. Deleting web's gate afterwards is not quite a no-op. I removed web-strict-date-headers and loaded this strptime:

  • Unchanged: all five of web's gate tests still pass. Those are the 31 Feb IMS and IUS, an IMS with trailing text, a lower-case IMS, and an IMS next to a rejected IUS.
  • Changed: a dashed cookie-date If-Modified-Since: Fri, 31-Dec-2100 23:59:59 GMT is ignored today (200) and honoured without the gate (304). HttpDate.parse accepts the dashed forms on purpose, for cookies; that is class 4 of carpentry-org/web#70's differential.

So the plan to "drop the strict date parser" still has that one row to decide when it is slated.

Verdict: merge

The change is correct. Its tests catch every removed check exactly as the PR's table says. The differential shows it rejects only what the description says it rejects, and both downstream suites are unchanged. It is a draft waiting on the :60 call, and either answer is a one-number change.

@hellerve
hellerve merged commit 73a7cc2 into master Oct 3, 2026
2 checks passed
@hellerve
hellerve deleted the claude/strptime-strict-fields branch October 3, 2026 07:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant