N-ary functions: make arity structural across the compiler - #8557
N-ary functions: make arity structural across the compiler#8557cristianoc wants to merge 6 commits into
Conversation
7d0341c to
8c402ae
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## codex/nary-parsetree #8557 +/- ##
=======================================================
Coverage ? 75.94%
=======================================================
Files ? 475
Lines ? 62856
Branches ? 0
=======================================================
Hits ? 47736
Misses ? 15120
Partials ? 0
🚀 New features to boost your workflow:
|
rescript
@rescript/belt
@rescript/darwin-arm64
@rescript/darwin-x64
@rescript/linux-arm64
@rescript/linux-x64
@rescript/runtime
@rescript/win32-x64
commit: |
|
Developer playground preview: https://rescript-lang.github.io/rescript/dev-playground/?version=pr-8557 |
|
@cknitt tried to put the entire overview together, in this draft PR, before splitting into individual PRs. |
It's already good that CI is green. I can also try to test against a large company project of ours tomorrow. |
|
Starting to extract the first couple of commits, which are just low risk bug fixes, into PRs. |
Done, seeing a single change in compiler output, and that is just the variable name:
for a function parameter |
| }); | ||
| }; | ||
|
|
||
| let non_terminate = g(x); |
There was a problem hiding this comment.
Lost some optimization / inlining here.
| ]); | ||
|
|
||
| if (!$eq$tilde(sort(u), [ | ||
| let x = sort(u); |
263febd to
a43dd97
Compare
19f6137 to
49ff009
Compare
49ff009 to
c2a8f61
Compare
Types.Tarrow carries a parameter list (Tarrow of arg list * type_expr);
Texp_function carries typed parameters {fp_lbl; fp_param; fp_pat;
fp_partial} and a body; Ttyp_arrow and Otyp_arrow follow. The arity
annotation and its int-option phantom state are gone from the compiler.
Type relations compare parameters pairwise; a length mismatch is
structural incompatibility (which also makes mcomp's arrow verdict
sound: arrows of different lengths can never unify). filter_arrow
becomes filter_arrow_n. type_function types all parameters against one
arrow, checking expected labels up front to preserve the dedicated
Abstract_wrong_label diagnostics; optional-parameter defaults desugar
to uniquified *opt_<label>* bindings stacked at the head of the body.
type_application is a single parameters-to-arguments matching loop
preserving the legacy commutation, optional auto-fill, eta-expansion
placeholder, and error-selection behavior.
translcore's push_defaults is deleted (defaults now sit in the body by
construction) and transl_function walks the parameter list, keeping the
active-pattern split. Downstream, the gather-until-arity walkers in
gentype and the outcome printer, reanalyze's two arity-corrective
helpers, and typedecl's structural arity fallback are all deleted.
The cmi and cmt magic numbers are bumped to Caml1999I023/Caml1999T023.
Generated JavaScript is byte-identical across the test suite except:
- a bug fix: defaults of optional parameters in curried functions are
now computed when their own parameter group is applied ((~x=d, y) =>
(~z=d, w) => ... no longer defers x's default to the inner
application); pinned by the uncurried_default.args snapshot;
- optional-parameter internals are named *opt_<label>* instead of
*opt* in the one unprettified case (mario_game).
Error-message improvements: method arity mismatches report unlabelled
argument counts precisely, and missing-argument lists print in source
order. Reanalyze no longer emits spurious empty optional-argument
references; genType recovers real parameter names after defaulted
parameters.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Cristiano Calcagno <cristianoc@users.noreply.github.com>
270640e to
63be65d
Compare
c2a8f61 to
def62af
Compare
- The Too_many_arguments error cannot be raised anymore: the expected type is committed to an arrow of the literal's shape before destructuring, so every legacy path now surfaces as a regular type clash or Uncurried_arity_mismatch (which is what the fixture credited to it in ERROR_VARIANTS.md was already producing). Remove the variant, its printer, and the ?in_function threading through type_expect/type_cases that existed only to decorate it. - Remove the function$-vs-arrow unification bridge in ctype (nothing produces a function$ type expression anymore), the structural arity counter Ctype.arity (no callers), the parsetree arity probes get_uncurry_arity/get_curry_arity (is_arity_one reads the params list directly), and Ast_async's redundant newtype double-dig. - Deduplicate the arrow-flattening step shared by the analysis extract_function_type helpers. - Delete Ast_compatible outright. Every member was a thin veneer over an existing Ast_helper constructor (rec_type_str/rec_type_sig had byte-identical signatures to Str.type_/Sig.type_), hand-rolling the records the canonical constructors build; the OCaml-version compatibility it existed for is long gone, and its fun_ silently hard-coded arity 1. Call sites use Ast_helper directly, with two let-bound app1/app2 shorthands kept local to ast_derive_js_mapper. Generated code is unchanged. The Pjs_fn_make no-op elision explored alongside these cleanups is deliberately left out pending a dedicated analysis of that primitive. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Cristiano Calcagno <cristianoc@users.noreply.github.com>
With structural arity, "function of arity n" is a construction invariant rather than a goal state: translcore builds every Lfunction with exactly the parameters its type declares. The arity-enforcement layer therefore disappears: - Function literals are emitted directly; the Pjs_fn_make wrapper that every function passed through was resolved as a no-op by lam_pass_alpha_conversion, but only *after* deep_flatten, simplify_exits and simplify_alias had run with the function hidden inside an Lprim, acting as an accidental optimization barrier. - Pjs_fn_make_unit was a one-bit metadata channel: its entire effect was setting one_unit_arg so js_exp_make drops the unit parameter. translcore now sets the attribute directly, gated on the parameter pattern binding no identifiers (a () or _ pattern) - a more principled test than the alpha pass's check that the parameter was named "param". - The active-pattern currying split in transl_function is deleted. It preserved pattern-effect timing across curried application steps, which no longer exist: total applications supply all arguments at once and explicit partial application eta-defers the entire call. The old output proves the point - the split's closures were immediately applied by the arity adapter, so only the allocations are gone (see mutable_uncurry_test). - The I<N> unboxed-record producer (the @this method-callback encoding) is removed: the general Record_unboxed translation already returns the single field unboxed, and the wrapped value is a literal of matching arity. With no producers left, both primitive constructors and every consumer arm are deleted, including the 230-line unsafe_adjust_to_arity (its only callers were the two Pjs_fn_make resolution sites). On recursive modules: removing the wrapper lets the static recursive-module compilation path see module members that are plain functions, replacing the Primitive_module.init/update bootstrap with hoisted function declarations. This is safe because the static path's own applicability check now sees the functions it was designed to check - the wrapper was hiding them, pessimizing compilation - and the bootstrap demonstrably remains for members that are not plain functions (rec_module_test keeps its lazy/value cases dynamic). Verified: stdlib byte-identical; full test suite green; JS output changes limited to removed adapter closures, removed no-op module bootstraps, better name preservation, and constant propagation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Cristiano Calcagno <cristianoc@users.noreply.github.com>
Replace the Pexp_newtype wrapper chains that the parser built for (type t, x) => ... arrow syntax with a structural field on the function node: Pexp_fun.newtypes carries each newtype name with its own attributes, hoisted in front of the value parameters as before. Pexp_newtype remains solely as the desugaring of [let f: type a. ...] annotations and for PPX-authored trees. Fidelity fixes visible in the formatter: - Attributes keep their association with their type parameter group: (@attr type t, x, @attr2 type s, y) round-trips as written instead of printing @attr @attr2 on the function. - Comments written next to a type parameter travel with it to the hoisted group instead of migrating onto the following value parameter. - Attributes written in front of the arrow now live on the function node, so built-in attribute processing (e.g. @this) sees them on type-first functions; previously they sat inert on the wrapper node. Typing follows the upstream OCaml 5.x design: the newtype machinery is extracted into a reusable type_newtype helper (mirroring OCaml's helper of the same name) and the function case peels one newtype at a time, mimicking the typing of the former wrapper chain; the typedtree output is bit-identical to before. The v0 PPX bridge expands the field back into a wrapper chain around Function$: each wrapper carries its own newtype's attributes, and the outermost wrapper separates function-node attributes from the first newtype's attributes with an internal _res.newtype_attrs marker (no marker means node attributes only, matching the historical wire). Newtype-free programs are wire byte-identical; for functions with newtypes the deltas are confined to wrapper-node locations and, for the rare attributed groups, per-wrapper attribute placement. Identity-PPX round-trips are AST-exact, verified against the previous compiler. Also: jsx_v4 and bs_builtin_ppx now carry newtypes (and their attributes) through their function rebuilds instead of dropping them, the sexp AST debugger emits the field, and dead parser plumbing (fundef param attrs/p_pos, arrow_start_pos, make_newtypes ~attrs) is removed. Signed-Off-By: Cristiano Calcagno <cristianoc@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the desugared encoding of [let f: type a. t = e] - a Ptyp_poly
pattern constraint plus a Pexp_newtype chain over a Pexp_constraint,
with the type stored twice and no AST invariant ensuring that the copies
agree - with a structural field on the binding:
pvb_constraint: {pvc_newtypes: string loc list; pvc_type: core_type}
Only the [type a.] form uses the field; plain constraints and explicit
polymorphic annotations keep their existing representation. The type is
stored once, and [varify_constructors] now runs in exactly one place,
inside the type checker.
With functions already carrying their locally abstract type parameters in
Pexp_fun.newtypes, this removes the last place where the parser constructs
Pexp_newtype. Delete the constructor from the current parsetree, along with
the Texp_newtype exp_extra, which had no consumer beyond no-op iterators and
the debug printer. The CMT magic number is bumped to Caml1999T024; the CMI
format is unchanged.
Type checking follows the same design as the function case (and OCaml
5.x): type_let introduces the locally abstract types into scope via
type_newtype, types the body against the constraint, and unifies with the
pattern's polymorphic type. This preserves the semantics of the former
desugaring.
The frozen v0 PPX bridge expands the field back into the historical
wrapper-chain encoding and recognizes well-formed instances of that
encoding on the way in, verified by unit tests. A v0 Pexp_newtype chain
that cannot be represented - such as one that does not enclose ReScript's
Function$ encoding, or one whose structure was changed by a PPX - now
becomes a located ocaml.error extension with an explicit message. This is
the only intentional reduction in accepted v0 PPX output.
Formatter bug fix covered by syntax fixtures: a trailing comment between
the constraint type and [=] is no longer dropped. An end-to-end GADT test
checks that refinement still works with the new binding field.
Signed-Off-By: Cristiano Calcagno <cristianoc@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Function labels in Outcometree were strings, with optionality encoded by a leading question mark. That forced printers to decode the spelling and left downstream consumers without the label structure already known by the type system. Store Noloc.arg_label directly on Otyp_arrow, update both printers to match it exhaustively, and remove the unproduced Octy_arrow constructor. The doc generator previously walked Types.type_expr independently and flattened every reachable constructor into one list. Nested arrows became outer parameters, tuples and type variables disappeared, labels and optionality were lost, and non-function values acquired fabricated zero-parameter signatures. Build details from the normalized Outcometree instead: parameters retain their metadata, constructors, variables, tuples, and functions form recursive nodes, uncommon forms remain visible through a rendered fallback, and only top-level arrows receive signature details. Update the published RescriptTools.Docgen types and snapshots for the intentionally breaking JSON shape, and correct the implementation's stale alias tag to match the signature tag declared by its interface. The documentation site drops value details before publishing its data, but third-party consumers of rescript-tools doc need the changelog warning. Focused fixtures cover labeled and optional parameters, generic variables, callbacks, tuple returns, returned functions, fallback rendering, and non-function values. Compiler, tools, analysis, syntax, roundtrip, and full test suites remain green. Signed-off-by: Cristiano Calcagno <cristianoc@users.noreply.github.com>
def62af to
4fce94c
Compare
Note
Stack tracking PR. This draft tracks the remaining work: its diff contains only the not-yet-extracted commits (items 5–10 in the PR series section below). Items 1–3 have landed on master (#8559, #8561, #8563) and item 4 is in review as #8566, which this PR is based on. As each remaining item is extracted, review it in its own PR rather than here.
Functions are n-ary, arity is structural
This branch completes the transition started by uncurried-by-default: functions and arrow types are now n-ary at every level of the compiler — parsetree, typedtree, and
Types— and a function's arity is the length of its parameter list rather than anint optionannotation on the head of a curried chain. Locally abstract types follow the same principle: a function's newtypes are a field on the function node, andlet f: type a. t = eis a structural field on the binding, replacing the wrapper-node encodings.Why. The curried encoding survived only in the frontend, inherited from OCaml, and every layer paid for it: arity lived in three places that could disagree (the term, the type, and the backend's
.cmj), and some 30 hand-rolled "walk the chain until the arity marker" loops existed across the compiler, genType, reanalyze, and the editor tooling. Where they disagreed, the results were real bugs (see below). With structural arity, "declared arity = runtime arity" holds by construction, and the machinery that existed to enforce, compensate for, or work around the old encoding is deleted rather than maintained.Benefits
Soundness and correctness fixes (each found by the refactor, each pinned by a test):
:>coercion ignored arity, so a curriedint => int => intcould satisfy(int, int) => int; a first-class use of such a value then miscompiled (typedint, runtime closure). Now a compile error with a dedicated message.(~x=d, y) => (~z=d, w) => ...deferredx's default to the inner application).~x: int => string(unparenthesized) printed identically to(~x: int) => stringbut did not unify with it.@res.asyncattributes, dropped arrow/awaitattributes (one crashing the formatter with a stack overflow), lost JSX closing tags,assert falseon PPX-emitted OCaml-stylefunction.Formatter fidelity (from the structural newtypes):
typeparameter group:(@attr type t, x, @attr2 type s, y)round-trips as written instead of printing@attr @attr2on the function.type a.constraint and=is no longer dropped.Correct docgen output: the structured
detailproduced byrescript-tools docwas corrupt — nested arrows flattened into extra parameters, labels and optionality were discarded, tuples and type variables vanished and shifted the parameter/return split, and non-functions got fabricated zero-parameter signatures. Details are now built from the same normalized outcome tree as the printed signature: parameters carrylabel/optional, types are recursive nodes (constructor/variable/tuple/function/rendered), and only functions get signature details. This is a breaking change to the publishedRescriptTools.DocgenJSON schema (see Risks).Better generated code:
(param => {...})(a)(b)); these are gone (seemutable_uncurry_test.mjs).Primitive_module.init/updateruntime bootstrap (seerec_module_test.mjs). The bootstrap remains where it is load-bearing.Pjs_fn_makewrapper acted as an accidental optimization barrier: early Lambda passes saw anLprimwhere a function was. With it gone, user variable names survive more often and constants propagate (e.g.param_0/param_1become the user'su/v).$staropt_dir$starinstead of$staropt$star$1) in the rare unprettified case.Better error messages: arity mismatches report precise unlabelled-argument counts; missing-argument lists print in source order; the confusing "This labeled function is applied to arguments in an order different from other calls" restriction is gone (labels commute for inferred functions too, soundly).
Less compiler, with test coverage added. Deleted outright: the parsetree arity annotation and
ast_uncurried.ml;push_defaults; thePjs_fn_make/Pjs_fn_make_unitprimitives and the 230-lineunsafe_adjust_to_arity; the gather-until-arity walkers in genType (×2), reanalyze (×2), the outcome printer, and the editor tooling; the unreachableToo_many_argumentserror and its?in_functionplumbing; the parser/printer mirrored@as-arity hacks; thePexp_newtype/Texp_newtypewrapper encoding, the parser'swrap_type_annotationdouble-type dance, and the'?'-in-string label smuggling inOtyp_arrow(plus the deadOcty_arrow).Better tooling output: signature help no longer includes the opening paren in the first parameter's range; genType recovers real parameter names after defaulted parameters; reanalyze stops emitting spurious empty optional-argument references.
Risks
detailJSON emitted byrescript-tools docand the publishedRescriptTools.Docgentypes changed shape (the old shape was unusable — see Benefits). The documentation site does not consumedetail; third-party consumers must adapt.Pexp_newtyperemoved from the current parsetree, a v0 locally-abstract-type wrapper that the bridge cannot represent (e.g. PPX-synthesizedfun (type a) ->with no arity wrapper, or atype a.molecule a PPX perturbed) becomes a locatedocaml.errorextension with an explicit message, instead of passing through. Compiler-produced shapes round-trip exactly (unit-tested, including the diagnostic).@this this => async arg => ...now means what it says (a method returning an async function) instead of the old chain-walk absorbing the nested lambda's parameter into the method. Relatedly, an attribute written in front of a type-first arrow (@this (type t, x) => ...) now lands on the function node and takes effect; it previously sat inert on a wrapper node.I023, cmt magic isT024; clean builds are required, and cmt-consuming tools must be rebuilt in lockstep (all in-tree consumers are updated here).type_functionandtype_applicationin typecore andtransl_functionin translcore. Mitigations: generated JS is byte-identical across the stdlib and the test corpus except for the deliberate improvements listed above; the full suites (syntax round-trip, super_errors, build tests, gentype, analysis, tools, ounit) pass at every commit; an adversarial corpus covers label commutation, optional inference, partial application, and the reject-side of every closed soundness hole._res.arrow_node_attrsand_res.newtype_attrsmarkers appear when a node's attribute split must survive the single v0 attribute slot;Has_arityNnow always equals the arrow-chain length (previously not true for@as-phantom externals); and the synthesized newtype/constraint wrapper nodes carry slightly different location values than the old parser produced (structure and attributes are exact; verified by loading both wires through the same frontend).make test-rewatchis red on master itself (the vendoredsuryuses the removedJsnamespace) — unrelated to this branch.PR series
Each commit builds and passes the full suite independently. Extraction proceeds bottom-up; as each PR below merges, this PR's base moves down the stack and its diff shrinks accordingly. (Item numbers are stable — cross-references like "rides with 5" refer to them.)
Merged:
@asarity fudge and its printer compensation)In review:
Remaining — this PR's diff, bottom to top:
Tarrow/Texp_functionparams; cmt+cmi bump; downstream tools adapt in lockstep)Pjs_fn_make,Pjs_fn_make_unit, andunsafe_adjust_to_arity(rides with 5; contains the recursive-module rationale)Pexp_newtype/Texp_newtype, bumps cmt toT024, contains the PPX-surface narrowing)Otyp_arrowlabels + the docgen schema change; landable any time after 5)Of the items originally deferred here: docgen precision and structured
Otyp_arrowlabels landed as commit 10; per-parameter newtypes resolved into commits 8–9 after design review (front-hoisting is intentional normalization, so newtypes became structural fields rather than positional parameters, mirroring what OCaml 5.1/5.2 did withPvc_constraintandtype_newtype); optionality-as-a-parameter-field was analyzed and declined — the churn outweighs the payoff, and the v0 wire keepsOptionallabels regardless.🤖 Generated with Claude Code