Skip to content

ci: split push.yml into reusable workflows, share build steps via composite action - #1710

Merged
johnnyreilly merged 18 commits into
mainfrom
ci-split-workflows
Oct 4, 2026
Merged

johnnyreilly merged 18 commits into
mainfrom
ci-split-workflows

Conversation

@johnnyreilly

@johnnyreilly johnnyreilly commented Oct 2, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Replaces the single push.yml with ci.yml, which calls comparison-tests.yml, execution-tests.yml and lint.yml as reusable workflows (workflow_call)
    • push now only triggers on main (PRs no longer run CI twice), and in-progress runs are only cancelled for PRs
    • ci.yml grants the token contents: read only
    • The old standalone lint job (install + lint) now builds first; there's no separate build job, since every CI job builds via the composite action
  • Extracts the install/build steps into a composite action, .github/actions/build-ts-loader/action.yml, used by every workflow that builds ts-loader (comparison tests, execution tests, lint, benchmark, release)
    • On Windows it copies the checkout to C:\source\ts-loader and installs/builds there, giving the consistent path the comparison tests' output normalisation expects; callers no longer pass a working-directory
    • The Windows Defender exclusions previously in the benchmark and probe were dropped: hosted Windows runners already have real-time protection off and C:\/D:\ excluded (confirmed with Get-MpComputerStatus on this branch)
  • Adds macOS comparison tests and a macOS benchmark job (macOS execution tests are present but commented out for now)
  • Collapses the three benchmark jobs into one OS matrix job (every step in bash; check names, artifact names and PR comment markers unchanged)
  • Raises the benchmark's per-scenario time budget from 60s to 90s so cold typeCheck builds complete all their iterations, and documents the budget in test/benchmark-tests/README.md
  • Pins Node 24 with setup-node (and cache: yarn) in comparison tests, lint, and the test probe, which previously used each runner image's default Node; execution tests keep their Node matrix and gain cache: yarn. The release workflow deliberately has no cache
  • Removes TypeScript 5.6.2 from the execution-test matrix (now 5.7.2–6.0.2)
  • Drops sudo from the Ubuntu comparison tests
  • Generalizes and renames windows-test-probe.yml to test-probe.yml, now running on Ubuntu, macOS and/or Windows via a new os workflow_dispatch input (default all)
  • Documents test-probe.yml in AGENTS.md, points its execution-test matrix reference at execution-tests.yml, and corrects its description of how the benchmark runs the two sides

Before merging

  • Update main's branch protection required checks: the old contexts (Comparison Tests Ubuntu, Comparison Tests Windows, Execution Tests … (24, next, …)) will never report again. New contexts look like comparison tests / ubuntu, execution tests / windows (24, 6.0.2, 5), lint / lint

Test plan

  • ci.yml's reusable-workflow jobs (comparison tests, execution tests, lint) run green on this PR
  • Benchmark runs green on Ubuntu, macOS and Windows
  • windows-test-probe.yml (pre-rename) dispatched successfully against this branch across all three OSes
  • test-probe.yml dispatches successfully once this merges to main (workflow_dispatch workflows must be registered on the default branch under their current filename before they're dispatchable)
  • release.yml builds via the composite action on the next release

🤖 Generated with Claude Code

…posite action

Breaks the single push.yml into comparison-tests.yml, execution-tests.yml,
and lint.yml, called from push.yml as reusable workflows. Each OS job's
install/build steps (plus the Windows Defender exclusion and the
copy-to-C:\source\ts-loader workaround) are now shared via the
build-ts-loader composite action, also reused by the Windows test probe.
Also generalizes the probe to run on Ubuntu/macOS/Windows via a new os
input, defaulting to all three.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Benchmark (Ubuntu)

Scenario transpileOnly PR branch median (ms) base branch median (ms) Δ vs base branch
Cold build false 766.4 755.7 +1.4%
Incremental rebuild (leaf touch) false 53.3 53.6 -0.7%
Incremental rebuild (hub touch) false 298.4 291.0 +2.5%
Cold build true 467.2 458.9 +1.8%
Incremental rebuild (leaf touch) true 28.0 28.0 -0.0%
Incremental rebuild (hub touch) true 28.3 29.2 -3.1%

PR branch = /home/runner/work/ts-loader/ts-loader, base branch = /home/runner/work/ts-loader/ts-loader-main. 2 warmup + 10 measured iterations per scenario, median reported. Report-only - no threshold fails this check.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Benchmark (Windows)

Scenario transpileOnly PR branch median (ms) base branch median (ms) Δ vs base branch
Cold build false 1024.0 1018.1 +0.6%
Incremental rebuild (leaf touch) false 68.0 67.2 +1.3%
Incremental rebuild (hub touch) false 412.4 397.0 +3.9%
Cold build true 621.8 617.1 +0.8%
Incremental rebuild (leaf touch) true 33.0 33.1 -0.5%
Incremental rebuild (hub touch) true 32.3 33.0 -2.3%

PR branch = C:\source\ts-loader, base branch = C:\source\ts-loader-main. 2 warmup + 10 measured iterations per scenario, median reported. Report-only - no threshold fails this check.

johnnyreilly and others added 3 commits October 3, 2026 09:59
Reflects that it now probes comparison tests on Ubuntu/macOS/Windows,
not just Windows.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Benchmark (macOS)

Scenario transpileOnly PR branch median (ms) base branch median (ms) Δ vs base branch
Cold build false 1100.3 1108.7 -0.8%
Incremental rebuild (leaf touch) false 90.5 87.3 +3.7%
Incremental rebuild (hub touch) false 442.5 438.7 +0.9%
Cold build true 608.1 616.6 -1.4%
Incremental rebuild (leaf touch) true 52.0 49.1 +6.0%
Incremental rebuild (hub touch) true 54.1 52.7 +2.7%

PR branch = /Users/runner/work/ts-loader/ts-loader, base branch = /Users/runner/work/ts-loader/ts-loader-main. 2 warmup + 10 measured iterations per scenario, median reported. Report-only - no threshold fails this check.

johnnyreilly and others added 14 commits October 3, 2026 13:44
…docs

- add cache: yarn to execution-tests setup-node steps
- give ci.yml's token contents: read only
- rename build-and-lint workflow to "build and lint" and strip trailing whitespace
- prefix benchmark job names and use "macOS" casing consistently
- update AGENTS.md: ci.yml replaces push.yml, C:\source copy is for path consistency (not Defender), list all workflows using the build action

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The standalone build job duplicated work every other CI job already does
via the build-ts-loader composite action, so drop it and keep just lint
(which builds before linting).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Ubuntu, macOS and Windows jobs were the same steps three times over
and had started to drift. A matrix with a per-OS ROOT (the workspace, or
C:/source/ts-loader on Windows where build-ts-loader builds) lets every
step run in bash on all three OSes. Check names, artifact names and PR
comment markers are unchanged so existing comments still update in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cold typeCheck builds were regularly hitting the 60s per-scenario budget
(e.g. stopping after 44/62 iterations). 90s lets them complete all
iterations while still bounding pathologically slow scenarios.

Also document the budget in the benchmark README, replace its stale
Windows Defender paragraph with how CI runs the benchmark now, and fix
AGENTS.md describing the two sides as running back-to-back in one process.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@johnnyreilly
johnnyreilly merged commit 5362a17 into main Oct 4, 2026
55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant