Skip to content

Automated doc/artifact sync pipeline for crowdsplit and contracts releases - #162

Open
adnanhq wants to merge 30 commits into
developfrom
feat/doc-sync-pipeline
Open

Automated doc/artifact sync pipeline for crowdsplit and contracts releases#162
adnanhq wants to merge 30 commits into
developfrom
feat/doc-sync-pipeline

Conversation

@adnanhq

@adnanhq adnanhq commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Sets up the automated documentation and artifact sync pipeline. When crowdsplit or contracts publishes a stable release, a workflow here regenerates everything mechanically derivable from the source repo and opens a reviewable PR against develop. Nothing merges automatically.

Contracts track

  • Implements the existing generate-abis.ts / check-abis.ts stubs. contract-map.ts is the single source of truth mapping the 8 vendored contracts to their SDK modules, README table rows and docs links.
  • contracts-sync.yml listens for the contracts-release-published dispatch (or a manual tag), downloads the release ABI bundle, verifies its checksums against the dispatch payload, regenerates contracts/*/abi.ts + abi-manifest.json + the README entities table, adds a minor changeset and opens a PR on branch sync/contracts-<tag>. Re-runs for the same tag update the same PR.
  • Change detection is by canonical ABI hash, so a release with unchanged ABIs is a green no-op instead of a churn PR. New deployable contracts are flagged in the PR body but never auto-added; they need an entity layer and tests first.
  • CI gains an abis:check step: generated ABI modules that were hand-edited or drifted from the manifest fail every PR. The manifest is seeded from the current committed ABIs (tag: sdk-baseline), so the first real contracts release will produce a large sync PR containing the genuine drift that has accumulated (UUPS/AccessControl members, removed Ownable functions). That PR will need entity-layer updates before merge and is expected to be breaking.

Crowdsplit track

  • scripts/sync-crowdsplit.mjs bundles the OpenAPI spec from a crowdsplit checkout at a release tag, snapshots it to packages/payments/openapi/openapi.bundled.yaml, regenerates the internal src/generated/api.ts (openapi-typescript, exact-pinned), computes an endpoint-level diff plus an oasdiff changelog and breaking verdict, and writes a patch changeset. Output is deterministic; re-runs for the same tag are byte-identical no-ops. Runnable locally for dry runs.
  • crowdsplit-sync.yml drives it on the crowdsplit-released dispatch and opens the PR (branch sync/crowdsplit-<tag>, labels api-sync/crowdsplit, plus breaking-api-change when applicable). The PR body lists new and removed endpoints as a checklist for the follow-up service work.
  • CI gains an api-types-freshness job that regenerates the types from the committed snapshot and fails on any drift.
  • Seeded from crowdsplit v2.1.1 (76 endpoints). The generated layer is internal only and not exported; services adopt it progressively. Includes a minor changeset for the payments SDK since the published dist/ now contains the generated declarations.

What stays manual

Merging sync PRs, writing service methods and tests for new endpoints, building entities for new contracts, escalating changesets for breaking releases, and all hand-written prose.

Setup required before the pipeline is live

  • Secrets on this repo: CROWDSPLIT_READ_TOKEN (fine-grained PAT, Contents read on oak-network/crowdsplit) and SDK_BOT_TOKEN (fine-grained PAT on this repo, Contents + Pull requests read/write; needed so bot-created PRs trigger CI). (Note: Already created)
  • The workflows arm once this reaches main via the usual develop promotion; repository_dispatch is ignored until then.
  • Companion PRs: oak-network/contracts (release artifact pipeline) and oak-network/crowdsplit (dispatch job).

Testing

  • 27 new unit tests for the ABI tooling (canonicalization, rendering, change detection, registry drift, README markers); full unit suites pass with existing coverage thresholds (807 contracts, 182 payments).
  • Sync scripts were run end-to-end locally against a real forge build --ast bundle and a real crowdsplit checkout at v2.1.1; second runs verified as no-ops. The freshness gate reproduces the committed generated types byte for byte.
  • All workflows pass actionlint. Dispatch payload fields are validated before any shell use; checkout credentials are not persisted.

devmahmud and others added 28 commits April 2, 2026 21:41
Rename contracts package to `contracts-sdk`
Release v1.1.0: Event Log System, Multicall Utility & Metrics Aggregation
Release v1.1.1: updated readme
feat(payments): expand SDK to cover Crowdsplit staging endpoints
feat(ci): add staging sync workflow with issue automation
feat: add countries API service with unit and integration test
[fix]: remove redundant fields from payment method
chore: restrict dependabot to major only updates #137
Migrate payments SDK API base URLs to Oak network domain
Updated sandbox url and provider registration types
Implements the generate-abis and check-abis stubs and wires them into a
repository_dispatch triggered workflow.

- contract-map.ts: single source of truth mapping the 8 vendored
  contracts to their SDK modules, README table rows and docs links.
  TimeConstrainedPaymentTreasury stays unmapped on purpose, it shares
  the PaymentTreasury ABI surface.
- generate-abis.ts: regenerates contracts/*/abi.ts from a release ABI
  bundle with canonicalized (sorted) entries and a dependency-free
  renderer, and writes abi-manifest.json (canonical-JSON sha256 per
  contract). Change detection is hash-based, so a release with
  identical ABIs is a no-op instead of a header-churn PR. New unmapped
  contracts are reported, never auto-added. DataRegistryKeys drift is
  checked via viem keccak256.
- check-abis.ts: manifest mode (offline, runs in CI on every PR)
  catches hand edits to generated modules; bundle mode diffs the
  working tree against a release bundle; the seed command bootstraps
  the manifest from the committed ABIs (tag: sdk-baseline) until the
  first real release sync records genuine provenance.
- update-readme.ts: renders the Available Entities table between
  AUTOGEN markers from the contract map.
- contracts-sync.yml: on a contracts-release-published dispatch (or
  manual tag), downloads the release bundle, verifies checksums and
  metadata, regenerates, adds a minor changeset and opens an
  idempotent human-reviewed PR against develop. A failing typecheck
  is surfaced as a PR warning instead of blocking the PR. Failures
  open a deduplicated contracts-sync issue.
- 27 unit tests cover canonicalization, rendering, change detection,
  registry drift and README marker handling.
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

Comment thread .github/workflows/ci.yml Fixed
Comment thread packages/contracts/__tests__/unit/scripts.test.ts Fixed
Comment thread packages/contracts/__tests__/unit/scripts.test.ts Fixed
…d types layer

- scripts/sync-crowdsplit.mjs: bundles the Crowdsplit OpenAPI spec at
  a stable release tag (redocly version taken from the crowdsplit
  checkout), snapshots it to packages/payments/openapi/, regenerates
  the internal src/generated/api.ts via pinned openapi-typescript,
  computes an endpoint-level diff natively plus an oasdiff changelog
  and breaking verdict when available, and emits a patch changeset and
  PR body. Deterministic output (no timestamps), unchanged specs are
  no-ops, and the working tree is only touched after every generation
  step succeeds.
- crowdsplit-sync.yml: on a crowdsplit-released dispatch (or manual
  tag), runs the script against a tag checkout and opens an idempotent
  human-reviewed PR against develop. Breaking releases get the
  breaking-api-change label and a PR warning. Failures open a
  deduplicated api-sync issue. The source commit is resolved from the
  tag checkout, not the dispatch payload.
- ci.yml: abis:check drift gate (contracts) and api-types-freshness
  job (payments) so generated artifacts cannot drift or be hand-edited
  unnoticed.
- Seeded from crowdsplit v2.1.1: spec snapshot, sync metadata
  (76 endpoints) and generated types. The generated layer is internal
  only, not exported; services adopt it progressively. Includes a
  minor changeset for the payments SDK.
- backend-sync.yml issue template now mentions the release-time sync
  PR; generated paths are marked linguist-generated; CLAUDE.md
  documents the regeneration commands.
@adnanhq
adnanhq force-pushed the feat/doc-sync-pipeline branch from 90a1c57 to 9194d9c Compare July 17, 2026 09:17
- ci.yml: explicit contents: read permissions on the api-types-freshness
  job.
- scripts.test.ts: build DataRegistryKeys.sol fixtures with a helper
  instead of single-occurrence string replacement on the closing brace.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants