Skip to content

Latest commit

 

History

History
473 lines (395 loc) · 25 KB

File metadata and controls

473 lines (395 loc) · 25 KB

Metadata dependencies: building on a model published elsewhere

Two or more projects — often in separate repositories, sometimes in separate languages — need to share part of one metadata model: a common Customer/Address shape, an audited-base entity, a set of enums. Copy-pasting the metadata is a fork the moment either side edits it. dependencies in .metaobjects/config.json is the alternative: declare a published metadata package as a dependency, meta deps sync it into a committed snapshot, and build on it the same way you build on any other metadata in your own tree — extends, overlay: true, and plain FQN references.

Phase 1a scope. This page describes what ships today: the path transport, the generated shared-model artifact, the lock + snapshot, default exclusion from your own codegen/schema/ledger, and the loader errors that fire when upstream moves. See Deferred for what is designed but not built yet — read that section before assuming a capability exists.

When to reach for this, versus sources or plain vendoring

metadata-sources.md already lets a sources entry point at metadata living anywhere, including a directory copied from another project (its "Vendoring" section). That is enough when you just need the files. Reach for dependencies instead when you also want:

  • A recorded version and a content hash, not just "whatever is in that directory today" — meta deps check / verify --deps tell you the moment the publisher's metadata moved out from under the hash you last accepted.
  • A publisher that controls what it exports. sources vendoring copies whatever directory you point it at, internal shapes included. A dependency's artifact is generated by the publisher's own sharedModelFile() (below) — the publisher decides the public surface, and it is drift-gated by the publisher's own verify --codegen.
  • A refusal instead of a silent drop when your own metadata declares a brand-new node into the publisher's package (ERR_DEPENDENCY_PACKAGE_NOT_OWNED, below) — plain vendoring has no such concept, because it has no notion of "whose package this is."

If you just need one directory of metadata read by two build tools in a monorepo with no separate publish/version step, plain sources vendoring is simpler and is not going away.

Declaring a dependency

// .metaobjects/config.json (consumer)
{
  "schema_version": 1,
  "sources": [],
  "dependencies": [
    { "name": "acme-common", "path": "../acme-common/metaobjects" }
  ]
}

A dependency spec carries a name (/^[a-z0-9][a-z0-9._-]*$/, unique across the array — this is the LOCAL alias your config, lock, and .metaobjects/deps/<name>/ snapshot directory key on) and exactly one transport key.

Only path resolves in Phase 1a. It is resolved relative to the directory holding .metaobjects/ and must point at a directory containing a publisher's metaobjects.pkg.json + artifact (below) — a sibling checkout, a git submodule, or any other directory your own build can reach on disk. npm and python transport keys are accepted by the config schema (reserved, the same precedent as sources' resource/package kinds) but meta deps sync refuses them today:

dependency "acme-common": transport `npm` is not supported by this toolchain yet; use `path`

dependencies is read at every rung of the source ladder — wherever your own sources come from (the default metaobjects/ directory, a declared sources list, or a port's native surface), the neutral config's dependencies array is read the same way on top of it.

meta deps sync, check, and list

meta deps sync [<name>…] [--dry-run] resolves each declared dependency's path, reads its metaobjects.pkg.json, verifies the sibling artifact's bytes hash to what the manifest records, and re-loads the artifact standalone with core providers only (a consumer loads it with its own providers, never the publisher's, so a Phase 1a export needing publisher-only vocabulary must fail at publish time, not here). It then writes the committed snapshot (.metaobjects/deps/<name>/<name>.metaobjects.json) and the lock (.metaobjects/deps.lock.json), and finally loads the whole updated collection once — so a sync that produces an unloadable model fails in sync itself, with the loader's own error, rather than surfacing on the next unrelated meta gen. Narrow to one or more names as positionals; --dry-run reports the plan and writes nothing.

$ meta deps sync
synced acme-common 1.0.0→1.1.0 (10fbf886→a1b2c3d4)

meta deps check re-resolves each declared dependency exactly as sync's first two steps do, and compares the artifact's currently installed hash against what the lock recorded — read-only, it never touches the lock or the snapshot. It needs the publisher's path reachable right now, which is why it is a separate command from loading (below): CI running against a committed snapshot does not need the publisher checked out, but a check run does.

meta deps list prints the lock's entries — name, version, hash, node count, owned packages — with no resolution at all; it just reads .metaobjects/deps.lock.json.

Every dependency-aware port loads the committed snapshot, never the live path

Once synced, TypeScript's and Python's collection resolvers — the two ports that implement dependencies in Phase 1a — load dependency artifacts from the committed snapshot first, then your own project files, offline, with no transport involved. At load time it checks the snapshot against the lock and refuses (ERR_DEPENDENCY_SNAPSHOT_STALE) when they disagree — a missing lock for a declared dependency, a lock entry with no matching declaration, a missing snapshot file, or an artifact whose bytes no longer hash to what the lock recorded (someone hand-edited the snapshot, or synced elsewhere and forgot to commit it). Every message names the fix: run \meta deps sync`. Two dependencies cannot export the same fully-qualified node either — that is ERR_DEPENDENCY_NODE_COLLISION, checked the moment they are both resolved — and a dependency published against a different metamodel **major** than this toolchain fails with ERR_DEPENDENCY_METAMODEL_INCOMPATIBLE(ADR-0035 Amendment 2: the metadata contract is promised on the major alone, so a dependency built against1.3loads fine at1.0; one built against 2.0` does not).

Reference, extend, and overlay a dependency's nodes

Dependency files load before your own, in the same loader.load(...) your project already runs — so, exactly as within one project:

Reference a foreign node by FQN from any ref-bearing attribute (@objectRef, @references, @via, …) — no different from referencing anything else in your own tree.

extends a foreign abstract — the cross-repo form of the BaseEntity pattern (abstracts-and-inheritance.md):

// metaobjects/meta.blog.json (consumer, package acme::blog)
{ "metadata.root": { "package": "acme::blog", "children": [
  { "object.entity": { "name": "Comment", "extends": "acme::common::Audited",
      "children": [
        { "source.rdb": { "@table": "comments" } },
        { "field.string": { "name": "body", "@required": true } },
        { "identity.primary": { "@fields": ["id"] } }
      ]
  }}
]}}

Overlay a foreign node (overlay: true, same package + name) to add presentation, validators, documentation attrs, or filter/sort flags — the local, unshared additions a consumer is expected to make. Put overlay-only contributions in their own file by convention (this doc uses meta.<dependency-name>.overlay.json; the corpus itself names these files meta.ov*.json) — it is not enforced, but it keeps "what did I add to someone else's node" visible at a glance. It is purely a readability convention: since ADR-0055 the loader applies every overlay in a deferred pass, so an overlay works from any file, in any position, including the same file as its base:

// metaobjects/meta.acme-common.overlay.json (consumer)
{ "metadata.root": { "package": "acme::common", "children": [
  { "object.entity": { "name": "Customer", "overlay": true, "children": [
      { "field.string": { "name": "email", "overlay": true,
          "@filterable": true, "description": "used for the support search box" } }
  ]}}
]}}

One requirement: say overlay: true. The parser merges a same-(type, package::name) redeclaration whether or not it carries the flag — but only the flagged form fails loudly (ERR_OVERLAY_NO_TARGET) when the target disappears upstream. Without it, an upstream removal turns your amendment into a silent new local object under the same name. Always flag a contribution to a node you did not declare.

Declaring a brand-new top-level node into the dependency's package is refused. This is the one thing you may not do:

// metaobjects/meta.ext.json (consumer) — REFUSED
{ "metadata.root": { "package": "acme::common", "children": [
  { "object.value": { "name": "Note", "children": [
      { "field.string": { "name": "text" } }
  ]}}
]}}

fails to load with:

ERR_DEPENDENCY_PACKAGE_NOT_OWNED
"acme::common::Note" is declared here, but the package "acme::common" belongs to a
metadata dependency this project imports — and "acme::common::Note" is not one of
the nodes that dependency exports. A node in an imported package would be excluded
from this project's own codegen and schema with no output and no error, so it is
refused instead.

with the fix named: declare it in a package you own and extends the dependency's node if you need its shape, or — if you meant to amend an existing node — give it that node's name and overlay: true. (An overlay's FQN is one of the dependency's exported nodes, so it merges and passes; only a genuinely new declaration is refused — this is what keeps the package-level rule from failing silently.)

What's excluded by default, and how to opt a package back in

A dependency's nodes are load-only by default. They load so your own model can resolve against them, but they are excluded from every action surface — codegen, schema, the requirements ledger — unless your own config names their package literally. In the maintainer's words: no metadata is imported in runtime, code-generated from, or used on the meta CLI unless explicitly imported for that purpose.

The mechanism is one predicate, built from the lock:

  • imported(fqn) is true when fqn's package is one that some resolved dependency owns (the union of every lock entry's packages — a set lookup on packages, not on individual node names).
  • explicitlyIncluded(pkg, patterns) is true when some pattern in the list names pkg literally — drop the pattern's final segment; what remains must be wildcard-free and equal pkg. So acme::common::** and acme::common::Address both name acme::common; acme::** and a bare ** do not — they match the package's nodes, which is a weaker statement than naming the package. This asymmetry is deliberate: a project that writes scope.include: ["**"] to mean "all of my own model" must not thereby start generating and migrating someone else's.

TypeScript's Collection.inScope — read by codegen, meta gen, verify --codegen, and the requirements ledger's denominator — composes both conjuncts:

inScope(fqn) = matchesScope(fqn, scope) && (!imported(fqn) || explicitlyIncluded(packageOf(fqn), scope.include))

Python's in_scope — read by run_gen's select and verify --codegen — is only the second conjunct. It never applies matches_scope to the project's own objects (the Python CLI has never scoped its own generated output); only the import-exclusion half is new behaviour there:

inScope(fqn) = !imported(fqn) || explicitlyIncluded(packageOf(fqn), scope.include)

So scope.include narrows a Python project's own codegen output not at all — it only ever widens which imported package is let back in. Do not read the TypeScript formula above as describing both ports.

migrate, verify --db, and offline generate compose the schema-side twin, which also removes an excluded import from the expected schema before drift is computed — so importing a table-backed entity can never turn the publisher's other tables into DROP candidates, and never proposes creating the imported table:

inMigrateScope(fqn) = (declaredMigrateScope(fqn) ?? true) && (!imported(fqn) || explicitlyIncluded(packageOf(fqn), migrate.scope))

Naming the package in scope.include (and, if you own its tables, migrate.scope) is how you take over a shared model — the legacy "I instantiate this metadata myself" case, expressed with declarations that already exist rather than a separate mode:

{
  "scope": { "include": ["acme::blog::**", "acme::common::**"] },
  "migrate": { "scope": ["acme::blog::**", "acme::common::**"] }
}

Naming an excluded import explicitly is refused, not silently ignored. meta gen Customer (or a Python entities: ["Customer"]), where every loaded Customer is imported and excluded, exits 2:

meta gen: 'Customer' (acme::common::Customer) is imported from dependency 'acme-common'
and is not generated here — add its package to scope.include in .metaobjects/config.json
to generate it

The requirements ledger's denominator follows the same rule. An imported entity is excluded from the ledger's coverage count — demanding a capability claim for a model you never declared into would report your project as failing to claim entities it does not own. Note this also applies with zero dependencies: inScope composes matchesScope(fqn, scope) regardless of imports, so a project that declares scope.include for its own codegen narrows its ledger denominator to that declared scope too, whether or not it has any dependencies at all.

A shared enum is emitted for the entities that are selected. An abstract field.enum reaches the generated enums.ts iff a selected entity's field resolves to it — decided by the using entity, never by the enum's own package.

What fails at load, when upstream moves

The loader's own validation already fails your build the moment a target you built on is gone or incompatible — no new machinery, just the existing errors, now also firing against a synced dependency after meta deps sync replaces the snapshot:

Upstream change Your construct Error
a node removed or renamed (including a package rename) overlay: true on it ERR_OVERLAY_NO_TARGET
extends it ERR_UNRESOLVED_SUPER
@objectRef / @references / @through / an origin head / a template ref to it ERR_UNRESOLVED_OBJECT_REF / ERR_INVALID_REFERENCE / ERR_INVALID_RELATIONSHIP / ERR_INVALID_ORIGIN / ERR_INVALID_TEMPLATE / ERR_PARAMETER_REF_UNRESOLVED
a member removed or renamed a dotted extends: Foo.member ERR_UNRESOLVED_SUPER
an identity/index @fields, a layout.dataGrid @columns, a @via path, @filter on an inherited member the existing validation-pass error for that construct
a member's type or subtype changed a dotted extends targeting it ERR_EXTENDS_TARGET_MISMATCH
an attr your overlay also sets is now set differently upstream the overlay ERR_MERGE_CONFLICT

What load cannot see: a member's subType, isArray, @required, @values, @objectRef, @default, or physical name changing while every name stays the same; a base's extends retargeted; a @maxLength narrowed. Nothing in Phase 1a classifies those as breaking automatically — see Deferred.

What fails at meta deps check / verify --deps

verify --deps runs meta deps check's comparison as one of verify's gated subverbs (never part of the bare default — it needs the publisher's path reachable, which CI checking out only your own repo may not have). Each declared dependency reports one of:

  • current — the installed artifact's hash matches the lock.
  • drifted — the publisher's artifact has changed since you last synced. Fix: review the artifact diff, then meta deps sync.
  • unresolved — the dependency could not be resolved at all right now (the path moved, or the manifest/artifact fails validation).

Any drifted or unresolved dependency fails the command with ERR_DEPENDENCY_UPSTREAM_DRIFT — a check that cannot check must not pass. This is distinct from ERR_DEPENDENCY_SNAPSHOT_STALE above: that one fires at load time, offline, when the committed snapshot disagrees with the lock; this one fires only when you explicitly ask whether the publisher has moved since, which needs the publisher reachable.

Publishing a dependency: sharedModelFile()

A publisher wires the shared-model generator (TypeScript only, in Phase 1a) to select a subset of its own metadata by the same include/exclude scope-pattern grammar scope already uses, and emit it as one canonical-JSON artifact plus a manifest:

// metaobjects.config.ts (publisher)
import { sharedModelFile } from "@metaobjectsdev/codegen-ts";

export default defineConfig({
  generators: [
    entityFile(), /* … */,
    sharedModelFile({
      name: "acme-common",
      include: ["acme::common::**"],
      // exclude: ["acme::common::InternalOnly"],
      // files: [...],       // default: the run's own source files
      // version: "1.2.0",   // default: the nearest package.json version
      target: "shared",
    }),
  ],
  targets: { shared: { outDir: "dist/shared-model" } },
});

It emits <name>.metaobjects.json (a metadata.root document with no root package — every top-level node carries its own — raw own-layer form, extends preserved) and metaobjects.pkg.json (the manifest: version, metamodelVersion, the artifact's sha256, and the sorted packages/nodes it exports). Along the way it:

  1. Loads files standalone with your project's own registry, strict.
  2. Selects top-level nodes matching include/exclude — always excluding requirement.* and template.* (a publisher's ledger and prompts are not a consumer's shapes).
  3. Checks the selection is closed — every extends target and every ref-bearing attribute of a selected node (and its descendants) must resolve to another selected node, or the build fails naming the (referrer → target) pairs. This is what stops an internal shape from silently leaking through a dangling reference.
  4. Re-loads the emitted artifact standalone with core providers only — your provider vocabulary is not carried into the export; a Phase 1a export that needs it fails here, at publish time, not on a consumer's meta deps sync.

Because it is codegen output, it is drift-gated: meta verify --codegen in the publisher's own build regenerates and diffs it, so a metadata edit that changes the public surface cannot ship unnoticed.

shared-model is registered and discoverable, but deliberately not ejectable. It shows up in meta gen --list like any other generator, but meta eject --list does not offer it (unlike entity/queries/routes/barrel, ADR-0034's four scaffold-and-own generators). The artifact it emits is a contract whose bytes a cross-port corpus pins and whose hash consumers verify — a user-owned, editable copy of the generator would invite an artifact that silently stops matching what consumers expect.

Only TypeScript publishes in Phase 1a — and only TypeScript and Python consume. TypeScript and Python are the only two ports that can consume a dependency at all today (load its snapshot, extend it, overlay it) — Java, Kotlin, and C# don't read dependencies yet (Phase 2). Of those two, only the TypeScript toolchain can publish: it alone can generate the artifact a publisher ships. A Python project that wants to publish a shared model runs a TypeScript sharedModelFile() generator against its own metadata the same way any TypeScript consumer would; a Java, C#, or Kotlin project cannot yet participate as either a publisher or a consumer.

Vendoring vs. depending

Plain sources vendoring dependencies
What you commit a copy of a metadata directory a generated artifact + manifest hash, pinned in a lock
Who decides what's exported you, by choosing what to copy the publisher's sharedModelFile() selection
Drift detection none — the copy is whatever you last copied meta deps check / verify --deps compare against the lock
Versioning none version + metamodelVersion recorded per sync
Declaring a new node into the shared package not meaningful — there is no "owned by" concept refused (ERR_DEPENDENCY_PACKAGE_NOT_OWNED) unless it's an overlay
Excluded from your own codegen/schema by default no — everything you vendor loads exactly as your own metadata does yes, unless scope.include / migrate.scope names the package

Use plain vendoring for a quick, single-repo or monorepo copy with no separate publish step. Reach for dependencies once more than one consuming project needs the same versioned, hash-verified artifact.

The legacy resources-JAR pattern, for readers who knew it

If you have shipped a JVM library whose jar carried nothing but a metadata/*.json resources directory, pulled by an ordinary dependency declaration and read off the classpath — this generalizes that pattern rather than replacing it with something unrecognizable. The artifact still travels inside the publisher's own ecosystem package (an npm package's files, in Phase 1a); it is still versioned with the package that implements it; a consumer still pulls it through its own package manager. What's different: the artifact is one flattened, generated file rather than a directory of hand-placed sources, its manifest is generated rather than hand-written, and meta deps sync copies it into a committed snapshot rather than reading it live off the classpath at build time — the classpath-at-build-time read is recorded as a Phase 2 JVM option, not gone, but the committed snapshot is the cross-port contract every consumer gets today.

Deferred

Designed (see docs/superpowers/specs/2026-09-11-fr-023-metadata-dependencies-design.md) but not built in Phase 1a — do not assume any of the following exists:

  • The npm and python transports (and maven/nuget in a later phase). Only path resolves; the others are refused with ERR_DEPENDENCY_UNRESOLVED.
  • A local co-development override (deps.local.json) that would point sync at an unpublished local checkout without editing the committed config.
  • A usage-aware breaking-change classifier at sync/check time. Nothing automatically distinguishes a widening change (say, @maxLength growing) from a breaking one — the committed artifact's diff in your own history is today's review, and the load-time errors above are today's safety net.
  • packageBindings — codegen importing a foreign named type across the dependency boundary (the FR-025 slice). A generated artifact that names a foreign type in Phase 1a does so however your own generator already resolves cross-package references.
  • A runtime ObjectManager scope predicate. Nothing at runtime refuses to serve an imported entity — "what's imported" is a load-time/codegen concept today, not a runtime one.
  • Java, Kotlin, and C# as dependency consumers or publishers — Phase 1a is TypeScript + Python only; the other three ports arrive in Phase 2.

Verified by

  • fixtures/dependency-conformance/ — 23 cases pinning resolution, the exhaustive imported/inScope/inMigrateScope sets, overlay/extends/reference behavior against a synced snapshot, and every load- and resolution-time error above, run by TypeScript (the reference implementation) and Python. Three artifacts under fixtures/dependency-conformance/artifacts/ are pinned by sha256 in the README so the corpus itself can never silently drift. Python's runner asserts all 23 cases identically to TypeScript's, with no exemptions — the two overlay cases that need a view child as incidental content use view.currency (the one concrete view.* subtype registered cross-port, per fixtures/registry-conformance/expected-registry.json), not view.text, so they exercise the full imported/selected/governed assertions like every other case.

See also

  • metadata-sources.md — where your own metadata comes from, and plain-directory vendoring
  • abstracts-and-inheritance.mdextends: and overlay: within one project; the cross-repository case builds on the same rules
  • own-your-codegen.md — generator ownership, and why shared-model is registered but not ejectable
  • cli.md — the meta deps command surface and verify --deps
  • CONFORMANCE.md — per-port pass status for dependency-conformance