Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@
- moved to omop-alchemy 1.x, oa-configurator 1.x and orm-loader 1.x
- concept resolver imports follow omop-alchemy's move from `cdm.handlers` to `toolkit.core.concepts`

# 0.7.1 (unreleased)
# 0.7.1
- `construct-contracts.toml`: the wheel-packaged, machine-readable D3 grain catalogue. Records the grain, intended logical key, ORM identity, surrogate stability, per-input fan-out, concurrent-refresh eligibility, 1.0 inclusion, and lung-report usage of all 43 registered constructs, plus a findings register
- `omop_constructs.core.contracts` loads and validates that manifest; `omop_constructs.core.catalogue` renders it into `docs/construct-catalog.md`
- new CLI subcommands `contracts` and `render-catalogue`
Expand All @@ -226,10 +226,10 @@
- reconciled the construct catalogue prose with the actual queries for surgery attribution, SACT and RT grain, the treatment envelope, and event attachment
- release validation normalises rendered SQL before checksumming: resolved concept-ID lists are emitted in hash-seed-dependent set order, and PostgreSQL stores the qualified schema name of every referenced object, so unnormalised checksums reported roughly a quarter of the registry as changed on every run

# 0.8.0 (unreleased)
# 0.8.0
- diagnosis-linked Measurement, Observation, and Procedure Occurrence constructs now use omop-alchemy's canonical event projection and explicit-first episode attachment policy
- valid explicit links require the event ID, Field-concept discriminator, episode, and person to agree; exact duplicate links collapse and linked events no longer also enter date-window fallback
- unlinked events select one eligible episode of care deterministically: already-started episodes first, then nearest episode start, then lowest episode ID; valid explicit multi-links remain intact
- custom event factories can select an `EpisodeAttachmentPolicy` and `TemporalRankingSpec`; deprecated Boolean and direct-helper calls issue warnings that identify the policy-based replacement
- ranked date-window fallback considers episodes of care only, so an unlinked event can no longer attach to a later-starting progression or metastatic episode; those episodes receive events only through valid explicit links. All-in-window and explicit-only policies are unchanged. Requires omop-alchemy>=1.2.1 for the separate explicit-link and fallback episode sources
- deployments must rebuild the affected event materialized views and `consult_window_mv`; an in-place refresh does not replace a PostgreSQL materialized-view definition
- deployments must rebuild the diagnosis-linked event views, leaf modifier views, `modified_conditions_mv`, `stage_modifier_mv`, and `person_demography_mv`, plus `consult_window_mv` and their dependent views in dependency order; ANALYZE each immediately after building. An in-place refresh does not replace a PostgreSQL materialized-view definition
12 changes: 10 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,12 +72,14 @@ Dependencies can point at constructs outside the currently imported set; those a

Diagnosis-linked procedures, measurements, and observations use one attachment rule across the construct family. A valid `Episode_Event` relationship is authoritative when its event identifier, OMOP Field concept, episode identifier, and person agree. Every valid explicit relationship is retained, so an event deliberately linked to two episodes produces one row for each relationship.

When an event has no valid explicit relationship, the event date determines its episode of care. Progression and metastatic episodes are never chosen by date; they receive events only through valid explicit links. The eligible window begins 90 days before the episode start and ends on the episode end date. An episode without an end date remains eligible for 365 days after its start. From the eligible episodes of care, the resolver chooses one using these rules in order:
When an event has no valid explicit relationship, the event date determines its episode of care. Progression and metastatic episodes are never chosen by date; they receive events only through valid explicit links. The default lower bound is the episode-of-care start minus 90 days. Its upper bound is the greatest of its own window end and the latest window end of its same-person nested progression or metastatic diagnoses. Each episode's own window end is its recorded end date, or its start plus 365 days when the end is absent. Extension changes fallback eligibility only: explicit links, root-start ranking, and recorded episode dates in the output remain unchanged. From the eligible episodes of care, the resolver chooses one using these rules in order:

1. Prefer episodes that have started by the event date.
2. Choose the episode start nearest to the event date.
3. If two starts are equally near, choose the lowest `episode_id`.

When the nearest eligible already-started episode began at least 365 days before the event, choose the earliest eligible upcoming episode starting within 60 days after the event, breaking equal starts by the lowest episode ID. Both thresholds are inclusive. A more recent started episode blocks this override, and a same-day start is already started. No concept list is used.

```mermaid
flowchart TD
event[Clinical event] --> explicit{Valid Episode_Event relationship?}
Expand All @@ -87,7 +89,10 @@ flowchart TD
eligible -- Yes --> started{Any eligible episode already started?}
started -- Yes --> prior[Rank the already-started episodes]
started -- No --> future[Rank all eligible future episodes]
prior --> nearest[Choose the nearest episode start]
prior --> override{"Started winner aged at least 365 days<br/>with an eligible upcoming start within 60 days?"}
override -- Yes --> upcoming[Keep upcoming starts within 60 days]
override -- No --> nearest[Choose the nearest episode start]
upcoming --> nearest
future --> nearest
nearest --> tie[Break an equal-distance tie with the lowest episode_id]
```
Expand Down Expand Up @@ -185,3 +190,6 @@ measure_temporal_window(
threshold_days=X,
)
```


Ranked attachment queries carry a private `_fallback_window_end` alongside `attachment_method`. `episode_relevant_window` consumes both, so its final filter honours the extension while materialized views retain recorded episode dates and their existing columns. Configure ranked open-end horizons with `EpisodeWindowSpec` at attachment time. All-in-window and explicit-only queries keep their original window path.
4 changes: 2 additions & 2 deletions docs/constructs/events.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ From `omop_constructs.alchemy.events`.
Event constructs attach individual clinical events to condition episodes. Unlike episode constructs (which follow OMOP episode hierarchy links), event constructs use a two-tier attachment strategy implemented in `event_factories.py`:

1. **Explicit link:** Accept an `Episode_Event` relationship only when its event ID, OMOP Field-concept discriminator, episode ID, and person all agree.
2. **Time-window fallback:** Attach by date only when the event has no valid explicit link. The default window is 90 days prior to episode start through the episode end date (or 365 days after episode start for episodes with no end date). Only episodes of care are candidates, and events falling outside this window for every episode of care the patient has do not appear in the view. Progression and metastatic episodes are never chosen by date; they receive events only through valid explicit links.
2. **Time-window fallback:** Attach by date only when the event has no valid explicit link. The default lower bound is the episode-of-care start minus 90 days. Its upper bound is the greatest of its own window end and the latest window end of its same-person nested progression or metastatic diagnoses. Each episode's own window end is its recorded end date, or its start plus 365 days when the end is absent. Extension changes fallback eligibility only: explicit links, root-start ranking, and recorded episode dates in the output remain unchanged. Only episodes of care are candidates, and events falling outside this window for every episode of care the patient has do not appear in the view. Progression and metastatic episodes are never chosen by date; they receive events only through valid explicit links.

The construct family uses the `explicit_first_ranked` policy. A valid explicit row suppresses date-based fallback for that table-scoped event, while multiple valid explicit links remain distinct. An unlinked event attaches to one eligible episode of care, preferring an episode already started at the event date, then the nearest start, then the lowest episode ID. Exact `(source table, event ID, episode ID)` duplicates are removed. For example, an unlinked spirometry result that falls within two lung cancer episode windows is assigned to one episode by these rules; a result explicitly linked to both episodes remains associated with both. No construct in this family has a database unique key; each construct's key is listed under [Contracts](#contracts).
The construct family uses the `explicit_first_ranked` policy. A valid explicit row suppresses date-based fallback for that table-scoped event, while multiple valid explicit links remain distinct. An unlinked event attaches to one eligible episode of care, preferring an episode already started at the event date, then the nearest start, then the lowest episode ID. Q4 adds one narrow preference: when the nearest eligible already-started episode began at least 365 days before the event, choose the earliest eligible upcoming episode starting within 60 days after the event, breaking equal starts by the lowest episode ID. Both thresholds are inclusive. A more recent started episode blocks this override, and a same-day start is already started. No concept list is used. Exact `(source table, event ID, episode ID)` duplicates are removed. For example, an unlinked spirometry result that falls within two lung cancer episode windows is assigned to one episode by these rules; a result explicitly linked to both episodes remains associated with both. No construct in this family has a database unique key; each construct's key is listed under [Contracts](#contracts).

---

Expand Down
10 changes: 8 additions & 2 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@
Typical runtime usage assumes:

- `oa-configurator` is installed and a shared OMOP stack config is available
- `omop-alchemy>=1.2.0,<2`, `omop-semantics`, and `orm-loader>=1.2.0,<2` are installed
- `omop-alchemy>=1.2.2.dev0,<2` (the local Q4 wheel selected by `tool.uv.sources`), `omop-semantics`, and `orm-loader>=1.2.0,<2` are installed
- `omop-semantics` runtime value sets are available
- a PostgreSQL database is available for materialized view creation and refresh

The modifier layer builds database-side concept predicates without opening a connection at import time. Other semantics-backed construct families still require database-backed resolver setup.

Diagnosis-linked measurements, procedures, and observations preserve every valid explicit episode link. An event without an explicit link attaches to one episode of care: an episode already started at the event date is preferred, followed by the nearest episode start and then the lowest `episode_id`. For example, an unlinked pathology observation recorded after one cancer episode began but shortly before a second episode began is assigned to the episode already in progress. A valid explicit relationship to the second episode takes precedence over this date-based choice. Progression and metastatic episodes are never chosen by date; they receive events only through valid explicit links. Custom queries can override the default with explicit `policy` and `ranking` arguments.
Diagnosis-linked measurements, procedures, and observations preserve every valid explicit episode link. An event without an explicit link attaches to one episode of care: an episode already started at the event date is preferred, followed by the nearest episode start and then the lowest `episode_id`. When the nearest eligible already-started episode began at least 365 days before the event, choose the earliest eligible upcoming episode starting within 60 days after the event, breaking equal starts by the lowest episode ID. Both thresholds are inclusive. A more recent started episode blocks this override, and a same-day start is already started. A valid explicit relationship to the second episode takes precedence over this date-based choice. Progression and metastatic episodes are never chosen by date; they receive events only through valid explicit links. Custom queries can override the default with explicit `policy` and `ranking` arguments.

## Configuration With `omop-config`

Expand Down Expand Up @@ -97,12 +97,15 @@ These helpers assume PostgreSQL materialized views and use `pg_matviews` for exi

The diagnosis-linked event constructs use an explicit-first attachment policy. A valid `Episode_Event` relationship must match the event ID, Field-concept discriminator, and person. It is emitted once and suppresses date-window fallback for that table-scoped event. An event without a valid explicit relationship is attached to one eligible episode of care using the deterministic ranking described above.

The default lower bound is the episode-of-care start minus 90 days. Its upper bound is the greatest of its own window end and the latest window end of its same-person nested progression or metastatic diagnoses. Each episode's own window end is its recorded end date, or its start plus 365 days when the end is absent. Extension changes fallback eligibility only: explicit links, root-start ranking, and recorded episode dates in the output remain unchanged. Custom `EpisodeWindowSpec` horizons and bound inclusivity also apply to the extended candidate. All-in-window and explicit-only policies retain their existing behaviour.

The attachment policy is part of each materialized-view definition. PostgreSQL `REFRESH MATERIALIZED VIEW` repopulates the definition already stored in the database, so installing a release with a changed attachment policy requires a rebuild. The affected definitions are:

- `dx_measurement_mv`, `dx_observation_mv`, and `dx_procedure_mv`
- the concept-specific measurement views (`weight_dx_mv`, `weight_change_dx_mv`, `height_dx_mv`, `bsa_dx_mv`, `creatinine_clearance_dx_mv`, `egfr_dx_mv`, `fev1_dx_mv`, `dtherm_dx_mv`, `ecog_dx_mv`, and `smoking_pyh_dx_mv`)
- `consult_window_mv`, which depends on `dx_observation_mv`.


For a registry-managed deployment, rebuild the managed construct set in dependency order:

```python
Expand Down Expand Up @@ -183,3 +186,6 @@ Together, these views compute episode-level referral-to-specialist and referral-
- Import-time resolver errors: check the active `oa-configurator` stack config before importing modifier-heavy modules.
- Schema mismatch during validation: the mapped class and underlying materialized view definition have drifted.
- Missing construct in downstream code: confirm the module containing that construct class has actually been imported.


Ranked attachment queries carry a private `_fallback_window_end` alongside `attachment_method`. `episode_relevant_window` consumes both, so its final filter honours the extension while materialized views retain recorded episode dates and their existing columns. Configure ranked open-end horizons with `EpisodeWindowSpec` at attachment time. All-in-window and explicit-only queries keep their original window path.
8 changes: 2 additions & 6 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ authors = [
requires-python = ">=3.12"
dependencies = [
"oa-configurator>=1.2.1,<2",
"omop-alchemy[postgres]>=1.2.1,<2",
"omop-alchemy[postgres]>=1.2.2,<2",
"omop-semantics>=0.6.0",
"orm-loader>=1.2.0,<2",
"sqlalchemy>=2.0.45",
Expand All @@ -18,10 +18,6 @@ dependencies = [
[project.optional-dependencies]
dev = [
"ipython>=9.8.0",
# <0.16 deliberately: ruff 0.16 widened its default rule set enormously
# (omop-alchemy reports 0 findings on 0.15.14 and 607 on 0.16.3). Every
# sibling is green only because its lockfile predates 0.16. Lifting this
# bound is a stack-wide decision, not a per-repo one.
"ruff>=0.15,<0.16",
"ty==0.0.61",
"pytest>=9.0.3",
Expand Down Expand Up @@ -72,4 +68,4 @@ unresolved-attribute = "ignore"
testpaths = ["tests"]
markers = [
"postgres: requires a configured PostgreSQL database via ENGINE_CDM or ENGINE",
]
]
Loading
Loading