Skip to content

Mapper: suggest the field mapping from the column headers #2907

Open
OpenConceptLab/oclmap
#106

Description

@paynejd

Part of OpenConceptLab/ocl_online#464 (critical user journey 1).

A column is recognised only when its header is exactly a mapped-field name (name, synonyms, property: …). Any other header starts unmapped with a red ✗ in Field Configuration, so a new user has to learn the mapped-field vocabulary before the first run: "Test Name" isn't suggested as Name, and "Specimen" isn't suggested as Property: SYSTEM on LOINC. The section's "Learn more" is plain text, not a link.

Acceptance criteria

  • On upload, headers that match common patterns get a suggested mapped field: name, term, test, description → Name; synonym → Synonyms; specimen, system, sample → Property: SYSTEM when the target has that property; class → Property: Class. Code-like columns stay unmapped.
  • Suggestions are visibly suggestions and can be changed. Nothing is stored until the project is saved.
  • A column that's left unmapped on purpose (for example a local code) doesn't show an error ✗.
  • Field Configuration's "Learn more" links to the Mapper docs.

Components: src/components/map-projects/ColumnMapTable.jsx, MapProject.jsx (isValidColumnValue, the columns set after upload), ConfigurationForm.jsx (map_project.field_configuration_description).

Activity

  1. added this to the P2 · Mapper: Beta milestone on Oct 13, 2026
  2. ocl-automation commented on Oct 13, 2026

    @ocl-automation
    Collaborator

    Automated Triage — 2026-10-13

    Dimension Score
    Complexity 3
    Scope 3
    Clarity 4
    Dependency 1
    Risk 2

    Decision: decision/ai-ready
    Type: type/feature
    Signals: signal/low-risk, signal/well-specified

    The issue gives header-to-field patterns, UI behavior (suggestions are changeable, nothing stored until save, no error ✗ for deliberately unmapped columns), a docs link, and names the components (ColumnMapTable.jsx, MapProject.jsx, ConfigurationForm.jsx). This is a moderate change inside the Mapper, with minor design choices on suggestion display and validity handling. The "Learn more" link target is described only as "the Mapper docs", which leaves a small gap. Risk is low because it only affects client-side defaults that users can change and nothing is persisted until save.


    Generated by ticket-triage pipeline v3. Model: anthropic/claude-sonnet-5.5 (openrouter).

  3. paynejd commented on Oct 15, 2026

    @paynejd
    MemberAuthor

    The sample's specimen values don't match LOINC's SYSTEM values. Suggesting Property: SYSTEM for the Specimen column, as this ticket asks, won't improve the sample's matches on its own.

    • The sample spreadsheet linked above uses Serum and Whole blood. LOINC's SYSTEM values are abbreviations. I looked up the usual LOINC code for each of the sample's ten tests in OCL's LOINC:

      • Ser/Plas: glucose, creatinine, sodium, potassium, cholesterol, LDL, ALT and TSH.
      • Bld: hemoglobin and HbA1c.

      Ser alone wouldn't match the eight Ser/Plas tests either. In OCL's LOINC, no concept has SYSTEM Serum or Whole blood. Ser has 12,852 concepts, Ser/Plas has 10,437 and Bld has 3,524.

    • The Mapper sends a Property: SYSTEM column to $match as properties__SYSTEM (MapProject.jsx).

    • $match adds that property as an exact, case-sensitive keyword clause on properties.SYSTEM.keyword, OR'd with the name clauses (oclapi2 core/concepts/search.py). So a value that matches nothing doesn't remove any candidates, and it doesn't help the ranking either.

    I ran $match against LOINC today. These are the top three, semantic, called directly with no Mapper filters:

    Row #1 #2 #3
    Glucose LP14635-4 Glucose LP32534-7 Glucose LP435977-6 Glucose
    Glucose, SYSTEM Serum LP14635-4 Glucose LP32534-7 Glucose LP435977-6 Glucose
    Glucose, SYSTEM Ser/Plas 2345-7 Glucose [Mass/volume] in Serum or Plasma 1518-0 Glucose [Mass/volume] in Serum or Plasma --2 hours post 75 g glucose PO 1507-3 Glucose [Mass/volume] in Serum or Plasma --1 hour post 75 g glucose PO

    The lexical $match calls showed the same thing. Sending Serum with Glucose, or Whole blood with Hemoglobin, returned the same top three as sending no specimen.

    Two ways to close the gap:

    1. Change the sample's values to Ser/Plas and Bld.
      • It's one edit in ocl-docs and needs no Mapper release.
      • The sample would then show the specimen column improving the matches.
      • The trade-off is that the codes are less readable for a new user than "Serum".
    2. Translate common specimen words into LOINC's SYSTEM codes when the target is LOINC, e.g. serum, plasma or "serum or plasma" → Ser/Plas, and blood or whole blood → Bld.
      • There's a precedent: the Mapper already adds PROPERTY = NFr for LOINC when a name contains %.
      • This is bigger than a lookup table. "Serum" alone could mean Ser or Ser/Plas, and $match takes one value per property.

    Option 1 is the quick fix for the sample. Option 2 is probably its own ticket.

  4. snyaggarwal commented on Oct 16, 2026

    @snyaggarwal
    Contributor
  5. added 2 commits that reference this issue on Oct 16, 2026
  6. paynejd commented on Oct 16, 2026

    @paynejd
    MemberAuthor

    Moved to P3 (Jon, 2026-10-16). The Mon Oct 19 cut for OpenConceptLab/ocl_online#464 keeps three Mapper changes: #2908, the first-run AI default, and #2851. This one waits because the sample spreadsheet's Name header already maps (header matching ignores case), and per the comment above, suggesting Property: SYSTEM for Specimen wouldn't improve the sample's matches. Until it lands, a new user on the sample sees one red ✗ on Specimen.

  7. filiperochalopes commented on Oct 18, 2026

    @filiperochalopes

    Options for the sample-file discussion: what CIEL can offer per domain

    Context: the discussion is about whether the Mapper sample (currently lab tests only) should also cover diagnoses, procedures and imaging, and which target repository each sample should point at. To ground it, I checked what CIEL holds locally for each domain and which target codes its mappings reach. Nothing below changes the ticket scope; these are inputs for a decision.

    1. Strategy options

    Option New user experience Cost Main risk
    A Keep one lab sample (LOINC) Works well with LOINC None Says nothing about diagnoses/procedures/imaging
    B One mixed sample, any target Can be run against any repo Low Most rows are a mismatch for the chosen target (e.g. diagnoses vs LOINC), so the tool looks bad
    C One sample per domain, each paired with a suggested target repo Rows always fit the target Several files + short docs More files to maintain
    D C plus a "General purpose (CIEL)" mixed sample, preselected when unsure Mixed rows are valid because CIEL covers all classes C + one file Still needs the target hint in the docs/UI

    D is the only option that satisfies both positions in the thread: a mixed sample is valid when the target is CIEL, and domain samples avoid sending a diagnosis to LOINC.

    2. What CIEL has locally, by concept class

    Local OCL stack, CIEL HEAD, non-retired concepts. Counts are concepts with at least one active mapping to the repository. Test, Procedure and Radiology/Imaging were scanned in full. Diagnosis is a sample (first 2,000 of 36,690), so read its percentages as indicative only.

    CIEL class Concepts LOINC SNOMED CT ICD-10-WHO ICD-11-WHO WHO-ICHI
    Test 700 522 (75%) 634 (91%) 3 3 1
    Radiology/Imaging Procedure 333 306 (92%) 333 (100%) 0 0 8
    Procedure 1,698 7 1,698 (100%) 6 0 182 (11%)
    Diagnosis (sample) 2,000 of 36,690 0 2,000 2,000 1,266 (63%) 0

    Reading: LOINC is a valid target for Test and Imaging only. ICD-10/ICD-11 are valid for Diagnosis only. For Procedure, SNOMED CT is the only broad target; WHO-ICHI reaches about one in nine. SNOMED CT and WHO-ICHI appear here as mapping targets in CIEL; I did not check whether the Mapper offers them as selectable repositories.

    3. Candidate rows per sample

    CIEL columns are the expected answer, for reviewers; the sample file itself would only carry Code,Name (plus context columns where useful). Map types are as stored in CIEL: ~ = SAME-AS, > = CIEL is broader than the target, < = CIEL is narrower than the target.

    3a. Diagnoses → ICD-11 / ICD-10 (suggested), SNOMED CT

    Row name CIEL ICD-11 ICD-10 SNOMED CT
    Malaria 116128 ~ 1F4Z ~ B54 ~ 61462000
    Hypertension 117399 > BA00.Z < I10 ~ 38341003
    Pneumonia 114100 < CA40.Z ~ J18.9 ~ 233604007
    Asthma 121375 > CA23.32 ~ J45.9 ~ 195967001
    Epilepsy 155 ~ 8A6Z ~ G40.9 ~ 84757009
    Anaemia 121629 < 3A9Z ~ D64.9 ~ 271737000
    Tuberculosis 112141 < 1B1Z < A16.9 ~ 373576009
    Diarrhoea 142412 < ME05.1 < A09.9 ~ 62315008
    Non-insulin dependent diabetes (pre-existing) 158474 ~ JA63.1 < E11.9 ~ 199230006
    Cerebral malaria 145851 > 1F40.0 > B50.0 ~ 53622003

    3b. Procedures → SNOMED CT (suggested); WHO-ICHI where available

    Row name CIEL SNOMED CT WHO-ICHI
    Appendectomy 161768 ~ 80146002 none
    Cesarean section 1171 ~ 11466000 ~ NME.JF.AA
    Blood transfusion 1063 ~ 116859006 < DIA.DG.AF
    Hysterectomy 159837 ~ 236886002 > NME.JK.AA
    Circumcision 161803 < 72310004 none
    Insertion of pleural tube drain 161726 ~ 264957007 none
    Extraction of cataract 161865 ~ 54885007 none
    Biopsy 170373 ~ 86273004 none
    Nasogastric tube placement 1766 ~ 87750000 < KBF.DL.AC
    Laparoscopic appendectomy 162908 ~ 6025007 none

    3c. Lab tests + imaging → LOINC (one sample), SNOMED CT as secondary

    Both classes are LOINC-mapped in CIEL (Test 75%, Imaging 92%), so a single LOINC sample can carry both. Lab rows are the existing sample's tests, resolved to CIEL.

    Row name Kind CIEL LOINC SNOMED CT
    Serum glucose Lab 887 > 2345-7 ~ 22569008
    Haemoglobin Lab 21 > 718-7 ~ 441689006
    Serum sodium Lab 1132 > 2951-2 ~ 104934005
    Serum potassium Lab 1133 > 2823-3 ~ 271236005
    Thyroid stimulating hormone test Lab 161505 ~ 14297-6 ~ 61167004
    Chest X-ray, PA and lateral Imaging 161251 ~ 36643-5 ~ 1290963007
    Ultrasound, abdomen Imaging 845 ~ 24558-9 ~ 418394000
    CT scan, head Imaging 846 ~ 30799-1, 24725-4 ~ 408754009
    Mammogram Imaging 163591 ~ 36625-2 ~ 71651007
    Obstetric ultrasound Imaging 159618 > 12130-1 ~ 268445003

    3d. General purpose (CIEL), mixed classes, 10 rows

    Row name CIEL class CIEL
    Malaria Diagnosis 116128
    Hypertension Diagnosis 117399
    Pneumonia Diagnosis 114100
    Serum glucose Test 887
    Haemoglobin Test 21
    Thyroid stimulating hormone test Test 161505
    Chest X-ray, PA and lateral Radiology/Imaging Procedure 161251
    Mammogram Radiology/Imaging Procedure 163591
    Cesarean section Procedure 1171
    Blood transfusion Procedure 1063

    4. Caveats

    • Row names are CIEL display names, i.e. best-case input. Real user files use looser wording (e.g. "high blood pressure"); if the sample should demonstrate that, row names need to be reworded and re-tested.
    • Many CIEL→LOINC/ICD links are broader/narrower, not same-as, so "expected code" in a sample is the best candidate, not always an exact equivalent.
    • Not checked: how the Mapper ranks any of these rows, and which repositories the Mapper UI lists as targets. The tables show only what CIEL's own mappings support.

    Method: read-only queries against the local OCL API (/orgs/CIEL/sources/CIEL/concepts/ with mappings, non-retired). Counts depend on the CIEL HEAD loaded in that stack.

  8. added 2 commits that reference this issue on Oct 19, 2026
  9. self-assigned this
    on Oct 19, 2026
  10. added a commit that references this issue on Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

component/mapperMapping toolsdecision/ai-readyReady for autonomous AI executionsignal/low-riskSafe to execute with minimal downsidesignal/well-specifiedClear requirements and acceptance criteriastage/triagedAI triage complete — scored and classifiedtype/featureNew or improved functionality

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions