Why FHIR-to-OMOP isn't mechanical, and whether niche use cases should shape the spec — Nov 21, 2023

DB Ignite — Spark schemas generated from HL7 JSON schema

A Databricks contributor introduced DB Ignite, an open-source project that began as FHIR-to-OMOP work. Rather than hand-writing classes, they took HL7's own JSON schema definitions and translated them into Spark schemas — around 900,000 lines of metadata covering 157 resources — so the mapping stays repeatable and the metadata is just the published metadata.

The honest part — where FHIR-to-OMOP stops being mechanical

A FHIR-to-OMOP mapping is not one-to-one: you cannot take every Claim and turn it into an Encounter, because whether a bundle represents a prior authorisation depends on a ServiceRequest sitting behind it.

They have the mapping working for C-CDA-derived records and deliberately stop there — solving every permutation is the equivalent of solving every healthcare organisation's data model.

The hardest bit — vocabulary normalisation

Every concept in OMOP gets a centrally assigned concept ID that comes out of mapping tables, and essentially nobody carries OMOP concept IDs in their FHIR resources — so the transform has to look each one up, keep both the original code and the concept ID, and fall back to zero when nothing matches.

That means calling a terminology service from inside FHIRPath, which is precisely what you would not want a plain database view doing. Add the awkward cases — no mapping found, or several mappings because a CodeableConcept carried several codings — and it starts to need specific handling.

Scope — should unusual use cases steer the spec?

One position: the unusual use cases raised recently — a client telling a server what to pre-optimise, for instance — are genuine and important, but letting them steer the spec is the tail wagging the dog, because they are nowhere near as central as the analytics cases ViewDefinition is for.

The compromise offered: write them up and document them, perhaps as a section on why people build runners the way they do, and let them inform things like the intent enum rather than the core design.

A side idea worth noting

Standardising a Parquet schema for FHIR, on the grounds that bulk export could then hand you Parquet directly and simplify a lot of downstream work.