---
{
  "title": "FHIR needs data quality checks",
  "description": "A green validator tells you nothing about whether a dataset is usable. OMOP solved this a decade ago with the Data Quality Dashboard. FHIR now has every piece needed to build its own — on SQLQuery plus a few extensions.",
  "date": "2026-07-15",
  "author": "Nikolai Ryzhikov",
  "reading-time": "16 min read",
  "tags": [
    "SQL on FHIR",
    "Data Quality",
    "Analytics"
  ],
  "tldr": "The FHIR validator answers 'is this resource well-formed?' It cannot answer 'is this dataset usable?' — that question is about counts, rates and distributions. FHIR profiles the instance but has no dataset profile. The OMOP world has run on the Data Quality Dashboard for a decade, and the whole analytics industry tests its data the same way: a check is just a query that returns the bad rows. So a dataset profile for FHIR needs no new resource and no new operation — an SQLQuery plus three extensions carrying category, threshold and severity, which any SQL on FHIR engine can run.",
  "utm-campaign": "analytics",
  "utm-content": "fhir-data-quality"
}
---

## Garbage in, garbage out

Someone wants to run a clinical quality measure over your FHIR data — a HEDIS-style measure written in CQL, say, or a diabetes-control dashboard, or a cohort for a study. They pull a few million resources through a Bulk FHIR export, point their logic at it, and get a number back.

Should they trust that number?

Everything validated against US Core. Every reference resolved. The validator was green from top to bottom. And yet: maybe 40% of the lab observations carry no numeric value. Maybe half the patients have no encounters at all. Maybe a batch of body weights arrived in pounds where the profile expects kilograms — a perfectly conformant `Quantity` holding a perfectly wrong number. Maybe every patient on diabetes medication is missing a diabetes diagnosis, because the source system kept those in a table nobody mapped.

None of that breaks a single profile rule. All of it flows straight into the measure and quietly moves the answer. Garbage in, garbage out — except here the garbage is invisible, because every byte of it is well-formed FHIR. And the honest response to "how much of this data is wrong?" today is a shrug.

And that still assumes the data is even where you went looking for it. FHIR gives you more than one valid way to record the same clinical fact. A patient's diabetes might live in a `Condition`, or as an `Observation` with a diagnostic code, or be implied only by a `MedicationRequest` for metformin or a `Procedure`. A measure that queries `Condition` and nothing else isn't *wrong* — it just silently misses every patient whose diabetes was modeled the other way. The data is valid and conformant, just not where the logic looked, and no validator will tell you so.

This is not a hygiene problem to clean up later. It's the thing standing between FHIR data and every use case that motivated collecting it — analytics, quality measurement, research, a model.

## FHIR profiles the instance, not the dataset

The instinct is to reach for profiles, and profiles genuinely are part of the answer — just not the part people assume. A profile is where the community agrees on *representation*: that diabetes belongs in `Condition`, coded from this ValueSet, with these elements present. That's exactly the ambiguity from the opening, pinned down — without a profile, every dataset is a different shape and no shared check is even writable. Profiles are the foundation this whole approach stands on.

But a profile is an *agreement*, not an audit — and FHIR profiles are, by design, permissive. Most elements stay optional; must-support asks a system to be *capable* of a field without ever requiring a value; bindings are frequently extensible; escape valves like data-absent-reason are built in. That looseness is deliberate, so real-world data can flow. It's also why a profile describes the *shape* of good data without saying how much of your data fills it in — a contract, not a measurement.

And even the rules a profile *does* enforce, its engine enforces **one resource at a time**. The validator has no notion of a second resource, let alone ten million. So the questions that matter most here are exactly the ones it cannot ask:

- What fraction of `Observation.value` is null? *(a rate, not a rule)*
- Does any identifier repeat across two patients? *(uniqueness spans records)*
- Is our diabetes prevalence 0.1% where it should be near 10%? *(a distribution)*
- Do patients on metformin have a matching condition? *(a join)*

A validator, by construction, cannot count. No profile will ever express "nulls under 5% is acceptable", because a profile has no concept of *how many*.

![Left: an instance profile validates one Observation — status, code, value, subject all structurally valid. Right: a dataset profile, the checks that run across millions of records — null rate, unique keys, distribution, cross-resource joins.](dq-two-engines.svg "FHIR profiles the instance — one resource against a StructureDefinition. What it has no name for is the dataset profile: the rates, joins, and thresholds that decide whether the data as a whole is usable.")

That asymmetry is the whole argument in one picture. FHIR gives you a rich **instance profile** — a StructureDefinition that says what one well-formed resource looks like — and nothing for the **dataset profile**: no standard way to state, or check, what a good *collection* of those resources looks like. Everything below is about building that missing half.

You can watch this play out in the community. There's a [109-message thread on chat.fhir.org](https://chat.fhir.org/#narrow/stream/implementers/topic/Exchanging%20non-conformant%20data) — nineteen participants, including some of the most senior people in the ecosystem — working through what a system should do with a `Period` whose end precedes its start. Real data, out of a real EHR conversion. The debate covers whether to send it, drop it, move it into an extension, or tag it unreliable. It's a careful, thoughtful discussion.

And every word of it is about **one bad period**. Not once does anyone ask what share of periods in the dataset are inverted, because there is no way to ask.

Worse, the conclusion the community keeps arriving at makes the gap wider. If the accepted practice for garbage is to move it out of the computable element into an extension, or flag the resource with an [integrity security tag](https://terminology.hl7.org/ValueSet-v3-SecurityIntegrityObservationValue.html), then a fully conformant dataset can be stuffed with unusable data **by design**. Conformance doesn't reveal the problem. It absorbs it.

### Mandatory, present, and empty

The sharpest version of this is the [data-absent-reason extension](https://hl7.org/fhir/extensions/StructureDefinition-data-absent-reason.html). Mark an element `1..1` and you'd think you've guaranteed a value. You haven't. A minimum cardinality is satisfied by the element merely being *present* — and an element carrying nothing but an empty `data-absent-reason` is present. The validator counts it, the resource passes, and no value was ever supplied.

This isn't a loophole someone forgot to close; it's inherited from US Core, deliberately, as an escape valve for legacy, external, and redacted data. Implementers have [run into it in the wild](https://chat.fhir.org/#narrow/stream/Da.20Vinci.20CRD/topic/data-absent-reason%20on%20mandatory%20elements%20in%20CRD%20profiles) — a required field satisfied by an absent-reason and nothing else — and the community's own read is that it quietly defeats the point of marking the field required. The proposed remedy is another instance-level invariant, written and enforced per IG, forbidding the extension where a real value is expected.

Notice what that costs. To know whether your `1..1` fields actually hold data, you can't trust the green checkmark — you have to ask *what fraction of them are standing in an absent-reason instead of a value.* That is a rate across the dataset. A profile cannot compute it. A query can.

The gap even shows up where you'd most expect a fix. Da Vinci DEQM — the IG for exchanging quality measure data — has a section titled [Data Quality](https://hl7.org/fhir/us/davinci-deqm/guidance.html#data-quality). Its content, in full, is that measures should use defined profiles like US Core or QI-Core so exchanged data is standardized and suitable for evaluation. No thresholds. No rates. No aggregates. Not a single mechanism for assessing quality — just profiles again, the same tool that can't answer the question.

So this isn't an argument against profiles — it's an argument for a second kind. The instance profile stays the source of truth for what a *valid resource* is; a dataset profile measures how much of your data actually lives up to it. **One owns the instance, the other owns the dataset.**

## Every other data stack already tests its data

Step outside healthcare and this problem isn't just solved — it's table stakes. Testing your data is a standard stage in any serious analytics pipeline, and the mechanism is always the same, and always this simple: **a check is a query that returns the rows that break a rule. Zero rows, the data passes. Any rows, those rows are the problem.**

The same idea ships under a different name in every major tool:

| Tool | What a check is |
|---|---|
| [**dbt**](https://docs.getdbt.com/docs/build/data-tests) | a `SELECT` that returns failing rows — with generic templates like `not_null`, `unique`, `accepted_values`, `relationships` |
| [**SQLMesh**](https://sqlmesh.readthedocs.io/en/stable/concepts/audits/) | an *audit*: a query that must return zero rows, or the pipeline halts |
| [**Amazon Deequ**](https://github.com/awslabs/deequ) | "unit tests for data" on Spark — completeness, uniqueness, distribution, across billions of rows |
| [**Great Expectations**](https://greatexpectations.io/) · [**Soda**](https://www.soda.io/) | validation-as-code: human-readable *expectations* run in CI and in production |

Look at what they all check: values present, keys unique, numbers in an accepted range, references that resolve, distributions that look right. The same short list everywhere — because data goes wrong in the same handful of ways regardless of domain. This is a mature, load-bearing part of data engineering, not a fringe practice.

## OMOP brought it to health data a decade ago

Healthcare analytics already made this jump. OHDSI's [Data Quality Dashboard](https://ohdsi.github.io/DataQualityDashboard/) takes that same query-per-check pattern and aims it at clinical data: point it at an OMOP CDM database, it runs thousands of checks, and hands back a graded report. Nobody in that world would publish a dataset without one.

What OMOP adds is a taxonomy for the ways health data specifically goes wrong — the [Kahn framework](https://pmc.ncbi.nlm.nih.gov/articles/PMC5051581/), which sorts every check into three questions:

| Category | The question | Example |
|---|---|---|
| **Conformance** | Is the data in the right shape? | `status` holds a value outside the allowed set |
| **Completeness** | Is the data there at all? | 40% of observations have no value |
| **Plausibility** | Can the data be believed? | A body weight of 1000 kg |

Two mechanics make it work. First, a check type is a **template**, not a query — one `not_null` template expands across every required field of every table, which is how roughly two dozen templates become thousands of concrete checks. Second, every check carries a **threshold**: bad rows under 5% passes, over 5% fails. That's what makes these checks *fuzzy* in a way an invariant can never be. An invariant is binary. A data quality check is statistical, and reality is statistical.

This taxonomy isn't an OMOP quirk, either. The [NCQA Bulk FHIR Quality Coalition](https://www.ncqa.org/bulk-fhir-api-quality-coalition/) grades Bulk FHIR data with exactly these three categories. Germany's [Medical Informatics Initiative](https://doi.org/10.3233/SHTI230117) assesses FHIR data quality with Kahn. PhUSE has [evaluated FHIR API data for FDA submissions](https://www.lexjansen.com/phuse-us/2024/ic/PAP_IC12.pdf) on the same framework. The vocabulary is settled — FHIR just never picked it up.

## FHIR now has the pieces: SQLQuery + extensions

![FHIR resources flatten into a table via a ViewDefinition, then a SQL query with extensions turns that table into a data quality dashboard.](dq-pipeline.svg "The whole pipeline is standard artifacts: a ViewDefinition flattens FHIR, a SQL query plus extensions turns the result into a dashboard.")

Read the diagram left to right and you have the whole idea. A `ViewDefinition` flattens FHIR into a table. A SQL query over that table returns the rows that break a rule — the same dbt-style check every other stack runs. A few **extensions** on that query — Kahn category, threshold, severity — turn a plain query into a graded check you can put on a dashboard.

That's the entire proposal: **a data quality check is an `SQLQuery` plus three extensions.** No new resource, no new operation, no new engine — a check is structurally identical to any other query, and the extensions are the only thing that make it a check.

None of the parts is new — every piece a data quality dashboard needs already exists in the spec:

| A DQD needs… | FHIR already has |
|---|---|
| A flat table to check | **ViewDefinition** — flattens FHIR into columns |
| A check | **SQLQuery `Library`** — a query over that view |
| A way to run it | **`$sqlquery-run`** — the existing operation |
| Composition, roll-ups | **`relatedArtifact: depends-on`** — query-to-query dependencies |
| Schema rules | **profiles** — already the source of truth |

That's what changed. Building a DQD used to be an infrastructure project — OHDSI needed its own SQL engine, its own flat data model, years of work. [SQL on FHIR](/blog/aidbox-becomes-the-first-fhir-server-to-pass-all-sql-on-fhir-tests) standardizes that layer, so in FHIR it's no longer an infrastructure problem. It's just content: write the queries.

And because a check is nothing but SQL over a standard flat view, it's a *specification*, not an implementation. The same `SQLQuery` runs on Postgres, DuckDB, or Spark — or compiles down to the engines every analytics team already runs: a dbt test, a Deequ constraint, a Great Expectations suite. That's the whole reason to standardize it. Not to build another data quality engine — FHIR doesn't need one — but to give the ecosystem **one portable, vendor-neutral way to state what a good FHIR dataset looks like**, authored once and run anywhere.

This isn't a thought experiment. At the recent HL7 Vulcan connectathon we ran it: a FHIR-to-OMOP transformation built only from these primitives, plus **258 DQD checks — each one a `Library(type=sqlquery)` carrying the three extensions above**, not the mockup from the previous section. The transformation passed all 172 golden cases and the 23-case answer key with **zero conformance errors**. The checks flagged 5 failures on our own output and 20 on the working group's gold tables — every one a completeness or plausibility signal that the WG had deliberately seeded, matched to the row (our `plausibleGender` check caught exactly their 6 benign-prostatic-hyperplasia and 4 prostate-cancer conditions on female patients).

Two things stood out. Porting a decade of accumulated data quality checks cost **essentially nothing** — a DQD check is a SQL query returning bad rows, and SQL on FHIR runs exactly that. And the checks earned their keep immediately: `plausibleStartBeforeEnd` caught a visit ending three days before it began, sitting in the working group's *own gold tables* — not in the 130 source encounters, not in anyone's predictions, an artifact no human had spotted. The [community debates one inverted `Period` by hand](https://chat.fhir.org/#narrow/stream/implementers/topic/Exchanging%20non-conformant%20data); the check finds them across the whole dataset, automatically.

## What it looks like

Everything below shares one source table — a ViewDefinition flattening `Observation`:

```json
{ "resourceType": "ViewDefinition", "name": "obs_flat", "resource": "Observation",
  "select": [{ "column": [
    { "name": "id",         "path": "getResourceKey()" },
    { "name": "status",     "path": "status" },
    { "name": "loinc",      "path": "code.coding.where(system='http://loinc.org').code.first()" },
    { "name": "patient_id", "path": "subject.getReferenceKey(Patient)" },
    { "name": "value",      "path": "value.ofType(Quantity).value" },
    { "name": "unit",       "path": "value.ofType(Quantity).code" },
    { "name": "effective",  "path": "effective.ofType(dateTime)" }]}]}
```

A check is an SQLQuery over that view. The extensions carry the semantics — this one says *completeness, warn above 5% missing*:

```json
{ "resourceType": "Library", "id": "dqc-obs-value-complete",
  "type": { "coding": [{ "code": "sql-query" }] },
  "extension": [
    { "url": ".../dq-category",  "valueCode": "completeness" },
    { "url": ".../dq-threshold", "valueDecimal": 0.05 },
    { "url": ".../dq-severity",  "valueCode": "warning" }],
  "relatedArtifact": [
    { "type": "depends-on", "resource": "ViewDefinition/obs_flat", "label": "obs" }],
  "content": [{ "contentType": "application/sql", "data": "<base64>" }]}
```

The SQL inside is deliberately boring, and that's the feature:

```sql
-- completeness: rows where the measurement is missing
SELECT id FROM obs WHERE value IS NULL
```

**Referential integrity** just adds a second view and a second dependency, `patient_flat` labelled `pat`:

```sql
SELECT o.id, o.patient_id
FROM obs o LEFT JOIN pat ON o.patient_id = pat.id
WHERE o.patient_id IS NOT NULL AND pat.id IS NULL
```

**Plausibility** is where this earns its keep — no profile can express any of it. OMOP's DQD has a whole family of plausibility checks, and they port to LOINC-coded `Observation`s directly. Three of the most useful.

*Value outside the physiologic range for its code* — DQD's `plausibleValueLow` / `plausibleValueHigh`. The bounds live in a small reference table, one row per LOINC code, which is exactly the template pattern from earlier: one check, thousands of concrete bounds.

```sql
-- 29463-7 body weight (kg)  0–650   |  8480-6 systolic BP (mm[Hg])  0–300
-- 8867-4  heart rate (/min) 0–300   |  4548-4 HbA1c (%)             0–20
SELECT o.id, o.loinc, o.value, o.unit
FROM obs o JOIN obs_range r ON o.loinc = r.loinc
WHERE o.value < r.low OR o.value > r.high
```

*Wrong unit for the measurement* — DQD's `plausibleUnitConceptIds`. A body weight recorded in anything but a mass unit is suspect no matter how sane the number looks:

```sql
SELECT id, value, unit FROM obs
WHERE loinc = '29463-7' AND unit NOT IN ('kg', 'g', '[lb_av]')
```

*A test that contradicts the patient's sex* — DQD's `plausibleGender`. A prostate-specific antigen result on a female patient (`patient_flat` carries `gender`):

```sql
SELECT o.id, o.patient_id
FROM obs o JOIN pat ON o.patient_id = pat.id
WHERE o.loinc = '2857-1' AND pat.gender = 'female'
```

Cross-resource rules land here too. "A patient on diabetes medication should have a diabetes diagnosis" is a join — routine in SQL, awkward to impossible in FHIRPath.

**Profiling metrics** aren't pass/fail at all, just the numbers a dashboard needs:

```sql
SELECT count(*)                              AS "rowCount",
       count(*) FILTER (WHERE value IS NULL) AS "nullCount_value",
       count(DISTINCT patient_id)            AS "distinctCount_patient",
       min(value) AS "min_value", max(value) AS "max_value"
FROM obs
```

And roll-ups compose through the same dependency mechanism, pointing at other checks instead of views:

```sql
SELECT category, count(*) AS checks, sum(failed) AS failed
FROM ( SELECT 'conformance'  category, (SELECT count(*) FROM c1) > 0 failed
       UNION ALL SELECT 'conformance',  (SELECT count(*) FROM c2) > 0
       UNION ALL SELECT 'completeness', (SELECT count(*) FROM c3) > 0 ) t
GROUP BY category
```

The payoff lands where FHIR already does its work: the Implementation Guide. An IG author today ships an **instance profile** — the agreement on what goes where and how it's coded. With this, the same IG carries its other half, a **dataset profile**, in the same bundle:

- the **ViewDefinitions** that flatten data conforming to those profiles into tables, and
- the **quality checks** — SQLQuery checks over those tables, each tagged with its Kahn category and threshold.

Now an IG says more than *"here is the shape your data should take."* It says *"here is the shape, here is how to query it, and here is how to tell whether your data lives up to it."* A consumer points the package at a Bulk export and gets a data quality dashboard back — *this dataset passes 94 of 100 checks from this IG* — without writing a line of bespoke validation code. Profiles, views, and checks travel together, authored by the people who understand the domain.

## Where this goes

Put the pieces together and the picture is simple. Today an Implementation Guide ships an **instance profile**, and increasingly **ViewDefinitions**. With this, it ships the missing half — a **dataset profile**: a curated set of data-quality checks that spell out what a good dataset *for this IG* actually looks like. Publish both together, and any conformant SQL on FHIR engine runs the checks out of the box. The author writes them once; every server grades data against them the same way — no bespoke tooling, no per-vendor setup.

One honest limit: these checks can't be auto-derived from a profile's invariants, because ViewDefinition's FHIRPath subset is smaller than what those invariants use. The base catalogue gets written by hand — a one-time job the community shares.

And that's the invitation. This isn't hypothetical — it's live work in the [SQL on FHIR working group](https://github.com/HL7/sql-on-fhir), with the extension definitions and a starter set of checks in [issue #375](https://github.com/HL7/sql-on-fhir/issues/375), worked forward on the group's calls. The taxonomy is settled and the machinery exists; what's left is building the catalogue, in the open. If you've built data quality tooling over FHIR — or ever wished FHIR had it — come help design it: bring your checks to the thread and join a call.