SQL on FHIR WG Meetings
SQL on FHIR WG Meeting — February 8, 2024
Feb 8, 2024

Topics discussed:

  • The strongest roadmap opinion was to stop adding things: publish what exists — the ViewDefinition resource, the documentation, a stable base with test cases that multiple implementations pass — call that version one and draw a line in the sand. No ambition for new features that haven't been tested; declaring intent might just squeak in. The group agreed on that scope, which leaves the tests as the real blocker: two efforts need merging before they drift to cross purposes, and the highest-value thing anyone who has implemented this can do is plug the tests in even if they fail, because that exercise is what validates the tests and surfaces the hairy parts.
  • How do you tell a FHIR server to run a ViewDefinition? One implementation does it implicitly — create the ViewDefinition and the view exists — but an explicit operation would let you parameterise what actually varies: temporary or unlogged table, materialised view or not. The most wanted piece was the ability to ask in advance whether a server would run a given view, rather than turning it on and finding out, much the way SubscriptionTopic lets a system declare up front what it will do. Not before v1, though.
  • Instead of exporting everything and flattening it afterwards, say "give me a bulk export using this ViewDefinition, as Parquet or CSV". You get a bespoke extract, the implementation details stay under the hood, and it sidesteps what makes bulk data expensive today — serialising enormous amounts of data just to hand it all over. A supporting number from one pipeline: rebuilding a view by reading from the FHIR server is 20 to 30 times slower than reading Parquet and applying the view to that. The database is the slow part.
  • The columnar and JSON schema guidance sits awkwardly inside a spec that is otherwise data-layer agnostic, so the proposal was a separate repository and spec — narrow, just the Parquet schema and recommendations, sitting between two standards — which SQL on FHIR and bulk data could both point at. One caution: whatever gets published as a "recommendation" or a "sample" becomes a de facto standard the moment people implement it, so be careful what goes out.
  • Whether Parquet is even good ground for standardisation was contested. Against: there's no single perfect generalised mapping, and even people who have built a generalised one find themselves reaching for implementation-guide-specific schemas to fit the actual use case. For: without a defined mapping, Parquet cannot be a bulk export output format at all — if you want it as an output, someone has to write down how extensions and the rest are represented. The two positions turned out to be about different things — a near-lossless rendering for passing data around, versus tuning the shape once it has landed — and both were accepted.