For AI agents: the documentation index is at /docs/aidbox/llms.txt. A Markdown version of this page is available at /docs/aidbox/notebooks.md or by requesting it with the Accept: text/markdown header.
Aidbox Docs
All notebooks

SQL on FHIR: analytics on form data

Description

45 cells · updated Feb 18, 2025

This is a read-only view. Responses shown were saved when the notebook was published — open it in your own Aidbox to run the cells against live data.

Load sample data

We generated 100 000 sample responses for the Risk Assessment Questionnaire provided in FHIR Analytics with Aidbox and Metabase sample project.

The following cell loads responses (QuestionnaireResponse resources) and extracted resources (Observation and Patient resources).

REST Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
POST /v2/fhir/$import
Accept: text/yaml
Content-Type: text/yaml

id: myimp
contentEncoding: gzip
inputs:
- resourceType: Patient
  url: https://storage.googleapis.com/aidbox-public/events/2025-02-philippines-bootcamp/patient.ndjson.gz
- resourceType: QuestionnaireResponse
  url: https://storage.googleapis.com/aidbox-public/events/2025-02-philippines-bootcamp/questionnaireresponse.ndjson.gz
- resourceType: Observation
  url: https://storage.googleapis.com/aidbox-public/events/2025-02-philippines-bootcamp/observation.ndjson.gz
Response: Body
Status: 200
{}

Data

The extracted observation are of three kinds:

  • Body mass index (BMI);
  • Smoking status (smoker, passive smoker, stopped smoking, never smoked);
  • Excessive alcohol consumption (yes/no)

Let's look at each observation more closely

BMI

BMI observation contain BMI value and SNOMED code defining observation type.

REST Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
GET /fhir/Observation/gen-obs-bmi-1
accept: text/yaml
Response: Body
Status: 200
meta:
  lastUpdated: '2025-01-28T14:56:39.967857Z'
  versionId: '81'
  extension:
  - {url: 'ex:createdAt', valueInstant: '2025-01-28T14:56:39.967857Z'}
derivedFrom:
- {reference: QuestionnaireResponse/gen-qr-1}
valueQuantity: {value: 24.064332204316088}
resourceType: Observation
effectiveDateTime: '2025-01-28T13:44:37.022833976Z'
status: final
id: gen-obs-bmi-1
code:
  coding:
  - {code: '60621009', system: 'http://snomed.info/sct', display: 'Body Mass Index '}

Smoking status

Smoking status observation contain SNOMED code 365981007 (Tobacco smoking behavior — finding) defining the observation kind and one of the following SNOMED codes:

  • 77176002 (smoker)
  • 266919005 (never smoked)
  • 160617001 (stopped smoking)
  • 43381005 (passive smoker)
REST Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
GET /fhir/Observation/gen-obs-smoking-1
Accept: text/yaml
Response: Body
Status: 200
meta:
  lastUpdated: '2025-01-28T14:56:39.967857Z'
  versionId: '81'
  extension:
  - {url: 'ex:createdAt', valueInstant: '2025-01-28T14:56:39.967857Z'}
derivedFrom:
- {reference: QuestionnaireResponse/gen-qr-1}
resourceType: Observation
effectiveDateTime: '2025-01-28T13:44:37.022833976Z'
status: final
id: gen-obs-smoking-1
code:
  coding:
  - {code: '365981007', system: 'http://snomed.info/sct', display: Tobacco smoking behavior - finding}
valueCodeableConcept:
  coding:
  - {code: '77176002', system: 'http://snomed.info/sct', display: Smoker}

Excessive alcohol consumption

Excessive alcohol consumption observation contains SNOMED code 719848005 (Disorder caused by alcohol) defining observation kind and one of the following SNOMED codes as value:

  • 373067005 (no)
  • 373066001 (yes)

Note that the questionnaire contains question: "Have you ever consumed alcohol?". And the observation is created only if alcohol has ever been consumed.

REST Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
GET /fhir/Observation/gen-obs-exc-alc-2
Response: Body
Status: 200
{
  "meta": {
    "lastUpdated": "2025-01-28T14:56:39.967857Z",
    "versionId": "81",
    "extension": [
      {
        "url": "ex:createdAt",
        "valueInstant": "2025-01-28T14:56:39.967857Z"
      }
    ]
  },
  "derivedFrom": [
    {
      "reference": "QuestionnaireResponse/gen-qr-2"
    }
  ],
  "resourceType": "Observation",
  "effectiveDateTime": "2025-01-28T13:44:37.022833976Z",
  "status": "final",
  "id": "gen-obs-exc-alc-2",
  "code": {
    "coding": [
      {
        "code": "719848005",
        "system": "http://snomed.info/sct",
        "display": "Disorder caused by alcohol"
      }
    ]
  },
  "valueCodeableConcept": {
    "coding": [
      {
        "code": "373067005",
        "system": "http://snomed.info/sct",
        "display": "No"
      }
    ]
  }
}

Create ViewDefinition resources

As we have seen above, Observation resources are quite nested and inconvenient to use in analytical SQL queries.

SQL on FHIR provides a way to extract relevant data in nice tabular format: ViewDefinition resources.

In our case all Observation resources have similar structure: each of them contains observed value and a code defining observation kind.

BMI

We will create a table with the following structure

Observation idBMI valueQuestionnaireResponse id

Note that Aidbox uses string type by default, therefore we need to specify datatype for the BMI value.

REST Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
PUT /fhir/ViewDefinition/observation_bmi_view
Content-Type: text/yaml
Accept: text/yaml

name: observation_bmi_view

resource: observation
select:
  - column:
      - name: observation_id
        path: getResourceKey()
      - name: bmi
        path: value.ofType(Quantity).value
        type: decimal
      - name: qr_id
        path: derivedFrom.first().getReferenceKey()
where:
  - path: code.coding.where(code = '60621009').exists()

status: active
id: observation_bmi_view
resourceType: ViewDefinition
Response: Body
Status: 201
where:
- {path: code.coding.where(code = '60621009').exists()}
meta:
  lastUpdated: '2025-02-17T13:20:38.778225Z'
  versionId: '107'
  extension:
  - {url: 'https://fhir.aidbox.app/fhir/StructureDefinition/created-at', valueInstant: '2025-02-17T13:20:38.778225Z'}
name: observation_bmi_view
resourceType: ViewDefinition
status: active
id: observation_bmi_view
resource: observation
select:
- column:
  - {name: observation_id, path: getResourceKey()}
  - {name: bmi, path: value.ofType(Quantity).value, type: decimal}
  - {name: qr_id, path: derivedFrom.first().getReferenceKey()}

Aidbox automatically create a view in the sof schema. Let's look at data.

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
select * from sof.observation_bmi_view
limit 10
Result
bmiqr_idobservation_id
24.064332204316088gen-qr-1gen-obs-bmi-1
23.320073211330087gen-qr-10gen-obs-bmi-10
30.760773314666764gen-qr-100gen-obs-bmi-100
24.825445454831456gen-qr-1000gen-obs-bmi-1000
27.455811393648887gen-qr-10000gen-obs-bmi-10000
21.7619619118027gen-qr-100000gen-obs-bmi-100000
23.233307017944394gen-qr-10001gen-obs-bmi-10001
28.6701388177837gen-qr-10002gen-obs-bmi-10002
28.247785615346345gen-qr-10003gen-obs-bmi-10003
32.117852778104975gen-qr-10004gen-obs-bmi-10004

Smoking status

This is similar to the previous case.

We will create a table with the following structure

Observation idSNOMED codeHuman-readable valueQuestionnaireResponse id
REST Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
PUT /fhir/ViewDefinition/observation_smoking_view
Content-Type: text/yaml
Accept: text/yaml

name: observation_smoking_view
resource: observation

select:
  - column:
      - name: observation_id
        path: getResourceKey()
      - name: code
        path: value.ofType(CodeableConcept).coding.first().code
      - name: display
        path: value.ofType(CodeableConcept).coding.first().display
      - name: qr_id
        path: derivedFrom.first().getReferenceKey()
where:
  - path: code.coding.where(code = '365981007').exists()

status: active
id: observation_smoking_view
resourceType: ViewDefinition
Response: Body
Status: 201
where:
- {path: code.coding.where(code = '365981007').exists()}
meta:
  lastUpdated: '2025-02-17T13:20:47.141847Z'
  versionId: '110'
  extension:
  - {url: 'https://fhir.aidbox.app/fhir/StructureDefinition/created-at', valueInstant: '2025-02-17T13:20:47.141847Z'}
name: observation_smoking_view
resourceType: ViewDefinition
status: active
id: observation_smoking_view
resource: observation
select:
- column:
  - {name: observation_id, path: getResourceKey()}
  - {name: code, path: value.ofType(CodeableConcept).coding.first().code}
  - {name: display, path: value.ofType(CodeableConcept).coding.first().display}
  - {name: qr_id, path: derivedFrom.first().getReferenceKey()}
SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
select * from sof.observation_smoking_view
limit 10;
Result
codeqr_iddisplayobservation_id
77176002gen-qr-1Smokergen-obs-smoking-1
266919005gen-qr-10Never smoked tobaccogen-obs-smoking-10
77176002gen-qr-100Smokergen-obs-smoking-100
266919005gen-qr-1000Never smoked tobaccogen-obs-smoking-1000
160617001gen-qr-10000Stopped smokinggen-obs-smoking-10000
266919005gen-qr-100000Never smoked tobaccogen-obs-smoking-100000
160617001gen-qr-10001Stopped smokinggen-obs-smoking-10001
77176002gen-qr-10002Smokergen-obs-smoking-10002
43381005gen-qr-10003Passive smokergen-obs-smoking-10003
43381005gen-qr-10004Passive smokergen-obs-smoking-10004

Excessive alcohol consumption observation

This is the same as smoking status, only observation kind code differs.

Observation idSNOMED codeHuman-readable valueQuestionnaireResponse id
REST Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
PUT /fhir/ViewDefinition/observation_alcohol_view
Content-Type: text/yaml
Accept: text/yaml

name: observation_alcohol_view
resource: observation

select:
  - column:
      - name: observation_id
        path: getResourceKey()
      - name: code
        path: value.ofType(CodeableConcept).coding.first().code
      - name: display
        path: value.ofType(CodeableConcept).coding.first().display
      - name: qr_id
        path: derivedFrom.first().getReferenceKey()
where:
  - path: code.coding.where(code = '719848005').exists()

status: active
id: observation_alcohol_view
resourceType: ViewDefinition
Response: Body
Status: 201
where:
- {path: code.coding.where(code = '719848005').exists()}
meta:
  lastUpdated: '2025-02-17T13:21:02.093064Z'
  versionId: '113'
  extension:
  - {url: 'https://fhir.aidbox.app/fhir/StructureDefinition/created-at', valueInstant: '2025-02-17T13:21:02.093064Z'}
name: observation_alcohol_view
resourceType: ViewDefinition
status: active
id: observation_alcohol_view
resource: observation
select:
- column:
  - {name: observation_id, path: getResourceKey()}
  - {name: code, path: value.ofType(CodeableConcept).coding.first().code}
  - {name: display, path: value.ofType(CodeableConcept).coding.first().display}
  - {name: qr_id, path: derivedFrom.first().getReferenceKey()}
SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
SELECT * FROM sof.observation_alcohol_view
LIMIT 10
Result
codeqr_iddisplayobservation_id
373066001gen-qr-100Yesgen-obs-exc-alc-100
373067005gen-qr-1000Nogen-obs-exc-alc-1000
373067005gen-qr-10000Nogen-obs-exc-alc-10000
373066001gen-qr-10001Yesgen-obs-exc-alc-10001
373067005gen-qr-10002Nogen-obs-exc-alc-10002
373066001gen-qr-10008Yesgen-obs-exc-alc-10008
373067005gen-qr-10009Nogen-obs-exc-alc-10009
373066001gen-qr-10011Yesgen-obs-exc-alc-10011
373067005gen-qr-10012Nogen-obs-exc-alc-10012
373067005gen-qr-10013Nogen-obs-exc-alc-10013

QuestionnaireResponse

We will take gender into account in our analytical queries. It is not extracted from the sample QuestionnaireResponse resource, so we will create ViewDefinition to extract it.

QuestionnaireResponse idgender
REST Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
PUT /fhir/ViewDefinition/qr_view
Content-Type: text/yaml
Accept: text/yaml

name: qr_view
resource: questionnaireresponse

select:
  - column:
      - name: qr_id
        path: getResourceKey()
      - name: gender
        path: >-
          item.where(linkId='Zicv0gIs').item.where(linkId='Jw_kMVir').answer.first().value.ofType(Coding).code
where:
  - path: questionnaire = 'http://forms.aidbox.io/questionnaire/cvd-form'

status: active
id: qr_view
resourceType: ViewDefinition
Response: Body
Status: 201
where:
- {path: 'questionnaire = ''http://forms.aidbox.io/questionnaire/cvd-form'''}
meta:
  lastUpdated: '2025-02-17T13:21:12.436424Z'
  versionId: '116'
  extension:
  - {url: 'https://fhir.aidbox.app/fhir/StructureDefinition/created-at', valueInstant: '2025-02-17T13:21:12.436424Z'}
name: qr_view
resourceType: ViewDefinition
status: active
id: qr_view
resource: questionnaireresponse
select:
- column:
  - {name: qr_id, path: getResourceKey()}
  - {name: gender, path: item.where(linkId='Zicv0gIs').item.where(linkId='Jw_kMVir').answer.first().value.ofType(Coding).code}
SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
select * from sof.qr_view limit 10;
Result
qr_idgender
gen-qr-1male
gen-qr-10female
gen-qr-100male
gen-qr-1000male
gen-qr-10000male
gen-qr-100000female
gen-qr-10001male
gen-qr-10002female
gen-qr-10003male
gen-qr-10004male

Creating custom views

SQL on FHIR v2 describes how to flatten a single resource. It is useful to create custom views for our cases.

First, let's create a schema for our views

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
CREATE SCHEMA IF NOT EXISTS u_analytics;
0 rows affected

Next views will prepare data for visualization

Combined smoking data

Let's join smoking observations view and questionnaire response view to get combined data, then transform machine-readable codes to human-readable text.

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
CREATE OR REPLACE VIEW u_analytics.smoking AS
SELECT
	qr.gender AS gender,
    CASE
    	WHEN obs.code = '77176002' THEN 'Smoker'
        WHEN obs.code = '266919005' THEN 'Never smoked'
        WHEN obs.code = '160617001' THEN 'Stopped smoking'
        WHEN obs.code = '43381005' THEN 'Passive smoker'
    END AS status
FROM sof.observation_smoking_view obs
JOIN sof.qr_view qr USING (qr_id);
0 rows affected

Result:

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
SELECT * FROM u_analytics.smoking LIMIT 10;
Result
genderstatus
maleSmoker
femaleNever smoked
maleSmoker
maleNever smoked
maleStopped smoking
femaleNever smoked
maleStopped smoking
femaleSmoker
malePassive smoker
malePassive smoker

Combined alcohol view

Alcohol observations are absent if a person never consumed alcohol. For our analytics it is useful to have this information present.

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
CREATE OR REPLACE VIEW u_analytics.alcohol AS
SELECT
	qr.gender AS gender,
    CASE
    	WHEN code = '373067005' THEN 'Not binge drinker'
        WHEN code = '373066001' THEN 'Binge drinker'
        ELSE 'Not drinker'
    END AS status
FROM sof.observation_alcohol_view obs
RIGHT JOIN sof.qr_view qr USING (qr_id);
0 rows affected

Result:

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
SELECT * FROM u_analytics.alcohol LIMIT 10;
Result
genderstatus
maleNot drinker
femaleNot drinker
maleBinge drinker
maleNot binge drinker
maleNot binge drinker
femaleNot drinker
maleBinge drinker
femaleNot binge drinker
maleNot drinker
maleNot drinker

Combined BMI view

And here it is useful to have both BMI and assessment (overweight or not)

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
CREATE OR REPLACE VIEW u_analytics.bmi AS 
SELECT
	qr.gender AS gender,
    CASE
    	WHEN obs.bmi >= 25 THEN 'Overweight or obese'
        ELSE 'Normal weight'
    END status,
    obs.bmi as bmi
FROM sof.observation_bmi_view obs
JOIN sof.qr_view qr USING (qr_id);
0 rows affected

Result:

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
SELECT * FROM u_analytics.bmi LIMIT 10;
Result
bmigenderstatus
24.064332204316088maleNormal weight
23.320073211330087femaleNormal weight
30.760773314666764maleOverweight or obese
24.825445454831456maleNormal weight
27.455811393648887maleOverweight or obese
21.7619619118027femaleNormal weight
23.233307017944394maleNormal weight
28.6701388177837femaleOverweight or obese
28.247785615346345maleOverweight or obese
32.117852778104975maleOverweight or obese

Analytics

Smoking statistics

What proportion of adults smoke (by sex and overall)?

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
select
	gender,
    status,
    count(*) * 100 / sum(count(*)) over () AS "percentage (total)",
    count(*) * 100 / sum(count(*)) over (PARTITION BY gender) as "percentage (group)"
FROM u_analytics.smoking
GROUP BY (status, gender)
ORDER BY gender asc, "percentage (group)" desc
limit 10;
Result
genderstatuspercentage (group)percentage (total)
femalePassive smoker38.15346375881975419.034
femaleNever smoked29.1472899294419514.541
femaleSmoker22.79906991661321311.374
femaleStopped smoking9.900176395125084.939
malePassive smoker38.02282886334610619.054
maleNever smoked28.99704661558109814.531
maleSmoker23.06034482758620611.556
maleStopped smoking9.919779693486594.971

Excessive alcohol consumption

What proportion of adult drank excessively in the past 6 months (by sex and overall)?

Note that we use RIGHT JOIN here, since not all responses generate observation, they don't if a person never drank.

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
select
	gender,
    status,
    count(*) * 100 / sum(count(*)) over () AS "percentage (total)",
    count(*) * 100 / sum(count(*)) over (PARTITION BY gender) as "percentage (group)"
FROM u_analytics.alcohol
GROUP BY (status, gender)
ORDER BY gender asc, "percentage (group)" desc
limit 10;
Result
genderstatuspercentage (group)percentage (total)
femaleNot binge drinker41.8918377164849320.899
femaleNot drinker32.83154265554842516.379
femaleBinge drinker25.27661962796664612.61
maleNot binge drinker41.5469348659003820.82
maleNot drinker33.1377713920817416.606
maleBinge drinker25.3152937420178812.686

Unhealthy BMI

What proportion of adults are overweight or obese (BMI > 25), by gender and overall?

SQL Send Cells run only inside Aidbox. Use Run in Aidbox above to open this notebook in your own instance.
select
	gender,
    status,
    count(*) * 100 / sum(count(*)) over () AS "percentage (total)",
    count(*) * 100 / sum(count(*)) over (PARTITION BY gender) as "percentage (group)"
FROM u_analytics.bmi
GROUP BY (status, gender)
ORDER BY gender asc, "percentage (group)" desc
limit 10;
Result
genderstatuspercentage (group)percentage (total)
femaleOverweight or obese53.3976106478511926.639
femaleNormal weight46.6023893521488123.249
maleOverweight or obese73.6550127713920836.91
maleNormal weight26.3449872286079213.202