Tutorial: Produce QuestionnaireResponse to Kafka topic

Stream FHIR QuestionnaireResponse data to Apache Kafka using Aidbox Forms and topic subscriptions.

Topic-based Subscriptions to Kafka

DEMO | Documentation

This example showcases Aidbox SubscriptionTopic producing data to Kafka.

Objectives:

  1. 1.
    Set up Aidbox and Kafka locally using Docker Compose.
  2. 2.
    Get FHIR QuestionnaireResponse via Aidbox Forms.
  3. 3.
    Learn how AidboxSubscriptionTopic and AidboxTopicDestination work with Kafka to handle the collected data.

Table of Contents

Prerequisites

Step 1: Set Up the Environment

Set Up Aidbox

  1. 1.

    Copy the .env.tpl file to .env:

    cp .env.tpl .env
  2. 2.

    Get a self-hosted Aidbox license from the Aidbox Portal .

  3. 3.

    Add the license key (AIDBOX_LICENSE) to the .env file.

Run Aidbox, Kafka & Kafka UI

docker compose up

The Docker Compose file initializes the environment for both Kafka and Aidbox with the following configuration:

  • Imports FHIR Questionnaire (see init-aidbox service).
  • Creates a Kafka topic for QuestionnaireResponse (see init-kafka service).

Step 2: Set Up Subscription and Destination

Create AidboxSubscriptionTopic Resource

To create a subscription on the QuestionnaireResponse resource that has a specific status, open Aidbox UI -> APIs -> REST Console and execute the following request:

POST /fhir/AidboxSubscriptionTopic
content-type: application/json
accept: application/json

{
  "resourceType": "AidboxSubscriptionTopic",
  "url": "http://example.org/FHIR/R5/SubscriptionTopic/QuestionnaireResponse-topic",
  "status": "active",
  "trigger": [
    {
      "resource": "QuestionnaireResponse",
      "fhirPathCriteria": "status = 'completed' or status = 'amended'"
    }
  ]
}
Using %current and %previous in fhirPathCriteria

When working with resource updates, you can use special variables in your fhirPathCriteria to compare the current and previous state of a resource:

  • %current - refers to the current state of the resource after an update
  • %previous - refers to the previous state of the resource before an update

This allows for creating more specific triggers based on changes in resource values. For example:

POST /fhir/AidboxSubscriptionTopic
content-type: application/json
accept: application/json

{
  "resourceType": "AidboxSubscriptionTopic",
  "url": "http://example.org/FHIR/R5/SubscriptionTopic/QuestionnaireResponse-status-change",
  "status": "active",
  "trigger": [
    {
      "resource": "QuestionnaireResponse",
      "supportedInteraction": ["update"],
      "fhirPathCriteria": "%previous.status = 'in-progress' and %current.status = 'completed'"
    }
  ]
}

On the create interaction %previous return {}. On the delete interaction %current return {}.

Create AidboxTopicDestination Resource

Creating this resource establishes a connection to the Kafka server. When the system produces an event, it will be processed to the specified Kafka topic.

POST /fhir/AidboxTopicDestination
content-type: application/json
accept: application/json

{
  "meta": {
    "profile": [
      "http://aidbox.app/StructureDefinition/aidboxtopicdestination-kafka-at-least-once"
    ]
  },
  "kind": "kafka-at-least-once",
  "id": "kafka-destination",
  "topic": "http://example.org/FHIR/R5/SubscriptionTopic/QuestionnaireResponse-topic",
  "parameter": [
    {
      "name": "kafkaTopic",
      "valueString": "aidbox-forms"
    },
    {
      "name": "bootstrapServers",
      "valueString": "kafka:29092"
    }
  ]
}

Step 3: Demonstration

Submit Form

Open the list of forms , click share -> enable 'allow amend' -> click attach -> copy the link -> open the link -> fill out the form, and submit it.

Check AidboxTopicDestination Status

Open the Aidbox REST Console and get the AidboxTopicDestination status:

GET /fhir/AidboxTopicDestination/kafka-destination/$status

See Messages in Kafka UI

Open Kafka UI -> Topics -> aidbox-forms -> messages and review the QuestionnaireResponse that was created after submitting the form.

Example of Kubernetes Setup

Also you can find example of k8s deployment:

Last updated: