Aidbox.bulk data import

Overview

aidbox.bulk API uses Aidbox JSON RPC API

Features of the aidbox.bulk API:

Asynchronous validation of uploaded resources & references
Human readable validation errors
Can process Aidbox & FHIR format

RPC methods

`aidbox.bulk/import-start`

Starts aidbox.bulk import.

Object with the following structure:

format input resources format, fhir or aidbox (default: fhir)
meta meta data that will be attached to each of the imported resources
on-conflict action to resolve id uniqueness constraint violation, update or override
input array of objects
- url string with input source url

Resource requirements for `aidbox.bulk/import-start`:

Operation	id	resourceType
`aidbox.bulk/import-start`	Required	Required

Example

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-start
params:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta: {source: app1}
  input:
  - {url: 'https://storage.googleapis.com/aidbox-public/synthea/100/corrupted-patient.ndjson.gz'}

status: 200
result:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta: {source: app1}
  input:
  - {url: 'https://storage.googleapis.com/aidbox-public/synthea/100/corrupted-patient.ndjson.gz', status: loaded, count: 124, errors: 1, time: 157}
  status: failed
  errors: 1
  time: 241

`aidbox.bulk/import-status`

Returns latest aidbox.bulk import info

Expects no parameters

Example

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-status
params: {}

Status: 200
result:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta:
    source: app1
  input:
    - url: >-
        https://storage.googleapis.com/aidbox-public/synthea/100/Patient.ndjson.gz
      status: loaded
      count: 124
      errors: 0
      time: 1205
  status: finished
  count: 0
  time: 1452

`aidbox.bulk/import-errors`

Returns latest aidbox.bulk import detailed errors list

Object with the following structure:

omit-resources? (default: false)

Example

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-errors
params:
  omit-resources?: true

Status: 200
result:
  - input_no: 0
    line_no: 1
    type: Patient
    id: e9adac47-eb98-4fce-b871-512226086c97
    errors:
      - path:
          - name
          - 0
          - given
        message: expected array
  - input_no: 0
    line_no: 20
    type: Patient
    id: b2d58f0f-4499-4392-ad3a-1c2141c8a6c1
    errors:
      - path:
          - address
          - 0
          - line
        message: expected array

`aidbox.bulk/import-cancel`

Cancels latest aidbox.bulk import

Expects no parameters

Example

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-cancel

Import local file

Sometimes you want to import local file into local Aidbox. Possible solutions for local development:

Add volume to the `aidboxone` container (not `aidboxdb`):

volumes:
- ./Encounter.ndjson.gz:/resources/Encounter.ndjson.gz
# url: file:///resources/Encounter.ndjson.gz

Use tunneling e.g. ngrok:

python3 -m http.server 
ngrok http 8000
# url: https://<...>.ngrok-free.app/Encounter.ndjson.gz

Overview

RPC methods

aidbox.bulk/import-start

Resource requirements for aidbox.bulk/import-start:

Example

aidbox.bulk/import-status

Example

aidbox.bulk/import-errors

Example

aidbox.bulk/import-cancel

Example

Import local file

Add volume to the aidboxone container (not aidboxdb):

Use tunneling e.g. ngrok:

`aidbox.bulk/import-start`

Resource requirements for `aidbox.bulk/import-start`:

`aidbox.bulk/import-status`

`aidbox.bulk/import-errors`

`aidbox.bulk/import-cancel`

Add volume to the `aidboxone` container (not `aidboxdb`):