Aidbox Docs

Aidbox.bulk data import

Overview

aidbox.bulk API uses Aidbox JSON RPC API

Features of the aidbox.bulk API:

  • Asynchronous validation of uploaded resources & references
  • Human readable validation errors
  • Can process Aidbox & FHIR format

RPC methods

aidbox.bulk/import-start

Starts aidbox.bulk import.

Object with the following structure:

  • format input resources format, fhir or aidbox (default: fhir)
  • meta meta data that will be attached to each of the imported resources
  • on-conflict action to resolve id uniqueness constraint violation, update or override
  • input array of objects
    • url string with input source url

Returns input params object with following attributes added:

  • status in-progress, finished, failed
  • count overall number of imported resources
  • time import execution time
  • input the import status of each specific input
    • status loaded, validated, failed
    • count number of loaded resources
    • errors number of validation errors
    • time input execution time
  • message "There is running import - wait until it finish or cancel it with aidbox.bulk/import-cancel"
  • import object identical to result object

Resource requirements for aidbox.bulk/import-start:

OperationidresourceType
aidbox.bulk/import-startRequiredRequired

Example

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-start
params:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta: {source: app1}
  input:
  - {url: 'https://storage.googleapis.com/aidbox-public/synthea/100/corrupted-patient.ndjson.gz'}
status: 200
result:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta: {source: app1}
  input:
  - {url: 'https://storage.googleapis.com/aidbox-public/synthea/100/corrupted-patient.ndjson.gz', status: loaded, count: 124, errors: 1, time: 157}
  status: failed
  errors: 1
  time: 241

aidbox.bulk/import-status

Returns latest aidbox.bulk import info

Expects no parameters

Same as aidbox.bulk/import-start result

message "No active imports"

Example

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-status
params: {}
Status: 200
result:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta:
    source: app1
  input:
    - url: >-
        https://storage.googleapis.com/aidbox-public/synthea/100/Patient.ndjson.gz
      status: loaded
      count: 124
      errors: 0
      time: 1205
  status: finished
  count: 0
  time: 1452

aidbox.bulk/import-errors

Returns latest aidbox.bulk import detailed errors list

Object with the following structure:

  • omit-resources? (default: false)

Result is an array of objects with following structure:

  • input_no
  • line_no
  • type
  • id
  • resource
  • errors

message "No active imports"

Example

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-errors
params:
  omit-resources?: true
Status: 200
result:
  - input_no: 0
    line_no: 1
    type: Patient
    id: e9adac47-eb98-4fce-b871-512226086c97
    errors:
      - path:
          - name
          - 0
          - given
        message: expected array
  - input_no: 0
    line_no: 20
    type: Patient
    id: b2d58f0f-4499-4392-ad3a-1c2141c8a6c1
    errors:
      - path:
          - address
          - 0
          - line
        message: expected array

aidbox.bulk/import-cancel

Cancels latest aidbox.bulk import

Expects no parameters

Returns object with following structure:

  • message "Import canceled"
  • import Same as aidbox.bulk/import-start result

message "No active imports"

Example

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-cancel
Status: 200
result:
  message: "Import Canceled"
  import: <. . .>

Import local file

Sometimes you want to import local file into local Aidbox. Possible solutions for local development:

Add volume to the aidboxone container (not aidboxdb):

volumes:
- ./Encounter.ndjson.gz:/resources/Encounter.ndjson.gz
# url: file:///resources/Encounter.ndjson.gz

Use tunneling e.g. ngrok: 

python3 -m http.server 
ngrok http 8000
# url: https://<...>.ngrok-free.app/Encounter.ndjson.gz

Last updated: