Merge operation

MDMbox provides two merge operations: server-computed $merge/v2, which builds the transaction with a sandboxed server-side algorithm, and client-plan $merge, which executes a transaction Bundle supplied by the caller. Both modes add Task and Provenance records and commit successful business and audit changes atomically.

Server-computed merge v2

$merge/v2 merges one duplicate FHIR resource (source) into the surviving resource (target). They must have the same resourceType, for example two Patients or two Organizations. Task, Provenance, AuditEvent, and Device are server-managed and cannot be a merge pair. The caller may supply the desired target result, select a server-side algorithm, and list the related resource types whose source references must be reassigned. There is no client-provided transaction plan.

POST https://<mdmbox-host>/api/fhir/$merge/v2
Content-Type: application/fhir+json

{
  "resourceType": "Parameters",
  "parameter": [
    {
      "name": "source",
      "valueReference": { "reference": "Patient/duplicate-123" }
    },
    {
      "name": "target",
      "valueReference": { "reference": "Patient/primary-456" }
    },
    { "name": "merge-algorithm", "valueString": "simple" },
    {
      "name": "result",
      "resource": {
        "resourceType": "Patient",
        "id": "primary-456",
        "name": [{ "given": ["John"], "family": "Smith" }]
      }
    },
    { "name": "related-resource-type", "valueString": "Encounter" },
    { "name": "related-resource-type", "valueString": "Observation" },
    { "name": "preview", "valueBoolean": false }
  ]
}

The built-in simple algorithm optionally updates the target from result, changes references to the source within the selected related resource types, and deletes the source. Source and target must both exist, must be different resources of the same type, must not be marked not-a-match, and must not form an active or circular merge. Related resources may have the same type as the pair: an Organization's partOf can be relinked during an Organization merge. A malformed request (including a cross-type or server-managed pair) or unknown algorithm returns 400 Bad Request, a missing resource returns 404 Not Found, invalid state or transaction data returns 422 Unprocessable Entity, and an algorithm or unexpected server failure returns 500 Internal Server Error.

result supplies the desired target content, not a version precondition. The built-in simple algorithm uses the current target version read by the server for ifMatch and the pre-merge history reference, even if result.meta.versionId contains an older version.

Operators may disable built-ins with MDMBOX_BUILT_IN_ALGORITHMS: unset enables all built-ins, empty disables all, and simple,strict enables only those ids. Disabled built-ins disappear from the catalog and cannot be invoked explicitly or implicitly as the default. If the selected id is unavailable, preview and execution both return HTTP 400 with an OperationOutcome. Custom Git and database algorithms are unaffected; see configuration reference.

All mutations generated by an algorithm are version-protected and limited to the source, target, and resources discovered in the merge snapshot. An algorithm cannot write server-managed Task, Provenance, AuditEvent, or Device resources. MDMbox adds Task and Provenance after the algorithm returns and executes them in the same transaction as the merge plan. Operation-level AuditEvent emission and failure auditing are deferred to a separate change.

Concurrent changes by ordinary FHIR writers can invalidate the computed plan. In that case, v2 returns 409 Conflict with an OperationOutcome and rolls back the complete business transaction; the caller can retry against the current state. Pair merges wait for each other before opening their database snapshot.

Custom v2 algorithms may create resources with an unconditional POST and a unique urn:uuid: fullUrl, except resources of the pair's type and server-managed types. Conditional creation with request.ifNoneExist (or an equivalent if-none-exist header) is forbidden: it can return an existing resource that unmerge would otherwise mistake for newly created data. Such an algorithm plan returns 500 Internal Server Error with an OperationOutcome before preview or execution. This restriction does not change the legacy client-plan endpoints.

In v2 plans, non-POST entries must omit fullUrl. Their resource identity must match the canonical request URL, including the result of a PATCH. Request fields are limited to method, url, ifMatch and ifNoneMatch; raw libox headers cannot override the validated preconditions.

The v2 algorithm-plan restrictions above do not apply to client plans for $merge, $unmerge, $link, or $unlink. Those operations retain their existing contracts; their own operation rules and FHIR transaction validation still apply.

Preview returns a Parameters resource containing outcome and the complete audited plan. Successful execution returns outcome and the new merge task. Preview creates no Device, Task, Provenance, AuditEvent, or business-resource versions. Post-write audit references cannot be resolved during preview; their actual ids and versions are recorded only during execution.

Merge and unmerge JavaScript algorithms both return {plan, outcome?}. The plan is a FHIR transaction Bundle, not the whole algorithm result. Existing custom merge scripts must change return bundle to return {plan: bundle}; the server does not convert the old shape implicitly. This does not change legacy v1 client plans.

An algorithm may omit outcome when it has no specific messages. MDMbox then supplies an informational OperationOutcome, so the HTTP response still always contains outcome. When supplied, the outcome must contain at least one issue: outcome: null and issue: [] are invalid and return HTTP 500. Algorithm warnings are preserved in both preview and execution with HTTP 200. An error or fatal issue returns the OperationOutcome directly with HTTP 409 and prevents execution, even if a plan was also supplied. plan: null is allowed only with a blocking issue; the plan key is required.

Optionality here belongs to the internal JS API, not a claim that FHIR Patient $merge permits omitting its response outcome. The published FHIR Patient merge operation specifies an OperationOutcome in its response, and OperationOutcome.issue has cardinality 1..*. MDMbox's v2 endpoints use their own operation contract.

The merge Task records the exact source and target versions, algorithm identity, and related-resource-type scope needed by $unmerge/v2. Keep the corresponding Task, Provenance, and FHIR history versions for as long as server-computed unmerge must remain available.

Merge v2 records both sides of each update: Provenance.entity.what references the pre-merge version, and Provenance.target references the actual post-merge version, for example Patient/123/_history/20. Created resources also have version-specific targets. Deleted resources retain an unversioned target and a versioned pre-delete removal entity. The operation Task target stays unversioned for discovery with GET /Provenance?target=Task/<id>. Provenance is created once with the final references; its failure rolls back the business writes and Task as well. Legacy v1 audit assembly is unchanged.

Git algorithm storage

For script inputs, all mdm functions, examples, and plan restrictions, see the JavaScript algorithm API. Its helpers are not HTTP endpoints and are not described by Swagger.

Merge and unmerge algorithms can be loaded from a public or private Git repository. Store one synchronous entry point per file:

merge/
  identifier-union.js    # function merge(input, mdm)
unmerge/
  custom-restore.js      # function unmerge(input, mdm)

The filename without .js is the algorithm id. IDs are 1–64 characters, start with a letter or digit, and otherwise contain letters, digits, ., _, or -. Use only regular .js files directly inside these two directories: nested files, symlinks, and submodules there are rejected. Other repository paths are ignored. There must be 1–100 algorithm files in total, each at most 1 MiB, with a combined source size of at most 16 MiB per repository. There are no modules, imports, or build steps; each file must be self-contained.

Example container environment for a private HTTPS repository:

MDMBOX_ALGORITHM_GIT_URL: https://git.example/team/mdm-algorithms.git
MDMBOX_ALGORITHM_GIT_REF: refs/tags/release-1
MDMBOX_ALGORITHM_GIT_USERNAME: deploy-user
MDMBOX_ALGORITHM_GIT_TOKEN_FILE: /run/secrets/algorithm-git-token

Mount the token file read-only and grant the container's app user permission to read it. Use a repository-scoped read-only token. Do not put credentials in the repository URL or in scripts. For an internal CA, configure MDMBOX_ALGORITHM_GIT_CA_FILE; do not disable TLS verification.

For SSH, use a URL such as ssh://git@git.example/team/mdm-algorithms.git. Mount a read-only deploy key and verified known_hosts under the runtime user's .ssh directory, or provide an SSH agent. OpenSSH runs in batch mode with strict host-key checking: interactive password and trust prompts are disabled. The Docker image includes Git and the OpenSSH client; standalone deployments need these commands on PATH when Git storage is configured. Ambient Git credential helpers and Git tracing/configuration overrides are not used by this loader.

Loading has a 60-second budget. MDMbox fetches into a disposable bare repository, reads committed blobs without a checkout, validates the appropriate entry points in the existing JS sandbox, and discards the temporary repository. No Git hooks or submodules are executed. All scripts must pass validation before merge and unmerge catalogs are published together in one database transaction. The published scripts are stored in the shared database: preview and execution do not fetch or depend on Git availability. Each operation resolves its script once; a concurrent sync cannot change an operation already in progress. The remote repository must remain small enough to fetch within the budget. Source-size limits do not limit the downloaded Git history or temporary disk usage; use a small dedicated repository and deployment resource limits.

Both merge and unmerge resolve algorithms in this order: built-in, Git, then database. The catalogs are separate, so the same custom id may name both entry points. Select them with merge-algorithm or unmerge-algorithm. Two Git sources cannot publish the same (operation, id): the conflicting sync fails without changing either published catalog. An unknown id is still HTTP 400. Git-backed algorithms have exactly the same capabilities and plan restrictions as other server-side algorithms.

The allowlist removes only built-in implementations from that precedence order. A Git script with a disabled built-in's id remains selectable, including by the operation's unchanged default id. MDMbox never substitutes a different id.

The operation Task records algorithm storage git, the Git source id, exact commit and file path, and the executed script's SHA-256. These use merge-algorithm-git-source, merge-algorithm-git-commit, and merge-algorithm-git-path inputs (or the corresponding unmerge- prefix). Repository URLs and credentials are not copied into Task or diagnostic errors. Git algorithms appear in the Admin UI as read-only: they can be inspected and duplicated into database storage, but not edited or deleted there. See configuration reference.

Managing algorithms in the Admin UI

Open Algorithms, then choose Merge or Unmerge. Each tab lists the effective algorithms for that operation, with built-ins first and storage labels for built-in and Git source.

• New Algorithm creates a database script in the selected catalog.
• Duplicate copies any selected script into a new, editable form in the same catalog. Choose a unique id and save it.
• Database scripts can be edited and deleted. Save validates the appropriate merge(input, mdm) or unmerge(input, mdm) entry point before storing source. Changes are available to subsequent operations without a restart.
• Built-in and Git editors are read-only, including keyboard editing. Source can still be selected and copied. Git details show its source id, path, and published commit.

Ids are unique within an operation, not across both catalogs. Existing database scripts remain merge algorithms after upgrading. An enabled built-in or a Git algorithm takes precedence over database source with the same id; the merge database id simple remains reserved even when its built-in is disabled.

The Configuration tab manages Git sources while MDMbox is running:

1. Choose Add Git source. Enter a unique source id, repository URL, and ref. For private HTTPS access, enter the username and absolute server-side paths to mounted token and optional CA files. Never paste a token into the form.
2. Choose Save source. Saving configuration does not fetch or publish scripts.
3. Choose Sync on the source card. The background job fetches and validates the selected revision. The card shows progress, the published commit, and the last successful sync. Failure shows a sanitized error and keeps the complete last good catalog, including its commit metadata.
4. After changing scripts in Git, choose Sync again. There is no automatic branch polling. A commit SHA stays pinned; a branch or tag resolves again on each sync.

Edit changes a runtime source's configuration. Published algorithms continue to use the previous configuration until a successful sync. Blank credential fields preserve existing settings; the explicit clear checkboxes remove token or custom CA settings. Stored credential values and file paths are never sent back to the browser. A concurrent configuration edit requires reloading the form, including when a source was deleted and recreated with the same id. An older running sync cannot publish over the newer configuration.

Remove deletes a runtime source and its published Git algorithms, with confirmation. It does not delete the remote repository, database-authored scripts, or previous operation Tasks. A previously shadowed database algorithm may become selectable after the Git source is removed. Already running operations retain the script they selected.

Runtime sources and published catalogs survive restarts in the shared database. All MDMbox instances using that database read the same published Git scripts. There may be at most 20 sources, including the environment source, and two simultaneous sync jobs per instance. Only one job may synchronize a particular source across instances. An interrupted job becomes retryable after its 90-second lease expires; expired or superseded jobs cannot start publishing.

The reserved environment source is configured with MDMBOX_ALGORITHM_GIT_*. It can be synchronized here, but editing or removing its configuration requires changing the deployment environment and restarting. Startup refreshes this source before serving requests; a failed startup refresh does not silently use a stale catalog. Simultaneously starting instances with the same configuration can wait for the same sync. Unsetting the URL removes only this source, not runtime sources. Keep the environment configuration consistent across instances, and make required secret files, CA files, SSH keys, and known_hosts available on every instance that may perform Sync.

Built-in availability remains deployment-controlled: change MDMBOX_BUILT_IN_ALGORITHMS and restart. The page shows the effective allowlist and whether credential files are configured, without disclosing their contents or stored paths. Viewing catalogs and configuration does not contact Git. Only trusted administrators should manage sources: repository code runs with the existing algorithm capabilities, and fetching uses the server's network and filesystem access.

Client-plan merge v1

The original $merge operation merges two FHIR resources by executing a client-provided FHIR transaction Bundle. The client controls exactly what changes are made — MDMbox executes them atomically and adds audit records.

How it works

1. The client identifies a duplicate pair (source and target)
2. The client builds a FHIR transaction Bundle describing the merge (update target, reassign references, delete source)
3. MDMbox validates the request, adds audit resources (Task and Provenance), and executes the Bundle as a single transaction
4. If anything fails, the entire transaction rolls back — including audit records

Request

POST https://<mdmbox-host>/api/fhir/$merge
Content-Type: application/json

{
  "resourceType": "Parameters",
  "parameter": [
    {
      "name": "source",
      "valueReference": { "reference": "Patient/duplicate-123" }
    },
    {
      "name": "target",
      "valueReference": { "reference": "Patient/primary-456" }
    },
    { "name": "preview", "valueBoolean": false },
    {
      "name": "plan",
      "resource": {
        "resourceType": "Bundle",
        "type": "transaction",
        "entry": [
          {
            "request": {
              "method": "PUT",
              "url": "Patient/primary-456",
              "ifMatch": "3"
            },
            "resource": {
              "resourceType": "Patient",
              "id": "primary-456",
              "name": [{ "given": ["John"], "family": "Smith" }],
              "birthDate": "1985-03-20"
            }
          },
          {
            "request": {
              "method": "PUT",
              "url": "Encounter/enc-789",
              "ifMatch": "1"
            },
            "resource": {
              "resourceType": "Encounter",
              "id": "enc-789",
              "subject": { "reference": "Patient/primary-456" }
            }
          },
          {
            "request": { "method": "DELETE", "url": "Patient/duplicate-123" }
          }
        ]
      }
    }
  ]
}

Parameters

Plan Bundle

The plan is a standard FHIR transaction Bundle. It typically contains:

• A PUT entry for the target resource (with updated data)
• PUT entries to reassign references from related resources (Encounters, Observations, etc.)
• A DELETE entry for the source resource

Use ifMatch headers (ETags) for optimistic locking. If a resource was modified between when the client read it and when the merge executes, the entire transaction rolls back with a 422 Unprocessable Entity response (code conflict).

Preview mode

Set preview to true to see the assembled Bundle (including audit resources) without executing it:

{
  "resourceType": "Parameters",
  "parameter": [
    {
      "name": "source",
      "valueReference": { "reference": "Patient/duplicate-123" }
    },
    {
      "name": "target",
      "valueReference": { "reference": "Patient/primary-456" }
    },
    { "name": "preview", "valueBoolean": true },
    {
      "name": "plan",
      "resource": {
        "resourceType": "Bundle",
        "type": "transaction",
        "entry": []
      }
    }
  ]
}

Preview response:

{
  "resourceType": "Parameters",
  "parameter": [
    {
      "name": "outcome",
      "resource": {
        "resourceType": "OperationOutcome",
        "issue": [
          {
            "severity": "information",
            "code": "informational",
            "details": { "text": "Merge plan is valid and ready to execute" }
          }
        ]
      }
    },
    {
      "name": "bundle",
      "resource": {
        "resourceType": "Bundle",
        "type": "transaction",
        "entry": ["... assembled entries including audit resources ..."]
      }
    }
  ]
}

Response

On success, the response is a Parameters resource containing:

{
  "resourceType": "Parameters",
  "parameter": [
    {
      "name": "outcome",
      "resource": {
        "resourceType": "OperationOutcome",
        "issue": [
          {
            "severity": "information",
            "code": "informational",
            "details": {
              "text": "Merge completed: Patient/duplicate-123 -> Patient/primary-456"
            }
          }
        ]
      }
    },
    {
      "name": "input-parameters",
      "resource": {
        "resourceType": "Parameters",
        "parameter": [
          {
            "name": "source",
            "valueReference": { "reference": "Patient/duplicate-123" }
          },
          {
            "name": "target",
            "valueReference": { "reference": "Patient/primary-456" }
          },
          { "name": "preview", "valueBoolean": false }
        ]
      }
    },
    {
      "name": "task",
      "resource": {
        "resourceType": "Task",
        "id": "generated-task-id",
        "status": "completed",
        "code": { "coding": [{ "code": "merge" }] },
        "businessStatus": { "coding": [{ "code": "merged" }] },
        "for": { "reference": "Patient/duplicate-123" },
        "focus": { "reference": "Patient/primary-456" }
      }
    },
    {
      "name": "result",
      "resource": {
        "resourceType": "Patient",
        "id": "primary-456"
      }
    }
  ]
}

Audit trail

Every merge creates two audit resources inside the same transaction:

Task — records the merge event:

• for — the source (merged away)
• focus — the target (survivor)
• businessStatus — merged
• code — merge

Provenance — records what was changed:

• target — all affected resources plus the operation Task
• entity[] — versioned references to all affected resources before the merge
• agent — Device/mdmbox
• activity — merge from http://terminology.hl7.org/CodeSystem/iso-21089-lifecycle

Find the audit record for a Task with GET /Provenance?target=Task/<task-id>. These audit resources enable future unmerge by preserving the pre-merge state of every affected resource. They also power Notifications — downstream systems can subscribe to merge and unmerge events via Topic-Based Subscriptions.

Unmerge

A completed merge can be reversed with $unmerge. The unmerge request points to the original merge Task and supplies a client-built reverse transaction Bundle. MDMbox executes that reverse plan atomically, creates its own audit Task and Provenance, and updates the original merge Task to businessStatus=unmerged.

See Unmerge operation.

Validation

MDMbox validates the merge request before execution:

Structural validation (400 Bad Request):

• Source and target must be different resources
• Plan must be a transaction Bundle with at least one entry
• No duplicate PUT/DELETE URLs in the plan

State validation (422 Unprocessable Entity):

• Both source and target must exist
• Source must not already be merged (no active merge Task)
• Target must not already be a source in another merge (no circular merges)

FHIR transaction validation (4xx OperationOutcome):

• Resources in the plan are validated when the FHIR transaction executes
• If a resource declares meta.profile, the corresponding FHIR package must be installed and the resource must satisfy the profile
• If validation fails, the transaction rolls back, including Task and Provenance audit records

Profiled resources in the merge plan

If a resource written by the merge plan declares meta.profile, install the package that contains the profile in Aidbox before calling $merge. For US Core 6.1.0:

POST https://<aidbox-host>/fhir/$fhir-package-install
Content-Type: application/json

{
  "resourceType": "Parameters",
  "parameter": [{ "name": "package", "valueString": "hl7.fhir.us.core@6.1.0" }]
}

During $merge, every profiled resource in the transaction is validated. If any resource violates its declared profile, MDMbox returns the FHIR validation OperationOutcome and rolls back the complete merge transaction.

Finding related resources

Use the $referencing operation to discover resources that reference a given resource. This is useful when building the merge plan — you need to know which Encounters, Observations, etc. point to the source and need to be reassigned.

See Referencing operation.