Matching models

A matching model defines how MDMbox compares records to find duplicates. It specifies which fields to compare, how to compare them, and what scores to assign.

MDMbox supports two types of models:

• MatchingModel — for online $match queries against individual resources
• BulkMatchingModel — shared by Bulk matching jobs and Continuous matching processes

Both are stored as FHIR resources. Manage MatchingModel through MDMbox REST endpoints, and BulkMatchingModel through the Admin UI or the Aidbox FHIR API.

Concepts

Variables

Variables extract values from FHIR resources using SQL expressions. They are referenced by blocks and features.

The normalization examples use canonical mdm_ names. If your installation only has legacy names, use the corresponding function, such as immutable_unaccent_upper for mdm_unaccent_upper.

{
  "variable": [
    { "name": "dob", "expression": "(#.resource->>'birthDate')" },
    {
      "name": "family",
      "expression": "mdm_unaccent_upper(#.resource->'name'->0->>'family')"
    }
  ]
}

The # prefix is replaced with the table alias at query time (l. for the left record, r. for the right).

public.mdm_unaccent_upper(text) removes accents, converts text to uppercase, and preserves SQL NULL. Wrap it in btrim(...) to ignore surrounding spaces. See SQL functions for all normalization helpers, examples, and handling of empty values.

Blocks

Blocks define how candidate pairs are selected before comparison. Each block is a condition that narrows the search space. Blocks are combined with OR — a pair only needs to match one block.

{
  "block": [
    { "name": "dob", "variable": "dob" },
    { "name": "fn", "variable": "family" }
  ]
}

A block with "variable": "dob" means "candidates must have the same date of birth". For custom logic, use "sql" instead of "variable":

{ "name": "custom", "sql": "l.some_column = r.some_column" }

Features

Features define the comparison logic. Each feature has a list of cases evaluated top to bottom. The first matching case determines the weight (log2 Bayes factor).

{
  "feature": [
    {
      "name": "dob",
      "case": [
        { "expression": "l.#dob = r.#dob", "weight": 10.59 },
        { "expression": "levenshtein(l.#dob, r.#dob) <= 1", "weight": 3.99 },
        { "else": -10.32 }
      ]
    }
  ]
}

The total match score is the sum of all feature weights.

See SQL functions for Jaro–Winkler, edit distance, phonetic codes, and trigram comparisons available to feature expressions.

Thresholds

Thresholds classify match results into grades:

{
  "thresholds": {
    "certain": 25,
    "probable": 16
  }
}

• Score >= certain — match grade certain (high confidence)
• Score >= probable — match grade probable (review recommended)
• Score < probable — match grade possible (returned when using a lower threshold or R6 onlySingleMatch)

MatchingModel

Used by the $match operation for online queries. Works directly against FHIR resource JSONB columns.

Create a model

POST https://<mdmbox-host>/api/models
Content-Type: application/json

{
  "resourceType": "MatchingModel",
  "id": "patient-model",
  "resource": "Patient",
  "thresholds": {
    "certain": 25,
    "probable": 16
  },
  "variable": [
    { "name": "dob", "expression": "(#.resource->>'birthDate')" },
    {
      "name": "given",
      "expression": "mdm_unaccent_upper(#.resource->'name'->0->'given'->>0)"
    },
    {
      "name": "family",
      "expression": "mdm_unaccent_upper(#.resource->'name'->0->>'family')"
    },
    { "name": "gender", "expression": "(#.resource->>'gender')" }
  ],
  "block": [
    { "name": "dob", "variable": "dob" },
    { "name": "fn", "variable": "family" }
  ],
  "feature": [
    {
      "name": "dob",
      "case": [
        { "expression": "l.#dob = r.#dob", "weight": 10.59 },
        { "expression": "levenshtein(l.#dob, r.#dob) <= 1", "weight": 3.99 },
        { "else": -10.32 }
      ]
    },
    {
      "name": "name",
      "case": [
        {
          "expression": "l.#given = r.#given AND l.#family = r.#family",
          "weight": 13.34
        },
        {
          "expression": "l.#given = r.#family AND l.#family = r.#given",
          "weight": 13.1
        },
        { "expression": "l.#family = r.#family", "weight": 2.4 },
        { "else": -12.37 }
      ]
    },
    {
      "name": "sex",
      "case": [
        { "expression": "l.#gender = r.#gender", "weight": 1.85 },
        { "else": -4.84 }
      ]
    }
  ]
}

Model CRUD

BulkMatchingModel

Defines columns, blocks, features, and thresholds for both Bulk matching jobs and Continuous matching processes.

Create and update these resources through Aidbox's FHIR API, or use the MDMbox Admin UI:

PUT https://<aidbox-host>/fhir/BulkMatchingModel/patient-bulk
Content-Type: application/fhir+json

Key differences from MatchingModel:

• column replaces variable — each column has a source SQL expression and a PostgreSQL type
• tableName specifies the flat table to create
• index defines indexes to create on the flat table after population
• In block entries, variable refers to a column name (not a variable — the field name is the same in both model types)
• In feature expressions, reference column names directly (e.g., l.dob instead of l.#dob)

{
  "resourceType": "BulkMatchingModel",
  "id": "patient-bulk",
  "resource": "Patient",
  "tableName": "mdm.patient_flat",
  "thresholds": {
    "certain": 23,
    "probable": 16
  },
  "column": [
    { "name": "dob", "type": "text", "source": "resource->>'birthDate'" },
    {
      "name": "family",
      "type": "text",
      "source": "mdm_unaccent_upper(resource->'name'->0->>'family')"
    }
  ],
  "index": [
    { "name": "idx_dob", "column": "dob", "type": "btree" },
    { "name": "idx_family", "column": "family", "type": "btree" }
  ],
  "block": [
    { "name": "dob", "variable": "dob" },
    { "name": "fn", "variable": "family" }
  ],
  "feature": [
    {
      "name": "dob",
      "case": [
        { "expression": "l.dob = r.dob", "weight": 10.59 },
        { "else": -10.32 }
      ]
    },
    {
      "name": "family",
      "case": [
        { "expression": "l.family = r.family", "weight": 13.34 },
        { "else": -12.37 }
      ]
    }
  ]
}

Admin UI

In the Admin UI at /mdmbox/admin on the Aidbox address, select a model in Models to edit its JSON, or choose New Model to create one. A bulk badge identifies BulkMatchingModel resources. Check the resource type before saving or deleting: the two model types can share an ID.

Tuning

The example weights above are for demonstration. Production deployment requires calibrating weights based on your data. Key considerations:

• Bayes factors reflect the discriminative power of each comparison. Higher absolute values mean the feature is more decisive.
• Thresholds control the tradeoff between precision and recall. Lower thresholds catch more duplicates but increase false positives.
• Blocks determine which pairs are compared. Too broad blocks slow down matching; too narrow blocks miss duplicates.
• Database indexes are critical for performance with large datasets. Create indexes on columns used in blocks.

Next steps