This guide explains how to evaluate a healthcare Master Data Management (MDM) or enterprise Master Patient Index (MPI) platform: which identity decisions it must support, how to test matching quality on your own data, and how to compare adaptability, integration, and long-term cost.
Get the full guide as a PDF
The complete evaluation framework with the vendor scorecard and proof-of-capability plan — to read offline or share with your team.
Executive brief
Healthcare organizations rarely have one authoritative record for every patient, practitioner, organization, or location. The same entity is created independently in EHRs, laboratory systems, claims platforms, registries, portals, acquired businesses, and analytics environments. Identifiers differ by source. Names, addresses, and affiliations change. Fields may be incomplete, outdated, or simply wrong. The result is a set of records that cannot be used safely until the organization can determine which ones refer to the same real-world entity.
Master Data Management, or MDM, provides the technology and operating controls used to resolve those identities and maintain trusted master data across healthcare systems. It covers the complete decision lifecycle: detecting possible duplicates, deciding whether records represent the same entity, applying the correct resolution action, preserving provenance, correcting mistakes, and publishing the result to other systems.
A Master Patient Index, or MPI, is the patient-identity part of healthcare MDM. An enterprise MPI performs this work across multiple systems, facilities, or organizations. Healthcare MDM can apply the same foundations to practitioners, organizations, locations, members, related persons, and the relationships among them.
Identity quality affects far more than duplicate counts. A missed match can fragment a clinical history and hide relevant information. A false match can combine data from different people. Duplicate provider and organization records can distort attribution, directories, referrals, network reporting, and claims. Analytics systems often continue to run when identity is wrong; they return plausible results based on inflated populations, fragmented timelines, and inconsistent relationships.
The appropriate solution depends on the organization's operating model. One health system may merge records inside a centrally governed EHR. A health information network may preserve every participant's source record and maintain links between them. A healthcare platform may need different matching models, review processes, and interfaces for each customer. Survivorship rules, governance boundaries, automation thresholds, integrations, and correction procedures can all vary.
This variation changes how a product should be evaluated. A general feature list does not show whether the platform can implement the required identity policy. A published accuracy percentage does not show how the product will perform on the buyer's data. A polished review screen does not show whether the workflow, evidence, permissions, and corrective actions fit production operations.
Five questions every evaluation must answer
- Can the platform make and explain reliable identity decisions? Matching quality must be measured on representative data, and reviewers must be able to see the evidence behind each result.
- Can it apply the required resolution policy? The product may need merge, link, golden-view, survivorship, non-duplicate, unmerge, and unlink behavior that differs by entity, source, tenant, or confidence level.
- Can it adapt to the organization? Workflows, interfaces, analytics, mappings, and decision boundaries often require customization and continue to change after go-live.
- Can it fit the healthcare architecture? Identity decisions must move through FHIR APIs, events, batch feeds, EHR integrations, analytics platforms, and governance boundaries without losing provenance.
- Can it operate predictably at scale? Performance, support, implementation effort, routine changes, and commercial terms determine the actual cost of ownership.
This guide explains the choices behind those questions, converts them into product requirements, and provides a concise method for comparing vendors. Health Samurai publishes the guide and develops MDMbox. That relationship is stated directly. The evaluation method should be applied to MDMbox with the same buyer-owned data, workflows, and acceptance criteria used for every other product.
Part I: Understand the core MDM choices
1. Identity resolution in healthcare
Matching answers one question: how likely is it that two records represent the same real-world entity? Resolution answers a different question: what should happen after that relationship is identified? A buyer needs to understand both before defining requirements.
Merge
A merge consolidates duplicate records into a surviving record. The operation may copy or recompute field values, redirect references from related resources, retire or delete the source, and notify downstream systems. Merge is most appropriate when one authority controls the records and can define the surviving state.
The difficult part is policy. A healthcare organization must decide which identifiers remain active, which demographic values survive, what happens to conflicting clinical or administrative information, and which related resources may be moved automatically. FHIR R5 defines a Patient merge operation, but the specification leaves many implementation choices to the server because legitimate policies differ.
A production merge must also be reversible. An incorrect merge may be discovered after new encounters, results, claims, or other resources have been attached to the survivor. The platform needs enough history to identify the original change, distinguish later activity, and execute a safe unmerge or route conflicts for review.
Link
A link records that two or more records belong to the same entity while keeping the original records separate. References remain attached to their source records, and each source can continue to manage its own data. Link is often safer across EHRs, organizations, legal entities, tenants, or data-sharing participants that do not share merge authority.
Links should be explicit, queryable, auditable, and reversible. A system that can create an identity cluster but cannot unlink one record from it has merely moved the correction problem into another data structure.
Golden view
A golden view presents a governed representation of the entity while retaining the contributing source records. It may be stored as a canonical record or assembled dynamically from linked records. The organization must define the provenance of every selected value and decide whether the golden view is used for analytics, operational workflows, downstream publishing, or all three.
Non-duplicate decisions
Similar records do not always belong to the same person or organization. Twins, relatives, providers with common names, shared household contact details, and reused facility addresses create legitimate near-matches. Once a reviewer confirms that a pair represents different entities, the platform should preserve that decision and prevent the same pair from repeatedly returning as an unresolved candidate.
Manual and automated resolution
Automation should reflect the cost of each error. A high-confidence pair may be merged or linked automatically. An ambiguous pair may require a data steward. A weak pair may require no action. The boundaries depend on the population, the available data, and the consequences of a false action.
The fine-tuning work performed for a large diagnostic laboratory network illustrates the point. False merges carried a particularly high privacy risk because they could disclose laboratory results to the wrong person. The initial operating model therefore favored manual review over aggressive automatic merging. Shared addresses, facility addresses, and abundant contact data also required the model to avoid treating repeated weak signals as independent proof.
| Action | Record behavior | Appropriate when | Required safeguard |
|---|---|---|---|
| Merge | One record survives and related data may be reassigned | One authority controls the records and survivorship | Preview, atomic execution, provenance, and unmerge |
| Link | Source records stay active and an identity relationship is maintained | Records cross systems, tenants, or governance boundaries | Explicit cluster membership, evidence, and unlink |
| Golden view | A canonical view is created from retained sources | Consumers need one governed representation without destructive consolidation | Field-level lineage and refresh rules |
| Keep separate | Similar records remain distinct | Evidence is insufficient or the pair is confirmed as different | Persisted non-duplicate decision |
Hierarchical identity
Enterprise networks often need more than one resolution action. Records can be merged within a tenant that owns them, while the tenant-level masters are linked across the network. This preserves local authority and still creates an enterprise identity view. The same pattern can serve health information exchanges, clinically integrated networks, acquisitions, and healthcare platforms with customer-specific governance.
Figure 1. A hierarchical merge-within / link-across operating model
The correct topology is a business and governance decision. A product should support it without forcing all records into one global merge policy.
2. Matching approaches
Healthcare identity matching is commonly implemented with deterministic rules, probabilistic models, external referential data, or a combination of these approaches. The buyer does not need to master the mathematics, but should understand what each approach assumes and where it fails.
| Approach | Useful for | Main limitations |
|---|---|---|
| Deterministic or rule-based | Exact identifiers, simple populations, explicit constraints, and high-certainty rules | Becomes brittle as exceptions grow and is highly sensitive to missing or inconsistent data |
| Probabilistic | Incomplete, noisy, conflicting, or multi-source healthcare data | Requires calibration, representative data, and expert review of errors and thresholds |
| Referential | Enriching local evidence with a vendor-maintained external identity dataset | Coverage, provenance, privacy, permitted use, cost, latency, and dependency on the reference asset |
| Hybrid | Combining strict constraints, probabilistic evidence, and optional enrichment by use case | Requires clear precedence and transparent evidence so the result remains governable |
Deterministic matching
Deterministic matching applies explicit rules such as exact agreement on a trusted identifier or agreement on a defined combination of name and date of birth. In this guide, deterministic and rule-based matching are treated as one category. The terms are not perfectly synonymous in every implementation, but separating them does not improve a buyer's decision.
Rules are easy to understand when data is stable and the number of cases is small. They become difficult to maintain when the organization adds sources, populations, name formats, transliteration, historical values, or contradictory identifiers. A growing rule set can also hide the trade-off between missed duplicates and false matches.
Probabilistic matching
Probabilistic matching evaluates the combined evidence in a pair of records. Agreement on a rare, reliable identifier can contribute more evidence than agreement on a common surname. A missing value can remain neutral instead of behaving like a mismatch. A transposed date or misspelled name can contribute partial evidence rather than failing an exact rule.
For complex healthcare environments, a probabilistic method such as Fellegi–Sunter is usually the strongest default. It handles imperfect data, produces a score that can support several decision zones, and allows the organization to measure precision and recall at different thresholds. Health Samurai's tuning work also shows why the model still needs local calibration. Shared addresses, placeholder values, family contact details, and source-specific errors change how much each signal should be trusted.
The algorithm label alone proves little. The buyer should focus on the resulting quality, the evidence available for each decision, and the effort required to adapt the model when data changes.
Referential matching
Referential services compare customer records with an external identity graph or demographic dataset assembled from commercial, public, or other sources. This can add evidence when local records are sparse or stale. It can also introduce material constraints.
The buyer should determine where the reference data came from, which populations and geographies it covers, how often it is updated, what evidence is returned, whether submitted records are retained, and whether the intended use is permitted under applicable contracts and law. In the United States, authorization for a business associate to operate a patient matching service depends on the governing agreement and permitted uses of protected health information. Other jurisdictions may impose different requirements. Legal and privacy review should therefore cover the specific data source and workflow rather than assume that access to a reference database establishes a lawful use.
Referential matching may also add per-query cost, network latency, coverage bias, and long-term dependency on the vendor's asset. These trade-offs can be justified for some populations. They should not be treated as automatically superior to a well-calibrated model operating on the organization's own evidence.
Hybrid matching
A hybrid design can combine probabilistic scoring with deterministic safeguards and, where justified, referential enrichment. For example, a model may use probabilistic evidence for candidate evaluation while a strict rule prevents automatic merge when two trusted identifiers conflict. The important requirement is that the product can explain which evidence and policy produced the final decision.
3. Explainability and decision evidence
An explainable system shows more than a total score. A reviewer should be able to inspect the source values, field-level agreements and disagreements, the strength of each contribution, the applicable threshold, and the model or rule version used for the decision.
This evidence serves several users:
- A data steward needs to decide whether the records belong together and what action is safe.
- A model owner needs to understand false positives, missed matches, and unexpected patterns.
- An auditor needs to reconstruct which configuration and user action produced the result.
- An integration team needs to explain why a downstream merge, link, or suppression event occurred.
Explainability must survive change. If the model is updated next month, the organization should still be able to reproduce or interpret a decision made today. That requires versioned configuration, preserved source values, structured audit data, and a link between the match evidence and the resolution action.
The interface matters as much as the underlying model. Field-level evidence that exists only in an internal database is not useful to a steward. A product should expose it through an appropriate review interface or API so that the organization can embed it in its own workflow.
The same requirement applies to automated decisions. A high-confidence action should retain the evidence, model version, threshold, initiator, affected resources, and outcome. Automation without a reconstructable decision record creates operational and audit risk.
Part II: Define the required operating model
4. Define the target MDM operating model
The organization can define its requirements once it understands the available identity actions and matching approaches. The target operating model should be short enough to use in a vendor workshop and specific enough to prevent every supplier from demonstrating a different problem.
| Decision | What to define | Example output |
|---|---|---|
| Scope | Entities, relationships, sources, populations, and first implementation phase | Patient and Practitioner in phase one, Organization and Location in phase two |
| Identity | What counts as the same entity and which evidence is trusted | Local MRN is unique only within its assigning organization |
| Resolution | Where to merge, link, create a golden view, or keep records separate | Merge within a hospital, link across network participants |
| Survivorship | Which values survive and how conflicts are handled | Verified identifiers outrank unverified feeds; historical values remain traceable |
| Automation | Which decisions are automatic and which require review | Automatic link above the approved threshold; merge requires steward approval |
| Governance | Ownership, permissions, audit, retention, and tenant boundaries | Tenant stewards manage local merges; enterprise users manage links only |
| Operation | Integrations, interfaces, analytics, volumes, latency, and support | Real-time registration match plus nightly bulk reconciliation and outbound events |
Scope and identity
Start with the entities that create real operational or analytical risk. Patient identity may be the first use case, but provider, organization, and location data often determine attribution, referral routing, network reporting, and claims. Each entity can require its own fields, model, thresholds, and stewardship process.
Define identity in the context of each source. The same identifier value can be reliable within one assigning organization and meaningless without its system in another. A provider's current name or address may be less useful than a stable NPI, yet the NPI registry itself can contain duplicate provider records. The requirement should describe which evidence is trusted, not merely list available fields.
Resolution and survivorship
For each use case, state the required action and authority. Define whether the system should merge, link, produce a golden view, or preserve a non-duplicate decision. Then define survivorship at the field level: source priority, recency, verification status, completeness, effective dates, and handling of conflicting values.
Related resources belong in the same decision. A patient merge may affect Encounters, Observations, Claims, Tasks, subscriptions, cached identifiers, and downstream calculations. A provider merge may need to preserve PractitionerRole history and point-in-time affiliation. The operating model should say which related data moves automatically and which requires a separate workflow.
Automation and stewardship
Define the acceptable risk and expected workload. How costly is a false merge compared with a missed duplicate? How many cases can be reviewed each week? Who owns a decision? Can the review occur in a dedicated interface, or must it be embedded in registration, health information management, provider operations, or a customer application?
The answers determine thresholds, queues, permissions, evidence, and interface requirements. They also determine whether a product that depends on a fixed data steward screen can fit the organization.
Governance and integration
Specify the boundaries that identity decisions must respect. Tenants, facilities, legal entities, source systems, and data-sharing participants may require different models and different permitted actions. Define which users can inspect evidence, approve changes, configure models, alter workflows, and reverse decisions.
List the systems that create, consume, or react to identity decisions. Include real-time and batch paths, required formats, downstream event handling, reconciliation, and failure recovery. An MDM that produces a correct decision but cannot propagate or correct it in connected systems does not solve the full problem.
The output should be a concise set of policies, production scenarios, and acceptance conditions. Detailed RFP questions can be derived later. The operating model is the stable reference against which every product is evaluated.
5. Translate the operating model into product requirements
Flexible identity and resolution model
The product must support the required healthcare entities and relationships without treating every new domain as a separate implementation. At minimum, buyers should determine whether patient, practitioner, organization, location, member, and related-person use cases can use the same identity foundation while retaining separate models and policies.
Resolution must be configurable at the action level. The buyer may need merge for one domain, link for another, and a hybrid hierarchy for a third. Survivorship, reference handling, source disposition, audit, and downstream effects should follow the organization's policy. Research and production experience across MPI implementations show that these choices vary materially between organizations; a single hard-coded merge algorithm cannot represent them all.
Correction is a product capability, not a support procedure. Require preview, atomic execution where appropriate, unmerge, unlink, provenance, and persisted non-duplicate decisions. Ask what happens when data changes after the original action and whether a reversal can preserve legitimate later activity.
Adaptable workflows, interfaces, and analytics
Healthcare MDM workflows are rarely identical. One organization may review possible matches in a central HIM team. Another may route provider identity conflicts to network operations. A platform company may expose review actions inside its own customer-facing application. Some deployments automate most decisions; others require approval for every merge.
The product should therefore expose the underlying actions and evidence through APIs as well as provide useful administrative interfaces. Buyers should distinguish among three levels:
- A built-in interface that works for the standard workflow.
- A configurable interface that can change fields, evidence, actions, roles, and queue behavior.
- An embeddable API or component model that allows the organization to build a workflow-specific experience.
Dashboards must also match the operating model. Useful measures include pending and completed reviews, queue age, decision yield, duplicate prevalence, automated resolution rate, latency, throughput, failures, model versions, and audit activity. If the product does not provide the required view, the buyer should be able to export structured operational data to an external analytics platform.
Interface and dashboard requirements will change after go-live. Evaluation should therefore include the cost and lead time for modifying them, not only the quality of the initial screen.
Architecture and integration
FHIR support should be tested as data behavior rather than accepted as a label. Determine whether FHIR resources are the primary identity model or whether the platform translates into a proprietary schema. Create a profiled resource with multiple identifiers and extensions, run a match, execute a resolution action, and retrieve the source, result, provenance, and audit data. The test shows what survives the round trip.
Integration requirements should cover FHIR APIs, batch ingestion, events, webhooks or subscriptions, mappings, related-resource handling, downstream reconciliation, and error recovery. Many healthcare environments also require HL7v2, C-CDA, flat files, SFTP, or custom feeds. Clarify whether the MDM product supports these formats directly or whether a separate integration layer is part of the proposed architecture.
Mappings and outbound schemas change. A sustainable design keeps them versioned, testable, and deployable without rebuilding the MDM. The buyer should know which changes its own team can make and which require vendor engineering.
Enterprise readiness
Enterprise readiness covers more than record count. The platform must support the expected mix of online matching, bulk reconciliation, continuous processing, concurrency, and noisy data. Performance evidence should document the dataset, candidate volume, model, infrastructure, worker configuration, timing boundaries, and software version.
Governance requirements include tenant isolation, tenant-specific policies, cross-tenant identity where permitted, role-based access, structured audit, authentication, encryption, retention, backup, recovery, and data residency. Deployment responsibilities must be explicit: software, database, infrastructure, upgrades, monitoring, availability, and disaster recovery may belong to different parties.
The organization should also be able to export identity relationships, configuration, decisions, and audit history. Exit is part of enterprise readiness because the identity graph becomes foundational infrastructure once downstream systems depend on it.
Part III: Evaluate vendor fit
6. Classify product fit and adaptability
A vendor can answer "yes" to a requirement in several materially different ways. The capability may be available in the standard product, exposed as configuration, achievable through custom development, dependent on another component, planned for a future release, or unsupported. Treating all of these answers as equivalent hides cost and delivery risk.
| Status | Meaning | Evidence to request |
|---|---|---|
| Native | Available in the supported product without project-specific development | Documentation and live demonstration |
| Configurable | Implemented through supported settings, models, rules, or templates | Buyer-led configuration change and repeatable deployment path |
| Customizable | Requires code, a custom UI, an integration, or a project-specific extension | Scope, owner, estimate, upgrade model, tests, and support terms |
| External component | Depends on another product or service in the proposed architecture | Component responsibility, data flow, license, SLA, and failure behavior |
| Roadmap | Planned but not available in the evaluated version | Dated contractual commitment if the capability is essential |
| Unsupported | Cannot be delivered within acceptable constraints | Record as a gap rather than converting it into a low-confidence promise |
Specific requirements can eliminate a vendor
Healthcare organizations often discover during evaluation that a core requirement is outside a product's operating model. A vendor may support merge but not link. It may allow one global survivorship policy but not tenant-specific policies. Its review UI may expose the wrong evidence or actions. Its integration layer may publish a merged identifier but not a reversible relationship or correction event.
These gaps are not secondary details. If the organization's policy requires local merge and cross-tenant link, a product that only creates one global golden record cannot be fixed through scoring weights. If data stewards must work inside an existing registration application, a vendor-only queue may be operationally unusable even when the matching engine is strong.
Identify these constraints before the weighted scorecard. A vendor that fails a required governance, safety, deployment, or workflow condition should not recover by accumulating points for unrelated features.
Test the ability to change
The initial implementation is only one version of the operating model. Sources change their file structures. New populations arrive. A business adds a tenant. Review responsibilities move between teams. A new regulation changes retention or access. Users learn that the first dashboard or screen does not support the work they actually perform.
Evaluation should therefore include a controlled change after the baseline demonstration. Ask the vendor to alter one material requirement, such as:
- change a match or review threshold for one population without affecting another;
- add a source-specific survivorship condition;
- change a merge flow into a link flow at a governance boundary;
- expose another piece of evidence and a new corrective action in the review interface;
- modify an inbound mapping or outbound event payload;
- add a dashboard measure or export needed by operations.
Record who performs the change, how long it takes, whether it requires a release, how it is tested, and how it moves from development to production. This exercise reveals the practical difference between a configurable platform and a professional-services dependency.
Evaluate workflow and UI separately
A good interface demonstrates the current workflow. It does not prove that the workflow can change. Determine whether the UI is fixed, configurable, or replaceable; whether every UI action is also available through an API; and whether the buyer can create different experiences for stewards, administrators, analysts, and application users.
The same applies to operational analytics. A dashboard should answer the questions the organization needs to manage quality and workload. If the built-in dashboard cannot, structured exports and documented data models become essential. Screenshots are weak evidence; the vendor should configure the required view or show how the buyer would build it.
7. Run a proof of capability
A scripted demonstration shows the path the vendor chose. A proof of capability tests the path the buyer needs. It should use representative data, agreed scenarios, and acceptance conditions defined before results are known.
Representative data
The sample should contain confirmed matches and non-matches from every important source and population. Include the cases that create real errors:
- missing and outdated demographics;
- transposed dates and digits;
- aliases, name changes, transliteration, and reordered names;
- shared phones, household addresses, facility addresses, and common provider names;
- placeholder or default values;
- conflicting trusted identifiers;
- multiple identifier systems;
- twins, relatives, and records previously resolved by stewards;
- each tenant, source type, geography, or population that may require different behavior.
Use a labeled validation set that is not consumed entirely during tuning. If the vendor configures the model against every evaluation pair, the final metric measures adaptation to the test file rather than performance on unseen production data.
Measure quality and workload
An overall accuracy figure can be dominated by easy non-matches. Require enough information to distinguish the error types and the operational burden.
| Measure | Decision it supports |
|---|---|
| Precision | How many records classified as matches are truly matches |
| Recall | How many known duplicates the process finds |
| False matches | Risk of joining different people or entities |
| Missed matches | Residual fragmentation after matching |
| Review volume | Number of cases the selected thresholds send to stewards |
| Review yield | Share of reviewed candidates confirmed as matches |
| Latency | Suitability for real-time registration or API workflows |
| Throughput | Time and infrastructure required for bulk or continuous processing |
Report results by meaningful segment, not only in aggregate. A model can perform well overall while failing on one source, minority population, tenant, or data-quality pattern. The chosen thresholds should also reflect the relative cost of false matches, missed matches, and manual review.
Required scenarios
The proof should cover six connected scenarios:
- Explain a decision. Match representative records and show the source values, field-level evidence, score, threshold, model version, and classification.
- Apply the resolution policy. Execute the required merge, link, golden-view, survivorship, and non-duplicate behavior rather than the vendor's default example.
- Run the human workflow. Route an ambiguous case to the intended user, expose the required evidence and actions, complete the decision, and inspect the audit record.
- Exercise the integration. Process one real inbound path and one outbound decision or correction event, including mappings and failure handling.
- Correct a mistake. Unmerge or unlink a completed action, preserve provenance, and show how related resources and downstream systems are handled.
- Change a requirement. Modify one model, workflow, interface, mapping, or dashboard requirement and repeat the affected test.
Test performance separately with a production-like volume and data profile. Require a benchmark statement that names the dataset, duplicate prevalence, model, indexes or candidate strategy, infrastructure, concurrency, workers, included processing stages, software version, and measurement boundaries.
Preserve evidence
For every requirement, record whether the result came from the standard product, supported configuration, custom development, an external component, or a roadmap commitment. Save the configuration, result exports, screenshots where useful, benchmark conditions, and acceptance decision.
The final contract should reflect essential conclusions from the proof. A successful prototype does not guarantee that the same customization is supported through upgrades, included in the license, or covered by an SLA.
Part IV: Compare the commercial reality
8. Cost, dependency, and growth
License price is only one component of healthcare MDM cost. The useful comparison is a three-year scenario based on the target operating model and expected growth.
Three-year cost includes software licenses, implementation, data profiling, model configuration, integration, mappings, custom workflows and interfaces, infrastructure, environments, stewardship labor, support, upgrades, change requests, data growth, and exit.
Ask what changes each component of the price:
- records, entities, API calls, match transactions, or bulk jobs;
- source systems, message types, mappings, and connectors;
- domains, matching models, tenants, facilities, or legal entities;
- production and non-production environments;
- steward, administrator, or API users;
- storage, retention, audit volume, availability, and support tier;
- custom development, optimization, and future migration.
Vendor dependency is a separate cost driver. A low license can become expensive if every threshold change, mapping update, workflow adjustment, or UI modification requires a paid services engagement. Estimate the number of routine changes expected each year, then price and time them using the vendor's actual delivery model.
Flat-fee licensing can make the software component more predictable as data volume and adoption grow. It does not remove implementation, infrastructure, integration, tuning, or operating costs. The proposal should separate those costs so the buyer can see which expenses scale with usage and which scale with complexity.
Exit should be priced before purchase. Confirm that the organization can retrieve source relationships, clusters, golden views, matching configurations, non-duplicate decisions, merge and link history, audit records, and the data needed to reconstruct downstream identity state.
9. Common buying traps
Selecting the highest accuracy claim
Vendors may use different datasets, definitions, thresholds, and denominators. Compare precision, recall, false actions, and review workload on the same representative sample.
Treating matching as the entire product
A strong score is not enough. Resolution, survivorship, correction, stewardship, integration, audit, and propagation determine whether the decision can be used safely.
Testing only clean duplicates
Easy examples conceal the behavior that matters. Include shared addresses, conflicting identifiers, missing data, transliteration, common names, and known production failures.
Assuming FHIR support means FHIR-native behavior
A FHIR endpoint may translate into a proprietary schema. Test profiles, extensions, assigning systems, history, relationships, provenance, and export through a complete round trip.
Accepting a fixed operating model
A product may have excellent defaults and still be wrong for the organization's merge policy, governance boundary, review process, or UI. Demonstrate the buyer's workflow and one future change.
Underpricing stewardship
A configuration that maximizes recall may flood the review queue. Estimate weekly volume, review yield, queue age, and steward hours at the proposed thresholds.
Confusing a prototype with the product
Identify custom code, external components, roadmap items, unsupported extensions, and upgrade responsibilities. Require commercial and support terms for anything essential.
Ignoring correction and exit
Merge, link, and golden-view decisions become dependencies for other systems. Test reversal and export before those dependencies exist.
10. Decision scorecard
Set weights before vendors present results. Score each criterion from zero to five, multiply by its weight, and divide by five. Use a knockout condition for any requirement that cannot be compromised.
| Criterion | Weight | Minimum evidence |
|---|---|---|
| Matching quality and explainability | 20 | Buyer-data results and field-level evidence |
| Resolution and survivorship | 20 | Required merge, link, golden-view, correction, and non-duplicate scenarios |
| Workflow, UI, and analytics adaptability | 15 | Production workflow plus one buyer-requested change |
| Architecture and integration | 15 | FHIR round trip, real integration, event handling, and export |
| Governance and enterprise readiness | 10 | Tenant policy, RBAC, audit, deployment, and recovery evidence |
| Performance and reliability | 10 | Production-like online and bulk benchmark statement |
| Three-year cost and vendor dependency | 10 | Priced growth, routine changes, services, and exit |
| Total | 100 |
Acceptable evidence includes product documentation, a live demonstration, a buyer-data proof, a customer reference, a security artifact, or a contractual commitment. A slide, verbal assurance, or uncommitted roadmap item should not receive the same score as a verified capability.
Suggested knockout conditions include inability to explain a decision, inability to reverse an incorrect action, failure to support a required governance boundary, unsupported deployment or PHI controls, inability to export identity state, and refusal to test representative buyer data.
Part V: Apply the framework to MDMbox
11. MDMbox against the evaluation criteria
MDMbox is Health Samurai's FHIR-native healthcare MDM and enterprise MPI product. Current product documentation describes a service deployed with Aidbox against shared FHIR data in PostgreSQL. Aidbox provides FHIR storage and APIs; MDMbox provides probabilistic matching, bulk and continuous matching, merge and unmerge, link and unlink, non-duplicate decisions, audit, and administrative interfaces.
The distinction matters during evaluation. MDMbox is the identity engine. Broader ingestion of HL7v2, C-CDA, files, or source-specific formats can be implemented through the surrounding Health Samurai stack, including Interbox and Aidbox, or through the buyer's existing integration layer. A proposal should identify which component owns every mapping, queue, transformation, and SLA.
| Buyer requirement | MDMbox approach | What to verify |
|---|---|---|
| Healthcare entity scope | Matching models can target Patient, Practitioner, Organization, Location, and other supported FHIR resource types | Configure and match the buyer's required non-patient resource |
| Probabilistic matching | Configurable Fellegi–Sunter models for online, bulk, and continuous processes | Run the buyer's labeled sample and inspect segment-level metrics |
| Explainability | Feature-level weights, scores, thresholds, model identifiers, and structured audit data | Reproduce one difficult decision and retain its evidence |
| Merge and survivorship | Client-provided transaction plan or supported server-managed algorithm | Implement the buyer's survivorship and related-resource policy |
| Link and golden view | Audited Linkage-based clusters can retain source records and an optional canonical view | Link, inspect, and unlink without changing source records |
| Safe correction | Unmerge, unlink, transaction rollback, provenance, and audit lifecycle | Reverse a decision after additional data has arrived |
| Non-duplicate decisions | A structured Task records that two entities are different and affects later operations | Confirm the pair is suppressed or labeled in later matching output |
| Workflow adaptability | API-first operations plus administrative and example steward interfaces | Demonstrate the intended production workflow and UI ownership |
| FHIR integration | FHIR resources, operations, transaction Bundles, Task, Provenance, AuditEvent, and subscriptions | Run a full input, decision, event, correction, and export path |
| Enterprise topology | Enterprise configuration can merge within governance boundaries and link across them | Execute the buyer's tenant hierarchy and permission model |
Healthcare identity foundation
MDMbox operates on FHIR resources rather than limiting identity resolution to a patient-specific proprietary record. The same matching and resolution foundations can be configured for patients, practitioners, organizations, locations, and other FHIR resource types. Relationships remain explicit FHIR references, which is important for provider affiliation, organization hierarchy, provenance, and related-resource correction.
In a broader healthcare MDM implementation, heterogeneous feeds can be normalized to FHIR, mastered by MDMbox, and published to analytics systems. Patient, Practitioner, Organization, and Location models can remain independently governed. This pattern is relevant to multi-domain programs, although the complete architecture may include Aidbox, Interbox, or the buyer's existing integration components rather than MDMbox alone.
Matching and explainability
MDMbox matching models define the resource type, variables or extracted columns, candidate rules, comparison cases, feature weights, and classification thresholds. Online MatchingModel resources and BulkMatchingModel resources are managed through APIs and the Admin UI. Bulk models prepare typed data and indexes for batch performance; continuous matching can retain a process that matches new records after the initial dataset.
The model produces decomposable evidence rather than only a black-box label. This allows an organization to inspect why a pair crossed a threshold and to tune the balance among false matches, missed duplicates, and manual review.
The Health Samurai tuning guide provides a useful example of this process. In a simplified labeled sample, raising the threshold and increasing the penalty for conflicting phone numbers reduced false positives from 2,374 to 37 while recall changed from 0.997 to 0.992. The example is not a general product benchmark. It demonstrates why thresholds and weights must be calibrated against the client's data, risk, and review capacity.
MDMbox is strongest where the buyer needs this transparency and control. The buyer should still require a representative validation set, an agreed error analysis, and a repeatable promotion process for model changes.
Resolution and correction
MDMbox supports two merge patterns. A client can submit a FHIR transaction Bundle that states exactly which resources to update, create, or delete. MDMbox validates and executes the plan atomically while recording Task, Provenance, and AuditEvent resources. Current releases also support server-managed merge and unmerge algorithms, including custom JavaScript algorithms stored in the database or synchronized from Git. The executed algorithm and version are recorded for audit.
This design separates policy from safety. The client or configured algorithm determines survivorship and reference handling. The server enforces transaction boundaries, optimistic locking, allowed mutations, audit assembly, and rollback. Preview can show the proposed plan before execution.
Link and unlink provide a non-destructive alternative. The client builds or updates a profiled Linkage resource, may include a canonical view, and retains the original records. MDMbox validates cluster membership, records the operation, and executes it atomically. A structured mark-not-a-match operation records that a reviewed pair represents different entities and prevents an incompatible merge or link.
The proof of capability should test the buyer's precise correction policy. Unmerge behavior is difficult when resources have changed since the original merge, and no product can remove that underlying complexity. MDMbox exposes restore and stricter conflict-aware patterns; the buyer must decide which policy is safe for its workflows.
Workflow, interface, and analytics adaptability
MDMbox provides an Admin UI for model management, matching jobs, continuous processes, and merge or unmerge algorithms. Health Samurai also provides a Data Steward UI example and API coverage for the underlying identity actions. This supports three implementation choices: use the supplied experience where it fits, customize an implementation, or embed the operations and evidence into an existing application.
The distinction between available product UI and project-specific workflow should remain explicit. Complex queue assignment, escalation, customer-specific dashboards, or role-specific screens may require configuration or custom development. In a typical delivery model, business users perform review and correction through a UI, while technical users own models, mappings, source onboarding, and workflow configuration. Project-specific interfaces and workflows should be scoped as implementation work.
Operational data is available through structured FHIR resources, matching exports, job status, and audit records. Buyers that require a specific dashboard should include it in the proof and classify it as native, configurable, or custom.
Architecture, integration, and governance
Current MDMbox documentation places MDMbox and Aidbox against the same PostgreSQL-backed FHIR data. This avoids maintaining a separate synchronized identity store inside the proposed stack. FHIR operations and resources carry the match request, transaction plan, Linkage, Task, Provenance, and AuditEvent data.
Notifications can use Aidbox Topic-Based Subscriptions to deliver merge and unmerge events to webhooks, Kafka, GCP Pub/Sub, and other destinations. Non-FHIR source conversion, durable ingestion queues, custom mappings, and outbound analytics feeds belong to the broader implementation architecture and can be delivered through Interbox, Aidbox, or existing buyer infrastructure.
Enterprise deployments can use separate policies inside tenant boundaries and link tenant-level masters across an enterprise layer. This capability belongs to the Enterprise edition because many single-organization deployments do not need hierarchical identity. Buyers should test the exact tenant isolation, permissions, model selection, merge authority, cross-tenant link, and tenant-exit behavior required by their architecture.
Authentication, role-based access, encryption, network controls, infrastructure, backup, recovery, and data residency span MDMbox, Aidbox, PostgreSQL, the hosting environment, and the implementation. The component-level responsibility matrix should be part of the proposal. A generic compliance statement is not a substitute for that review.
Performance and operating evidence
Health Samurai publishes three scale indicators for MDMbox: match operations measured in milliseconds, a deployment scale exceeding 100 million records, and a bulk example of approximately 20 minutes for five million patient records. These are vendor-reported indicators, not guarantees for another model or dataset.
A buyer should reproduce the relevant workload with its own fields, candidate logic, duplicate prevalence, data quality, infrastructure, workers, concurrency, and persistence requirements. Online, bulk, and continuous matching have different operating characteristics. Current continuous-matching documentation also identifies product-specific behavior and limitations that should be tested for updates, deletions, resets, and result consumption.
Evaluation standard
MDMbox should pass the same proof required of every vendor. Bring difficult pairs and representative source data. Define the desired merge, link, survivorship, stewardship, and tenant policies. Inspect the evidence. Change a model or workflow requirement. Execute a real integration. Reverse a decision. Benchmark the relevant volume. Then compare the verified result, implementation effort, future change cost, and three-year cost using the scorecard in this guide.
Methodology and disclosure
Health Samurai publishes this guide and develops MDMbox. Product capabilities are based on current public documentation, the MDMbox product page, Health Samurai technical articles, the fine-tuning guide, and practical implementation experience. Product behavior and documentation can change. Buyers should evaluate the version proposed for their deployment.
Performance figures and claims about MDMbox deployments are vendor-reported. They should be reproduced under buyer-relevant conditions.
Legal and regulatory requirements vary by jurisdiction, contracting structure, data source, and use case. The discussion of referential matching identifies questions for counsel and privacy teams; it is not a legal conclusion about a particular vendor or dataset.
Test MDMbox on your own data. Bring representative records, your merge and link policies, and one change request. We will run the proof of capability described in this guide with your team: health-samurai.io/mdmbox. The full guide is also available as a PDF.
