Issued April 16, 2024 — one day after Data Profiling Results released. These rules define how the identity resolution platform will classify and handle each matching category. This is the decision logic that governs the steward workflow and determines which records advance to the worklist for human review.
Deterministic, Probabilistic, and Clerical Review Classification
Contents
1. Matching Categories Overview
Every record in the identity resolution queue is classified into one of four categories based on the confidence and verifiability of the match. Each category has distinct handling rules, decision authority, and record state on completion.
| Category | Decision Authority | Confidence | Action |
|---|---|---|---|
| Deterministic | System (automated) | >95% (exact match) | Auto-merge, no review |
| Probabilistic Auto | System (tuned threshold) | 92–95% | Auto-merge, logged |
| Clerical Review | Data Steward | 65–91% | Worklist entry, documented decision |
| Non-Match | System (or Steward if ambiguous) | <65% | Separate records, flag as duplicate candidate |
The threshold bands are configured in the matching engine and locked for the duration of the integration phase. Configuration changes require formal change control and re-validation of all previously classified records. No mid-stream threshold tuning is permitted.
2. Deterministic Matching Rules
Trigger: A record from Cumberland Valley (source) matches a record in the ACME base (target) on one of the following exact identifiers:
Rule D1: Social Security Number (SSN) + Date of Birth (DOB), both exact. Zero tolerance for partial SSN, transposed digits, or approximate DOB. This is the highest-confidence match class.
Rule D2: Member ID exact match from either organization's prior assignment. If both organizations have member ID systems and the ID is carried forward into the integration (no re-issue), an exact ID match is deterministic.
Rule D3: Full Legal Name + DOB + sex, all exact. Name comparisons use standardized formatting (leading/trailing spaces trimmed, middle name variations consolidated), but the match must be character-for-character on the surname, given name, and date fields.
Handling: Deterministic matches are merged programmatically with no human review required. The merge is logged with the matching rule invoked (D1, D2, or D3), timestamp, and system user. A report of all deterministic merges is generated weekly and reviewed by Dr. A. Ravindran (Data & EMPI) for drift or unexpected patterns.
Challenge period: Deterministic records may be challenged by a steward during the clerical review phase if additional context surfaces a data quality issue (e.g., a record marked as "deceased" should not merge with an active member). Such challenges are escalated to the Data Steward Supervisor and must be documented with the steward's name, timestamp, and reason.
3. Probabilistic Auto-Match Rules
Trigger: A record pair that does not meet deterministic criteria but scores above the configured auto-match threshold on the probabilistic matching engine. Typical scoring inputs include:
Rule P1: Name similarity (Jaro-Winkler distance, with weight 0.35) + Address similarity (CASS/NCOA with weight 0.25) + Phone match (Levenshtein, weight 0.15) + DOB proximity (weight 0.15) + Sex match (weight 0.10) = aggregate score. The configured threshold is 92%. Matches at or above 92% are accepted automatically.
Rule P2: Soundex/Double Metaphone phonetic matching is applied to names that fail string-distance matching. Names that are phonetically identical and have matching first letter are scored as a "phonetic match candidate." This category is scored at 0.88+ aggregate score (lower threshold because phonetic matches are inherently more conservative).
Rule P3: Blocking rules prevent unnecessary comparisons and reduce false positives. Records are blocked (not compared) if: (a) surname begins with different first letter, (b) DOB differs by more than 4 years, or (c) sex indicators conflict. Blocked pairs are flagged for manual review only if other signals suggest a match.
Handling: Probabilistic auto-matches are merged programmatically. Unlike deterministic matches, each auto-match is scored and logged with the aggregate confidence. A parallel report by steward team verifies that the auto-accept threshold is not drifting: if the proportion of records in the 92–93% band suddenly increases, it may indicate a configuration issue and triggers an escalation to Dr. Ravindran.
Threshold tuning is forbidden mid-integration. Raising the threshold to reduce review band width sacrifices accuracy at the margin where decisions become hardest. Lowering it increases false positives, the error type that does not surface in normal operations. The thresholds are frozen as configured on April 16, 2024 (the day this rule set is issued) and do not change without a formal change request and retesting.
4. Clerical Review Band Rules
Trigger: A record pair scoring between 65% and 91% on the probabilistic engine, or any pair flagged for manual review due to data quality concerns, business rules, or analyst challenge.
Rule C1: Every clerical record receives a worklist entry in the EMPI steward dashboard. The entry displays (a) the pair of records side-by-side with all fields, (b) the confidence score and matching-rule inputs, (c) the steward's prior decisions on similar pairs (for consistency), (d) any flags from data quality profiling (e.g., "SSN appears in multiple records"), and (e) a suggested decision based on heuristics (not binding).
Rule C2: The steward reviews the pair and makes a binary decision: MATCH or NO MATCH. A MATCH decision requires the steward to document the reasoning in one of the following categories: (a) high-confidence name and address combination despite score, (b) DOB confirmed by secondary source, (c) known variation (nickname, name order), or (d) other (free text). A NO MATCH decision requires explanation only if the score is >80% (the "near-miss" band where automated suggestions might mislead).
Rule C3: Survivor-per-attribute merge rule. If the steward decides MATCH, the system does not select a "winning" record; instead, each attribute (name, address, phone, DOB, sex, emergency contact, etc.) is reviewed independently and the steward confirms which source's value is most recent or accurate. This produces a merged record composed of attributes from both sources, with lineage preserved (audit trail records which source contributed which attribute).
Rule C4: False-positive audit. Once per week, Dr. A. Ravindran (Data & EMPI) samples 5% of the steward's MATCH decisions (weekly sample size ~n determinedfrom weekly volume) and independently reviews the pairs. If the steward's match decision is questionable (score <75% and reasoning insufficient), the pair is marked for re-review and the steward is notified. Repeated failures trigger a 1:1 coaching session with the Steward Supervisor.
Handling: Clerical decisions are final once approved by the Steward Supervisor (end-of-day review). The merged record is written to the integration layer with a merge decision ID and timestamp. The steward's name, timestamp, and reasoning are retained in the audit log for post-close forensics.
5. Outcome Handling & Record State
| Outcome | Source State | Target State | Merged Record ID |
|---|---|---|---|
| Deterministic Match | Retire source ID (redirect to merged ID) | Become merged master | Target ID retained (canonical) |
| Probabilistic Auto-Match | Retire source ID | Become merged master | Target ID retained |
| Clerical Match | Retire source ID | Become merged master | Target ID retained |
| Non-Match | Remain distinct (create internal dup entry) | Remain distinct | Both IDs retained, linked as potential duplicate |
In all MATCH outcomes (deterministic, auto, or clerical), the target system's member ID becomes the canonical identifier for the merged record. The source system's member ID is retired (marked as "historical" or "retired-on-[date]") and all references are repointed to the target ID. This ensures a single golden record per member in the go-forward ACME system.
Why target ID is retained: The acquirer's ID system is the canonical one post-integration; retaining it removes the need to re-issue member identifiers (ID cards, system references, benefit statements). The source ID is preserved in the historical record for forensic traceability.
6. Conflict Resolution & Exceptions
Conflict Type 1: Attribute Conflicts — If the steward marks MATCH but the source and target records have conflicting values for a critical attribute (e.g., name, DOB, sex), the survivor-per-attribute rule applies: each attribute is decided independently. If the conflict is irreconcilable (e.g., DOB differs by >1 year and neither source is clearly more reliable), the steward escalates to the Steward Supervisor, who reviews the source data quality flags and decides. If still unclear, the record is flagged for post-close research (Dr. Ravindran's team handles this after cutover, with time pressure removed).
Conflict Type 2: Multiple Candidate Matches — A source record scores above 92% match on multiple target records. The system flags this as a one-to-many conflict and sends it to clerical review instead of auto-matching. The steward's decision: which target is the true match, or are these genuinely separate records? If the steward confirms one true match, the system demotes the other candidates to non-match and flags them as potential duplicates within the target system (if any of those duplicates are confirmed, they are managed separately).
Conflict Type 3: Steward Override of Automation — A steward may challenge a deterministic or auto-match decision. The challenge is logged with the steward's name, reason, and timestamp. If the challenge is sustained by the Steward Supervisor, the match decision is reversed and the pair is sent to clerical review or marked as non-match. This is rare but permitted to capture steward knowledge of data anomalies that the rules do not anticipate.
Escalation path: Steward → Steward Supervisor (T. Vandiver, daily) → Dr. A. Ravindran (weekly or urgent) → Program Manager (if policy change needed)
Related artifacts: 31 — Data Profiling Results · 33 — Steward Standard Operating Procedures · 19 — Retention Plan (T. Vandiver, Steward Supervisor)