Quality on an integration program means something narrower and harder than quality on a build program. Almost nothing here is new functionality being validated against a specification — it is existing functionality being moved, and the question is whether it still behaves the same way. This plan sets out how that is tested, how a defect's severity is judged by what it does to a member rather than by where it appears, and what must be demonstrated before each gate. Approved December 18, 2023.
The central distinction: this program tests for equivalence, not correctness. Nobody is asking whether ACME's claims platform adjudicates correctly — it has been doing so for years for 1,800,000 members. The question is whether it adjudicates a Cumberland Valley claim the same way Cumberland Valley's platform did, and where it does not, whether the difference is intended. That is a fundamentally different test design: the oracle is not a specification, it is the outgoing system.
Table of Contents
Part I — Approach
Part II — Specialist Assurance
Part III — Control
1. Three Kinds of Quality
The word covers three different things on this program, with different owners, methods and failure modes. Conflating them produces a test plan that tests the easiest one thoroughly and the others not at all.
| Kind | Question | Owner | Method |
|---|---|---|---|
| Functional equivalence | Does the surviving system behave as the retiring one did? | W. Ferriday | Parallel run with reconciliation |
| Data quality | Did the right data arrive, correctly matched and complete? | Dr. A. Ravindran | Profiling, control totals, audited sampling |
| Operational readiness | Can the organization actually run it — including when it breaks? | B. Trammell | Runbooks, recovery testing, incident rehearsal |
Operational readiness is the one that gets tested last and least, and it is where a first-time cloud organization is weakest. Functional testing has an obvious method and an obvious finish line. "Can we recover this at two in the morning with the people who are actually on call" has neither, so it slips — and it is the only one of the three whose failure is discovered during an incident rather than during a test cycle.
2. Equivalence Testing and Parallel Run
2.1 Parallel run is the test; reconciliation is the assertion
For a defined period before each cutover, the same input is processed by both systems and the outputs are compared. The comparison — not the successful completion of the run — is the test result.
| Compared | Tolerance | Reasoning |
|---|---|---|
| Adjudication outcome (pay / deny / pend) | Zero variance | A different decision on the same claim is either a defect or an intended change. There is no third category. |
| Allowed amount and member liability | Zero variance | Money. Any difference is investigated to root cause before cutover. |
| Denial and adjustment reason codes | Mapped, not identical | Code sets differ between platforms; the mapping is the deliverable and the mapping is what is tested. |
| Processing timeliness | Within regulatory limits | ⚠ Not "same as before" — the prior market conduct finding makes timeliness a regulatory measure, not a comparison. |
| Accumulator balances | Zero variance | Running totals. A drift here becomes a wrong member liability on the next claim. |
Every intended difference has to be documented in advance, or the parallel run produces noise instead of signal. Some differences are deliberate — a benefit interpretation being corrected, an edit rule being applied that the target's platform never had. If those are not recorded before the run, the reconciliation returns hundreds of variances, the team triages them by hand, and the genuine defects hide among the intended ones. The expected-difference register is written first and is itself reviewed, because it is the list of places the program is knowingly changing a member's outcome.
2.2 Duration
Parallel run length is set by claim cycle rather than by calendar convenience: long enough to include a full month-end, a full payment cycle, and at least one instance of the low-frequency, high-complexity claim types that never appear in a two-week sample.
3. Risk-Based Sampling
You cannot test 420,000 member records, and pretending otherwise produces a plan that fails silently. Exhaustive testing at this volume is not a resourcing problem to be solved with more people — it is arithmetically impossible inside any migration window. So the program samples, and the only decision that matters is how the sample is chosen. A random sample of member records is dominated by the simple, healthy, single-coverage majority — which is precisely the population that was never going to break.
| Sample stratum | Why it is over-represented relative to its share of the population |
|---|---|
| Members matched in the clerical review band | Identity resolution decided these by human judgment. If matching is wrong anywhere, it is wrong here. |
| Members with coverage under both entities | The only records where coverage history is a union rather than a copy |
| Members with active accumulator balances | Running totals crossing the boundary mid-year |
| Coordination of benefits and secondary coverage | Multi-payer logic differs most between platforms |
| High-cost claimants and active care management cases | Small population, disproportionate clinical and financial consequence |
| Complex claim types — transplant, dialysis, long inpatient stays | Rare enough to miss in a random sample, expensive enough to matter when wrong |
| Retroactive terminations and reinstatements | Effective dating is where platform logic diverges most quietly |
The defensible summary if challenged on coverage: the sample is not representative, and it is not meant to be. It is deliberately weighted toward the populations where an error is both most likely and most consequential. A statistically representative sample would give a comforting pass rate and would systematically under-test exactly the members whose claims are complicated — who are also, in a health plan, the members whose claims matter most.
4. X12 Transaction Assurance
Contracted to Gallatin EDI Assurance under a fixed-price statement of work, because X12 testing is a specialist result rather than general quality assurance.
| Transaction | Assurance scope |
|---|---|
| 834 | Enrollment and maintenance. Full-file versus change-file semantics per trading partner; effective dating; termination handling. |
| 837 | Institutional, professional and dental variants tested separately — they are different rule sets, not one format with options. |
| 835 | Remittance reconciled to the payment actually issued, across both paying platforms during coexistence |
| 270 / 271 | Eligibility inquiry and response under load, including timeout behavior |
| 820 | Premium payment reconciled against billing at group level |
The acceptance criterion is deliberately not "the file passes validation." A functional acknowledgment confirms the file was syntactically well formed — it says nothing about whether the receiving system accepted it, loaded it, or loaded it into the right group. A pipeline that treats acknowledgment as success reports green while enrollment silently fails, and the failure surfaces weeks later as members who cannot prove coverage at a pharmacy counter. Every EDI interface therefore carries a business-level control total alongside its acknowledgment.
5. Data Quality Gates
| Ref | Gate | Criterion |
|---|---|---|
| DQ-01 | Source profiling complete | Completeness, format and duplicate rates measured — not assumed — before any match rules are tuned |
| DQ-02 | Match rule configuration accepted | Thresholds set, review band sized, survivorship rules documented per attribute |
| DQ-03 | Clerical review queue within capacity | ⚠ Queue clearable inside the migration window at staffed throughput. A queue that cannot be worked is not a control. |
| DQ-04 | Audited false-positive rate below threshold | Independent re-review of a sample of auto-matched pairs, by the Privacy Officer's team rather than by the workstream that produced the matches |
| DQ-05 | Migration control totals balanced | Record counts and financial totals reconciled per load, per source |
| DQ-06 | Coverage history complete | No member loses a period of coverage in the merge. Union, never overwrite. |
DQ-04 measures the error that does not announce itself, which is why it is audited rather than reported. A false negative leaves a duplicate record — visible, annoying, eventually noticed and fixed. A false positive merges two people, and nothing in normal operations surfaces it: the member does not know, the system does not flag it, and it is found by accident months later. An error that produces no symptom must be gone looking for, and it cannot be measured by the team whose thresholds produced it.
6. Recovery Acceptance Criteria
Recovery is a quality requirement with acceptance criteria, not an infrastructure setting. These criteria apply at every workload go-live and are evidenced, not asserted.
| Tier | RTO | RPO | Acceptance |
|---|---|---|---|
| Tier 0 — critical | 4 h | 15 min | Recovery demonstrated under production conditions |
| Tier 1 — essential | 24 h | 1 h | Restore tested; runbook rehearsed by the on-call team |
| Tier 2 — important | 72 h | 24 h | Restore tested with referential integrity verified, not merely file presence |
| Tier 3 — deferrable | 120 h | 24 h | Backup configuration verified; restore tested at least once |
6.1 The enterprise data warehouse
The warehouse is the program's only new build. It inherits no backup regime, no rehearsed runbook and no restore history, so its recovery position must be constructed and demonstrated rather than assumed. It is Tier 2 — RTO 72 hours, RPO 24 hours — and does not go live until all seven conditions below are evidenced.
- Documented RTO and RPO agreed with the business owner and recorded here
- Geo-redundant backup configured to the Azure paired region
- Immutable or soft-delete protection against ransomware and accidental deletion
- Business Associate Agreement confirmed to cover the backup location and secondary region
- Full restore tested end to end, with evidence retained for audit
- DR runbook with named roles and a named accountable owner
- Recovery obligations carried in the risk register with a residual rating
Backup retention is seven years, above the HIPAA floor, driven by state insurance record-retention rules. Restore is tested annually thereafter.
For a warehouse, durability matters more than availability, and the acceptance criteria are weighted accordingly. A 72-hour outage is survivable — no claim goes unpaid, no member is denied care. But the warehouse is the system of record for quality measurement and feeds statutory reporting, and losing a measurement year cannot be recovered at any recovery time objective, because the source transactions have aged out of the systems that produced them by the time anyone notices. Condition five is the one that gets skipped and the only one that proves anything: a backup that has never been restored is a hypothesis.
7. Defect Severity and Management
Severity is judged by consequence to a member or a regulator, not by which system the defect appears in. A cosmetic error on an internal report and a cosmetic error on a member's explanation of benefits are the same class of technical fault and completely different problems. Severity scales that classify by component rather than by effect systematically under-rate the ones that reach a person.
| Severity | Definition | Examples |
|---|---|---|
| S1 | A member is harmed, denied access, or has their information exposed. A regulatory obligation is breached. | Two members merged into one record · a claim paid against the wrong member · eligibility unavailable at point of care |
| S2 | Financial or regulatory consequence without immediate member harm | Incorrect member liability · claims timeliness outside limits · accumulator drift |
| S3 | Operational impact absorbed by internal workaround | Report failure · interface requiring manual reprocessing |
| S4 | Cosmetic or internal-only | Formatting on an internal screen |
7.1 Management
- S1 stops the gate. No cutover proceeds with an open S1, regardless of schedule pressure. This is not delegable to the Program Manager.
- S1 root cause is reported to the Steering Committee individually, not as a count in a defect summary.
- S2 requires a documented workaround and a fix date before the gate.
- ⚠ Defect counts are reported alongside defect age — a stable count with rising age is a backlog, not a plateau.
- Defects found in parallel run are triaged against the expected-difference register before being raised.
8. Quality Gates by Milestone
| Gate | Must be demonstrated |
|---|---|
| Day 1 | Payroll parallel cycle reconciled · claims adjudicating on both platforms · eligibility responding · all Business Associate Agreements executed · no open S1 |
| Each migration wave | Performance against pre-migration baseline · interfaces reconciled · policy compliance clean · restore tested in the new location · source decommissioned or dated |
| Warehouse go-live | The seven recovery conditions in §6.1, evidenced |
| Core admin cutover | Parallel run reconciled to zero variance on adjudication outcome and amounts · EDI trading partners re-registered · identity resolution complete · no open S1 or S2 |
| TSA exit | Receiving function operating unaided · knowledge transfer accepted by reverse-shadowing · no residual dependency |
Every gate on this list is expressed as something demonstrated rather than something completed, and that is the plan's single organizing idea. "Migration finished" is a status; "performance measured against the pre-migration baseline and reconciled" is evidence. The distinction costs time at every gate and it is what allows the program to state, at closeout, that the combined entity adjudicates claims the way it did before — rather than that it believes so.
Related artifacts: 20 — Application Disposition Matrix (AD-20) · 22 — TSA Schedule & Exit Plan · 23 — Data Migration & EMPI Strategy · 25 — Integration Architecture · 28 — Risk Register · 34 — Cloud Landing Zone Design · 35 — Cloud Migration Wave Plan · 30 — Day 1 Go/No-Go