How a migration is tested when the specification is the system being retired — adjudication of differences, the claims-edit inventory, in-flight work, user acceptance, regression across the interfaces, and the cutover go/no-go.
1. What This Document Adds
The Quality Plan already establishes how equivalence is measured: the parallel run is the test, the reconciliation is the assertion, adjudication outcome carries zero tolerance, and sampling is risk-based because 420,000 member records cannot be tested exhaustively. None of that is restated here. Two documents that describe the same tolerance will eventually describe it differently.
What the Quality Plan does not answer is what happens when the comparison produces a difference. It establishes that a different decision on the same claim is either a defect or an intended change and that there is no third category — but not who decides which, on what evidence, or inside what window. That question, and the five that follow from it, are this document's scope.
2. The Oracle Problem
BRD-02 states the governing fact: the oracle is not a specification, it is the system being retired. The target platform adjudicates correctly — it has done so for 1,800,000 members for years. The question is not whether it is right. It is whether it decides a Cumberland Valley claim the way Cumberland Valley's platform decided it, and where it does not, whether the difference was intended.
⚠ That inverts the usual testing relationship. Ordinarily a test asserts behavior against a written requirement, and a system that behaves better than the requirement has passed. Here there is no such requirement for most of the surface, so the assertion runs against observed legacy behavior instead — including the parts nobody can explain.
Two consequences follow, and they are the spine of everything below:
- Reproducing a known error is a pass. If the retiring platform applied an edit incorrectly for years, the target platform applying it the same way is a correct result. It is a business problem, and it is not this program's problem to fix without a decision.
- Improving on it without a decision is a defect. A tester who finds the target platform producing the better answer has found a variance, not a success. Silent improvement is indistinguishable from silent breakage at the point a member disputes a claim.
⭐ This is uncomfortable and it is deliberate. The alternative — correcting legacy behavior as it is discovered, mid-migration — means the program changes what members receive without anyone having decided to, and without the change being visible to anyone who could object.
3. Adjudicating a Difference
Every parallel-run variance enters a single queue and leaves it with a recorded disposition. No variance is closed by a tester.
| Disposition | Meaning | Who decides | Consequence |
|---|---|---|---|
| Defect | The target platform is wrong. Legacy behavior was correct and was not reproduced. | E. Nakagawa — Director, Core Admin Platform | Fixed and re-run before the gate. |
| Intended change | The difference was designed. A documented exception exists in BRD-02. | W. Ferriday — VP Claims Operations (Workstream Lead) | Closed against the exception reference. |
| Known-wrong legacy | Legacy behavior was incorrect and the target platform reproduces it faithfully. | ⚠ Escalated — never closed by test | Logged, referred to the business, and the program does NOT fix it. |
| Undetermined | Neither platform's behavior can be explained from documentation. | D. Okpara — Manager, Claims Configuration (CVHP) | Held open; blocks the gate if unresolved at cutover. |
⚠⚠ The known-wrong path is the one that matters and the one most programs lack. Without it, a tester who discovers that the retiring platform has been mis-applying an edit has only two available actions: raise it as a defect against the target platform, which is wrong, or close it as equivalent and say nothing, which buries it. Both outcomes end with the program silently owning a decision it never made.
A known-wrong finding is therefore routed out of the test process entirely — to R. Lattimore (VP Claims, Cumberland Valley) as the business owner of the behavior, with C. Tyrrell notified as Program Manager. The program reproduces it, records that it did, and the business decides separately whether to change it after Day 1. Correcting it inside the migration is out of scope even when the correction is obvious.
Adjudication runs to a fixed window: a variance is dispositioned within five business days of being raised, and one held longer is reported to the workstream lead by exception. A queue that ages is a gate that will slip without anyone deciding it should.
4. The Claims-Edit Inventory as a Test Problem
BRD-02 records that Cumberland Valley's claims edits accreted over years, and several have no documented owner or original reason. FR-09 requires the inventory to be completed early rather than during build, and the reason is a testing reason as much as a configuration one.
An edit with no documented reason cannot be tested against intent, because there is no recorded intent. It can only be tested against behavior. So the inventory is the test basis: each edit is exercised against claims that trigger it, and the target platform's handling is compared to the retiring platform's, edit by edit, rather than in aggregate.
⚠ Aggregate comparison hides exactly the case that matters. A sample that reconciles to zero variance overall can still contain an edit that never fired in either system during the window — which is not evidence that it works, only evidence that it was not tested. Each edit in the inventory therefore carries an explicit status: exercised and matched, exercised and varied, or not exercised. The third is a finding in its own right and is reported to the gate rather than left as an absence.
Edits whose owner cannot be established are flagged at inventory time, not at test time. J. Whitfield (Configuration Analyst, retiring platform) and M. Prideaux (Configuration Analyst, Core Administration) hold the inventory jointly, and an unowned edit is escalated on the known-wrong path in §3 before it reaches a tester who has no way to judge it.
5. In-Flight Work at Cutover
A migration boundary is not a moment; it is a period during which work exists in both systems. Claims received before cutover and adjudicated after it, appeals opened on the retiring platform and continued on the target, authorizations spanning the boundary, and accumulators that must carry a partial year — each is a test case, and none is covered by comparing two systems processing the same input.
| In-flight case | What is tested | Owner |
|---|---|---|
| Claim received pre-cutover, adjudicated post-cutover | The claim adjudicates on the target platform to the same outcome the retiring platform would have reached, against the benefit configuration in force on the date of service — not the cutover date. | W. Ferriday — VP Claims Operations |
| Appeal open at cutover | The appeal continues with its history intact and its clock unreset. | R. Lattimore — VP Claims, CVHP |
| Authorization spanning the boundary | Remaining visits or units carry across at the correct balance. | D. Halloran — BA, enrollment & eligibility |
| Accumulators mid-year | Deductible and out-of-pocket carry the partial-year balance, and no member's accumulator resets at cutover. | R. Sedgwick — BA, benefit configuration |
| EDI submission in transit | A transaction submitted before cutover and acknowledged after it is not duplicated and not lost. | P. Devarakonda — Test Lead, Integration |
⚠ Accumulators are the case that produces member-visible harm fastest. A reset deductible is not a reconciliation variance a tester will notice in a sample; it is a member paying twice, discovering it at a pharmacy counter. It is tested explicitly rather than left to the parallel run.
6. User Acceptance Testing
UAT is not a second functional test run by non-technical staff. It answers a different question: whether the people who will operate the platform on Day 1 can do their work on it. A system that passes every equivalence test and cannot be operated by the claims floor has not passed.
Each area has a named business owner who signs, drawn from the roster rather than from a role title. A sign-off with no name attached is an opinion, and it cannot be relied on at a gate.
| Area | Business owner who signs | What acceptance means |
|---|---|---|
| Claims adjudication and edits | W. Ferriday — VP Claims Operations, ACME | Adjudicators process a representative day's work on the target platform without reference to the retiring one. |
| Claims, acquired book | R. Lattimore — VP Claims, Cumberland Valley | Cumberland Valley's own adjudicators confirm their claim types behave as they expect. |
| Benefit configuration | R. Sedgwick — Business Analyst, benefit configuration | Plan behavior matches the configuration record for every target product. |
| Enrollment and eligibility | D. Halloran — Business Analyst, enrollment & eligibility | Effective dating, retroactivity and group structure behave as operated today. |
| Core administration platform | E. Nakagawa — Director, Core Admin Platform, ACME | Platform operations, batch and exception handling are runnable by the operating team. |
| Data quality | Y. Abegunde — Data Quality Analyst | Migrated records meet the agreed quality gates before UAT begins, not during it. |
⚠ UAT begins only after data quality gates are met. Running user acceptance against records that have not passed migration quality produces defects against the data rather than the system, consumes the business owners' limited availability, and teaches them to distrust the platform before it has been fairly tested.
Acceptance is recorded per area and is not aggregated into a single sign-off. One area's refusal is a gate condition, not a percentage.
7. Regression Scope
The acquiring platform serves 1,800,000 existing members who did not ask to be part of this transaction. The largest untested risk in the program is not that the migration fails — it is that the migration succeeds and breaks something for them. Regression exists for those members.
Regression scope is defined by change, not by convenience:
- The fourteen interfaces described in BRD-03. Each is re-tested end-to-end after any change to the integration layer, because an interface that was not changed can still break when what it talks to does.
- Configuration shared between books. Where target-platform configuration is extended to accommodate acquired products, existing products are re-tested against it. Shared configuration is the most common path by which a migration reaches members it was never supposed to touch.
- Batch and financial cycles. Full cycles are run rather than sampled, because a batch defect is not proportional to the sample that found it.
- X12 transaction sets already assured under the Quality Plan, re-run against the post-migration configuration.
Regression runs on the existing book at full volume where the cycle permits it. B. Nkemdirim (Test Manager) owns the regression suite; S. Mahadevan (QA Engineer) owns its automation and its currency — a regression suite that has not been updated since the last release is evidence about the last release.
8. Cutover and Rollback Testing
The cutover runbook is tested as an artifact in its own right, twice, before it is used. A runbook rehearsed once has been read; a runbook rehearsed twice, the second time by a different operator, has been tested.
| Rehearsal | What it proves | Exit condition |
|---|---|---|
| Dress rehearsal 1 | The sequence is complete and the timings are real, not estimated. | Every step executed; elapsed time recorded against the window. |
| Dress rehearsal 2 — different operator | The runbook is followed rather than remembered. | Completed without reference to anyone who wrote it. |
| Rollback rehearsal | ⚠ The program can return to the retiring platform with no data loss and a known position. | Rollback executed to completion and the resulting state reconciled — not merely started. |
⚠⚠ Rollback is rehearsed to completion, not described. An untested rollback is not a contingency; it is a sentence in a document that everyone will discover is wrong at the worst possible moment. The rehearsal establishes the one number that matters on the night — how long rollback takes — because the decision to roll back has to be made while there is still time to complete it.
9. The Go/No-Go
Cutover readiness is decided against the Day 1 gate on Oct 2, 2023. The conditions below are conditions, not scores: each is met or the gate is not passed. A weighted readiness percentage lets a program go live with the one thing that mattered unmet.
| # | Condition | Evidence |
|---|---|---|
| 1 | Parallel run reconciles within the Quality Plan tolerances | Reconciliation report; adjudication outcome at zero variance. |
| 2 | No open variance of Defect or Undetermined disposition | Adjudication queue empty of both categories. |
| 3 | Every claims edit in the inventory is exercised and matched, or explicitly dispositioned | ⚠ Edit inventory with a status against every line — not exercised is a reported finding, not a blank. |
| 4 | In-flight cases tested, accumulators included | In-flight test results signed by each named owner. |
| 5 | UAT accepted by every named business owner | Six area sign-offs, unaggregated. One refusal is a no-go. |
| 6 | Regression clean on the existing book | Fourteen interfaces re-tested; batch cycles run in full for 1,800,000 existing members. |
| 7 | Two cutover rehearsals and one rollback rehearsal completed | Rehearsal records with elapsed times; rollback reconciled. |
| 8 | Known-wrong findings logged and referred, none silently corrected | Known-wrong register, countersigned by the business owner. |
The go/no-go recommendation is prepared by B. Nkemdirim (Test Manager) with C. Tyrrell (Program Manager), and the decision is taken at the Day 1 gate by the Steering Committee. ⚠ The test function recommends; it does not decide. A test manager who can stop a go-live alone will eventually be persuaded not to, and the pressure will not be recorded anywhere.
Condition 8 is the one that would be dropped first under schedule pressure and is the one worth keeping longest. It is the only evidence that the program migrated 420,000 members without quietly changing what any of them receives.