The governing quality document for the program: how a vendor-delivered platform upgrade, a full conversion of member, policy, provider and claims data, and six downstream integrations are verified before they carry live enrollment and claims traffic. It applies to every workstream and every release, and sits beneath the Project Charter and Project Management Plan.
8
QA staff, dual-shore
6
Downstream integrations
3
Conversion dry runs
Hybrid
Waterfall + agile testing
Fixed
Open-enrollment go-live
The constraint that governs everything. Assumption A-004 states that the program must complete Go-Live before the start of the next open-enrollment period. That date does not move: a health plan cannot open enrollment on a half-migrated platform, and it cannot defer open enrollment. Every other schedule variable — scope sequencing, environment availability, even conversion dry-run count — is negotiable against that fixed point, and testing is planned backwards from it rather than forwards from today.
01 Purpose, Scope & Objectives
In scope
- The vendor-delivered platform upgrade — acceptance testing of vendor-supplied functionality.
- Conversion of member, policy, provider and claims data from the legacy platform.
- Six downstream integrations: Billing, Underwriting, Claims Administration, Commission Management, Data Warehouse/Reporting, and Payment Processing.
- Non-functional verification: performance at enrollment-peak volume, security, and HIPAA/PHI handling.
- SOX-relevant controls where the platform affects financial reporting or payment.
Out of scope
- Vendor-internal testing of their own product build — evidenced by vendor test attestation and verified through acceptance testing, not re-performed.
- Defects in downstream systems themselves, except at the interfaces this program consumes or writes to.
- Legacy platform remediation beyond what conversion fidelity requires.
Objectives
| Objective | How it is met |
|---|---|
| No member, policy or claim is lost or corrupted in conversion | Staged dry runs with full reconciliation and exception accounting (Section 05) |
| Go-Live occurs before open enrollment | Testing planned backwards from the fixed date; conversion dry runs sequenced early, not late |
| All six integrations behave correctly on day one | Interface-level verification plus end-to-end business-process testing (Section 07) |
| The business can operate the new platform | UAT executed by named business SMEs against real operational scenarios (Section 09) |
| Financial and regulatory integrity is provable | SOX control testing with independent external assessment per decision D-004 |
02 Testing Under a Hybrid Delivery Model
Decision D-001 established a hybrid methodology — waterfall for requirements and vendor selection, agile sprints for integration build. Testing follows that split rather than imposing a single approach across work that behaves differently.
| Workstream | Delivery approach | Test approach | Why |
|---|---|---|---|
| Requirements & vendor selection | Waterfall | Review-based verification: requirements traceability, acceptance criteria definition | Nothing executable exists yet; defects here are found by inspection |
| Integration build (6 systems) | Agile sprints | In-sprint testing with automated regression; interface contracts verified per sprint | Integrations are built incrementally and benefit from fast feedback |
| Data conversion | Phase-gated dry runs | Staged full-volume rehearsals with reconciliation gates | Conversion is a rehearsed event, not an iterative build — it either runs clean at full volume or it does not |
| Vendor platform upgrade | Vendor-delivered | Acceptance testing against contracted functionality | The program verifies fitness, it does not build the product |
| UAT & cutover | Phase-gated | Formal cycles with business sign-off | Acceptance is a decision point, not a continuous state |
The practical consequence. Integration testing runs continuously from early sprints, while conversion testing happens in a small number of high-stakes rehearsals. These need different governance: sprint testing is governed by definition-of-done discipline, conversion testing by formal gate criteria. Treating them identically — the common failure on hybrid programs — either smothers the sprints in ceremony or lets the conversion drift without a gate.
03 Test Organization — Dual-Shore Operation
Decision D-003 adopted a dual-shore QA model rather than onshore-only, ratified alongside CR-007. It buys extended coverage and cost efficiency, and it introduces a coordination burden that is managed explicitly rather than assumed away.
| Name | Role | Location | Responsibility |
|---|---|---|---|
| R. Whitfield | QA / Test Lead | Onshore | Owns this strategy, test planning, exit criteria, overall quality reporting |
| P. Sundaram | Offshore QA Coordination Lead | Offshore | Offshore execution, handoff discipline, throughput management |
| A. Ferreira | QA Automation Engineer | Onshore | Regression automation, CI integration |
| D. Kowalczyk | QA Automation Engineer | Onshore | Integration and interface test automation |
| E. Solis | QA Automation Engineer | Onshore | Conversion reconciliation automation |
| V. Rao | QA Manual Tester | Offshore | Functional and regression execution |
| S. Banerjee | QA Manual Tester | Offshore | Functional and regression execution |
| N. Fernandes | QA Manual Tester | Offshore | Functional and regression execution |
Supporting quality roles
| Name | Role | Quality contribution |
|---|---|---|
| T. McCormick | Data Conversion Lead | Owns conversion validation approach and reconciliation results |
| K. Larsson | Data Architect | Mapping correctness, data model integrity |
| R. Sandoval | ETL Developer (Data Conversion) | Conversion logic, exception handling |
Dual-shore only works with handoff discipline. Risk R-002 anticipated that onshore/offshore coordination gaps could reduce testing throughput, and issue I-003 confirmed a version of it — intermittent VPN connectivity to the offshore QA environment measurably affected throughput. The controls are deliberate: a written end-of-shift handoff with blocked-item detail, defect records complete enough to action without a conversation, a daily overlap window for live escalation, and environment access treated as a program-level dependency rather than an IT ticket. Follow-the-sun testing fails quietly when handoffs are verbal.
04 Test Levels
| Level | Owner | Scope | Approach |
|---|---|---|---|
| Component / unit | Developers | Custom integration code and ETL routines | Automated in CI |
| Interface | QA automation | Each of the six downstream interfaces in isolation | Contract and field-level verification, automated |
| Vendor acceptance | QA + BAs | Vendor-delivered platform functionality against contracted scope | Scenario-based acceptance testing |
| System integration | QA (Whitfield) | End-to-end business processes spanning platform and downstream systems | Business-process scenarios |
| Conversion validation | Data Conversion (McCormick) | Migrated data completeness, accuracy, reconciliation | Staged dry runs (Section 05) |
| Parallel run | QA + Business | Legacy and new platform processing the same input | Output comparison and variance investigation |
| Performance | QA + Infrastructure | Enrollment-peak and claims-batch volumes | Load and endurance testing |
| UAT | Business SMEs | Real operational scenarios | Formal cycles with sign-off |
05 Data Conversion Validation
Conversion is the program's largest quality exposure. Risk R-001 — historical claims data quality extending conversion if root-cause cleansing proves insufficient — is rated 9 (High/High), the highest on the register, and issue I-001 confirmed the concern early: profiling revealed higher-than-expected duplicate member records in historical claims data.
Dry-run strategy
| Run | Purpose | Exit criteria |
|---|---|---|
| Dry Run 1 | Prove the conversion executes end-to-end at representative volume; surface the true defect population | Run completes; all exceptions categorized and root-caused |
| Dry Run 2 | Verify remediation of Dry Run 1 findings; establish reconciliation baselines | Reconciliation within tolerance; no unexplained variances; timing fits the cutover window |
| Dry Run 3 contingency | Additional cycle if root-cause cleansing proves insufficient | Clean reconciliation before production conversion is authorized |
The third dry run is pre-funded, deliberately. Decision D-002 pre-approved contingency utilization of up to $150,000 for a third Data Conversion Dry Run cycle should root-cause cleansing of historical claims data prove insufficient. That decision matters more than its cost: it means that if the data is worse than hoped, the program does not have to choose between an unproven conversion and an emergency funding request under deadline pressure. The option is already bought. Pre-authorizing a contingency cycle before you know you need it is how a fixed go-live date is protected in practice.
Validation activities per run
| Activity | What is verified | Tolerance |
|---|---|---|
| Record count reconciliation | Members, policies, providers, claims — source vs target | 100% — any variance is explained, never accepted as rounding |
| Financial control totals | Claim amounts, premium, accumulators, balances | Exact reconciliation; SOX-relevant |
| Field-level sampling | Statistically valid sample compared field by field | Zero defects on critical fields |
| Duplicate resolution | Duplicate member records identified, merged per rule, and auditable (issue I-001) | Every merge traceable and reversible |
| Exception handling | Unconvertible records quarantined, reported and owned — never silently dropped | Exception population fully accounted for before Go-Live |
| Cutover timing | Conversion completes within the available outage window | Run duration plus contingency fits the window |
A member whose record converts incorrectly may be unable to prove coverage at the point of care, and a claim that converts wrong may be paid or denied incorrectly. Conversion defects are treated at the highest severity for that reason, not because of data-hygiene principle.
06 Vendor-Delivered System Acceptance
The platform itself is delivered by the vendor. The program's job is not to re-test the vendor's product but to verify that what was delivered matches what was contracted, and that it works in this environment with this data.
| Activity | Approach |
|---|---|
| Vendor test evidence review | Vendor's own test results reviewed for coverage and credibility before acceptance testing begins |
| Acceptance scenario testing | Contracted functionality exercised against realistic enrollment and claims scenarios |
| Configuration verification | Plan, benefit and rule configuration verified — the most common source of "the software works but the setup is wrong" |
| Environment fitness | Vendor-provided environments confirmed production-representative in both configuration and data volume |
| Defect routing | Vendor-attributable defects raised through the contractual channel with evidence; program-attributable defects handled internally |
Environment fitness is a test prerequisite, not a detail. Issue I-002 records that the initial vendor-provided test environment lacked production-representative data volume, delaying early integration testing. An environment that behaves correctly at small volume proves very little about a claims platform; volume is what surfaces timeouts, batch-window overruns and index problems. Environment data-volume adequacy is therefore an explicit entry criterion for integration testing, verified rather than assumed — and assumption A-003, that the vendor's approach will not require a database platform change, is monitored as part of the same activity.
07 Integration Testing — Six Downstream Systems
| Integration | Primary test focus | Notable risk |
|---|---|---|
| Billing | Premium calculation, invoicing, adjustment handling | Financial accuracy; SOX-relevant |
| Underwriting | Risk data flow, rating inputs, policy issuance | Correct policy terms at issue |
| Claims Administration | Claim intake, adjudication data, status flow | Highest transaction volume |
| Commission Management | Commission calculation and payment triggers | I-004 — discrepancy found between the integration spec and legacy calculation logic |
| Data Warehouse / Reporting | Extract completeness, reporting accuracy, regulatory reporting | Downstream state filings depend on it (risk R-004) |
| Payment Processing | Payment initiation, reconciliation, failure handling | Money movement; SOX-relevant |
Specification versus behaviour. Issue I-004 is the classic integration finding: the Commission Management integration specification and the legacy system's actual calculation logic disagreed. The specification described what the system was believed to do; the code did something else. Integration testing on this program therefore validates against observed legacy behaviour as well as written specification, because on a replacement program the legacy system is the de facto requirement — and where they differ, the difference is a decision for the business, not a defect for QA to resolve.
08 Parallel Run & Reconciliation
Before cutover, legacy and new platforms process the same inputs and their outputs are compared. Parallel running is the only test that answers the question the business actually cares about: will the new system produce the same answers as the old one, except where we intended it to differ?
| Element | Approach |
|---|---|
| Scope | Representative claims and enrollment transaction mix, including edge cases and high-value claims |
| Comparison | Automated output comparison; variances classified as expected (intended change), explained (data or timing), or defect |
| Acceptance | Zero unexplained variances; every intended difference documented and approved by the business |
| Duration | Sufficient cycles to cover monthly processing events, not only daily transactions |
09 User Acceptance Testing
- Executed by named business SMEs from enrollment, claims, billing and underwriting operations — not by IT proxies, and not by QA on the business's behalf.
- Scenario-based, built from real operational cases including the awkward ones: retroactive enrollment changes, coordination of benefits, claim adjustments and reversals.
- Two formal cycles plus regression confirmation, with defect fixes retested before sign-off.
- Operational readiness assessed alongside functionality — can staff actually complete their work, with the runbooks and training provided?
- Sign-off is a business decision recorded against named approvers, and it gates Go-Live.
The usual failure mode. UAT slips when business SMEs are not genuinely released from their day jobs. Assumption A-002 — that named resources are available at planned allocation with no extended unplanned absences — applies as much to business SMEs as to the delivery team, and is the assumption most likely to be tested by an operations manager under pressure during enrollment season.
10 Non-Functional, Security & Compliance Testing
| Area | What is verified | Acceptance basis |
|---|---|---|
| Performance | Enrollment-peak concurrency; claims batch completes within its window; reporting extract timing | Meets targets at projected peak with headroom |
| Security | Access control, encryption in transit and at rest, penetration testing | Zero high or critical findings open at Go-Live |
| HIPAA / PHI | Minimum-necessary access, audit logging, PHI handling in all environments | Compliance sign-off |
| SOX controls | Controls over financial reporting and payment paths tested and evidenced | Independent external assessment per D-004 |
| Regulatory reporting | State filing and reporting outputs verified against requirements | Business and compliance verification (risk R-004) |
| Disaster recovery | Backup, restore and failover rehearsed | Successful rehearsal before Go-Live |
Issue I-005 — an external penetration test finding on staging TLS configuration — illustrates why security testing runs against environments that mirror production configuration rather than only against production itself. A staging weakness is a real finding when staging holds production-representative data.
Decision D-004 engaged an external firm for SOX assessment rather than relying on internal audit alone. On a program touching premium, claims payment and commission calculation simultaneously, independent assessment is proportionate to the financial-reporting exposure.
11 Test Environments & Data
| Environment | Purpose | Data |
|---|---|---|
| Development | Component and unit testing | Synthetic |
| Integration | Interface and system integration testing | De-identified, referentially complete |
| Conversion staging | Dry runs and reconciliation | Full-volume production copy under controlled access |
| Performance | Load, batch and endurance testing | Production-volume synthetic |
| UAT | Business acceptance and parallel run | De-identified production-derived, production-representative volume |
| Offshore access | Offshore QA execution | De-identified only; no PHI accessible offshore |
- PHI boundaries are absolute. Offshore testers work exclusively with de-identified data; conversion runs involving real member and claims data are executed and reconciled onshore.
- Environment availability is a tracked dependency, not an assumption — risk R-003 flags vendor SOW negotiation delay pushing back Environment Provisioning, which compresses every downstream test window.
- De-identification preserves realistic distributions, including the duplicate and malformed records that make conversion testing meaningful.
12 Entry & Exit Criteria
| Stage | Entry criteria | Exit criteria |
|---|---|---|
| Integration testing | Interfaces deployed; environment production-representative in volume (I-002); test data loaded | All six interfaces verified; no open critical or high defects; contracts stable |
| Conversion dry run | Conversion code complete; source data profiled; reconciliation automation ready | Reconciliation within tolerance; exceptions fully accounted; run fits the cutover window |
| System integration test | Integration and vendor acceptance complete | End-to-end business processes pass; performance targets met |
| UAT | System testing exit met; UAT environment loaded; SMEs released and trained | All scenarios executed; no open critical or high defects; business sign-off recorded |
| Go-Live | All above complete; DR rehearsed; rollback plan tested; SOX assessment complete | Steering Committee Go/No-Go approval |
13 Defect Management & Metrics
| Severity | Definition | Response |
|---|---|---|
| Critical | Data loss or corruption in conversion; incorrect claim payment or denial; PHI exposure; system unavailable | Immediate; blocks Go-Live absolutely |
| High | Core business process unusable; financial calculation incorrect; no workaround | Fixed before the applicable gate |
| Medium | Function impaired with acceptable workaround | Scheduled; disclosed at gate review |
| Low | Cosmetic or minor usability | Backlog |
Metrics
| Metric | Purpose | Target |
|---|---|---|
| Conversion reconciliation variance | The single most important quality number on this program | Zero unexplained variance |
| Requirements coverage | Traceability from requirement to executed test | 100% of critical-path requirements |
| Defect escape to UAT | Effectiveness of system testing | Zero critical; downward trend |
| Offshore test throughput | Whether the dual-shore model is delivering (R-002, I-003) | Stable; investigated when it dips |
| Automated regression coverage | Sustainability across repeated cycles | Critical paths automated before UAT |
| Open high/critical defects by gate | Gate readiness | Zero at each gate |
14 Risks to the Test Approach
| Risk | Effect on testing | Mitigation |
|---|---|---|
| R-001 — historical claims data quality | Conversion cycles extend; test windows compress against a fixed date | Early profiling; pre-funded third dry run (D-002); exception handling designed in |
| R-002 — onshore/offshore coordination gaps | Throughput drops silently | Written handoffs, daily overlap window, throughput tracked as a metric |
| R-003 — vendor SOW delay pushes Environment Provisioning | Every downstream test window compresses | Environment readiness tracked as a critical-path dependency, escalated early |
| R-004 — mid-program regulatory change | Unplanned compliance test scope | Regulatory monitoring; reporting test scope kept modular so additions are contained |
| R-006 — key resource attrition (Solution Architect) | Technical decision-making and test design authority disrupted | Documented design decisions; deliberate knowledge sharing across the technical leads |
| Fixed Go-Live compressing test windows | Pressure to reduce coverage on the highest-risk scope | Exit criteria are not negotiable; scope is descoped before quality is |
Governing relationship. This strategy sits beneath the Project Charter and Project Management Plan, and governs the test plans produced for conversion, integration and UAT. Changes with cost or schedule impact route through the program's change control process. Where a change affects conversion validation or SOX-relevant controls, it additionally requires Steering Committee awareness — because both bear directly on Go-Live authorization.