Writing a Test Plan & Test Strategy — QA Guide

Test Strategy vs Test Plan — The Essential Distinction

The terms "test strategy" and "test plan" are used interchangeably in many organisations, which creates significant confusion. They are distinct documents that operate at different levels of abstraction and serve different audiences.

A test strategy is a high-level, organisation-wide (or programme-wide) document that defines how the testing function operates across all projects. It answers: What testing approaches and types do we use? What tools and frameworks are standard? What are our quality principles? How does testing fit into our SDLC? The test strategy changes rarely — perhaps when the organisation adopts a new methodology, a major new tool, or shifts from Waterfall to Agile.

A test plan is a project-specific document that applies the strategy to a particular product release or feature set. It answers: What exactly are we testing for this release? What are the in-scope and out-of-scope features? What are the entry and exit criteria? What is the test schedule and who owns what? The test plan changes with every project or major release cycle.

Both are needed. The strategy provides consistency and organisational standards. The plan provides project-specific detail. Many teams have test plans but no strategy, which means every project reinvents the same decisions. Many consulting engagements begin with a strategy document precisely because the teams have been solving the same problems inconsistently across projects for years.

Test Document Hierarchy — Architecture Diagram

The relationship between business requirements, strategy, plans, and test artefacts:

Each level derives from the level above. Test cases implement the plan's scope. Reports measure against the plan's exit criteria. The strategy governs how all plans are written and all cases are managed. When a project's test plan is written, it should explicitly reference the org-level strategy and note any approved deviations.

Test Strategy Components

A complete test strategy document covers the following sections:

• Testing scope and objectives: What types of applications does this strategy cover? What are the quality objectives — reliability, security, performance, usability? What is the expected defect escape rate threshold?
• Testing levels: Which levels of testing are performed — unit, integration, system, acceptance? Who is responsible for each level? (Unit: developers. Integration: QA. System: QA with stakeholder sign-off. Acceptance: Product/business.)
• Test types: Functional, regression, performance, security, compatibility, accessibility. Which types are standard for every project? Which require explicit justification to include or exclude?
• Tools and environments: The approved automation frameworks, test management tools, performance tools, and environment configurations. Changes to this list require a strategy update.
• Entry and exit criteria templates: The standard conditions that apply to all projects unless the test plan overrides them with project-specific criteria.
• Defect management approach: Severity and priority classification scheme, SLA for bug resolution by severity, escalation paths, the tool used for defect tracking.
• Metrics and reporting: Standard metrics collected across all projects, report frequency, audience for each report type.

IEEE 829 Test Plan Structure

IEEE 829 (now superseded by ISO/IEC/IEEE 29119 but still widely referenced) defines an eight-section test plan structure that remains the foundation of formal test documentation in regulated industries (finance, healthcare, aerospace):

1. Test Plan Identifier: Unique ID, version number, date. Required for change control and auditing.
2. Introduction: Project overview, testing objectives, references to requirements documents and the test strategy.
3. Test Items: The specific software items (modules, services, releases, builds) being tested. Include version numbers — vague references to "the system" are not auditable.
4. Features to be Tested: Explicit list of features covered in this test plan. Each feature should map to a requirement ID.
5. Features NOT to be Tested: Equally important. Why are these features excluded? Third-party component (vendor is responsible)? Covered in a separate test plan? Out of release scope? Document the reason — auditors ask.
6. Approach: How the testing will be conducted. Test design techniques (boundary value, equivalence partitioning, exploratory). Automation strategy. Environment setup. Data management approach.
7. Pass/Fail Criteria: The measurable conditions for passing or failing the test effort. "Zero P1 bugs open at release" is a criterion. "Tests feel good" is not.
8. Suspension/Resumption Criteria: Conditions under which testing is suspended (environment down, critical build blocker, key personnel unavailable) and conditions for resuming (environment restored, blocking bug fixed).

The formal IEEE 829 structure also includes Test Deliverables, Testing Tasks, Environmental Needs, Responsibilities, Staffing and Training, Schedule, Risks and Contingencies. For regulated environments, all sections are required. For most commercial software projects, the eight above are the minimum viable structure.

Modern Agile Test Plan — The One-Page Format

For Agile teams, the full IEEE 829 structure is excessive overhead for a two-week sprint cycle. The modern Agile test plan fits on one page and covers the essentials without becoming a document that is outdated before the sprint ends:

• Scope: Which user stories and features are in scope for this sprint/release
• Risk: Top 3 testing risks and mitigation approach
• Approach: Testing types (functional, regression, exploratory), tools, and coverage focus
• Test types: Manual vs automated, which frameworks, which environments
• Environments: Dev, staging, production-like environment status and readiness
• Schedule: Key milestones — test start, regression run dates, sign-off date
• Team: Who is testing what, including developer unit testing responsibilities
• Tools: TMS, bug tracker, automation framework, performance tool

This one-page document can be updated at the start of each sprint in 30 minutes. It is shared in the team wiki, linked from the sprint board, and referenced in the sprint planning ceremony. Unlike a 40-page IEEE 829 document, the Agile test plan is actually used.

Writing Test Scope — In and Out

Scope is one of the most important and most poorly written sections of test plans. It must be explicit in both directions: what is in scope and what is out of scope. The out-of-scope section is equally important — it prevents scope creep and sets clear expectations with stakeholders.

In-scope features (example for a user account management release)

• User registration — email/password and Google OAuth flows
• Login and logout — web and mobile applications
• Password reset — email link flow
• Profile editing — name, avatar, notification preferences
• Account deletion — soft delete with 30-day grace period
• Session management — concurrent session handling, token expiry

Out-of-scope features (with reasons)

• SSO enterprise integration — separate workstream, covered in Q3 release plan
• Two-factor authentication — not in this release scope per product roadmap v2.3
• Admin user management — separate admin portal, covered in admin test plan
• Third-party email delivery (SendGrid) — vendor-managed service, functional testing only (email arrives) not delivery infrastructure testing
• Performance testing — separate performance test cycle scheduled two weeks before release

When scope is unclear, bugs discovered in out-of-scope areas create conflict: "Why didn't QA catch this?" The answer — "it was explicitly out of scope and documented" — requires that the scope was documented clearly and agreed upon.

Risk Analysis

Every test plan should include a risk analysis. Not a perfunctory list of risks — an actual likelihood × impact assessment that drives testing prioritisation. High-risk areas get more test coverage, more exploratory sessions, and more automation priority.

Risk analysis format: identify the risk, rate likelihood (1–3), rate impact (1–3), calculate risk score (likelihood × impact), define mitigation strategy.

Entry Criteria

Entry criteria define the conditions that must be met before testing begins. Testing before entry criteria are met wastes QA time and produces false results. Standard entry criteria for a release testing cycle:

• Build successfully deployed to the test environment
• Smoke test suite passing — the 10–15 most critical paths are green
• Test data prepared and verified — user accounts, sample data, edge case data
• Test environment stable for minimum 4 hours since last deployment
• All P1 bugs from previous test cycle resolved and verified
• Test cases reviewed and approved
• Required test tools configured and accessible
• Acceptance criteria confirmed for all in-scope stories

If any entry criterion is not met, testing does not begin. The test lead escalates to the project manager, and testing starts when the criterion is satisfied. This sounds rigid, but it prevents the common scenario where QA spends two days testing against an unstable build, generating meaningless results that must be repeated after the build is stabilised.

Exit Criteria

Exit criteria define when testing is complete enough to recommend release. They are the measurable conditions against which a go/no-go decision is made:

• 100% of planned test cases executed (not passed — executed; blocked tests need a documented reason)
• 95% test case pass rate (5% may be waived with documented exceptions approved by PO)
• Zero P1 bugs open
• Zero P2 bugs open (or explicit documented acceptance by product owner and sign-off)
• Regression suite passing — no new failures introduced by the release changes
• UAT completed and signed off by product owner
• Test summary report reviewed and approved
• All P3/P4 bugs logged in the backlog for next sprint

Test Levels

A test plan should specify which test levels are included and who owns each:

• Unit testing (developer responsibility): Individual functions and methods. The test plan should specify minimum coverage targets (e.g., 80% line coverage for business logic components). QA verifies coverage reports, not individual unit tests.
• Integration testing (QA and developer shared): API contracts, database interactions, service-to-service communication. Defines what integration test scenarios QA will execute and what framework (pytest, RestAssured, Postman collections) will be used.
• System testing (QA ownership): End-to-end user journeys through the fully integrated system. The primary QA responsibility. Includes functional, regression, exploratory, and compatibility testing.
• Acceptance testing (product/business ownership, QA-facilitated): UAT performed by product owners, business stakeholders, or beta users. QA sets up the environment, prepares test accounts, and facilitates the session. The product owner signs off.

Test Deliverables

The test plan should list what QA will produce during and at the end of the testing cycle:

• Test plan document: This document itself — reviewed, approved, and version-controlled
• Test cases in TMS: All test cases created and linked to requirements in TestRail, Zephyr, or equivalent
• Bug reports: All defects filed in JIRA (or equivalent) with structured, reproducible descriptions
• Daily status report: One-page daily update — test cases executed, pass/fail, bugs found, blockers (shared via email or Slack during test execution period)
• Test summary report: End-of-cycle report covering total scope, execution statistics, bug summary, recommendation (go/no-go)
• Automation results: CI pipeline test results linked from pull requests and builds

Writing the Test Summary Report

The test summary report is the QA team's end-of-cycle deliverable. It should be concise, data-driven, and clearly state a recommendation. Key sections:

• Date and scope: Release version, testing period, features tested
• Test execution statistics: Total test cases: 142. Executed: 142 (100%). Passed: 138 (97.2%). Failed: 2 (1.4%). Blocked: 2 (1.4%). Not run: 0.
• Bugs found: Total bugs filed: 23. By severity: P1: 0, P2: 1 (resolved), P3: 8 (7 resolved, 1 deferred), P4: 14 (8 resolved, 6 deferred).
• Deferred bugs: List of bugs accepted for deferral with PO sign-off, including the justification for deferral and the target sprint for resolution.
• Risks: Any remaining risks from the risk analysis — were mitigations effective? Any new risks discovered during testing?
• Recommendation: Clear go/no-go with rationale. "Recommend release. All exit criteria met. One deferred P3 bug (cosmetic UI issue on Android tablet landscape mode) accepted by PO for next sprint."

Common Mistakes in Test Plans

• Too detailed (becomes stale): A test plan that specifies every test step becomes a test case document. It is outdated within days of being written and requires continuous maintenance that nobody performs. Keep the plan at the approach level; put details in test cases.
• Too vague (useless): "QA will test all features thoroughly." This provides zero useful information. What features? What testing approach? What does "thoroughly" mean? A vague plan cannot be actioned or measured.
• No risk section: Missing risk analysis means high-risk areas are not identified and testing effort may be misallocated — thorough coverage of low-risk features, shallow coverage of high-risk ones.
• Incorrect scope: Features listed as in-scope that cannot actually be tested (missing test environment, missing data, missing access permissions) or missing features that stakeholders assumed were in scope.
• No sign-off process: A test plan that QA writes and never reviews with the project manager, product owner, or development lead is not a project document — it is a personal note. Sign-off creates shared accountability for the testing approach.

Traditional vs Agile vs No Plan

Full Test Plan Template

A copy-ready Markdown template for a lightweight Agile test plan. Adapt the sections to your project context: