Plan Your Test Strategy

Application testing should always begin with a little planning. Take a closer look at what planning a test strategy entails.

Introduction

A test strategy document includes the testing approach of your development lifecycle. Various stakeholders of the test strategy include the project managers, testers and developers. The strategy lists out some of the primary issues to be considered for the testing process.

At the minimum, the test strategy includes

  • Objectives of the test process
  • Approaches for testing new functionalities
  • Regression test approaches
  • Test environments
  • Risk analysis and mitigation plans
A major reason for developing a test strategy is to minimize the risks involved. The test strategy is more or less a static document – the document is rarely updated if any – and created at the beginning of a project using the high level system documents. In phased development, a strategy document may be created for each development phase.

Test Plan vs. Test Strategy

When talking to development teams, we see a lot of confusion between a test plan and test strategy. A test plan document is considered as somewhat superset of the strategy document. Usually a test strategy goes as an addendum to the test plan document or atleast referred from it when maintained as a separate document. Various teams/organizations follow their own processes for maintaining these documents.

A test plan defines the project scope and focus of testing. The document is more granular than a strategy document and deals with issues like test coverage, features that need to be tested or ignored, estimation of effort and time.

A test strategy focuses on the approach for attaining the test objectives, various test types and environments, tools & techniques, automation requirements, risk analysis and mitigation plans.

James Bach, Satisfice Inc. 

Test Plan: the set of ideas that guide a test project
Test Strategy: the set of ideas that guide test design
Test Logistics: the set of ideas that guide the application of resources to fulfill a test strategy

Do we Need a Strategy Document?

The short answer is: it depends. We do not create a blueprint when building a dog-house. At the same time we do not even think of building a house without a blueprint. Whether we need a test strategy document and how much detail the document should have depends on the type of project our team is handling.

Test Strategy

As a thumb rule, consider creating a test strategy/approach document for any project that has a long life and that is delivered to external customers. If the project size is small or medium (the team has less than 3 designated testers) include the test approach as an appendix into the test plan document itself. If the project is large, you should consider creating a separate document and refer to it from your test plan.

Developing a Good Test Strategy Document

Each project is different and so should be your test strategy document. Do not just blindly follow an available template or use an existing document. Review each of the sections to check whether they are valid in the current project context.

Some Tips for Writing a Strategy Document

  • Do not blindly follow an existing template.
  • Include a project background and intended users in the overview section. This will help in identifying issues and prioritizing tasks that need to be taken care of from high level perspective.
  • List out the test types that will be used for your project. - Unit Tests, Functional Tests etc. For each test list who owns the test and what are the entry and exit criteria.
  • Usually the test team develops the test strategy document with inputs from the project manager. However, the document belongs to all stakeholders. Do mention and describe the validation efforts handled by development team as in Unit Tests.
  • Testing strategies should be reviewed by the developers and all the test leads for all types of testing. Ensure that there is complete coverage of the application with not much overlapping among the test types. The test strategy document should get a sign-off from both the test manager and development managers before testing starts.
  • The test strategy document should answer How the testing is performed. List out the tools that are used. Are you going to automate tests? How testing is conducted? Is it integrated with the Continuous Integration (CI) servers? All these topics should be discussed and mentioned in the strategy document.
  • Discuss the modalities of a test life cycle. Are you going to use a Test Management System(TMS)? Are you going to use a Bug Tracking tool? Who can open or close a bug? Clearly mention the roles and responsibilities of team members.
  • Discuss how the testing progress tracked? Is code coverage being used as a metric? Do you perform a triage?
  • List the artifacts produced during the test process. Clearly mention the versioning of the artifacts. If they are live artifacts mention the common server where they will be available.
  • Describe the risk analysis and mitigation/contingency plan.

Sections of a Test Strategy Document

The following is a list of sections that typically exists in a test strategy document. Like mentioned earlier, each project is different. Some sections may not be needed in your document and some extra sections may be needed. This is not a comprehensive list – keep that in mind if you like to use this structure.

Overview and Scope

The overview section discusses the product requirements, the background of the product, the intended users. This section should help in deciding which of the tasks need to be prioritized and which can be ignored.​

Test Types

The test strategy lists and discusses the test types to be performed. There are primarily three types of testing: unit testing, integration testing, and system testing. Usually, the developers are responsible for unit testing. Integration and System testing performed by individual tester(s).

Test Environments

Test Environments is an important part of the strategy document. You need to clearly mention the operating systems and other software like database systems, network stack used (including version numbers and patch levels).

The more clear the environment description, it is that much more helpful in setting up test beds at a later stage.

Roles and Responsibilities

This section details the roles and responsibilities of all the team members. Specifically, the roles and responsibilities of the test leader(s), individual testers and project manager should be clearly defined.

Risk Analysis

Document all the risks that effects the testing process in this section. A risk that is documented can be anticipated. Either it can be prevented or reduce the damage to a significant extent by working on the mitigation strategy.

Risks can either be associated with a delay of the upfront processes like requirement, design or coding stages. It might also be associated with the limitations of the tools, support availability of the testing environments etc.

Testing Approaches

You should document various testing approaches in this section. A testing approach for a new feature differs from the one for a regression. For each of the test types - mention how the responsible team approach creating/modifying a test.

Tools and Techniques

The tests can be either manual or automated. In this section, define test management and automation tools required for test execution. If planned for describe the test approach and tools required for performance, load, security testing. Do list out the approach for UX and UI testing.

Test Groups and Priorities

This section should detail how tests of each type are grouped together. If your team uses Behavior Driven Development, this might have been already decided by the testing tool. How the bugs are tackled in what order is decided by the priority of a bug.

Reporting Progress

The senior management will be interested in the summary of the progress. The development team requires a daily progress. Each of the stakeholder requires a different type of report with different details. Note the various requirements and the procedure to generate the reports.

You may also maintain the test records for posterity - you need to mention where they are stored and how long they are kept.

Some Concluding Remarks

Test Strategy is not just a document. It embodies the wisdom required for executing a successful test process. Refer to the document while the development is taking place. Ensure that all the stakeholders are aware of its existence and they can access it easily.

In most cases the test strategy document is static. Once it is created it is not modified till the completion of the project. However, there are times when you need to modify the same. For example, your test environment might have changed. Or you find that the test tool is restrictive. In those cases, modify the document and ensure that all the stakeholders are consulted and made aware of the changes.

A Shout for MarathonITE

Automating Functional Tests with MarathonITE 

Marathon Integrated Testing Environment – MarathonITE, is an affordable, easy-to-use, cross-platform test automation tool for Java/Swing™, Java/FX™ and Web applications. Using MarathonITE you can quickly automate your daily tests.

Close Menu