Software Test Plan (STP)- describes plans for qualification testing of Computer Software Configuration Items (CSCIs) and software systems. It describes the software test environment to be used for the testing , identifies the tests to be performed , and provides schedules for test activities.

There is usually a single STP for a project . The STP enable the acquirer to assess the adequacy of planning for CSCI and , if applicable , software qualification testing

Components of STP

Scope

• Identification
• System overview
• Document overview

Referenced documents

Software test environment

• Name of test site(s)

Test identification

• General information
• Planned tests

Test schedules

Requirement traceability

Notes

These pages provide some guidelines that will ease your way through the software testing process. Our suggestions are general, so you can adapt them to the needs of your development environment.

Although testing is described here as a linear process, it is clear that more than one person is involved. Some of the steps in the process are managerial (defining testing goals, performing analysis at different stages), and some are more technical (creating and running tests).

Many quality experts believe that planning is the most important stage of the testing process. With a good test plan, you can assess the quality of your application at any time.

The planning process consists of five main stages.

The first step in the software testing process is to define your testing goals and outline the general strategy you will use to achieve them. This important step is key to a successful testing endeavor.

Your testing goals should answer three basic questions: what to test, how, and by whom?

What to test?

• What are the major features of the Application Under Test (AUT)?
• What are your testing priorities, based on risk analysis? What is the relative importance

of each element of the software functionality?

• What are your overall quality goals (usability, functionality, reliability, performance, etc.)?

How to test?

• Which testing techniques will be used? You can find information about testing techniques in many testing books. Here are some examples:

System tests to test the system as a whole across functional areas.

Stress tests to test how the software reacts under repeated execution of the

same operations.

Installation to test that the system can be installed smoothly

Security to test access to the system without authorization

Boundary tests to exercise code at the data and system boundaries.

Performance and load tests to test the performance of the system under

load conditions.

• How will defects/bugs be handled (severity classification, authorization to

open/close bugs)?

Who/When?

• What are the resources needed (human resources, hardware)?
• When will the various tasks be completed?

Since the typical application is too large to test as a whole, it is necessary to break it down into more manageable units. This breakdown can be extended several levels into sub-systems, and should be based on the testing goals and the functional units of the application.

To help determine how to break down your application into test subjects, you can look at the user interface of the application, or refer to system specifications or business requirement documents. But in most cases, a familiarity with your application will suffice.

During this stage, you further break down each test subject and sub-subject into individual tests (or test cases). Each test should have a distinct objective, such as to verify a specific function or system requirement.

The tests you define should be based on the testing goals you determined at the beginning of the process. You can derive tests by examining various design documents, such as data-flow diagrams (DFDs), entity relationship diagrams (ERDs), structure charts, functional specifications, business requirements, or user documentation.

If no formal documentation exists, you can collect input from subject matter experts, developers, and project leaders. As a last resort, you can perform reverse engineering and write system specifications by examining the software.

At this point, you design steps for each test you defined in the previous stage. Test steps provide detailed, step-by-step instructions on how to perform the test: the actions to be per-formed, input to be entered, and the expected output.

Example: For the test, you could design the following test steps:

Step	Input	Expected Output
1	Open the Login window.	Login window appears
2	Type "bart" in the Agent Name field, "mercury" in Password field, and Press OK.	Login successful. Main window of application appears.
3	Type "bart" in the Agent Name field, "abcd" in Password field, and Press OK.	Login fails. Error message appears.

By now you have completed a test plan and test design. Using the test steps you can begin manual test execution. During this final planning stage, you automate the tests using WinRunner.

Following are some guidelines as to which tests to automate, and how to assign testers to automation tasks:

• Frequency of test execution. It pays to automate tests that will be executed repetitively with each release of the software-for example, sanity-level tests that check the basic functionality of the application.
• Volume of input data. Tests which involve inputting large volumes of data (such as data-base operations) will benefit from automation.
• Length of execution time. Tests that take a long time to perform may need to be run during breaks or overnight. Automating these tests maximizes the use of your time.
• Data Driven vs. recording/UI tests. Within your testing group, the more technically-oriented testers should be assigned to developing the complex, data-driven automated tests. Testers that are less technically oriented should concentrate on developing user interface tests, using recording and point-and-click verification.

• Analysis and validation of your test plan is an ongoing task that you should perform through-out the testing process.
• Validating your test plan means checking how well it meets the goals that you defined at the beginning of the process. For instance, do the test cases actually test the highest risk areas? Do they include enough 'negative' types of input?
• If you're a QA manager, you may also want to examine the progress of test development, and the division of the work load among testers.

Test Execution is the heart of the testing process. Each time your application changes, you will want to execute the relevant parts of your test plan in order to locate defects and assess quality.

The execution process consists of three main stages. Click on one of the blocks in the flow chart below, or just move on to the next page to read on.

During this stage you decide the subset of tests from your test database you want to execute.

Usually you do not run all the tests at once. At different stages of the quality assurance process, you need to execute different tests in order to address specific goals. A related group of tests is called a test cycle, and can include both manual and automated tests

Example: You can create a cycle containing basic tests that run on each build of the application throughout development. You can run the cycle each time a new build is ready, to determine the application's stability before beginning more rigorous testing.

Example: You can create another set of tests for a particular module in your application. This test cycle includes tests that check that module in depth.

To decide which test cycles to build, refer to the testing goals you defined at the beginning of the process. Also consider issues such as the current state of the application and whether new functions have been added or modified.

Folloare examples of some general categories of test cycles to consider:

• sanity cycle checks the entire system at a basic level (breadth, rather than depth) to see that it is functional and stable. This cycle should include basic-level tests containing mostly positive checks.
• normal cycle tests the system a little more in depth than the sanity cycle. This cycle can group medium-level tests, containing both positive and negative checks.
• advanced cycle tests both breadth and depth. This cycle can be run when more time is available for testing. The tests in the cycle cover the entire application (breadth), and also test advanced options in the application (depth).
• regression cycle tests maintenance builds. The goal of this type of cycle is to verify that a change to one part of the software did not break the rest of the application. A regression cycle includes sanity-level tests for testing the entire software, as well as in-depth tests for the specific area of the application that was modified.

Once you have created cycles that cover your testing objectives, you begin executing the tests in the cycle. You perform manual tests using the test steps. WinRunner executes automated tests for you. A test cycle is complete only when all tests-automatic and manual-have been run.

With Manual Test Execution you follow the instructions in the test steps of each test. You use the application, enter input, compare the application output with the expected output, and log the results. For each test step you assign either pass or fail status.

During Automated Test Execution you create a batch of tests and launch the entire batch at once. WinRunner runs the tests one at a time. It then imports results, providing outcome summaries for each test.

You analyze and validate test results after every test run. You want to identify all the failed steps in your tests and to determine whether a bug has been detected, or if the expected result needs to be updated.

Locating and repairing software bugs is an essential part of software development. Bugs can be detected and reported by engineers, testers, and end-users in all phases of the testing process. Information about bugs must be detailed and organized in order to schedule bug fixes and determine software release dates.

Bug Tracking involves two main stages: reporting and tracking. Click on one of the boxes below, or turn the page to read on.

.

Once you execute the manual and automated tests in a cycle, you report the bugs (or defects) that you detected. The bugs are stored in a database so that you can manage them and analyze the status of your application.

When you report a bug, you record all the information necessary to reproduce and fix it. You also make sure that the QA and development personnel involved in fixing the bug are notified.

The lifecycle of a bug begins when it is reported and ends when it is fixed, verified, and closed.

• First you report New bugs to the database, and provide all necessary information to reproduce, fix, and follow up the bug.
• The Quality Assurance manager or Project manager periodically reviews all New bugs and decides which should be fixed. These bugs are given the status Open and are assigned to a member of the development team.
• Software developers fix the Open bugs and assign them the status Fixed.
• QA personnel test a new build of the application. If a bug does not reoccur, it is Closed. If a bug is detected again, it is reopened.

Communication is an essential part of bug tracking; all members of the development and quality assurance team must be well informed in order to insure that bugs information is up to date and that the most important problems are addressed.

The number of open or fixed bugs is a good indicator of the quality status of your application. You can use data analysis tools such as re-ports and graphs in interpret bug data.

Software Test Plan (STP)

Related Documents:

Changes List:

Scope

• Identification
• System Overview
• Document Overview

Software Test Environment

• Test Site(s)
• Required Equipment
  • Software Items
  • Hardware Items
  • Personnel
  • Orientation Plan
• Tests to be Performed

Test Identification

• General Information
  • Test Classes
  • General Test Conditions
  • Test Progression
• Planned Tests

Test Schedules

• Where/When
• Test Site Activities

Requirements Traceability

Scope

Identification

The software to be tested is Import Utility version x.x. For itemized testing description, please refer to paragraph 3.1.2 of this document.

This test plan relates to the application known as Import Utility version x.x. Testing will be performed for both the planned new functionality of the release as well as maintenance issues.

System Overview

Purpose

The Software Test Plan (STP) describes plans for qualification testing of software. It describes the software test environment to be used for the testing, identifies the tests to be performed and provides schedules for test activities.

Functionality

Import Utility's functionality is described in the functional design specification (see Related Documents section of this document).

Structure

Import Utility is a stand-alone application which uses

Document Overview

This document describes plans for qualification testing of Import Utility version x.x. It describes the software test environment to be used for the testing, identifies the tests to be performed and provides schedules for test activities.

The document is originally intended for internal review and guidance, but may be shown to personnel if authorized by one of the individuals designated as an "authorizer" on the title page.

Software Test Environment

This section describes the software test environment at each intended test site. Reference may be made to the High Level Design Document (HLD) for resources that are described there.

Test Site(s)

All testing will take place in the QA department.

Required Equipment

Software Items

Hardware Items

Personnel

During the test period, one QA Engineer will be required.

Orientation Plan

Prior to the start of full-scale testing, the functional spec describing the new functionality will be distributed. A handoff meeting between Development and the QA/Documentation group is scheduled to demonstrate functionality and to answer any questions which may arise. The QA Engineer will also be given a review of the testing procedure and batch run instructions beforehand.

Test Identification

The list of WinRunner tests to be performed is located in Appendix A of this document.

The descriptions of all other tests can be found in the Software Test Description document (see Related Documents section on cover page for location).

General Information

Test Classes

General functionality tests and erroneous input tests will be performed.

General Test Conditions

Tests must be performed for three operating environments: Windows for Work Groups, Windows 95 and Windows NT

The following table describes the testing matrix:

RDBMSs	Oracle
Operating Systems	Windows 95
Servers	Novell NT

Test Progression

The test include the following main steps:

• Select Module for mapping and flag the checkbox "Is For Import" in Admin for the Module
• Create header of CSV file
• Generate the testing data for CSV file
• Complete Import Utility
• Check the new data in the database

Planned Tests

Here is more detailed scheme for the testing:

1. Choose the database profile ( Oracle, SQL Any MU or SQL Any Local)
2. Choose themodule (e.g. Customers, Vendors, Employees etc.)
3. Enter to Admin Application with the same profile, select Modules option from Topics Menu Item, find the name of the Module in the list and flag the "Is For Import" checkbox
4. Choose the type of field separator ( comma or tab)
5. Choose the fields for the header of CSV files from or using Generate Header Option of Import Utility.

For example, for Warehouses Module it possible to choose the following columns: node_id,is_tracked_inventory,is_sa,sa_person_id,rb_stock_type,rb_receive_ type,is_allow_trans,descr,warehouse_type_id,address_id,is_primary,inv_type_id,warehouse_id

The full list of Modules, their mandatory fields and additional fields for testing see Appendix A.

The list of columns is separated by a defined character ( comma or tab ).

Repeat the test when selecting different set of header columns:

	Description	Input Values	Expected Results
1.	Select only the mandatory columns	Select the columns which are listed in the right box	Shouldn't be any problem
2.	Select mandatory and additional columns of different types (checkbox, text, DDDW etc.)	Drag the columns which were from the very beginning in the left box and which are of different types	Shouldn't be any problem

1. Generate the records (about 1000), according to the created header to the CSV file. For example, for the following header

node_id,is_tracked_inventory,is_sa,sa_person_id,rb_stock_type,rb_receive_

type,is_allow_trans,descr,warehouse_type_id,address_id,is_primary,inv_type

_id,warehouse_id

the data could be:

BALTIMORE,Y,Y,,0,0,Y,Gary Ruckle's Mobile arehouse,sa,1634,N,all,gary

TPMS-LOG,Y,N,,0,0,Y,TPM Service Main ,CENT,1544,N,all,TPMS-MAIN

The sequence is very important and empty values should be represented by close separators;

Generate the data for testing the following situations:

	Description	Input Values	Expected Results
1.	Import of duplicate records	Type two or more the same rows in the file	Utility should return warning or error message during Import process
2.	Import the data, which length is more than it is defined in the database	Type a data that takes more characters than it should be

1. Choose one of the Import mode

• New Records Only - when selected, adds new record only to the database, existing data is not overwritten;
• New and Update Records - when selected, adds new record to the database and also updates existing once;

1. Complete the Import process, paying attention to all the messages that should appear;
2. Check the new data in the current database

Test Schedules

Where/When

The testing will take place at QA department on 1997

Test Site Activities

A schedule for each test site depicting the activities and events listed below (as applicable) in chronological order:

On-site test period assigned to major portions of the testing

Pretest on-site period needed for setting up the software test environment and other equipment, system debugging, orientation, and familiarization

Collection of database/data file values, input values, and other operational data needed for the testing

Conducting the tests, including planned retesting

Preparation, review and approval of the Software Test Report (STR)

Requirements Traceability

By now we have the following requests:

1. "Prepare LISA to import the contract information from the legacy contract system" (see the C:\ Cont.doc ).
2. "While importing the contract, we need the ability to import scheduled work orders" (see the C:\ Cont.doc ).

All the test stands for checking Import Utility functionality and fulfilling the requests, but as far the information that has to be added to the Lisa - type database is absolutely new, so the most related step of the test is checking New Records Only Mode of Import Utility (step 6) and all the situations from the Creating Header ( step 4) and Generating data (step 5).

Appendix A.

Module Name	Columns	Label	Field Type	Mandatory
Employees	employee_person_id employee_node_id person_last_name person_first_name	Employee ID Node Last Name First Name	Varchar2 30 Varchar2 30 Varchar2 40 Varchar2 40	Y Y Y Y
Customers	company_descr main_company_id company_is_site	Name Customer ID Site	Varchar2 60 Varchar2 30 Boolean	Y Y NN