Software Testing Documentation: Overview

Ok, we didn`t really want it but seems like it is high time to dwell upon not that joyful stuff. Our team has never been into bureaucracy, yet documents remain crucial for our cooperation with clients globally. In particular, we realized the testing artifacts are often underestimated, although poorly created, their effect on project flow is disastrous. That`s the reason why we devote today`s post to the testing documentation overview and the role of each for your product success.

Test Plan

Probably, one of the most comprehensive reports you`ll get when outsourcing QA services. Here you`ll find the outline describing software scope and activities, the way QA engineers test a particular product together with the risks related. You may consider the plan of high quality if it includes:

• What to test? The first part outlines the task communicated with the client. It contains a project description with all the details covered: system specifics, application, hardware.
• Which software parts will undergo testing? This section represents a list of functions and system description (each separate component) that the QA team is going to check. What you`ll get here is a kind of task description for team members to work on along with the development.
• How will the project be tested? Here QA engineer provides with the fullest description of the software testing process. You`ll find types of testing QA experts choose to check the features/functions from the previous section. You`ll know the ways QA specialists apply the selected tests to the system and understand how things are going to work with your product.
• When to test? This part schedules working sequence. The section defines the time frame for preparation, the testing process itself, test result analysis along with the project development iterations.
• Defying starting criteria for testing. QA engineers begin if the testing platform is ready, the required function is developed, all the necessary documentation is available.
• Defying exit criteria in software testing. If the project meets the requirements for the number of open bugs available and if the testing results meet the product quality goals, QA engineers consider the testing completed.

Bug Report

A technical document that requires the use of specific terminology. The report contains a full description of defects, severity, and priority classified, as well as the conditions creating the problems.

A typical report contains Bug Summary, Severity/Priority, Steps to Reproduce, Actual Result, and Expected Result. The most important is, however, the way a QA engineer combines all the parts and makes the bug description clear for a developer to fix the issues. Our team recommends answering three questions when delivering a useful bug report to the developers.

• Where? QA engineer indicates the problem be it user interface or software architecture defect.
• What? QA expert defines the event inconsistent with the operational requirements of the software product.
• When? Place, event, conditions that influence the software and, therefore, trigger buggy behavior of the system.

Besides, the report will classify the software defects per Severity and Priority criteria. Often confused, yet these characteristics stay at the core of every bug description.

Severity is an attribute that characterizes the impact a defect has on the software behavior.

• S1-Blocker. A blocking error entirely hinders software functioning, making it impossible to use. The solution to the problem is critical for further project growth.
• S2-Critical. A critical defect refers to incorrect functioning of a particular software area. Drawbacks in security, temporarily crashed server, complete failure of a feature, unsuccessful app installation are the key examples of critical bugs.
• S3-Major. A significant error affects major functionality or data. The error might be not critical, sometimes, there are other input points to use the system.
• S4-Minor. A minor error that does not hinder product flow, often minor issues refer to the inconsistencies in the user interface.
• S5-Trivial. A trivial error isn`t always evident even through the user interface. It is a problem of third-party libraries or services, an error that does not affect the overall quality of the product. E.g. grammar, spelling mistakes, some layout discrepancies.

Priority, by contrast, indicates urgency or importance of fixing an error. The higher the priority, the faster a developer has to fix the defect. Though a software tester may set the criterion initially, it is a project/product manager who takes a final decision here.
The category falls into the following levels:

• High (P1). The error should be fixed as soon as possible, critical to the project.
• Medium (P2). The error must be corrected, although not critical, requires an urgent solution.
• Low (P3). Not critical, may or may not be fixed at all.

Priority is quite a subjective decision, always determined by business needs, defect severity, available resources in the team to fix and perform regression tests, deadlines, etc. Yet the task of your product manager is to carefully manage bug priority to follow smooth software development.

Test Case & Test Scenario

At the test design stage, QA engineers create tests relevant to the initial software requirements:

• analysis of existing design artifacts: documentation (specifications, requirements, plans), models, etc.
• test design specification
• creation of test cases

Test Scenario is a one-line statement that describes the area in the software to be tested. Product complexity usually defines the number of scenarios each area might have: from one to a few hundred. The term is often confused with a test case, although scenarios always require several steps to perform. In other words, a test scenario includes several test cases with the sequence they are executed.

Test Case is a series of simple steps, conditions, and inputs required to check a particular functionality. The key intent of writing a test case is to show whether the software passes or fails its functional requirements. The following structure is typical for a test case:

• Action taken by a QA engineer (e.g. open page “login”)
• Expected Result – the required behavior of the software (e.g. “login” page opens)
• Test Result – showcases the actual software behavior (passed/failed/blocked)

Checklist

Often prepared at the beginning, this is a list of tests to run on a particular software part. We complete a checklist during the testing process, include the scenarios we checked and describe the results. You`ll get it after the testing process is finished for your development team to fix the detected errors.

This is it. Now you know the types of docs QA team is to prepare while working on your project. We advise requesting all the above to make sure QA engineers check your software properly.

If you managed to read it till the end, you`re one of those lucky to get our best wishes to your project growth 🙂