TestRail is a comprehensive web-based test case management software to efficiently manage, track and organize software testing efforts. Its intuitive web-based user interface makes it easy to create test cases, manage test runs and coordinate the entire testing process. Track and follow the status of individual tests, milestones and projects with dashboards and activity reports. Get real-time insights into the testing progress and boost productivity with personalized todo lists, filters and email notifications.
Getting started with TestRail
TestRail allows team members to enter test cases, organize test suites, execute test runs and track their results, all from a modern and easy to use web interface. Teams benefit from four main advantages when adopting TestRail:
-
Manage test cases, suites and test runs
The traditional way of managing and documenting test cases on paper, in wikis, bug tracking tools or general document management systems quickly reaches its limits. TestRail enables you to create, manage and organize test cases and suites within an optimized user interface and application structure.
-
Boosting testing productivity
TestRail’s todo lists, filters and email notifications help coordinate test runs and increase the productivity and responsibility of testers. The goal is to make sure that every team member knows his tasks at all times and that team leads can assign new tasks to testers depending on their workload.
-
Real-time insights into your testing progress
To make important project decisions, it is critical to have access to detailed information about the testing progress and test results. Questions such as “How many percent of a test run have we successfully completed?”, “How are we progressing on the upcoming milestone?” or “Which tests have been executed in the past 24 hours?” can only be answered efficiently by using a comprehensive test case management software such as TestRail, and TestRail makes these details readily available.
-
Organize and track software testing efforts
The organization and coordination of multiple parallel tests, be it for different projects or releases, is often complex and time-consuming. To not lose track of your testing efforts, TestRail helps you manage important software-specific data and structures such as project milestones and makes it easy to integrate with bug tracking tools.
The dashboard
After logging in to TestRail, the first page you usually see is the dashboard. The dashboard provides you with an overview of available projects, recent project activities and your todos. When you navigate to a project (by clicking on a project title), you switch to the project view and land on the project’s overview page, showing project details such as test suites, active test runs, project activity and so on. Whenever you need to switch to another project, just return to the dashboard by clicking the link in the upper left corner.
Test cases and suites
A test case in TestRail consists of a description of the test’s prerequisites, a list of test steps and the expected result. A test case can ideally be verified by a single tester in a short period of time and confirms a specific functionality, documents a task or verifies a project artifact.
In TestRail, test cases are organized into test suites. A test suite is a collection of test cases and is often created for specific project modules or areas. How you organize your test suites depends on the size of your project. If you have a lot of test cases for a project module, it is recommended to further break down your test suites and create suites for specific features or functionality. Ideally, a test suite consists of 50-1000 test cases.
Test runs and tests
To run a test and enter test results for the cases you added, you start a test run for a particular test suite. While a test suite is just like a plan that specifies how an application is tested, a test run is an actual test you are conducting.For most projects you will most likely start multiple test runs for a particular test suite over time. For example, if you are releasing multiple versions of a software program, you might want to conduct a test run for each new release. Likewise, you can have multiple test runs for a particular test suite active at the same time. This can make sense if you want to execute a particular test suite for multiple configurations (such as different operating systems). You can then start a test run for each different configuration you want to test against.
The following test statuses are available by default:
-
Untested
By default, new tests have the status Untested. Once a test result has been added to a test, it can never receive the Untested status again.
-
Passed
A test is marked as Passed when a tester verified the test steps and the expected results.
-
Failed
A tester marks a test as Failed if one of the specified test steps resulted in an error or if the expected result differs from the actual test result.
-
Retest
If a developer or tester determines that a test should be tested again, she marks the test as Retest. E.g., if a test originally failed and the developer fixed the problem, the developer can mark it to be retested.
-
Blocked
The Blocked status is used to signal that a test cannot be executed currently because of some external dependency (such as a bug that needs to be fixed before being able to complete to test). It is often used in combination with the Retest status.
Projects and milestones
Projects are the main organizational unit in TestRail. It is generally recommended to create a TestRail project for every real software project you want to manage within TestRail. All other data such as test suites, runs, test results, milestones etc. are directly associated with a specific project. See the following illustration to see how the different entities relate to each other:
You can also add your project milestones to TestRail. A milestone can be any project milestone you deem to be important, such as a planned public software release, an internal test version or a new beta release for an important customer. Once you have added your milestones to TestRail, you can assign test runs to specific milestones
Test plans and configurations
When you need to manage multiple test runs and configurations for a single project, TestRail’s test plans come in handy. A test plan allows you to start multiple test runs at once, either if you have many test suites or if you want to test against multiple configurations (where a configuration can be anything you need to test your project against, such as different operating systems or web browsers).
To create a test plan, simply click the Add Test Plan button from the Test Runs & Results page. Now just select one or more test suites from the sidebar to add them to the plan. Just like you do with a single test run, you can configure all properties of the test runs such as the name, the cases you want to include and so on.
To dos and notifications
Tests can be assigned to team members. You can either assign tests when a run is created or do so afterwards from the run or test pages. Once a test has been assigned to a user, the test appears on the user’s todo list. Every user has its own todo list for each project she’s working on, while the Dashboard provides a quick way to see the user’s todos for all projects. The todo page can also be used to view the todos of other team members or the entire team at once, providing an easy way for team leads to assign tests to team members with the least workload.
Another relevant feature are email notifications. Email notifications help test owners to learn about test changes made by other users. For example, if a test has been assigned to a user (i.e. the test owner) and another user adds a comment or result to the test, the test owner is notified by email. It is also possible to subscribe to tests of other users or even entire runs by clicking the small email icons on the relevant pages.
Tracking progress and activity
Besides making it easier to manage test suites and enter test results, providing an easy way for all team members to track the test progress is one of TestRail’s most useful features. You can view the test results and testing activity of runs, milestones and entire projects on the individual resource pages.
You can choose between the status, activity and progress view on the test run and milestone pages from the sidebar, while you can switch between the project history and test results on the project’s overview page. Additional reporting features can be found on the Reports tab
