Browse Author: Abdul Shaikh

TestRail-Test Case Management Tool.

TestRail is a comprehensive web-based test case management software to efficiently manage, track and organize software testing efforts. Its intuitive web-based user interface makes it easy to create test cases, manage test runs and coordinate the entire testing process. Track and follow the status of individual tests, milestones and projects with dashboards and activity reports. Get real-time insights into the testing progress and boost productivity with personalized todo lists, filters and email notifications.

Getting started with TestRail

TestRail allows team members to enter test cases, organize test suites, execute test runs and track their results, all from a modern and easy to use web interface. Teams benefit from four main advantages when adopting TestRail:

  • Manage test cases, suites and test runs
    The traditional way of managing and documenting test cases on paper, in wikis, bug tracking tools or general document management systems quickly reaches its limits. TestRail enables you to create, manage and organize test cases and suites within an optimized user interface and application structure.
  • Boosting testing productivity
    TestRail’s todo lists, filters and email notifications help coordinate test runs and increase the productivity and responsibility of testers. The goal is to make sure that every team member knows his tasks at all times and that team leads can assign new tasks to testers depending on their workload.
  • Real-time insights into your testing progress
    To make important project decisions, it is critical to have access to detailed information about the testing progress and test results. Questions such as “How many percent of a test run have we successfully completed?”, “How are we progressing on the upcoming milestone?” or “Which tests have been executed in the past 24 hours?” can only be answered efficiently by using a comprehensive test case management software such as TestRail, and TestRail makes these details readily available.
  • Organize and track software testing efforts
    The organization and coordination of multiple parallel tests, be it for different projects or releases, is often complex and time-consuming. To not lose track of your testing efforts, TestRail helps you manage important software-specific data and structures such as project milestones and makes it easy to integrate with bug tracking tools.

The dashboard

After logging in to TestRail, the first page you usually see is the dashboard. The dashboard provides you with an overview of available projects, recent project activities and your todos. When you navigate to a project (by clicking on a project title), you switch to the project view and land on the project’s overview page, showing project details such as test suites, active test runs, project activity and so on. Whenever you need to switch to another project, just return to the dashboard by clicking the link in the upper left corner.

Test cases and suites

A test case in TestRail consists of a description of the test’s prerequisites, a list of test steps and the expected result. A test case can ideally be verified by a single tester in a short period of time and confirms a specific functionality, documents a task or verifies a project artifact.

In TestRail, test cases are organized into test suites. A test suite is a collection of test cases and is often created for specific project modules or areas. How you organize your test suites depends on the size of your project. If you have a lot of test cases for a project module, it is recommended to further break down your test suites and create suites for specific features or functionality. Ideally, a test suite consists of 50-1000 test cases.

Test runs and tests

To run a test and enter test results for the cases you added, you start a test run for a particular test suite. While a test suite is just like a plan that specifies how an application is tested, a test run is an actual test you are conducting.For most projects you will most likely start multiple test runs for a particular test suite over time. For example, if you are releasing multiple versions of a software program, you might want to conduct a test run for each new release. Likewise, you can have multiple test runs for a particular test suite active at the same time. This can make sense if you want to execute a particular test suite for multiple configurations (such as different operating systems). You can then start a test run for each different configuration you want to test against.

The following test statuses are available by default:

  • Untested
    By default, new tests have the status Untested. Once a test result has been added to a test, it can never receive the Untested status again.
  • Passed
    A test is marked as Passed when a tester verified the test steps and the expected results.
  • Failed
    A tester marks a test as Failed if one of the specified test steps resulted in an error or if the expected result differs from the actual test result.
  • Retest
    If a developer or tester determines that a test should be tested again, she marks the test as Retest. E.g., if a test originally failed and the developer fixed the problem, the developer can mark it to be retested.
  • Blocked
    The Blocked status is used to signal that a test cannot be executed currently because of some external dependency (such as a bug that needs to be fixed before being able to complete to test). It is often used in combination with the Retest status.

Projects and milestones

Projects are the main organizational unit in TestRail. It is generally recommended to create a TestRail project for every real software project you want to manage within TestRail. All other data such as test suites, runs, test results, milestones etc. are directly associated with a specific project. See the following illustration to see how the different entities relate to each other:

 

You can also add your project milestones to TestRail. A milestone can be any project milestone you deem to be important, such as a planned public software release, an internal test version or a new beta release for an important customer. Once you have added your milestones to TestRail, you can assign test runs to specific milestones

Test plans and configurations

When you need to manage multiple test runs and configurations for a single project, TestRail’s test plans come in handy. A test plan allows you to start multiple test runs at once, either if you have many test suites or if you want to test against multiple configurations (where a configuration can be anything you need to test your project against, such as different operating systems or web browsers).

 

To create a test plan, simply click the Add Test Plan button from the Test Runs & Results page. Now just select one or more test suites from the sidebar to add them to the plan. Just like you do with a single test run, you can configure all properties of the test runs such as the name, the cases you want to include and so on.

 

 

To dos and notifications

Tests can be assigned to team members. You can either assign tests when a run is created or do so afterwards from the run or test pages. Once a test has been assigned to a user, the test appears on the user’s todo list. Every user has its own todo list for each project she’s working on, while the Dashboard provides a quick way to see the user’s todos for all projects. The todo page can also be used to view the todos of other team members or the entire team at once, providing an easy way for team leads to assign tests to team members with the least workload.

 

Another relevant feature are email notifications. Email notifications help test owners to learn about test changes made by other users. For example, if a test has been assigned to a user (i.e. the test owner) and another user adds a comment or result to the test, the test owner is notified by email. It is also possible to subscribe to tests of other users or even entire runs by clicking the small email icons on the relevant pages.

Tracking progress and activity

Besides making it easier to manage test suites and enter test results, providing an easy way for all team members to track the test progress is one of TestRail’s most useful features. You can view the test results and testing activity of runs, milestones and entire projects on the individual resource pages.

 

You can choose between the status, activity and progress view on the test run and milestone pages from the sidebar, while you can switch between the project history and test results on the project’s overview page. Additional reporting features can be found on the Reports tab

Working with Calabash

Hi All,

Now we know how to install and run Calabash and its all requirements and prerequisites.

Now let’s see how to work with it.

Open the “calabash-android-0.9.0” folder. It resides at path C:\Ruby23\lib\ruby\gems\2.3.0\gems\calabash-android-0.9.0. The folder names will change in synch with the ruby/ calabash version you install on your machine.

Open the feature skeleton folder. Look out for this basic framework.

  • The *.feature file contains scenarios that we are going to automate.
  • The method used by the feature file is written in *.rb file inside “step_definitions” folder.
  • Common methods, environment setup, app installation and hooks should be placed inside “support” folder.

Resign & Build the app

  • Calabash-android resign *.apk
  • Calabash-android build *.apk

Attach the device to the system /Open the emulator

Check device attached. Type command >>adb devices

Attached devices list should be displayed.

How to Find the Element Locator

  • Open the console. Type the Command.>>calabash android console “APK Path”>>start_test_server_in_background
  • Above command launch the app on the device. To find the element locator use following command.>query “*”

Creating New Scripts

Open the feature file and following lines

Feature: Login feature
 Scenario: As a valid user I can log into my app
 When I press "Login"
 And I enter my username 
 And I enter my password
 Then I see "Welcome to coolest app ever"

Open the Step Definition file & Define the method into *.rb file.

Given /^I am on the login windows$/ do
 wait_for(:timeout =>100) { element_exists("* id:'loginInput;")}
 check_element_exists("* id:'loginInput;")
end.
Now you can automate your UI testing using the Calabash.

Automation testing Using Calabash

Calabash is one framework that enables automated UI Acceptance Tests written in Cucumber to be run on iOS and Android applications. While Calabash integrates tightly with Xamarin.iOS and Xamarin. Android project, it can also be used with iOS and Android projects written in the indigenous languages of Objective-C and Java.

Calabash is an open source Acceptance Testing framework that allows you to write and execute tests for iOS and Android Apps. It’s an Automated User Interface Framework that allows tests to be written in Ruby using Cucumber. Calabash works by enabling automatic UI interactions within an application such as pressing buttons, inputting text, validating responses, etc. It can be configured to run on different Android and iOS devices, which provides real-time feedback and validations.

Calabash and BDD

  • Calabash is Behavior Driven Development (BDD). It is same as Test Driven Development (TDD), but instead of creating tests to describe the shape of APIs, application behavior is specified.
  • BDD is a process in which multiple stakeholders weigh in to create a common understanding of what has to be built.
  • BDD is helpful in building the right software and designing from the perspective of the business owner.

How to install Calabash

Part I) Install Java JDK – Refer this guide – /install-java.html

Part II) Download and install Ruby.

Step 1) Download Ruby from the URL http://rubyinstaller.org/downloads

Step 2) Open the exe, follow the instructions on screen. Once install is complete you will see the following screen. Click Finish.

Step 3) Verify whether Ruby is installed successfully

  • Start Command Prompt with Ruby on Windows 10
  • Type ruby –v

Part III) Download and install Android

Step 1) Download Android Studio at http://developer.android.com/sdk/index.html

Step 2) Open the exe, follow the on-screen instructions and complete installation. Click the finish button once done

Part IV) Install Calabash Android

Step 1) In the console type install calabash-android. Install will start and will take some time to complete

Step 2) Once installation is done Type calabash-android version 

if if shows calabash-android -0.9.0 then you have successfully installed calabash.