Bianor blog

Me and my colleagues had a task to research advanced and effective tools for test management.

Background
According to our internal Quality Management System, and in particular its section concerning testing activities, we had to fill test cases and results from their execution in Word & Excel format as they were suitable and easily understandable from our customers. The maintenance of these documents was taking significant amount of the time for testing activities on the project.

Goal
The research goal was to find a suitable system, which to allow the QA engineers to create, perform, manage test scenarios in a centralized repository, and to generate reports from the execution of each test in same comprehensive way as our Excel template did. The latter is very important, since it would serve as an evidence for our work to customers.

Result
During the research, four systems for managing the testing process were evaluated and examined: Bugzilla Test Runner, Testlink, TestLog and RTH.

We chose Testlink.

General information
Testlink is a system for management of the testing process, and according to the statistics on the Internet discussion forum for open source software testing tools (http://www.opensourcetesting.org/testmgt.php) it is the most downloaded system with an open source.

The system is platform independent; it is fast and easily installed on a Web server and offers a web based installation. The necessary resources for configuration of Testlink are: MySQL 4.1.x + , PHP 4.3.x + Apache 1.3.x or 2.0.x + or IIS 3 and higher, etc.

Testlink provides user-friendly interface, which can be accessed through different web browsers (Mozilla, Firefox, IE).

The system offers tools palette, similar to the Formatting palette on Microsoft Word, thanks to which all created documents, can be formatted suitably.

Almost every page of Testlink contains convenient static help, thanks to which the users can receive information for the respective functionality.

Account management
With Testlink, users can easily create accounts with different access levels to the system, respective project and a test plan (administrator, QA Lead, tester, etc.). It is possible to configure different user roles and to choose access rights for the respective role.

Functionalities
Testlink offers an opportunity to deactivate different components of the system – project, test plan, test scenario, users, build. For example, if a given test scenario is deactivated, it will not be included in the testing plan and could not be executed.

The system provides a separate module for managing customers’ requirements. Each customer requirement can be associated to one or many test scenarios. At the end of the testing, QA engineers will have a status report for the tested requirement.

In Testlink you can easily manage your test scenarios – different versions of the test scenarios can be created and merged in Test Suite. Users could also copy, move, arrange and search scenarios in the Test Suite. All scenarios can be printed or saved in HTML and Word format and you could also export/import them from XML file. You can attach different files to scenarios, suites, and requirements. Another thing that I liked is that you can paste texts from Word and thus continue following the format of your test scenarios, created up to this moment in MS Word. By default, the scenario in Testlink can have the following status during execution time: Pass, Fail, Not Run and Blocked.

During the testing, you could generate different reports for the status of the build. The reports can be saved in MS Word and in HTML email format, which is automatically sent to the user via e-mail. Except for these reports, you can generate also graphic reports. The graphic reports are generated for the whole project and the only disadvantage is the format of the generated file (swf). Possible solution is these reports to be saved as screenshots subsequently stored on the file system.

Testlink offers integration interfaces to different defects control systems - Bugzilla, Mantis, Eventum, Jira. That way, each defect found can be linked directly to the respective test scenario and become easily accessible. Testlink offers also a report for the number of defects found in each scenario, as well as detailed information about each of the found problems.

The system supports configuration of keywords, Custom fields and Milestones.

Flow
1. The first step after you log into the system is to create a test project. The system allows editing and deleting a project. If you don’s see your test project when you login to the system, make sure you have appropriate project user role assigned.

2. After you already have a project in Testlink, you need to create a test plan. The system offers editing, erasing and deactivating of test plans and test plan versioning. The test plan can be created from already existing test plan in the project. An option for assigning test scenarios for execution from different users is offered.
All test scenarios included in the test plan can be saved in MS Word or HTML format.

3. After you have created the test plan and the scenarios, the next thing is to create builds (equivalent as version of your development builds). Once your build is created you are ready to execute all test cases assigned to the current test plan.

Disadvantages
As a weakness in Testlink I had to mention the insufficient information for integration of the system with instruments for automated testing and performance testing. There is little information about possible integration with TestComplete but I haven’t investigated that further. In one of my next posts I would try to share more on that topic.

Another weakness of the instrument is that, it lacks capability to send notifications at different events.

During the evaluation of the system, I came to the conclusion that the simultaneous execution/update of one test scenario from several testers is not recommended. If more than one QA is working on your project, it is better to assign the scenario to a particular QA. If you have to update already executed scenario, my suggestion is to create a new version of the test plan.

—
In conclusion, I can say that Testlink is one of the best free test cases management systems that could be found nowadays. There is a lot of information on the Internet and it has its own website (http://testlink.org/wordpress/). Some demo, numerous documentations and system for management of defects found in the system can be found here: http://www.testlink.org/mantis/main_page.php.

Testlink is suitable for QA engineers, which have not installed and administered up to now any test process management systems. You will save a lot of time with it, which you normally use for creation of documents and reports.

Ask your favorite administrator to help you in the configuration of Testlink and action! ☺