What is Xray’s Test Repository?

The Test Repository in Xray for Jira allows software tests to be organized hierarchically within Jira projects, facilitating efficient management of both manual and automated tests.

The QA team can create folders and subfolders to organize their tests, aligning them either with the functional architecture of the software or with the test strategy defined by the team. This structure is intuitive, since it resembles the way files are organized in any operating system.

What does the Test Repository provide us?

At WATA Factory, we find several advantages in using the Test Repository to improve the organization and quality of our tests:

• Hierarchical structuring of tests: Organize tests into folders and subfolders, making it easier to navigate and locate specific cases. For example, test cases can be segmented by modules, functionalities, versions, or test types.
• Greater visibility and control: The hierarchical organization provides a clear overview of the status and location of each test within the project, improving process monitoring and traceability between requirements, test cases, and executions.
• Compatibility with Agile methodologies and Continuous Integration: Adapts to Agile and CI practices, enabling test management aligned with fast development cycles and continuous delivery.
• Improved collaboration: Functional and technical teams can work in parallel, quickly finding the tests relevant to their scope.
• Scalability: Ideal for large projects with hundreds or thousands of test cases.

Tips to optimize the use of Test Repository

• Design the hierarchy carefully: Plan the folder structure in advance, since each test can only belong to one folder.
• Complement it with Test Sets: While the Test Repository provides a hierarchical structure, Xray’s Test Sets can be used to group tests across projects or based on specific criteria, offering additional flexibility.

Test Repository as a bridge toward automation

Organizing the test suite with Test Repository also serves as a bridge between manual and automated testing.

The QA team can begin by defining manual test cases within the folders of the Test Repository, organizing them according to requirements or acceptance criteria defined in Jira tickets.

As QA executes manual tests through Test Executions, they can identify which ones are most suitable for automation (e.g., repetitive tests, regression tests, or high-risk tests).

At that point, once the candidates for automation have been identified, Xray allows linking automated results to existing Tests through integration with frameworks such as Cucumber, JUnit, or TestNG.

Once automated, the same manual Test can continue to be used to store evidence of execution, simply by marking it as an automated test through the Test Type, while maintaining the same Test Issue in Jira and ensuring traceability.

Test Repository at WATA Factory

At WATA Factory, we consider a tool like Xray’s Test Repository essential for our QA teams. It improves test management in software projects not only by providing order and clarity, but also by making our test suites more scalable and easing the transition toward automation.

The post Test Repository Organization. A Little Order Before Automation appeared first on WATA Factory.

When it comes to defining the software quality and testing basics of a project, especially for web applications, deciding which technologies to use to automate our tests becomes a real challenge. This is even more true when we have to choose a technology to test our front-end.

If we focus on E2E (end-to-end) testing, simulating the behaviour and interaction of a real user by testing the user interface of our project, we find well-known technologies like Selenium, Protractor or Playwright.

But if there is one testing framework that is becoming more and more popular thanks to its ease of use and its advantages over the previous ones, it is Cypress.

What is Cypress?

Cypress is an open source framework for automatic front-end testing of JavaScript-based web applications. Its use is not limited to E2E tests, but also allows the creation of integration or unit tests.

Since it has its own assertion libraries, custom commands and other interesting features (e.g. a dashboard for debugging and running our tests), Cypress has proven to be a versatile framework adapted to the characteristics of the modern web.

All of this is based on an architecture that was built from the ground up: You don’t have to start with Selenium.

This allows us to work with a framework that does not require the installation of external tools and libraries to start building our tests.

Why Cypress?

Due to the development of web applications and the emergence of current and modern JavaScript-based frameworks such as Angular, Vue or React, E2E testing frameworks have also evolved. Considering the versatility and ease of use that Cypress offers, the popularity it has gained since its development is to be expected.

Another interesting feature of Cypress is that this framework runs the logic of our tests in the same execution cycle as the application. Cypress runs a NodeJS process that reacts to the events of our application in real time, without us having to use WebDriver to run our tests.

Cypress also makes it easier for us to develop tests, as you only need basic knowledge of JavaScript to get started. Assertions from other well-known test frameworks such as Chai and Mocha are also used.

Another important reason for using Cypress is, of course, the dashboard. A graphical interface where we can follow the process and execution of our tests in real time, as if we were in the place of the user, so that we can debug and have a direct overview of the errors that occur in our tests.

Relevant features of Cypress

Cypress has many features that are advantageous over other frameworks and that can help us decide to use it as an E2E testing tool for web projects.

1. current state of the framework and learning curve

Cypress is a modern and up-to-date all-in-one framework with good documentation, an active community and continuous updates.

It offers a quick setup of the framework in our project, as installation is fast and very little configuration is required. The tests are written in JavaScript and the Cypress library has descriptive and concise commands.

All this contributes to a flat learning curve for both the development team and the QA team.

2. Integration (or not) with other technologies.

Cypress does not require integration with Selenium because the architecture is built from scratch. In general, it does not need to integrate with other external libraries and frameworks, but we can do so if we want to extend the functionality of the framework with well-known technologies such as Mocha, Chai or jQuery.

It also works with any JavaScript-based library or framework that runs in a browser.

3. interaction with browsers

With Cypress, there is no need to install a browser as the headless browser (no graphical interface) allows for faster and less time-consuming execution. This would eliminate the need to install a browser to run the E2E tests through continuous integration.

On the other hand, we do not need to run a WebDriver in Cypress, as our tests are executed within a browser by a NodeJS process.

4. automatic wait times

When using Cypress, there is no need to insert arbitrary wait times between commands and assertions as this is done automatically, so there is no need to use wait and async in our tests.

5. useful and simple dashboard

Cypress has an easy to use and intuitive dashboard that we can interact with between each test step.

The dashboard provides us with dynamism as there is no need to restart the tests. Changes are automatically executed and are immediately reflected as soon as we save the progress of our file.

6. readable and versatile reports

The results of the console and dashboard tests in Cypress are very descriptive. In addition, there are many options for reporting the results of our tests, as well as the ability to easily take screenshots and record videos.

Cypress at WATA FACTORY

At WATA Factory, we focus on ensuring the quality of our software. Therefore, we consider it important to study well the testing tools we use in our projects, and we believe that the inclusion of Cypress is an up-to-date and optimal choice for testing our JavaScript-based web applications.

The post E2E test automation with Cypress: an elegant and affordable solution appeared first on WATA Factory.

Therefore the use of test management solutions and tools is crucial for the creation of testing plans and strategies for our software.

At WATA Factory, Xray provides us with the optimal tools to structure, organise and log the execution of manual and automated tests in our systems.

What is Xray for Jira?

Xray is a Jira add-on that helps us track the entire test lifecycle: Planning, design, execution and reporting From all this, we get very detailed diagrams and reports as a result. Thanks to this solution, team leads and testers can improve project management (or product development) and workflow by checking the status of requirements and the progress of test runs.

Xray’s integration with Jira allows us to index tests in real time and take control of the entire testing process to achieve full coverage and quickly identify issues. In addition, Xray supports BDD methodology and continuous integration (CI) through technologies such as Cucumber and Jenkins.

What types of issues does Xray offer us?

The integration of Xray with Jira opens up the possibility of creating new types of issues that can be used to define the key stages of the test lifecycle.

Depending on which phase of the test plan we are in, we can use the following types of issues for testing:

Test Planning

• Test Plan. Enables better management of our tests and provides us with a table of all tests added to the test plan.

Test Designing

• Precondition. Helps to define the conditions that must be met for a test to be performed.
• Test. This is the main issue to create our tests as a sequence of steps, paired with the conditions, inputs and expected outcome of the test.
• Test Set. Enables the organisation of a series of tests.

Test Execution and Reporting

• Test execution. This issue enables the monitoring of a set of tests to check whether said tests work as expected in a certain context and environment. This issue also creates, when running the tests, the test reports that show the requirements coverage of our project.

How is Xray used?

The following steps are usually followed to create a test environment for an Xray project:

1. Test Specification

Three different types of tests can be created with Xray: Manual, Generic and Cucumber.

With Manual tests in Xray, one manually defines the list of individual steps, formulates a description of the test, specifies the software version and reports the results.

The creation of automatic tests in Xray can be done with Generic and Cucumber tests.

While Generic tests allow us to create unstructured and step-less tests, Cucumber tests can be defined in a way that is understandable to both the technical team and the non-technical business side, based on Behaviour Driven Development (BDD) with the Gherkin language.

It should be noted that Generic tests can also be used for manual testing (e.g. exploratory testing).

2. Organisation of tests

The organisation of tests is an important step in managing the testing environment of our project, as the number of tests usually increases significantly. A good organisation will allow us to find each test in an efficient way.

With Xray, we organise our tests with the Issue Test Set and/or using Test Repository.

The Issue Test Set allows us to create test lists grouped by logic, which can be used as the basis for creating Test Executions or Test Plans. We can create multiple combinations of Test Sets depending on the requirements of our project’s test environment.

Using the Xray Test Repository helps us to organise our tests based on a hierarchical tree structure of folders and subfolders, which provides a better overview of our tests and their execution.

3. Planning of tests

The test planning phase is crucial for defining a good test strategy in our project based on certain aspects, such as the requirements we want to validate and how we want to validate them, the types of tests to be developed, the resource allocation or the execution plan.

For this process, we can use the Issue Test Plan in Xray, where we can select all the tests we want to place in a particular plan. In this issue we find a table that shows the tests assigned to the plan and their executions.

4. Execution of tests

Once we have defined our tests and test suites, we can execute them. Xray’s Issue Test Execution monitors and checks whether the tests selected for execution have passed or failed and validates the execution if the former is the case.

The execution result informs us of the progress of execution for each test that passed, failed, or was selected for execution. Each test has its own execution details so that progress and execution results can be easily tracked.

With manual tests, the status of the execution is defined manually in the Test Execution after the execution in the corresponding environment.

If, on the other hand, the automatic test execution is done by Continuous Integration (CI), the Test Executions are automatically generated with the test results.

5. Analysis of test runs

Xray provides an intuitive and descriptive dashboard for analysing and evaluating the test results. The Tests Executions automatically generate statistics and charts showing a detailed summary of the passed and failed tests in our project.

This allows developers and testers to closely review the execution of our tests and monitor the coverage of our test environment.

On the other hand, the Xray Dashboard is an ideal way to present the project status to our customers to promote transparent and trustful communication and collaboration.

Xray at WATA Factory

At WATA Factory, we define our testing environments around Xray for Jira tools and solutions to deliver quality software based on automated, organised as well as resilient testing strategies.

The post Xray for Jira: How we efficiently structure and organise our test environment appeared first on WATA Factory.