Automated UI Testing: Framework Development and Integration

This is a detailed guide on how Alin Groza (Principal Test Engineer at Visma) created an automated UI framework using BrowserStack. It is a three-part series, in which Alin will deep dive into:

• Framework development and integration
• Implementation and testing
• Results and achievements

In this guide let us look at framework development and integration.

Background

In 2020 Visma.net ERP created a project to move the payment service from Monolith to the Microservice architecture. From the start of the newly created service, a new technical solution had to be designed for performing automated UI tests.

Since this is a technical solution for a full CI/CD process, the framework is designed to include all elements that make it an integrated part of the automated deployment process.

The framework is integrated with the tools already in place, for example – Github, TeamCity, Slack, and Microsoft.NET. In other words, the solution is a brand-new implementation but still compatible and integrated with the existing tools on Visma.net ERP.

When running tests, the main requirement for the framework is to be a cross platform solution—allowing tests to be executed on OS X (the 10th version of Mac OS) and Windows on any web browser.

To make the framework complete and to complement the tools mentioned above, I needed to find a cloud platform that could be used as both test infrastructure (with operating systems and browsers) as well as test execution (integrated with CI/CD tools, debugging, reporting, logging, etc.) Keeping all this in mind, I chose to work with BrowserStack.

Read how Visma uses BrowserStack to improve release speed and quality.

Overview of the UI Automated Testing Framework

Below you find an overview of the framework components:

This is how the framework operates:

1. GitHub stores C# Selenium tests code which will be used by TeamCity when creating the build.
2. TeamCity gets the code from Github, creates the build, and sends it to BrowserStack.
3. BrowserStack runs the tests and creates reports.
4. BrowserStack sends a notification on Slack when reports are available.
5. Slack shows a link to BrowserStack reports.

Below you can see a diagram with the framework operation steps mentioned above:

For the framework to create actual tests, the following are required:

1. Programming language: I have selected C# as Object-oriented programming (OOP) since the entire ERP platform is developed in C#
2. Page Objects Model as a design pattern: Page Objects Model is a highly efficient pattern for UI automation reducing code duplication and improving test case maintenance.
3. Selenium: Selenium is an open-source tool for web automation on cross-platforms. I have selected it because it supports the Safari browser (one of the main requirements) and provides support and functionalities generally for any topic related to UI automation.

Integrating BrowserStack

BrowserStack is a cloud-based cross browser testing platform that allows you to test applications across 3000+ different browser versions, devices, and operating systems. It has built-in features for the following items:

• Running tests in any browser and operating system
• Generating and storing reports including videos, visual logs, and step-by-step test execution details in a user-friendly UI
• Debugging tools as part of reports (e.g. networks logs)
• Integration with Slack for sending notifications
• Creating local tunnels for testing internal network URLs

The UI automation framework uses all these features and by using them, a lot of time was saved when I created the framework from scratch. Access to any combination of browsers and operating systems is a huge advantage because it enables running tests without any limitations.

BrowserStack gives the following benefits:

1. BrowserStack is fully replacing VMs (Virtual Machines) for running tests. This saves the time, and effort involved in creating and maintaining the VMs.
2. Basically, any combination of OS and browser is available to run tests on BrowserStack (including the latest and legacy versions). Here is a complete list of browsers and versions supported by BrowserStack.
3. BrowserStack has a wide range of built-in integrations for Slack, TeamCity, GitHub, and even Jira.
4. BrowserStack comes with built-in features for reporting. The user-friendly reports were already created, and ready to be integrated into our framework. As a result, anyone on the team can check reports such as data validations, asserts, visual logs, network logs, a video recording of the test run, etc.

In the following four images, you will see an example of a test report.

The BrowserStack dashboard provides information about the build such as version tested, environment, list of executed tests, and status.

 BrowserStack Dashboard

The test report can be accessed by clicking on a test in the dashboard. The report contains information about OS, browser version and test status. You can also view the video recording of the test.

 Test report

By zooming in on the tab “Text logs”, you can see the step-by-step details of the test execution. Each step shows the validation status. By clicking on the time for each step it will show the video performing at that step/point in time. 

Text logs screen

By clicking on the tab “Network Logs”, you can see the details of the network logs. These logs show the interaction between the front end and the back end. It is a useful tool for debugging failed tests. The network logs will also display bugs/problems/issues found elsewhere.

Network logs

In this project, there is an internal environment that can be accessed only inside the company’s internal network. However, BrowserStack offers a NuGet package called “BrowserStackLocal” which contains methods to create a local tunnel. Basically, the Set Up method of each test contains the call of the method for creating a local connection tunnel, while the Tear Down contains the corresponding one to close the local connection tunnel.

Try BrowserStack now

Coming up next

In my upcoming guide, I will zoom in and present how tests are implemented and the way they are running, including BrowserStack reporting.