Automated Testing with Azure DevOps: Tutorial

Ever since cloud computing came into existence, it has witnessed exponential growth in usage, within all aspects of technology. Testing too, like other domains, has caught up with implementations on the cloud. Multiple cloud service providers are now offering various test automation products in the form of cloud services. 

Azure, AWS, GCP, Oracle Cloud services, and many more cloud service providers have integrations to multiple open source and enterprise test automation tools. Selenium is the most used framework that has integrations into all the major cloud platforms. Also, there are test platforms like BrowserStack, which offer test automation tools as SaaS (Software-as-a-Service) solutions. 

This article aims at getting people started with running Automated Testing on Microsoft’s Azure. It highlights what Azure DevOps is, and then understands how to run automated tests with Azure DevOps. 

What is Microsoft Azure and Azure DevOps?

Microsoft Azure, previously called Windows Azure, is the cloud computing platform by Microsoft. It is a huge collection of servers and networking hardware that hosts a complex set of distributed applications. These applications are exposed as services and range from cloud computing, analytics, networking, and storage solutions.

Azure DevOps is the service of Azure that helps users set up CI/CD pipelines for their software projects. It provides various developer services that help teams to plan work, collaborate on code development, and build and deploy applications.

Introduction to Azure Mobile Apps

Azure Mobile Apps helps developers and system integrators with a mobile application development platform. The platform is highly scalable and provides your mobile application with authentication, data query, and offline data synchronization.

Using the Azure Mobile apps platform you can:

• Build native apps
• Build cross platform apps
• Authenticate and connect users to your enterprise data stores
• Build offline apps with data sync

Run Selenium Tests on Azure DevOps

To run a Selenium test suite on Azure DevOps, listed below are some of the pre-requisites that you will need:

• Azure DevOps account: Free account can be created
• Knowledge of Selenium, Maven, Java, and TestNG
• A working test suite that uses the above technologies
• Github repo to store the test suite code

Step 1: Create an Azure DevOps account (If you already have an account, proceed to step 2)

Account can either be created using your email id or if you already have a Github account, Azure DevOps account can be created with the Github account. Once an account is created, sign in to Azure DevOps.

Step 2: Create a new Project 

Once you log in to your account, click on Create Project. This will open a dialogue box where you can fill in the project name, a description and also set the visibility (public/private) of the project. 

Once the project is created, you will land on the project home page. This will have options to create pipelines, setup dashboards, test plans, wiki pages, and tracking boards. 

Step 3: Create a Pipeline

As a next step, we will create a pipeline that we will link with our test repo and issue commands to perform the tests. 

Step 4: Link your code repo

The test automation code repo needs to be stored in a version control system and in this step, we will need to integrate that into the pipeline. Azure offers integration with Azure Repos, Bitbucket Cloud, Github, and Subversion. For this demo, we will go ahead with the Github option. 

Once you have linked your Github account with the Azure DevOps account, all the repos in your organization start to appear on the Select a repository page. 

Run Azure Selenium Tests on Real Devices

Step 5: Select the task 

For this demo, let us select the Maven task that will help us build java projects and run tests with Apache Maven. Upon selecting Maven, the review page will appear. Click on Save and run.

Step 6: Configure Maven goals

Configure the Maven goals. Add any options you would wish to pass to maven. Also, pass the test report location.

Step 7: Pipeline run

Click on Run. This will open a dialogue box as below. Select the agent pool, agent, and the branch from which Azure DevOps needs to execute the build. A commit ID can also be given to build a specific commit. On leaving this field empty, the latest commit on the branch shall be built. 

Check the Enable system diagnostics checkbox and click Run. Once the pipeline is triggered, it will queue the build and wait for the executor to be available. Once we have the executor, the build starts, and the progress can be viewed by clicking on the Job.

The job progress can be viewed by clicking on the jobs section. 

This will run the pipeline and display the results on the same page. Cumulative results are also shown. 

Integrating Browserstack with Azure Pipelines

Azure Pipelines is a Microsoft product. It is a Continuous Integration (CI) server, which automates the build and deploys process of your native web apps. There is an easy way to integrate BrowserStack Automate to run Selenium Tests with Azure Pipelines.

By executing your Appium test suite in Azure Pipelines, you can automate the testing of Mobile Applications as a part of the build process. It enables continuous testing, build, and deployment of iterative code changes. CI/CD tools help in catching failures ahead of the production stage and mitigate them as they occur. Adding BrowserStack to the mix, your native apps can be tested across 3000+ real devices. 

BrowserStack lets you integrate your Appium tests with Azure Pipelines using an extension that connects the CI server to the BrowserStack Appium Grid.

Start running tests on 3000+ real browsers and devices on BrowserStack’s real device cloud. Run parallel tests on a Cloud Selenium Grid to get faster results without compromising on accuracy. It helps in detecting bugs before users do it by testing software in real user conditions with BrowserStack Automate.

Features of BrowserStack Automate:

• Executes hundreds of concurrent tests
• Integrates with popular languages like Python, Java, and top CI/CD tools like Jenkins, CircleCI, etc.
• Offers instant access to 3000+ Real Devices and Browsers
• Comprehensive Debugging using video recordings, automated screenshots of errors
• Enterprise-Grade Security & GDPR Compliance

Try BrowserStack for Free