Booking.com leverages BrowserStack to improve testing experience and product quality

Before the team started using BrowserStack, they maintained an inventory of physical devices in the office, which was reviewed periodically, and then bought new devices per requirement. “It was pretty difficult to enable everyone to test because of this,” says Georgiana. Hence, they wanted a cloud platform to support their testing.

The testing team conducted a survey internally. Georgiana explains, “The purpose of the survey was to understand how multiple teams perceive using BrowserStack because testers and developers are equally responsible for testing at Booking.com.”

The team also probed other requirements. “We wanted to see if BrowserStack really improved the speed of delivery, or if there were fewer issues on particular browsers in particular locations since we were doing more testing. We also wanted to see how accurate the test results were,” Georgiana says.

The team wanted to perform both manual and automated tests covering websites and mobile apps. They started using BrowserStack in the Cars and Rides unit for manual testing in 2016 and later incorporated Automate and Percy for automated compatibility testing and visual reviews, respectively. In 2021, they consolidated their BrowserStack subscription for their Accommodation business unit to perform manual web and mobile app compatibility testing.

Describing the SDLC today, Georgiana explains, “We have multiple product teams that are responsible for different parts of the booking journey. When they have a new feature, they start with manual testing for the particular feature on different browsers, devices, and OS using Live and App Live. Once they deploy to test and development environments, they perform integration regression tests using Automate and App Automate. Here they want to check that all the components interact correctly with each other while ensuring our top browsers, devices, and languages are covered.”

In addition, the team uses Percy in the Cars and Rides unit to perform visual reviews for four browsers: Google Chrome, Mozilla Firefox, Apple’s Safari, and Microsoft Edge.

Examining the test insights for Jan 2022 – May 2022 period, the team observed that they ran tests on more than 200 browsers-OS and 150 real device-OS combinations across 13 and 9 Operating Systems for their manual and automated tests, respectively.

The team conducted an extensive survey among their BrowserStack users to understand their perception of the service across four main key areas: Release quality, Test coverage, Enablement, and Cycle time.

“Our internal survey revealed that 91% of our BrowserStack Live users indicated improved release quality and test coverage, and 95% agreed it enabled the whole team to perform compatibility testing. Similarly, 71% of BrowserStack App Live users championed improvement in the three parameters. The essential advantage of using BrowserStack was the elimination of effort spent on handling physical devices, which included buying, storing, delivering, and managing the inventory of devices,” she explains.

“Of the BrowserStack Automate users, 100% said in the internal survey that the product increased their team’s release quality, test coverage, and speed of delivery. Their main advantages were reduced manual effort for running regression tests on multiple browsers and devices, high speed in running compatibility tests due to parallel testing, and increased confidence during releases due to improved test coverage,” Georgiana adds.

With regards to visual testing results, Georgiana says, “100% of Percy users taking our internal survey agreed that Percy increased visual review accuracy. Percy enabled the whole team to perform visual reviews and allowed them to focus on more complex testing. The main advantage highlighted was the speed of execution compared to manual testing and the accuracy of the results, which could have been missed at first glance but could have a significant impact on the user experience or from a compliance point of view.”

Encouraged by the promising results of their internal survey and multiple benefits as a result of using BrowserStack, Booking.com aims to continue its journey with BrowserStack in the future. “We will continue encouraging more teams to integrate automated tests with BrowserStack to increase their test coverage, reduce their feedback loops and deliver high-quality products faster,” Georgiana says.