Vimeo enhances QA Automation with BrowserStack

John Mondi, QA Manager at Vimeo, recalls, “Before BrowserStack, we relied on a local grid with a handful of Android and iOS devices and our desktop machines for automated and human-action testing. Scaling our QA efforts was a significant challenge.”

The limitations of the local grid setup restricted the QA team from conducting comprehensive tests across various platforms and devices. The team also had to deal with maintenance overheads and ad hoc updates.

Andrew Katsikas, Principal QA Automation Engineer at Vimeo, elaborates, “Managing a Selenium grid service and ensuring the availability of real devices for testing was both cumbersome and resource-intensive. Achieving the desired speed and reliability in our testing processes was a constant struggle [before BrowserStack].”

To overcome these challenges, Vimeo partnered with BrowserStack for instant access to a wide range of real devices on cloud, eliminating the need to maintain a local grid. This transition allowed Vimeo to significantly enhance its automated testing capabilities.

The concurrency functionality of BrowserStack’s parallel tests enabled Vimeo to run multiple test suites simultaneously, significantly increasing the velocity and coverage of their tests. The team also benefited from BrowserStack’s reporting and diagnostic capabilities, which enabled them to take deterministic action on errors and improve resolution time.

Using BrowserStack helped overhaul Vimeo’s QA processes and significantly improve overall software quality. Automated testing became more robust and scalable, allowing the QA team to replicate and expand their test suites across multiple environments. This shift enabled Vimeo to run a higher volume of automated regression tests with greater efficiency and reliability, so human testers could focus on contextual and exploratory testing.
One of BrowserStack’s key benefits is its comprehensive diagnostic tools. “BrowserStack makes it very easy [to diagnose errors]. We get the error logs, screenshots, and videos that we can replay,” John notes. These features make it easy for the QA team to diagnose and address test failures promptly. This capability has enhanced their ability to distinguish between test failures, environmental issues, and code defects, ensuring that appropriate corrective actions are taken swiftly.

Andrew adds, “We want our automated tests to run successfully 100% of the time. When they’re not, we want to be able to easily diagnose the failed tests and distinguish between a test failure, an environmental failure, or a code failure. BrowserStack makes that very scalable.”

Other teams have also widely appreciated the ease of use, performance, and velocity of running tests on BrowserStack. 10 to 12 teams within the company use BrowserStack, facilitating consistent and efficient testing practices. “I would recommend BrowserStack to testing teams that want to test a wide variety of devices and platforms and don’t want to spend the time and money setting it up themselves,” says Andrew.

BrowserStack’s customer support and customer success teams have also played a crucial role in Vimeo’s success. “BrowserStack’s support has also been great. Our customer success managers have always been very accommodating to us as we have our monthly sessions, and whenever we come up on renewals, they’ve always worked with us to provide the best solutions for us,” John shares.

With BrowserStack, Vimeo has achieved the ability to scale up testing capabilities without additional pressure on the team while also increasing their confidence in the reliability of their tests. The move from a local grid to BrowserStack’s cloud-based platform has provided Vimeo with the scalability, reliability, and efficiency needed to meet its QA goals.

As Mondi summarizes, “BrowserStack has definitely impacted the company’s QA success. Over time, we’ve seen data trends showing that the QA or engineering teams are discovering more issues in test environments.”
Andrew Katsikas echoes this sentiment, saying, “The biggest benefit of working with BrowserStack is that they can provide any number of devices and platforms. All we need to do is specify what we want via configuration rather than having to maintain those environments ourselves. It’s effective and cost-efficient.”