Nykaa's Test Automation Transformation with BrowserStack

As per estimates, Nykaa had about ~9 million visits in the quarter ending June 2023. There were also 5 million app downloads in April 2023. These massive volumes made covering all customer devices in their day-to-day testing became a significant challenge for the QA team. Though they initially set up an in-house device farm for testing, limitations of the approach soon became apparent.

The ever-increasing device-platform diversity made it impossible to scale the farm, while maintenance and troubleshooting existing devices started consuming more and more bandwidth. All this negatively impacted the QA team, diverting their focus from delivering exceptional customer experiences. Eager to find a solution to the team’s testing challenges, Nykaa decided to try BrowserStack. Scaling test automation across multiple devices-browser-OS combinations without the burden of maintenance was an enticing proposition for the team.

“The [quality of] support and collaboration from BrowserStack has elevated our experience with BrowserStack,” says Ayushi Arora, SDET Lead at Nykaa. She and her team are responsible for end-to-end testing, automation, and release management.

The team uses App Automate for regression and end-to-end automated testing on a staggering 60 devices, on both iOS and Android platforms. For manual edge cases and specific device or browser version testing, the team uses App Live. Automate became their weapon of choice for testing on various browsers like Edge, Firefox, Chrome, Safari, etc, and their versions. Using BrowserStack’s network simulation, push notification testing, and network log checking features, the team enhanced their edge case and device-specific scenario testing.

Owing to BrowserStack’s parallel testing capabilities, fast test execution, and advanced features, testing at Nykaa transformed significantly. “The most major impact I have noticed is that we are able to deliver fast. We are also able to maintain the robustness and stability of our tests; the tests are more reliable now. And if anything fails, we have the video recording to go back and analyse where, when and why the test failed – is it a genuine issue or flakiness associated with our test case,” says Ayushi. The improved observability into test suite health and enhanced reliability boosted the team’s confidence in delivering top-notch website and app experiences to their customers.

The ability to capture video recordings of test failures enabled Nykaa’s QA and development teams to collaborate seamlessly. “Our developers are also getting comfortable with using BrowserStack,” she says. They could easily identify the root cause of any issue and ascertain whether it was a genuine defect or an anomaly in the test case. This streamlined the debugging process and ensured swift resolution of any issues. BrowserStack also helped the team identify trends and capture edge cases, leading to more reliable and robust test cases, resulting in higher-quality deliverables.

Another area of improvement was speed. Test cases that once took hours were now wrapped up within minutes. “We are able to wrap up our tests on BrowserStack within 2-5 minutes. Paralellisation enables us to run around 650 tests within 4 hours,” Ayushi elaborates. This speed allowed them to run an impressive number of tests. “On a average, we run 1500 test cases on BrowserStack every night. We are able to achieve this thanks to parallel threads on BrowserStack. During releases, we run this regression during the daytime as well, touching around 2000 test cases,” Ayushi explains.

Today, BrowserStack is an indispensable ally for Nykaa’s Dev and QA teams. The fast and reliable execution of tests on various devices and browsers has bolstered their product quality, coverage, and release velocity, besides improving collaboration between teams, productivity, and more! Nykaa’s partnership with BrowserStack has taken testing to new heights, reinforcing their commitment to delivering exceptional experiences to all their customers.