Things every CxO should know about testing

Tech startups and companies have revolutionized the world economy. There is such a huge range of tech products that are applied in different disciplines in different contexts, resulting in improved outcomes and establishing a clear superiority of the modern agile tech product. 

Behind every successful tech product, there is a list of common ingredients, among which quality assurance or testing is one of the important ones. So having a clear understanding of executive and technology leaders’ perspectives on testing can greatly help optimize and streamline resources and efforts applied in the direction.

Various types of application testing over the years, Source

Layered individual and team responsibilities

Teams in tech companies are arranged in hierarchies, relatively flatter but still present. Based on the level of experience and responsibility. Tech companies are among the most flexible work environments in the current economy, mostly because of the fact that the market itself is a very dynamic and fast-changing system.

Considering four slices of the overall arrangement of resources in a typical testing team, it would consist of:

• QA engineers 
• QA Leads/Managers
• Chief technology officer (CTO)
• Chief executive officer (CEO)

At each level we have slightly different set of responsibilities and perspectives. You can think of it like this; at the bottom level, the responsibility is mostly technological implementation and at the executive level the responsibility is to define constraints, direction and broad goals. You can visualize this understanding with the help of the following diagram.

At the highest level, the responsibility is setting direction and monitoring outcomes, and at the bottom level, the main responsibility is the execution. In the middle levels, members end up doing more than one thing and participating in workflows going in both directions. 

This is key to understanding the fact that the same team takes different perspectives during planning and implementation. Workflows going in both directions need contributions from most team members.

Let us now break down and understand some important things that executives and technologists should know about testing and how It can also help guide, define and manage the roles and responsibilities of a test lead.

Things executives should know about testing

• Importance of testing is proportional to business’ dependence on technology 

When saying dependence, it means identifying what percentage of the business revenue directly or indirectly depends on the application stack. 

The reality of tech companies and products is quite complex. You will find some companies have one application, which is their core offering and all the resources are dedicated working on that one particular application, In another scenario you will find one tech company having a buffet of applications feeding into and working with each other. 

Further more, in other cases you will find a hybrid situation where a real-world process is tied in to a technical work flow with the help of one or more than one applications.

So understanding first of all how important a particular application is for any given business use case, and then within the applications finding the most critical workflows to be tested and then developing test strategies on top of this understanding is ideally how it should be done. 

BrowserStack’s real device cloud can help you setup real test environments without the need to spend money on physical devices.

• Starting application development without testing can be an issue

In a typical tech startup situation, you will find the founders themselves or together with early team members will build application prototype initially without having proper architectural and quality assurance related considerations. Which is quite understandable because at that time the fate and features of the product and business are unknown, so the process has to be lean.

But once the business idea is validated and real production ready version has to be developed, at that point itself testing should be given considerable thought and based on its importance infrastructure should be established. Hiring a QA team to work on a project which is already in motion will result in the development team running ahead of the QA team from the get go. 

With BrowserStack cloud testing suite, you can start for free and then scale as your needs expand over time easily through your dashboard.

• High degree of automation is rare 

It’s a real popular myth that automation testing eliminates the need for humans or it helps you avoid the much-needed reviews and deep dives into existing bugs. It is important to understand that automation is only helpful in reducing the amount of regression related activity to maintain a state of functionality for an application. In other words, it is more like a tool for building processes rather than an algorithm to replace human testers.

Degree to which tests are automated in any given scenario are going to be different. Rarely you can find anyone who has automated more than 50% of their workflow. The reason is simple, application development is a recursive and messy process, so what has to be tested and what outcome leads to success are very non-intuitive things. So translating them into test cases usually turns out to be much harder than it seems.

With modern features provided by BrowserStack Automate and App automate, you get advanced visual and system level logs which help you capture all issues.

• You might need more than one kind of testing 

As we hinted on earlier, the fact that many real-world processes have technology woven into them and it plays a crucial role in ensuring the success of the whole workflow, you can see scenarios which might need more than just application testing. 

Take for instance a grocery delivery app, there are many moving parts to such operation including the app, warehouse and inventory management, logistics etc. from an executive perspective when you talking about testing, it can’t be just about whether or not you are able to place the order on the app, you also have to ensure that the order reaches the warehouse, the items are present in the inventory and the logistics are in shape to deliver to the right location at the right time. So making sure that the right kind of testing is being performed at the right step is the responsibility of executive layer.

With manual, automation and visual tests running on cross-browser and cross-device test environments, BrowserStack lets you manage the overall test flow and outputs seamlessly. 

• Your processes need to withstand the test of time

After successful prototyping and before the launch of the first production version of a tech workflow, it is recommended that a whole bunch of experimentation regarding the architectural and procedural design of the workflow takes place during this critical phase. 

The complete set of plans that will be used for finding, reporting and analyzing defect, bugs and errors has to be thought through. Most efficient systems allow for configurations and extensions, so we do not mean that you have to lock you or your team into a workflow, we are just saying that you have to be smart about the definition of the workflow such that it is a configurable and extendable system which can scale with your product.

Things technologists should know about testing

• Before testing comes proper documentation

Before starting work on actual testing it is important to ensure that business expectations are recorded clearly in a well documented format for both application design and testing. Specifications for the technical aspects of the application also need to be clearly stated and discussed with the team including all the specifics of tools that will be used for quality assurance and automation.

Recognising promising innovative solutions to problems and incentivizing the team members to learn and use those new technologies is also an important part of a technical lead’s job. Artificial intelligence, Machine Learning  and cloud technologies are pushing the test automation industry to a new height.

• Developers and QA engineers both participate in testing

Unit testing, despite being a hot discussion on whether it is an essential or not, has historically been considered the first step of application testing which gives way to higher levels of testing like system, integration etc. 

Unit test code is usually part of a branch of the core repository, with a continuous integration system unit test execution and application deployment are more or less carried out on the same triggers. 

System level testing is where the handover happens from developers to QA professionals. After which acceptance testing is the final step before product release. Hence keeping this seamless flow in consideration while designing the architectural specifications of a project is very important.

• Avoid elaborate set-ups, high stress development and low test coverage

In a fast paced development environment it’s very common to find teams with established testing processes working on application components on rapid pace, but failing to build meaningful and cohesive test plans resulting in a slightly vague and traffic signal style green-yellow-red pass and go mentality. 

In other words, for such teams running scheduled tests with each delivery is more of a regressive activity where they will stop and analyse failures and warnings, but not have significant depth, which is meaningful to business, in the written test cases. 

Especially in agile development, when new features are designed and implemented in a short window of time, it is common to see resources focusing more on writing functional code without sufficient test coverage.  It is essential to ensure a maximum test coverage. Test case prioritization is essential for regression testing to ensure maximum efficiency.

• Effective near term planning is an important goal

The two main steps in the work cycle of a QA lead are: 

1. The routine team management around reporting, writing new test cases and executing the existing ones. 
2. The broad spectrum analysis of complete testing resources, infrastructure and planning for present and near future. 

It is crucial for the technical leadership to be able to balance both and ensure team led smooth functioning of the first and upward communication based focus on the second. An effective QA leader can greatly improve outputs by adapting new technological innovations to gain leverage over limited resources.

Greater adoption of technological innovations could be hindered by an unclear understanding of the relevance of technologies one can put inside the stack. So being able to separate real-world advantages from time consuming low value additions is also crucial.

• Visual output is the most comprehensive

Various development methodologies like agile, combine, scrum etc. are processes designed to keep the uncertainties of the development process in check. Similarly, managing the results and reports coming out of a big testing effort can easily get intimidating if not handled properly. 

Using visualisations, screenshots, video and other means to capture feedback and issues, is a very critical component of a successful analysis.

Conclusion

Testing is essential to deliver a high-quality software, and it is everybody’s responsibility in the team to ensure that QA process is streamlined with development. While, QAs and devs are actively involved in the testing process, CxO (CEOs and CTOs) also hold prominence in the decision making. Hence, a couple of things they should know about the testing process are listed in the article, so that they can take them into consideration, when deciding over the process of software delivery. 

One such thing is testing on real devices and browsers to ensure that the results are accurate. In this case, it is upto CxO to decide whether they would want to build an in-house infrastructure or buy subscription of real device cloud. Since it involves parameters like cost, reliability, maintenance, security, and efficiency, the decision between build vs buy is crucial.

Real Device cloud like BrowserStack offers 3000+ real devices and browsers to test on. It integrates well with popular test automation frameworks like Selenium, Cypress, Appium, Playwright, Puppeteer, etc. making it easy for the teams to integrate their existing test suites with BrowserStack.

Try BrowserStack for free