How to perform SAP Testing

SAP applications have become a buzzword in the field of Enterprise Resource Planning, helping businesses to improve organizational control and customer relations and deliver an optimal experience. The applications enable users to comprehensively access both organizational and customer experience information, leading to faster and more efficient decision-making. 

This provides them with a competitive advantage in the industry, and any misalignment of the highly customized SAP offerings can result in unforeseeable losses in reputation and money for the clients. Thus, fool-proof test methods are a must while striving to develop new SAP versions.

The article provides an overview of SAP applications, the reasons for testing SAP applications, types of SAP testing in SAP applications, and those created using the new SAPUI5 framework, moving to automated tests for these, as well as best practices for SAP testing.

An Overview of SAP

SAP (alternatively known as SAP AG and SAP SE) is an acronym for Systems, Applications, and Products. It’s a German-origin MNC that has proved itself to be the world’s leading ERP (Enterprise Resource Planning) business application software maker. As per a 2020 Report, SAP ERP applications have cornered a majority of the market share, and the worldwide ERP application market is poised to reach a valuation of 100.7 Billion USD by 2025, with a CAGR of 1.1%.

Source

This makes SAP a pivotal player in this industry and further underlines the importance of getting SAP application deployment right in the first place. 

SAP’s platform is made up of 6 core products (along with several sub-products) which are: 

• ERP and Finance
• CRM and Customer Experience
• Network and Spend Management
• Supply Chain Management
• HR and People Engagement
• Business Technology Platform

SAP offerings are usually highly customized to meet the unique operational needs of a business. One of the most-common integrations can be found with SAP ERP products and a CRM platform like Salesforce.

SAP offers not only its diverse suite of products and subproducts but also IDEs. Tools, design systems, specifications (SAP Fiori) UI Framework (SAPUI5) to enable application developers and testers to implement business requirements seamlessly.

Given the diversity of SAP’s offerings, it is important to consider this article in the light of:

• SAP Application testing (specific to SAP ERP, SAP S/4 HANA)
• Websites/Web Applications created using the SAPUI5 framework

Because of the sheer diversity of SAP’s offerings and the number of customizations and personalizations possible, it is important to ensure that the business processes implemented in SAP and their integrations with external systems work seamlessly to ensure that they are secure and bug-free.

Why is SAP Testing essential?

Even if a single bug goes undetected in an SAP module, the consequences can be both far-reaching and disastrous. The case of the launch of SAP for the National Grid in the US serves as a stern reminder of what can happen when testing is suboptimally conducted. 

In 2012, the US-based utility company was due to launch its SAP implementation and was under intense pressure to go live as soon as possible. The cost of missing the due date would have cost millions, but unfortunately for them, Hurricane Sandy hit, and certain processes were rushed to meet this emergency. 

However, the effects of poor testing took their toll soon after, and chaos ensued. Incorrect paychecks, underpaid employees, unprocessed vendor invoices, and non-functional financial reporting completely hamstrung the organization. 

While SAP is an incredibly complex ERP suite, this example just reiterates the case for early detection of bugs as the cost for bug fixes can become astronomically high once the system goes live.

What to test in SAP Applications?

The SAP instances that are vulnerable to errors can be segmented as below:

1. Customizations

SAP terms its customizations as WRICEFs. This acronym covers any form of custom development or enhancement and stands for Workflow, Report, Interface, Conversion, Enhancement, and Forms. As default SAP suite features cannot serve the particular business needs of an organization, such customizations are common. These must be mandatorily tested to ensure there are no errors.

2. Updates/Configurations 

Regardless of whether a business uses SAP S/4 HANA, which tends to have more frequent updates, or an SAP module, these are inevitable. Depending on the scale of the company and its processes, the frequency of these will differ. However, a system update can cause a business process to break. To prevent this from occurring, it is imperative to perform SAP regression testing of features before releasing them to a production environment. Read more about Regression Testing.

3. Integrations 

Every SAP module usually supports a business process. However, a process can cross over to different modules and even beyond SAP to a third-party application. These integrations need to be tested to ensure a smooth flow of information and the seamless operation of business processes.

Some common examples are:

• TPP order system integrations ensure a new purchase order is immediately made available and transmitted to SAP.
• CRM integrations necessitate the integration of SAP’s ERP systems and a CRM system like Salesforce. For example, in the case of new customer onboarding, the CRM data must sync with financial, performance management, and other ERP module data.

4. Migration/Transitioning to SAP S/4 HANA 

SAP ECC has a deadline of 2027 for a complete transition to S/4 HANA to enable easier use and increase data-handling capacity. Businesses will need to thoroughly test their overall solution, individual processes, and all integrations before making the transition. This will ideally require end-to-end automation within and outside of SAP applications.

Types of Testing for SAP ERP Applications

The following key types of testing are commonly implemented when SAP Applications undergo a QA process:

Unit Testing in SAP

The SAP ERP suite is developed using a high-level programming language called ABAP (Advanced Business Application Programming). SAP’s extensive set of ABAP code development, testing, and analysis tools integrated with the ERP tool promotes optimality in syntax and functional code aspects.

SAP Unit testing is carried out whenever any change takes place in the WRICEF mentioned earlier. Some tools used in ABAP-based development and unit testing are, 

• ABAP Workbench and ABAP Test Cockpit: Used by developers to create standard SAP ERP application logic and logic for custom applications. ATC is the test framework that comes along with the workbench. This allows developers to execute and observe results on SAP application objects.
• ABAP Test Double Framework: This is handy for testing using mock objects to substitute and imitate the behavior of dependent/external objects to promote a focus on the modularity of the objects under test.

How to write ABAP Unit tests in SAP

ABAP Unit is a framework for executing automated tests written in local classes. To write ABAP Unit tests, follow the below steps:

1. Open the Test Classes tab at the bottom of the ABAP Editor.
2. Then write a test class using the predefined language elements. Alternatively, a template can be inserted for ABAP Unit test classes.
3. In the pop-up menu with suggested entries, select testClass – Test class (ABAP Unit) to insert the template.
4. The test methods can now be created using the static methods of the class CL_ABAP_UNIT_ASSERT to check assumptions.

Integration Testing in SAP

Once the unit tests are complete, the test team progresses to integration testing. This is an End-To-End process that moves from one business process to another to make sure they work as expected. This can extend over different SAP modules as well as non-SAP components. 

It usually takes multiple test cycles and teams of dedicated testers, consultants, and business users. 

Although integration tests can be as varied as the business scope and nature of the SAP modules in question, the following phases are ubiquitous in all integration tests.

Preparation Phase:

1. Identify the scope of testing, including all relevant business scenarios, any interfaces with legacy systems, and period-end scenarios
2. Load scripts in the testing tools
3. Identify the master data set and relevant organizational data for testing.
4. Identify the testers with the required skills and schedule a test plan
5. Setup the test environment
6. Create master data sets
7. Define a defect management procedure and identify focal points for process-wise defect resolution.
8. Perform a final check on the master data before testing commences

Execution Phase:

1. Testers execute the test cases, record the results in test management tools, and raise defects as appropriate.
2. Review meets are usually set up with the SAP process consultants to discuss focal points, defects, and legacy interface issues.
3. Daily status review meetings help review the test execution progress with the individual process teams and set periodic targets. Any blocking defects are raised and undergo escalation as required.
4. Reporting is performed on a periodic basis to cover visibility of the testing progression and satisfy stakeholders 

Defect Management Process:

1. Any issues discovered during test execution should be logged as defects in a defect management tool.
2. Defect definitions should be provided as per severity and priority
3. Recommended resolution times should be determined based on these
4. Fields should be set up in the defect management tool to capture the inputs required to test, analyze, and resolve them in the future.
5. Escalation points and timelines should be agreed upon for contingency.

Functional Testing in SAP

Functional tests are critical in evaluating the functionality of all business modules in SAP and aligning them with business requirements. The SAP ERP application is extremely flexible and offers a wide amount of configurability and customizability for its users. QA engineers must go through the design documentation, understand the functional features of a module and create test scenarios and test suites accordingly. 

Traditionally tools like SAP TAO Test Acceleration and Optimization) and eCATT (extended Computer Aided Test Tool) were used for this purpose. SAP TAO is more effective for automated end-to-end testing, whereas eCATT is more useful to record and parameterize test objects for automation testing by providing a graphical user interface along with an ABAP script editor.

SAP S/4 HANA-based solutions are more conducive for Selenium test automation due to their web-based component. And for a long period, legacy modules for SAP were considered unsuitable for Selenium as these were purely desktop-based.

With a full transition to SAP S/4 HANA with set a deadline for 2027 and potentially some more years away, having all customers on the cloud and ready for Selenium Testing is not a realistic scenario at this given moment, and the reliance on traditional tools remains. To get better accuracy in tests, it is highly recommended to test under real user conditions, which is why real device cloud like BrowserStack should be used to run functional tests.

BrowserStack offers a Cloud Selenium Grid that is connected to 3000+ real devices and browsers for testing. Users simply have to sign up and start testing on a robust Selenium grid that is constantly updated for the best possible results.

Run SAP Tests on Real Devices

Performance Testing in SAP

The overarching goal of any performance testing exercise is to ensure that the business modules of the SAP ERP application remain operational even when the load fluctuates beyond expected levels. This spike in load can be caused by :

• Increased number of users
• High transactional volume of business operations
• Scheduled background jobs to satisfy business requirements

Usually, this stress testing is done in tandem with integration testing to evaluate the performance of the individual components and the entire pipeline once the load requirements are put in motion. 

It is imperative for some large organizations to have dedicated performance test teams to carry out the infrastructure setup and other tasks required for performance testing.

Regression Testing in SAP

As discussed before, Regression tests for SAP are critical for testing the functionality of the SAP business processes after an update or during migration. The same tools used for functional testing of SAP applications can be used here again.

With many companies focused on migrating from legacy SAP modules to S/4 HANA, it is imperative to understand the scope of regression testing to be performed and how best to leverage regression test results to hasten the process.

Types of Testing involved in SAPUI5-based Applications

SAPUI5 is an enterprise-ready UI development framework for creating web applications using HTML5 standards. It supports features such as internationalization, localization, Data-driven UIs, routing, a comprehensive suite for performance analysis, and testing tools that support both unit testing and end-to-end testing approach by leveraging an MVC (Model View Controller) approach for developing web applications.

Testing SAPUI5 Applications

The following testing types are possible for SAPUI5-based sites and applications –

Unit testing of SAPUI5 Applications

A JS-based test framework called QUnit is used to carry out unit testing for SAPUI5-based applications. This framework comes with in-built features such as :

• Test Runner to execute test suites
• Test Suites to execute one or more test cases
• Test Cases which contain the actual tests written
• Assertion options to verify actual and expected results.
• Test run reports to document the test results

Integration testing of SAPUI5 Applications 

The OAP5 (One Page Acceptance Tests) included with the SAPUI5 offerings is essential in making the integration test execution faster and easier. However, there are some major limitations of this library.

• Screen capture is not possible with OAP5, and thus testers can face problems with cosmetic or visual tests. In such cases, a visual testing tool such as BrowserStack Percy can be included to help verify that the user interface (UI) appears correctly to all users. Visual tests check that each element on a web page appears in the right shape, size, and position. It also checks that these elements appear and function perfectly on a variety of devices and browsers. Percy by BrowserStack is one of the best-known tools for automating visual testing. It captures screenshots, compares them against the baseline images, and highlights visual changes. With increased visual coverage, teams can confidently deploy code changes with every commit.

Run SAP Visual Tests on Percy

• OAP5 cannot test more than one page at a time. And while this is usually sufficient, it sometimes becomes far too time-consuming for a test run if there are multiple pages to test and especially if they are to be tested across multiple devices. In such cases, Parallel testing can be leveraged to speed up test execution. Parallel Testing is a process to leverage automation testing capabilities by allowing the execution of the same tests simultaneously in multiple environments, real device combinations, and browser configurations. The overarching goal of parallel testing is to reduce time and resource constraints.

Best Practices for SAP Testing

Here are some of the best practices to follow during SAP Testing:

• Adopt Test Automation from the get-go: Automation helps save time and resources by allowing the speedy execution of repetitive tasks like SAP Regression testing, leaving manual testers to test edge cases, and focusing more on test strategy.
• Look beyond risk-based testing: Most companies only focus on those segments of their systems that can impact other segments if any change has been made. And while this seems reasonable from a cost-perspective, this exposes the system to increased interconnectivity risks and potential business process details. 
• Shift Left: This means testing early and more often in the earlier part of a project’s test lifecycle. Since SAP projects have inherently high complexity and inter-dependencies, they require a greater than normal clarity of process flows and test planning at an early stage to prevent delayed release cycles and defect-ridden software. Hence, Shift Left Testing is highly recommended.
• Integrate testing with CI/CD: SAP testing should start as soon as the code is ready for deployment. Ideally, it should be integrated into CI/CD workflows when the level of integration-test readiness is reached. Incorporating the testing process with Continuous Integration and Deployment workflows helps to foster a greater sense of trust in the product and shorten delivery cycles.

Conclusion

SAP is the leading ERP solution provider with a large feature set that can be customized and reconfigured to suit an organization’s business needs. However, only focussing on developing business process modules is not enough. SAP testing is critical in ensuring customer satisfaction and delivering processes that work seamlessly and enable the maximum leverage of SAP’s unique capabilities to ensure customer success.

Using BrowserStack’s real device cloud that provides 3000+ real browsers and devices makes testing Agile with wider coverage. The cloud allows parallel testing and supports integrations with popular CI/CD tools such as Jira, Jenkins, CircleCI, TeamCity, and Travis CI, to ensure streamlined Automation Testing.