How to report bugs in Appium UI Testing?

Appium, like Selenium, depends on test reporting tools, like TestNG and JUnit, that generate built-in reports. However, tools like Extent Report are preferred as they provide more detailed reporting for Appium testing.

This guide teaches you how to report bugs effectively in Appium UI testing using tools like App Automate, Test Management, Extent Report, and more.

Extent Reporting in Appium UI testing

Though you can generate in-built reports by JUnit, NUnit, and TestNG for Appium, the Extent report is more useful as it focuses on delivering test reports that are more detailed.

Advantages of Extent Report

• Step-by-step test execution information with pie charts and screenshots representation.
• Show the time taken for every test case execution.
• It can track multiple test cases within a single suite.
• Provide responsive UI.

Extent reports are written in HTML. Mainly the announcements contain two classes- ExtentReports class and ExtentTest class.

ExtentReports class: It generates HTML reports. This report will be created within your own folder means the path is user-specified. It can overwrite an existing HTML report. The default value becomes TRUE when a report is overwritten.

Syntax:

ExtentReports reports = new ExtentReports("Path of the directory to store the resultant HTML file", true/false);

ExtentTest class: This class log tests the previously generated HTML report.

Syntax:

ExtentTest test = reports.startTest("TestName");

Both these classes have some built-in methods. They are:

• startTest: Executes preconditions of a test case.
• endTest: Executes postconditions of a test case.
• Log: It’s a method. It logs each test step’s status onto the generated HTML report. It produces four value types- pass, fail, skip, and info, which we are elaborating on below.
• Flush: Creates an absolute new copy of the report and erases old data from the targeted report.

These values indicate the test status in the extent report-

• PASS
• FAIL
• SKIP
• INFO

Syntax

reports.endTest();
test.log(LogStatus.PASS,"Test Passed");
test.log(LogStatus.FAIL,"Test Failed");
test.log(LogStatus.SKIP,"Test Skipped");
test.log(LogStatus.INFO,"Test Info");

Generate extent report with NUnit

1. First you need to create an issuetrackerUTIL.java class. Thus we need to import the below packages. Then you will get an auto-generated Java document.

package <packagename>;

import io.restassured.RestAssured;
import io.restassured.builder.RequestSpecBuilder;
import io.restassured.http.ContentType;
import io.restassured.response.Response;
import io.restassured.specification.RequestSpecification;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testng.IInvokedMethod;
import org.testng.ITestResult;
import org.testng.SkipException;
import org.testng.annotations.Test;

import java.util.ArrayList;
import java.util.HashMap;

import static io.restassured.RestAssured.*;
import static io.restassured.path.json.JsonPath.from;
import org.apache.commons.lang3.StringUtils;

2. Create a listener to check the issue status.

package <packagenamehere>


import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;
import org.testng.SkipException;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

public class MethodIssueStatusListener implements IInvokedMethodListener {


@Override
public void afterInvocation(IInvokedMethod invokedMethod, ITestResult result) {
Issue issue = invokedMethod.getTestMethod()
.getConstructorOrMethod()
.getMethod()
.getAnnotation(Issue.class);

if (null != issue) {

IssueTrackerUtil.IssueStatus currentJiraStatus = IssueTrackerUtil.getStatus(issue.value());


if (IssueTrackerUtil.IssueStatus.OPEN==currentJiraStatus
|| IssueTrackerUtil.IssueStatus.IN_PROGRESS==currentJiraStatus
|| IssueTrackerUtil.IssueStatus.BLOCKED==currentJiraStatus
|| IssueTrackerUtil.IssueStatus.REOPENED==currentJiraStatus)
{
switch(result.getStatus()){
case ITestResult.FAILURE:
// no need to fail as we might have expected this already.
result.setStatus(ITestResult.SKIP);

if (issue.value()!= null) {
result.setAttribute( "issue", issue.value() );
}

break;

case ITestResult.SUCCESS:

}
}
}
}
@Override
public void beforeInvocation(IInvokedMethod invokedMethod, ITestResult result) {
// nothing to do

}

}

3. Create a @Issue interface to do the test with Jira

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface Issue {
String value() default "";
}

4. After that, use the Listener

@Listeners({MethodIssueStatusListener.class})

Screenshot capture in the Extent report

Since Appium supports mobile app testing, the camera is an obvious feature to be tested. Almost every mobile application uses a camera, including taking pictures, screenshots, and other images oriented tasks.

Here is a piece of code for taking screenshots.

Also, some basic methods used in this code are described-

test.log(LogStatus.FAIL,test.addScreenCapture(capture(driver))+ "Test Failed");
public static String capture(WebDriver driver) throws IOException {
File scrFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
File Dest = new File("src/../BStackImages/" + System.currentTimeMillis()
+ ".png");
String errflpath = Dest.getAbsolutePath();
FileUtils.copyFile(scrFile, Dest);
return errflpath;
}

Methods:

addScreenCapture()

This method is a member of the Extent Test class. It helps to get and add screenshots to the extent report. Thus, the log method uses this method to get a screenshot.

test.log(LogStatus.FAIL,test.addScreenCapture(capture(driver))+ "Test Failed");

getScreenShotAs()

This method captures screenshots of WebDriver instances. You must have to use this method when you’re taking screenshots of web drivers. It stores the screenshot in different output forms.

Here, the output type is a FILE.

File scrFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);

Then the output file will be stored in the’BStackImages’ folder in the ‘src’ folder. The  below statement creates the folder path-

File Dest = new File("src/../BStackImages/" + System.currentTimeMillis()+ ".png");

getAbsolutePath()

Use this method to place all error images in the destination folder.

Here is the complete statement for this-

 String errflpath = Dest.getAbsolutePath();
FileUtils.copyFile(scrFile, Dest);
return errflpath;

Now you have your extent report. Finally, you can get this report design from the dashboard

• Total number of test cases and associated test steps become visible.
• You can see the start time, end time, and execution time of a test case. You can check it by clicking on a test case name.
• It provides the number of test cases passed, failed, and skipped.
• Display the percentage of the total number of test cases passed.
• The number of steps in each test case (both the passed and failed test cases).

Appium Bug report with JIRA

Several automation tools support JIRA integration to report a bug similar to a Selenium JIRA integration.

Here is a sample extent report by JIRA integration with the test Automation framework.

The framework contains the below components-

• Appium
• Selenium
• Java 8 + TestNG
• Rest Assured

Updating the bug reports in JIRA becomes easy when your bug is already fixed.

Sample report by JIRA

compile group: 'com.aventstack', name: 'extentreports', version: '4.1.3'
use below extent-config.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<extentreports>
<configuration>
<!-- report theme -->
<!-- standard, dark -->
<theme>standard</theme>

<!-- document encoding -->
<!-- defaults to UTF-8 -->
<encoding>UTF-8</encoding>

<!-- protocol for script and stylesheets -->
<!-- defaults to https -->
<protocol>https</protocol>

<!-- title of the document -->
<documentTitle>ExtentReports 2.0</documentTitle>

<!-- report name - displayed at top-nav -->
<reportName></reportName>

<!-- report headline - displayed at top-nav, after reportHeadline -->
<reportHeadline>Automation Report</reportHeadline>

<!-- global date format override -->
<!-- defaults to yyyy-MM-dd -->
<dateFormat>yyyy-MM-dd</dateFormat>

<!-- global time format override -->
<!-- defaults to HH:mm:ss -->
<timeFormat>HH:mm:ss</timeFormat>


<enableCategoryView>true</enableCategoryView>
<enableAuthorView>true</enableAuthorView>
<enableExceptionView>true</enableExceptionView>
<enableTestRunnerLogsView>true</enableTestRunnerLogsView>

<!-- custom javascript -->
<scripts>
<![CDATA[
$(document).ready(function() {

});
]]>
</scripts>

<!-- custom styles -->
<styles>
<![CDATA[

]]>
</styles>
</configuration>
</extentreports>

BrowserStack App Automate for Bug reporting in Appium

You can opt for BrowserStack App Automate to report a bug for Appium testing. App Automate supports four types of integration: JIRA, Trello, GitHub, and Slack.

Here is how the Jira integration works with App Automate-

Integration with Jira

• First, go to Report Bug>Integrate with Jira>Connect to my Jira Account to start the integration process.
• For this purpose, you must provide your Jira account credentials, like hostname, email address, and Atlassian password.

Integrate App Automate with Jira

Bug reporting with Jira

• Now, the JIRA integration with BrowserStack is completed. When you find a bug in your Appium test script, click on the ‘Report Bug’ button in the toolbox. This time you are filing a bug in Jira with this button.
• Then add annotation with the ‘Annotation Toolbar’ to the screenshot.
• After that, fill in all the details about the bug. Specify the project name, type of issue, and summary. Also, you add some description about the issue if you wish so.
• You can fill in additional fields with the ‘More Fields’ button. This is optional.
• Finally, click the ‘Create Issue’ button to report the issue.

Apart from this, also you can use the Jenkins integration by App Automate for Appium testing. Creating a report is simple with this plug-in.

Steps are:

1. Go to Post-build Actions>Add post-build action.
2. Next select BrowserStack Test Report.
3. Finally, click on Save.

Now, you can see the result of the Appium test

BrowserStack Test Management for Bug Reporting in Appium

You can utilize BrowserStack’s Test Management tool to report bugs in Appium. To do this, you need to create a JUnit-XML test report. Appium provides this report by default, which can be found in the test-output folder.

Once you run the tests, you can upload this report into BrowserStack’s Test Management tool using the curl command. This action lets you import the test results onto the platform and find reports with detailed insights on the test execution.

To view the test report on Test Management, you can follow the steps below:

• Log in to the BrowserStack Test Management dashboard.
• Select the relevant project that has the test report uploaded.
• Click Test Runs.
• Open the Test Run generated from automation test execution.
• You will find all the test cases and their respective results here.

Conclusion

Additionally, it is quite straightforward to use Appium with BrowserStack and access a wide range of debugging tools, including text logs, network logs, visual logs, crash logs, and contextual log information.

BrowserStack App Automate enables you to test native and hybrid mobile applications using the Appium automation framework on thousands of real Android and iOS devices.

Get on board with the BrowserStack infrastructure to run and debug Appium tests on a real device cloud.

Debug in Appium with BrowserStack

Useful Resources for Appium

Tutorials

• How to perform Parallel Test Execution in Appium?
• Appium Visual Testing: The Essential Guide
• How to run Appium iOS Tests on Real Devices?
• How to perform Debugging in Appium
• How to Run Your First Appium Test Script
• How to report bugs in Appium UI Testing?
• How to run Appium Tests on macOS?
• XPath in Appium: Tutorial
• How to Analyze Appium Logs
• How to perform Drag and Drop using Appium
• How to test mobile app in Landscape or Portrait mode using Appium
• How to Run Same Script in Multiple Devices using Appium
• How to change Time Zones for Mobile App Testing using Appium
• How to Perform Localization of Mobile Apps using Appium
• What is Appium Inspector? (Benefits & How to Use it?)
• How to run Appium tests on Android devices
• How to scroll down to an element in Appium
• How to Download and Install Appium
• How to set up your Appium Grid
• How to test Biometric authentication using Appium?
• How to use touch actions in Appium?
• How to Automate a Real E2E User Flow involving App and Browser with Appium
• How to Inspect Element using UIAutomatorViewer in Appium
• How to Test Flutter Apps Using Appium Automation
• Understanding Appium Desktop: Tutorial
• Appium Tutorial for Mobile Application Testing
• React Native and Appium Tutorial
• Understanding FindElements in Appium

Best Practices, Tips, and Tricks

• Appium Best Practices Every Developer Must Know
• Effective Locator Strategies in Appium
• Top Appium Commands every Developer must know
• Desired Capabilities in Appium
• Overcoming Top Challenges faced in Appium Automation

Getting Started with

• Getting Started with Appium and NUnit framework
• WebdriverIO Tutorial: Getting started with Test Automation using Selenium and Appium
• Appium with Python: Getting Started with App Automation Testing
• Appium with Java: Getting Started to Run Automated Tests
• Test Windows Desktop App using Appium-Compatible WinAppDriver