Maven Cucumber Reporting

In software development, particularly in projects following Behavior-Driven Development (BDD), testing is as much about communication as validation. Cucumber Reports mainly bridge the gap between the technical team (developers, testers) and non-technical stakeholders (product owners, business analysts).

These reports are generated after executing Cucumber tests and provide a detailed analysis of the results. They go beyond simple pass/fail statistics, offering granular insights into individual test scenarios, steps, and outcomes.

What are Cucumber Reports?

Cucumber Reports are generated after running Cucumber test cases and provide an overview of the test results. They contain an overall Execution Summary, which collects findings from all test runs and provides a high-level overview of the testing progress.

Cucumber reports categorize findings by Feature-Certain Details, which provide insights for each feature file, and Tag-Based Analysis, which allows for modular reporting on certain capabilities or categories.

Detailed Execution Logs improve the usability of these reports by explicitly showing the stages that succeeded or failed, as well as crucial data for troubleshooting. These reports serve as a critical communication bridge, assisting both technical and business teams in understanding test coverage, identifying faults, and assessing the application’s overall health.

What is Maven?

Maven is a powerful build automation and dependency management tool used in Java-based projects, though it can support other languages as well. It simplifies project management by standardizing the way projects are built, tested, and deployed. Maven uses a declarative XML configuration file called pom.xml (Project Object Model), where developers specify dependencies, plugins, and build profiles.

One of Maven’s standout features is its ability to download and manage project dependencies from a central repository, saving developers the hassle of manually locating and maintaining libraries. It also defines a consistent project lifecycle, encompassing phases like compile, test, and package, ensuring a streamlined workflow.

With its extensive plugin ecosystem, Maven can integrate seamlessly with testing frameworks, reporting tools, and deployment solutions. This makes it an essential tool for teams looking to maintain scalability, improve collaboration, and automate complex build processes in modern software development.

What is Maven Cucumber Reporting?

Maven Cucumber Reporting integrates Cucumber’s test execution results with Maven to generate detailed and visually appealing reports. By leveraging Maven plugins, such as the maven-cucumber-reporting plugin, teams can produce reports that provide insights into test outcomes in multiple formats, including HTML, JSON, and XML.

These reports go beyond simple test pass/fail statuses, offering a comprehensive view of execution logs, scenario-level results, step-by-step breakdowns, and even aggregated summaries of multiple test runs.

Maven Cucumber Reporting simplifies the process of automating report generation as part of the Maven build lifecycle, making it ideal for CI/CD pipelines. Additionally, it supports advanced features like combining reports from parallel executions, embedding screenshots for failed steps, and customizing report metadata.

This integration enhances transparency, improves debugging efficiency, and makes test outcomes more accessible to both technical and non-technical stakeholders, ensuring better collaboration and quicker resolution of issues.

Benefits of Integrating Cucumber with Maven

Here are some of the top benefits of integrating Cucumber with Maven:

• Streamlined Test Execution: Automated test execution and report creation as part of the build process, reducing manual work and mistakes.
• Enhanced reporting capabilities: Creates complex, shareable reports in HTML and JSON formats, with compatibility for third-party analytics tools.
• Easy Dependency Management: Automatically downloads and updates needed libraries, assuring compatibility and saving time.
• Customizable Reports: Allows you to add tags, information, and branding to your reports, making them more targeted and professional for stakeholders.
• CI/CD Integration: Integrates seamlessly with CI/CD systems like Jenkins and GitLab, allowing for real-time test automation and reporting.

Prerequisites for Maven Cucumber Reporting

Before setting up Maven Cucumber Reporting, ensure the following prerequisites are met:

1. Java Development Kit (JDK): Install a compatible version of the JDK (Java 8 or above) and set up the JAVA_HOME environment variable.
2. Maven Installation: Install Apache Maven and add it to the system’s PATH to run Maven commands from the terminal or command prompt.
3. IDE for Development: Write and manage test scripts using an Integrated Development Environment (IDE) like IntelliJ IDEA, Eclipse, or VS Code.
4. Cucumber Feature Files: Prepare Cucumber .feature files that outline the test scenarios written in Gherkin syntax.
5. Test Runner Setup: Configure a test runner class in your project using JUnit or TestNG to execute Cucumber tests.
6. pom.xml Configuration: Ensure your pom.xml includes dependencies for Cucumber, JUnit/TestNG, and a reporting plugin like maven-cucumber-reporting.
7. Build Tool Knowledge: Familiarity with basic Maven commands (for example, mvn clean test) and lifecycle phases (clean, compile, test, package).
8. Optional Tools: For advanced reporting, tools like BrowserStack or Selenium WebDriver may be required for testing across different environments.

You can set up Maven Cucumber Reporting easily and generate insightful test reports by ensuring these prerequisites are in place.

Setting Up Maven for Cucumber Reporting

Follow the below steps to set up Maven for Cucumber Reporting:

Step 1. Add Cucumber Dependencies

These dependencies include the core libraries required for Cucumber tests.

<dependencies>

    <!-- Cucumber Java Dependency -->

    <dependency>

        <groupId>io.cucumber</groupId>

        <artifactId>cucumber-java</artifactId>

        <version>7.15.1</version>

    </dependency>



    <!-- Cucumber JUnit Dependency -->

    <dependency>

        <groupId>io.cucumber</groupId>

        <artifactId>cucumber-junit</artifactId>

        <version>7.15.1</version>

        <scope>test</scope>

    </dependency>



    <!-- JUnit Dependency -->

    <dependency>

        <groupId>junit</groupId>

        <artifactId>junit</artifactId>

        <version>4.13.2</version>

        <scope>test</scope>

    </dependency>



    <!-- Cucumber Reporting Dependency -->

    <dependency>

        <groupId>net.masterthought</groupId>

        <artifactId>maven-cucumber-reporting</artifactId>

        <version>5.7.0</version>

    </dependency>

</dependencies>

Step 2. Add Maven Surefire Plugin

The Surefire plugin is used to run tests and ensure compatibility with the reporting tools.

<build>

    <plugins>

        <!-- Maven Surefire Plugin -->

        <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-surefire-plugin</artifactId>

            <version>3.0.0-M7</version>

            <configuration>

                <includes>

                    <include>**/*Test.java</include>

                </includes>

            </configuration>

        </plugin>

    </plugins>

</build>

Step 3. Add Maven Cucumber Reporting Plugin

This plugin generates advanced reports in formats like HTML, JSON, and more.

<build>

    <plugins>

        <!-- Maven Cucumber Reporting Plugin -->

        <plugin>

            <groupId>net.masterthought</groupId>

            <artifactId>maven-cucumber-reporting</artifactId>

            <version>5.7.0</version>

            <executions>

                <execution>

                    <id>generate-report</id>

                    <phase>verify</phase>

                    <goals>

                        <goal>generate</goal>

                    </goals>

                    <configuration>

                        <projectName>Cucumber Project</projectName>

                        <outputDirectory>${project.build.directory}/cucumber-reports</outputDirectory>

                        <jsonFiles>

                            <jsonFile>${project.build.directory}/cucumber-reports/Cucumber.json</jsonFile>

                        </jsonFiles>

                        <reportTargetDirectory>${project.build.directory}/cucumber-reports</reportTargetDirectory>

                    </configuration>

                </execution>

            </executions>

        </plugin>

    </plugins>

</build>

Step 4. Complete Example pom.xml

Here is an example code for setting up Maven.

<project xmlns="http://maven.apache.org/POM/4.0.0"

         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>

    <artifactId>cucumber-reporting</artifactId>

    <version>1.0-SNAPSHOT</version>

    <dependencies>

        <dependency>

            <groupId>io.cucumber</groupId>

            <artifactId>cucumber-java</artifactId>

            <version>7.15.1</version>

        </dependency>

        <dependency>

            <groupId>io.cucumber</groupId>

            <artifactId>cucumber-junit</artifactId>

            <version>7.15.1</version>

            <scope>test</scope>

        </dependency>

        <dependency>

            <groupId>junit</groupId>

            <artifactId>junit</artifactId>

            <version>4.13.2</version>

            <scope>test</scope>

        </dependency>

        <dependency>

            <groupId>net.masterthought</groupId>

            <artifactId>maven-cucumber-reporting</artifactId>

            <version>5.7.0</version>

        </dependency>

    </dependencies>

    <build>

        <plugins>

            <plugin>

                <groupId>org.apache.maven.plugins</groupId>

                <artifactId>maven-surefire-plugin</artifactId>

                <version>3.0.0-M7</version>

                <configuration>

                    <includes>

                        <include>**/*Test.java</include>

                    </includes>

                </configuration>

            </plugin>

            <plugin>

                <groupId>net.masterthought</groupId>

                <artifactId>maven-cucumber-reporting</artifactId>

                <version>5.7.0</version>

                <executions>

                    <execution>

                        <id>generate-report</id>

                        <phase>verify</phase>

                        <goals>

                            <goal>generate</goal>

                        </goals>

                        <configuration>

                            <projectName>Cucumber Project</projectName>

                            <outputDirectory>${project.build.directory}/cucumber-reports</outputDirectory>

                            <jsonFiles>

                                <jsonFile>${project.build.directory}/cucumber-reports/Cucumber.json</jsonFile>

                            </jsonFiles>

                            <reportTargetDirectory>${project.build.directory}/cucumber-reports</reportTargetDirectory>

                        </configuration>

                    </execution>

                </executions>

            </plugin>

        </plugins>

    </build>

</project>

Generating Cucumber Reports

Cucumber reports provide a detailed summary of test execution results, including passed and failed scenarios, tags, and feature-specific details. These reports help stakeholders visualize the testing outcomes and identify areas requiring improvement.

To generate Cucumber reports:

1. Execute the Cucumber tests.
2. Ensure a JSON file (Cucumber.json) is created as output.
3. Use Maven’s reporting plugins to format the JSON into readable reports like HTML or XML.

Running Cucumber Tests with Maven

Running Cucumber tests with Maven is simple once your pom.xml and test configurations are in place.

Steps:

Step 1. Navigate to the Project Directory: Open your terminal and go to the project directory containing the pom.xml.

cd path/to/your/project

Step 2. Run Tests: Use the Maven command to execute tests.
bash

mvn clean test 

• clean: Removes previously compiled files.
• test: Executes tests based on your configuration.

Step 3. Generate Reports: After tests are executed, run the following to generate reports (if configured in the pom.xml):

mvn verify 

Generating Basic Cucumber Reports with Maven

Basic reports include details of feature execution and a breakdown of scenarios, tags, and steps.

Ensure JSON Output Configuration: Configure your test runner to generate a JSON file.
Example (JUnit Runner):

@CucumberOptions(  

    plugin = {"json:target/cucumber-reports/Cucumber.json"},  

    features = "src/test/resources/features",  

    glue = "stepdefinitions"  

)  

public class TestRunner {  

}

Then, use the above steps to run the test and generate a report, and view it.

Types of Reports (HTML, JSON, XML)

Cucumber supports multiple report formats to cater to diverse needs:

HTML Reports:

• Description: User-friendly, interactive, and visually appealing.
• Usage: Great for sharing results with stakeholders.
• Generated by: Maven plugins like maven-cucumber-reporting.

JSON Reports:

• Description: A structured format for machine-readable data.
• Usage: Useful for integration with analytics tools or custom parsers.
• Generated by: Specifying json in the @CucumberOptions.

XML Reports:

• Description: Hierarchical and structured, suitable for tool integrations.
• Usage: Commonly used in CI/CD pipelines for report aggregation.
• Generated by: Configuring plugins like Surefire or custom settings.

Example: Specifying Multiple Formats in @CucumberOptions

@CucumberOptions(  

    plugin = {  

        "pretty",  

        "html:target/cucumber-reports/cucumber.html",  

        "json:target/cucumber-reports/cucumber.json",  

        "junit:target/cucumber-reports/cucumber.xml"  

    },  

    features = "src/test/resources/features",  

    glue = "stepdefinitions"  

)  

public class TestRunner {  

} 

With these configurations, you can generate detailed reports in our desired formats and use them effectively for analysis and communication.

Talk to an Expert

How to Customize Cucumber Reports?

Customizing Cucumber reports allows teams to align test results with organizational standards and enhance their usability. With customization, you can add meaningful details like tags, metadata, branding, and formatting to improve clarity and professional presentation.

1. Configure Report Plugins (example: Cucumber HTML Report)

Plugins transform raw test output (JSON, XML) into user-friendly formats like HTML or interactive dashboards.

Steps to Configure Cucumber HTML Report Plugin:

Step 1. Add Plugin Dependency in pom.xml: Add the maven-cucumber-reporting plugin to your project.

<plugin>  

    <groupId>net.masterthought</groupId>  

    <artifactId>maven-cucumber-reporting</artifactId>  

    <version>5.7.0</version>  

    <executions>  

        <execution>  

            <id>generate-report</id>  

            <phase>verify</phase>  

            <goals>  

                <goal>generate</goal>  

            </goals>  

        </execution>  

    </executions>  

</plugin>

Step 2. Run Tests and Generate Reports: After running your tests, use the mvn verify command to generate HTML reports automatically.

Step 3. View the Report: Open the HTML file in your browser, typically found at:

target/cucumber-reports/advanced-reports.html

2. Adding Tags, Scenarios, and Custom Metadata

Tags and metadata make reports more descriptive and easier to filter.

Steps to Add Tags and Metadata:

Step 1. Using Tags in Feature Files: Assign tags to scenarios or features for better categorization.

@Login @SmokeTest  

Feature: Login functionality  

    Scenario: Successful login  

        Given the user is on the login page  

        When they enter valid credentials  

        Then they are redirected to the dashboard  

Tags like @SmokeTest allow filtering tests during execution and help categorize reports.

Step 2. Custom Metadata in Plugin Configuration: Add metadata (for example, browser, environment, test type) to your report by configuring the plugin in pom.xml.

<configuration>  

    <projectName>My Project</projectName>  

    <buildNumber>1.0</buildNumber>  

    <tagsOverview>true</tagsOverview>  

    <metadata>  

        <browser>Chrome</browser>  

        <environment>Staging</environment>  

        <platform>Windows</platform>  

    </metadata>  

</configuration>

3. Custom Report Formatting Options

Enhance the readability and branding of your reports by configuring the layout and content formatting.

Options:

1. Set Theme or Layout: Some plugins allow theme selection (light or dark) for improved readability.
2. Add Branding (Logos/Names): Incorporate company logos and custom titles for a professional look using the plugin’s configuration options.
3. Include Detailed Logs: Enhance debugging by configuring the plugin to include stack traces and screenshots for failed steps.

4. Including Screenshots in Reports

Screenshots add significant value for failed tests. Use hooks in your step definitions to capture screenshots.

@After  

public void embedScreenshot(Scenario scenario) {  

    if (scenario.isFailed()) {  

        final byte[] screenshot = ((TakesScreenshot) DriverFactory.getDriver()).getScreenshotAs(OutputType.BYTES);  

        scenario.attach(screenshot, "image/png", "Screenshot");  

    }  

}

Advanced Maven Cucumber Reporting with BrowserStack Test Management

Integrating Maven Cucumber reporting with BrowserStack’s Test Management capabilities unlocks advanced reporting and efficient test execution on real devices. This integration combines the power of detailed Cucumber reports with BrowserStack’s ability to run tests on a vast range of browsers, devices, and operating systems.

Use Maven’s maven-cucumber-reporting plugin to generate advanced HTML reports enriched with test execution data from BrowserStack, such as device details, OS, and browser versions. Reports can also include tags, scenarios, and metadata for context-specific analysis.

To create a report with BrowserStack, you need to perform the below setups :

1. Add BrowserStack and Cucumber Dependencies

Add the required dependencies for BrowserStack and Cucumber in your pom.xml.

<dependencies>

    <!-- Cucumber Dependencies -->

    <dependency>

        <groupId>io.cucumber</groupId>

        <artifactId>cucumber-java</artifactId>

        <version>7.12.0</version>

    </dependency>

    <dependency>

        <groupId>io.cucumber</groupId>

        <artifactId>cucumber-junit</artifactId>

        <version>7.12.0</version>

    </dependency>

    <!-- BrowserStack Local Testing Dependency -->

    <dependency>

        <groupId>com.browserstack</groupId>

        <artifactId>browserstack-local-java</artifactId>

        <version>1.0.4</version>

    </dependency>

    <!-- Maven Cucumber Reporting -->

    <dependency>

        <groupId>net.masterthought</groupId>

        <artifactId>maven-cucumber-reporting</artifactId>

        <version>5.7.0</version>

    </dependency>

</dependencies>

2. Set Up BrowserStack Capabilities

Create a class to configure BrowserStack-specific capabilities.

import org.openqa.selenium.remote.DesiredCapabilities;

import org.openqa.selenium.remote.RemoteWebDriver;

import java.net.URL;

public class BrowserStackSetup {

    public static RemoteWebDriver driver;

    public static void setupBrowserStack() throws Exception {

        DesiredCapabilities caps = new DesiredCapabilities();

        caps.setCapability("os", "Windows");

        caps.setCapability("os_version", "10");

        caps.setCapability("browser", "Chrome");

        caps.setCapability("browser_version", "latest");

        caps.setCapability("name", "Cucumber Test");

        caps.setCapability("build", "Cucumber-BrowserStack-Integration");

        caps.setCapability("browserstack.debug", "true"); // Debug mode for screenshots and logs

        // BrowserStack Credentials

        String USERNAME = "your_browserstack_username";

        String ACCESS_KEY = "your_browserstack_access_key";

        driver = new RemoteWebDriver(

                new URL("https://" + USERNAME + ":" + ACCESS_KEY + "@hub-cloud.browserstack.com/wd/hub"),

                caps);

    }

    public static void tearDown() {

        if (driver != null) {

            driver.quit();

        }

    }

}

3. Modify Step Definitions

Incorporate BrowserStack’s debugging and session URLs into the reports.

import io.cucumber.java.After;

import io.cucumber.java.Before;

public class Hooks {

    @Before

    public void setUp() throws Exception {

        BrowserStackSetup.setupBrowserStack();

    }

    @After

    public void tearDown() {

        // Print BrowserStack session details for reporting

        System.out.println("BrowserStack Session URL: https://automate.browserstack.com/sessions/" +

                            ((RemoteWebDriver) BrowserStackSetup.driver).getSessionId());

        BrowserStackSetup.tearDown();

    }

}

These changes allow you to run tests on real devices via BrowserStack, generate advanced Maven Cucumber reports, and embed BrowserStack session details for enhanced debugging and traceability.

To upload the above execution report, which is generated in JSON follow this URL.

Why use Test Management for Reporting in BrowserStack Cucumber?

Integrating BrowserStack’s Test Management capabilities with Maven Cucumber reporting offers a unified approach to monitoring, analyzing, and optimizing your test execution process. Here’s why this combination is beneficial.

• Centralized Reporting: Test management in BrowserStack provides a single platform for storing and organizing test results from multiple runs across devices, browsers, and environments. This centralization enhances collaboration among teams and eliminates the need to combine test data manually.
• Detailed Debugging Artifacts: BrowserStack automatically captures detailed logs, screenshots, and video recordings of test executions. These artifacts are essential for debugging failures and are easily embedded into Maven Cucumber reports, making the reports actionable and insightful.
• Traceability and Accountability: Each test session on BrowserStack generates a unique session ID, enabling precise tracking of individual tests. Linking this data with Maven Cucumber reports ensures clear traceability, especially for large test suites with multiple scenarios.
• Support for Parallel and Cross-Browser Testing: BrowserStack enables parallel execution of tests across various browser and OS combinations. Test management tools consolidate these results, making it easier to analyze trends and pinpoint platform-specific issues.
• Real-Time Insights for Stakeholders: Integrated reporting provides real-time updates on test execution, ensuring that stakeholders can access comprehensive and up-to-date information. This accelerates the decision-making process and enhances collaboration between QA and development teams.

Conclusion

Cucumber reporting plays a main role in transparency, detailed insights, and actionable feedback throughout the test lifecycle. When combined with powerful tools like BrowserStack, it transforms how teams approach testing and reporting.

BrowserStack Automate enables parallel testing, allowing teams to execute multiple test cases simultaneously across a wide range of devices and browsers. This drastically reduces test execution time and ensures faster feedback cycles, especially for large test suites. It also integrates seamlessly with CI/CD pipelines for a smooth workflow and helps to debug artifacts (screenshots, logs, videos)..

Integrating BrowserStack Test Management with Maven Cucumber Reporting enables centralized reporting and real-time insights into test progress. These features help teams stay organized, pinpoint issues efficiently, and make informed decisions faster. This eliminates risks associated with emulators or simulators, delivering reliable and accurate results.

Try BrowserStack Now