How to use Coverage.py to Measure Code Coverage in Python Projects

Building robust software goes beyond writing code; it requires creating tested, reliable code that functions as expected. As the codebase grows, tracking what works and what could go wrong becomes challenging. While testing helps catch bugs, measuring test effectiveness can be tricky.

This is where code coverage tools like Coverage.py are useful. They allow developers to see which parts of the code are tested and highlight areas for improvement. With Coverage.py, you can measure test coverage and enhance the quality of your Python projects.

This article will guide you in using Coverage.py to measure code coverage in Python projects.

What is Code Coverage?

Code coverage is a metric that evaluates the extent to which your source code executes during testing. It helps identify untested areas of a program and ensures the codebase is thoroughly validated.

It covers statements, branches, and functions. This helps detect bugs and ensures full validation. High coverage does not guarantee a bug-free application, but it strengthens testing strategies.

What is Coverage.py?

Coverage.py is a tool that measures code coverage in Python programs. It tracks which parts of the code are executed during testing, helping developers identify untested sections.

By integrating with testing frameworks like unit test and pytest, Coverage.py provides reports in various formats, including text and HTML, to visualize coverage data.

This insight aids in improving test effectiveness and overall code quality.

Key Features of Coverage.py

The main features of Coverage.py are:

• Execution Monitoring: This module uses tracing hooks in Python to track what parts of the code are being executed during the test runs.
• Analysis: It analyzes the code to determine which lines might have been executed and helps identify sections not covered by tests.
• Reporting: This module produces reports in several formats, such as text and HTML, to visualize the data on code coverage.
• Integration: It supports testing frameworks like unit test and pytest.

Why is Code Coverage Measuring important in Python projects?

Code coverage in Python projects should be measured for several reasons:

• Quality assurance: Higher code coverage signifies proper testing with minimal chances of undetected bugs reaching production.
• Risk mitigation: If the developer does not test codes, it allows focusing testing on specific critical areas.
• Code Maintenance: This aids in code maintenance by providing insurance, allowing regressions to be ruled out while any changes are incorporated.
• Documentation: The reports generated by code coverage tools could be used to document the areas of testing where the code lacked coverage and where more attention was necessary.

Installing Coverage.py

To install Coverage.py, follow these steps:

Prerequisites

Ensure Python is installed on your system. You can verify this by running the Python –version or python3 –version in your terminal.
Confirm that pip, Python’s package installer, is up to date.

Update it using:

python -m pip install --upgrade pip

Installation:

Install Coverage.py using pip:

python -m pip install coverage

For enhanced performance, Coverage.py includes a C extension. To ensure it’s installed:

On Linux, you may need to install development tools:

sudo apt-get install python3-dev gcc

After installation, verify the C extension is active:

coverage --version

Verification:

Check the installed version to confirm successful installation:

coverage –version

You can also invoke Coverage.py as a module:

python -m coverage --version

Both commands should display the version number and confirm the installation.

How to use Coverage.py to Measure Code Coverage

Understanding Coverage.py can be done with a simple example of Python code that performs basic addition and subtraction.

Running Tests with Coverage.py

To begin, ensure you have the necessary dependencies. Open your terminal and run the following command to install coverage and pytest:

pip install coverage pytest

You’ll need two Python files:

main.py (the code to test) and test_main.py (the test file).

main.py

This file contains the functions add() and subtract(). These are simple functions that perform addition and subtraction of two numbers.

# main.py



def add(a, b):

    return a + b



def subtract(a, b):

    return a - b

test_main.py

This file contains the test functions that use pytest to test the functionality of the functions in main.py.

# test_main.py


from main import add, subtract



def test_add():

    assert add(2, 3) == 5



def test_subtract():

    assert subtract(5, 3) == 2

To run the tests and measure code coverage, use coverage.py. In the terminal, navigate to the project directory and run:

coverage run -m pytest

This will run the tests from test_main.py while measuring which lines of code were covered by the tests.

Generate the Coverage Report

You can generate a code coverage report after running the tests. First, run this command to generate a simple report in the terminal:

coverage report

This will show a coverage summary indicating which parts of your code were tested. For example, the output might look like this:

This indicates that the tests covered all the code in both main.py and test_main.py.

For a more detailed, visual coverage report, you can generate an HTML report using the following command:

coverage html

This will create an htmlcov/ directory with an index.html file inside it.

Open the index.html file in your browser to see a visual representation of your code coverage, highlighting covered green and uncovered red lines.

Integrating Coverage.py with CI/CD Pipelines

Integrating Coverage.py into your CI/CD pipeline automatically measures test coverage with every commit, providing continuous feedback on the effectiveness of your tests.

This integration is essential for maintaining high code quality and ensuring that new changes don’t reduce overall test coverage.

To integrate Coverage.py into your pipeline, follow these general steps:

• Choose a CI/CD Platform: Popular CI/CD services like GitHub Actions, GitLab CI, CircleCI, Travis CI, and Jenkins offer easy integration with testing tools like pytest and Coverage.py. Choose a platform based on your project’s needs.
• Create a CI/CD Configuration File: Each CI/CD platform requires a configuration file that defines the steps to run tests and gather code coverage. For instance, in GitHub Actions, you would create a .github/workflows directory and a YAML file specifying the steps to install dependencies, run tests, and generate the coverage report.
• Install Coverage.py: You need to ensure that Coverage.py is installed in your CI pipeline. You can do this by including it in the dependencies section of your configuration file or by running pip install coverage during the pipeline setup.
• Run Tests with Coverage: To gather coverage data, the pipeline should run your test suite (typically pytest) with Coverage.py. The command coverage run -m pytest can run tests with coverage measurement.
• Generate Coverage Reports: Coverage.py can generate coverage reports once the tests are complete. These reports can be output to the terminal (using coverage report), or you can generate an HTML report using coverage html. The HTML report is useful for visually reviewing the areas of the code that are well-tested and those that require more tests.
• Store and Display Coverage Data: After running the tests and generating the coverage report, store the report as an artefact. Most CI/CD platforms, like GitHub Actions, allow you to upload the coverage report as an artefact, making it available for download or review.
• Display Code Coverage Badge: Many CI/CD services, like Codecov and Coveralls, allow displaying a badge that shows your current code coverage percentage. You can add this badge to your repository’s README file to provide an at-a-glance view of your test coverage.

Best Practices for Code Coverage in Python

The following tips can help you improve code coverage in Python:

• Strive for high test coverage (e.g., 80-90%), but remember that 100% coverage does not guarantee the quality of your tests. Focus on meaningful tests for critical code paths.
• Ensure your tests cover edge cases, error handling, and exceptional conditions to verify that your code behaves as expected under various scenarios.
• Regularly generate and review coverage reports to identify untested areas of your codebase and improve test coverage where necessary.
• Skip tests for simple or trivial code (e.g., setters and getters, boilerplate code) unless it is required for logic or integration.
• Prioritise tests that cover your application’s core business logic, as these are more likely to affect functionality and user experience.
• Test the integration points with external libraries to ensure they work correctly in your environment.
• If parts of the codebase are untested, consider refactoring them to improve testability and add tests for the newly refactored code.
• Run tests across various environments, configurations, or operating systems to ensure the application works in all scenarios.
• Integrate code coverage tools with your CI/CD pipelines to track coverage on every commit and catch issues early.
• While code coverage is a helpful metric, don’t rely on it alone to assess the quality of your tests. Focus on test quality and relevance rather than just achieving high coverage percentages.

Test on Real Devices with BrowserStack

BrowserStack is a cloud-based testing platform that allows you to run automated and manual tests on real devices and browsers. It lets you test your Python applications on real environments without setting up your device farm.

Here’s how you can use BrowserStack for testing your Python projects:

• Access Real Device Cloud: BrowserStack provides access to a wide range of real devices, including smartphones, tablets, and desktop devices. This is crucial for testing mobile responsiveness and ensuring your application works seamlessly across different devices and screen sizes.
• Run Automated Tests: You can integrate BrowserStack with your test automation tools like pytest or Selenium. This integration allows you to run your automated tests on real devices, ensuring your application behaves correctly in different environments.
• Live and Interactive Testing: In addition to automated testing, BrowserStack offers live testing capabilities where you can manually test your Python application on real devices. This feature helps identify UI/UX issues, performance bottlenecks, and device-specific bugs.
• Cross-Browser Testing: You can test your Python application across various browsers like Chrome, Firefox, Safari, and Edge to ensure compatibility and consistent performance.
• Integrate with CI/CD: BrowserStack supports integration with CI/CD tools to run tests on real devices as part of your build pipeline. This integration ensures that your application is thoroughly tested with each deployment.
• Generate Test Reports: BrowserStack allows you to generate detailed test reports for automated and manual tests. These reports include logs, screenshots, and video recordings, making analyzing and debugging issues easier.

Talk to an Expert

Conclusion

Measuring code coverage in Python projects with Coverage.py is essential for maintaining high-quality code. By tracking which parts of your code are tested, Coverage.py helps identify gaps and improve test coverage.

Integrating it into CI/CD pipelines ensures continuous testing with each commit. Best practices include aiming for high but realistic coverage, focusing on critical code paths, and using coverage reports to guide test improvements.

Tools like BrowserStack further enhance testing by enabling real-device testing, ensuring your Python application performs well across various environments.