Static testing – Tools and Techniques

To deliver high-quality software, it is essential to test the application in various ways. Static testing involves analyzing the software documentation, code, and functional specifications.

By performing static tests in the early stages of development, you can avoid potential defects and improve Code Quality. Since you identify bugs early, it can save you the cost and deliver quality software in production.

What is Static Testing?

Static testing involves reviewing and testing when the code is in a static state rather than in an execution state. In this type of testing, you will identify the issues earlier in the starting phase of the software development life cycle.

By catching defects early, Static Testing can help improve the software’s overall quality and reliability. Also, it enhances maintainability and ultimately saves time and money in the long run.

Why should you perform Static testing?

Static testing is involved during the early stage of Software development, which allows you to identify issues early and avoid the complexity of the bug.

Since you can identify bugs early, this will save cost. You can improve the coding quality by going through the code and documentation as a part of static testing. When the code is still in development, you can proceed with this testing.

You can validate the requirement and review documents under this testing to understand the product while it is being developed and avoid rework.

Techniques for Static Testing

There are two techniques for static testing – Review Analysis and Static Analysis.

Review Analysis

With this analysis, you will remove the error from the software’s design. These techniques help detect errors early in the development process, improve quality, and reduce costs. The main techniques for review analysis include:

• Informal review: An unstructured and casual review process where team members, often peers, provide feedback on the artifact without a formal meeting.
• Walkthrough: A semi-formal review where the author leads a discussion about the content (for example, code or document) to clarify and gather reviewer feedback.
• Inspections: A formal and rigorous process where a trained moderator leads a team in examining a document or code to identify defects systematically.

Static Analysis

Static analysis technique involves analyzing the software code, and functional specifications. By performing static tests in the early stages of development you can avoid potential defects and improve Code Quality. Below are the main techniques of static analysis.

• Technical review: A structured review process involving technical experts who evaluate the product for adherence to standards, architecture, and design quality.
• Static code analysis: Automated tools that analyze source code for potential defects, Code Quality issues, and coding standards violations without executing the code.

When should you perform Static Testing?

Static software testing should be performed in the early stages of the Software Development Life Cycle. This allows you to identify and fix bugs before the software is executed, which saves time when identifying and fixing bugs later on.

Below are the different stages where you need to perform static testing.

• Stage 1: Requirement phase: When the requirements are documented and available for review, you can start with static testing. With this, you can make sure that the requirements are clear, complete, consistent, and unambiguous. This reduces the risk of misunderstandings or missing requirements later in development.
• Stage 2: UX design phase: Once the design documents are prepared and the software development is about to start, you can perform static testing here. To catch design flaws, identify potential performance bottlenecks, and ensure that the design adheres to technical and functional requirements.
• Stage 3: Coding phase: As the software is being developed, you can perform static testing, especially before proceeding with dynamic testing.To find coding errors, and syntax violations, and ensure adherence to coding standards without executing the code. This can be done through peer reviews, code walkthroughs, or static code analysis tools.
• Stage 4: Test planning phase: After you get the test case plan, you can proceed with static testing. This will ensure that the test cases cover all the requirements and that there are no ambiguities or missing test scenarios. Reviewing test documentation ensures that the tests will be effective in detecting defects.
• Stage 5: Maintenance phase: Post getting the complete software, whenever there are any changes made during the maintenance phase, you can perform static testing. This is to ensure that changes or enhancements are properly documented and that they do not introduce new issues.

Types of Static Testing

Static testing can either be done manually or automated through coding.

Manual Static Testing

Manual static testing is the software testing process where software artifacts such as requirements, design documents, and source code are reviewed and analyzed without executing the software.

This type of testing is essential in identifying defects early in the software development life cycle, contributing to better quality assurance, reduced rework, and cost savings. It relies on human effort and expertise to inspect documents and code, ensuring alignment with project requirements and design expectations.

• Casual, unstructured evaluations where team members provide quick feedback on the software artifacts.
• Colleagues review each other’s work, particularly source code, to find defects or areas for improvement.
• The author of a document or code presents their work to a group, explaining its purpose and functionality. The group then provides feedback, helping to identify any discrepancies or areas of confusion.
• A formal and rigorous review process led by a moderator, where participants follow a structured approach (often using checklists) to detect defects. Inspections have defined roles, including the Moderator, Reviewers, Scribe, and Author.

Automated Static Testing

Automated static testing is a non-execution-based testing process that involves using specialized software tools to analyze code, documents, or configurations for potential defects, vulnerabilities, and violations of coding standards. Unlike manual testing, automated static testing applies predefined rules, algorithms, and models to examine artifacts without human intervention. It is particularly valuable in large-scale projects, continuous integration (CI) environments, and when frequent testing is required.

• Static code analysis tools such as BrowserStack Code Quality, SonarQube, ESLint, Checkstyle, and PMD are used to perform this testing. With this, you can identify errors such as coding standard violations, security vulnerabilities, and common programming mistakes.
• You also have other tools that could identify security vulnerabilities in the code that you write. Tools such as OWASP Dependency-Check, Fortify, and Checkmarx automatically scan the code for known security issues, such as SQL injection, cross-site scripting (XSS), and buffer overflows.

Top Tools for Static Testing

There are various tools available in the market for proceeding with static testing. Let us view a few of those tools one by one.

1. BrowserStack Code Quality tool

With BrowserStack Code Quality tools, you can monitor the Code Quality while they are being developed. You can import our code from the remote repositories and start doing the analysis. Get a thorough code repository analysis in minutes.

This tool identifies design anti-patterns, vulnerabilities, code issues, metric violations, and duplication. Critical vulnerabilities like memory leaks won’t go unnoticed in our code base.

Key features:

• Continuous Quality monitoring: You can set up automated quality checks in our CI/CD pipeline and benchmark our code using industry-leading KPIs.
• Security and Compliance: BrowserStack Code Quality helps identify security vulnerabilities and ensure applications meet industry standards
• Multi-dimensional Scan: It detects design anti-patterns, vulnerabilities, code issues, code metric violations, and code duplication.
• Find issues through plugin: Fix code issues before making commits—simply download the plugin for your IDE below, and uncover bugs, security issues, and code smells with fast scanners that get you quick feedback.
• AI recommendation: Machine Learning engine that suggests probable solutions to our code issues based on insights from our past fixes.
• Refactoring code: Recommendations by a partitioning assistant on how to break down large classes to improve maintainability.

Why choose Browserstack Code Quality?

BrowserStack Code Quality comes with the latest features, such as plugins for correcting the code from our IDE, such as IntelliJ, VScode, and Android Studio.

It empowers developers to maintain high coding standards by detecting vulnerabilities, anti-patterns, and code metric violations across repositories. Integrated with CI/CD pipelines, it automates quality checks and it ensures issues are caught early to improve code robustness and security.

Multi-dimensional scanning and real-time IDE feedback support continuous monitoring and compliance with industry standards.

Also, it has AI recommendations to suggest the possible best solutions to our code issues based on insights from our past. Also, it supports various languages. All these features make Browserstack Code Quality unique and attract us to make use of it in our code more than regular static testing tools.

2. Checkstyle

A static code analysis tool called Checkstyle was created mostly with Java applications in mind. Automatically examining the source code and highlighting any violations of these rules assists developers in adhering to a set of coding standards.

Checkstyle’s primary goal is to make code more readable, maintainable, and high-quality by making sure it adheres to the best standards.

Key features:

• CI/CD integration: Checkstyle can generate reports detailing the violations found in the code. These reports can be integrated into continuous integration (CI) pipelines using tools like Jenkins, Maven, Gradle, etc. so that developers are immediately aware of any issues after each build.
• Coding standards and rules: Checkstyle checks whether the code follows a predefined set of coding conventions. Developers can also configure Checkstyle with custom rules to enforce specific coding guidelines.
• Plugin support: Checkstyle integrates with popular IDEs like Eclipse, IntelliJ IDEA, and NetBeans, allowing developers to check their code in real-time while writing. This helps catch issues as they code rather than waiting until later in the development process.

Pros of checkstyle:

• Checkstyle helps enforce consistent coding practices across the entire codebase, making the code easier to read and understand. This is especially beneficial for large teams, where multiple developers contribute to the same project.
• By enforcing proper formatting and best practices, Checkstyle improves the readability of code, making it easier to review, debug, and maintain.
• Teams can enable or disable specific rules based on their coding preferences. For example, some teams might enforce strict naming conventions, while others may only focus on formatting.

Cons of checkstyle:

• Checkstyle doesn’t focus on the security part of coding and it only focuses on the coding standard perspective.
• Checkstyle supports only Java, limiting its usefulness in multi-language projects.
• Only focuses on coding style and doesn’t cover metrics like code complexity or test coverage.

3. SourceMeter

SourceMeter is a powerful static analysis tool that provides deep insights into the quality, maintainability, and security of code. It helps measure key software metrics, identify code smells, and estimate technical debt, helping developers improve Code Quality and reduce technical debt over time.

Its multi-language support, CI/CD integration, and IDE plugins make it a valuable addition to development pipelines for teams looking to enforce high coding standards and maintain a healthy codebase.

Key features:

• Code quality analysis: SourceMeter provides information about code duplication, function length, and cyclomatic complexity, which helps developers assess how easy the code is to modify or extend.
• Static code metrics: SourceMeter calculates various static metrics related to software design and implementation. These metrics include Cyclomatic complexity, Line of Code, and Depth of inheritance.
• Security threat detection: SourceMeter helps in detecting common vulnerabilities like SQL injection, cross-site scripting (XSS), or buffer overflows in C/C++ code.

Pros of SourceMeter:

• SourceMeter provides multiple language support such as Java, C/C++, Python, and more, making it versatile for multi-language projects. It also provides a broad set of code metrics such as maintainability index, complexity, size metrics, and others, which help in understanding the code’s quality.
• SourceMeter provides detailed reports on Code Quality, with customizable charts and graphs that help developers and managers make data-driven decisions. It can track Code Quality trends over time, helping to identify areas of concern before they become critical.
• It integrates smoothly into CI/CD pipelines, enabling continuous monitoring of Code Quality throughout the development lifecycle.

Cons of SourceMeter:

• While basic features may be available for free or under open-source licenses, some advanced features or capabilities may require a paid license, which could be a concern for small teams or companies with budget constraints.
• For teams or individuals unfamiliar with static code analysis tools, SourceMeter can have a steep learning curve, especially with its advanced metrics and detailed reporting. Interpreting the results, particularly more complex metrics like maintainability index or technical debt may require a deeper understanding of software quality principles.
• SourceMeter focuses on static code analysis and doesn’t provide runtime analysis or test coverage features like dynamic analysis tools. Teams may need to use additional tools for those purposes.

4. Soot

Soot started as a Java optimization framework. By now, researchers and practitioners from around the world use Soot to analyze, instrument, optimize, and visualize Java and Android applications.

Soot offers a range of analysis, from basic control-flow and data-flow analysis to more sophisticated points-to analysis. This variety makes it a valuable tool for identifying potential bugs, optimizing performance, and verifying code structure, all without executing the program.

Key features:

• Soot provides these intermediate representations for analyzing and transforming Java bytecode: Baf, a streamlined representation that is simple to manipulate; Jimple, a typed 3-address intermediate representation suitable for optimization; and Shimple, an SSA variation of Jimple.
• Soot helps in doing various static code analyses such as Data flow analysis, call graph construction, and Control flow graph construction.
• Soot allows the transformation and optimization of Java bytecode. Developers can modify the program at different IR levels, such as optimizing loops, eliminating dead code, or inlining methods.

Pros of Soot:

• Soot’s flexibility with different IRs, such as Jimple and Shimple, makes it adaptable for various static analyses and transformations.
• Soot’s ability to directly transform Java bytecode makes it a powerful tool for both static analysis and optimization.
• Due to its flexibility and extensibility, Soot is commonly used in academic research and educational settings to teach static analysis and optimization techniques.

Cons of Soot:

• Soot focuses primarily on Java bytecode, meaning it may not be suitable for projects that involve other languages.
• Soot can be complex to understand and use for beginners, especially those new to static code analysis or bytecode-level analysis. The extensive API requires a good understanding of Java bytecode and static analysis concepts.
• Static analysis using Soot, especially on large projects, can be slow because of its detailed analysis of bytecode. This may require additional optimization when working with large-scale software.

5. Lint

Lint is a static code analysis tool that identifies potential errors, bugs, stylistic errors, and suspicious constructs in source code.

Originally developed for C programming, the term “lint” has since become generalized and refers to any static analysis tool designed to flag questionable patterns in code across various programming languages.

Key features:

• Lint tools can catch basic syntax errors like missing semicolons, unclosed brackets, or invalid function calls and also it identifies potential logical errors, such as misuse of conditionals, faulty loop constructs, or problematic expressions that could lead to unexpected behavior during runtime.
• Modern Lint tools often include static security analysis, identifying common security flaws like SQL injection vulnerabilities, improper input validation, or unprotected API keys in the code.
• There are language-specific lint available that help us to capture errors for different languages.

Pros of Lint:

• With Lint added, you can identify bugs during the early development stage and avoid getting impacted during the execution phase. By enforcing style guides, Lint tools ensure that teams write code that is consistent, clean, and easier to read. This is especially useful in large teams working on shared codebases.
• Linting tools are easily integrated into continuous integration/continuous delivery (CI/CD) pipelines, ensuring that every commit meets quality standards before being merged into the main codebase.
• Most linting tools allow developers to customize the set of rules or style guides they want to enforce, enabling flexibility depending on the project’s specific requirements.

Cons of Lint:

• Lint tools can raise red flags for valid code, leading to unnecessary corrections. This can be especially frustrating if the tool isn’t configured properly for the specific project.
• While linting is effective for catching many common issues, it can miss more complex runtime errors or logical bugs that would only surface when the program is executed. Dynamic testing is required to catch those.
• Lint tools focus primarily on surface-level issues such as syntax, code style, and simple logical errors.

6. SonarQube

SonarQube is a popular open-source platform designed to continuously inspect Code Quality. It performs static code analysis to detect bugs, code smells, vulnerabilities, and other quality issues.

SonarQube integrates with various programming languages, CI/CD pipelines, and development tools to provide comprehensive Code Quality monitoring throughout the development lifecycle.

Key features:

• SonarQube supports almost 25 programming languages, which makes it versatile and suitable for projects using multiple technologies.
• SonarQube is a perfect tool for determining Code Quality analysis which can detect security vulnerabilities, and assess the complexity of code (for example, cyclomatic complexity, duplications, etc.).
• SonarQube can analyze code changes in pull requests, providing early feedback on Code Quality before the changes are merged into the main branch.

Pros of SonarQube:

• SonarQube offers a wide range of metrics, including maintainability, security vulnerabilities, code coverage, duplications, and complexity, ensuring a holistic view of Code Quality.
• The ability to define custom rules and gates ensures that each team or project can tailor SonarQube to their specific needs and coding standards.
• SonarQube scales well for large codebases and multiple projects, making it suitable for enterprises managing many repositories and large development teams.

Cons of SonarQube:

• Running SonarQube on large projects or within CI pipelines can slow down the build process due to the extensive code analysis it performs, especially for multi-language projects or if many rules are activated.
• The free Community Edition has limited features compared to the paid versions, especially in terms of advanced security analysis, multi-branch support, and enterprise-level scalability.
• Like most static analysis tools, SonarQube can generate false positives, flagging code as problematic when it may not be. Proper rule configuration and fine-tuning are necessary to reduce unnecessary warnings.

7. PMD

PMD is an extensible multi-language static code analyzer. It finds common programming flaws like unused variables, empty catch blocks, unnecessary object creation, and so forth.

It’s mainly concerned with Java and Apex but supports 16 other languages. It is primarily used to improve Code Quality, and maintainability, and to enforce coding standards.

Key features:

• PMD identifies code that might work but is written in a way that could cause problems in the future. It detects potential bugs and logical errors in the code, such as uninitialized variables, unreachable code, and unused code.
• PMD uses a large set of predefined rules to detect issues in the code. These rules are customizable, and developers can create their own rules based on the specific needs of their projects.
• PMD includes a tool called Copy-Paste Detector (CPD), which helps in identifying code duplication across the codebase. Code duplication is a common source of technical debt as it increases the risk of inconsistencies and future bugs.

Pros of PMD:

• PMD is relatively lightweight and fast compared to other static analysis tools, making it suitable for integration into IDEs and CI/CD pipelines without significant performance overhead.
• PMD is free and open-source, making it accessible for developers and teams looking for a robust static analysis tool without licensing costs.
• PMD’s rules are fully customizable, and users can create new rules based on their project requirements, making them adaptable for various use cases.

Cons of PMD:

• While PMD supports multiple languages, the depth of support and the number of rules for languages other than Java are limited. For example, support for languages like JavaScript or Python is not as robust as Java.
• PMD primarily focuses on Code Quality, design issues, and performance. While it can detect some basic security vulnerabilities, it is not as focused on security analysis compared to tools like SonarQube or Checkmarx.
• PMD focuses solely on static analysis and does not offer dynamic analysis capabilities like runtime behavior checks or performance profiling, meaning it won’t catch issues that occur during code execution.

8. Findbugs

FindBugs is an open-source tool used to perform static analysis on Java code. It scans Java bytecode (compiled code) to detect potential bugs, code smells, and other issues that might affect the quality, maintainability, or security of the application.

Unlike source-level tools, FindBugs analyzes the compiled bytecode rather than the source code itself, giving it unique strengths in detecting certain types of errors that are hard to spot by source-level tools.

Key features:

• FindBugs detects a wide variety of bug patterns, which are recurring issues in code that often lead to bugs or poor performance. It can detect bad practices, performance issues, multithreading issues, security bugs, and code vulnerabilities.
• Unlike tools that analyze source code, FindBugs inspects Java bytecode. This gives it the ability to catch bugs related to how the Java compiler transforms source code into executable code, which might be missed by source-level tools.
• FindBugs supports plugins that allow users to extend their capabilities or add custom rules to detect specific issues not covered by the standard ruleset.

Pros of Findbugs:

• The tool covers various types of issues, from correctness to performance and multithreading problems, making it a well-rounded choice for Java code analysis.
• Analyzing bytecode allows FindBugs to find issues that source-level tools might miss, especially those related to how the Java compiler translates code into executable form.
• As an open-source tool, FindBugs offers a powerful set of features without licensing costs, making it accessible to individual developers and large teams alike.

Cons of Findbugs:

• FindBugs is restricted to Java code. This makes it less versatile for teams working with multiple programming languages.
• While FindBugs can detect some security-related issues, it lacks the comprehensive security analysis found in dedicated security tools such as Checkmarx or SonarQube.
• Since FindBugs analyzes bytecode, it cannot directly check the source code style, coding standards, or other source-level concerns. Tools like PMD and Checkstyle, which work at the source level, are better suited for enforcing coding standards.

How to perform Static Testing?

Static testing is performed through techniques like code reviews, walkthroughs, inspections, and the use of automated static analysis tools. Here’s a step-by-step guide on how to perform static testing effectively:

Step 1: Prepare a checklist on what needs to be tested: It may be the requirement document, test case, test plan, or source code.

Step 2: Next, prepare the necessary artifacts, such as source codes, design documents, requirements documents, and test cases. Having these artifacts at the beginning of the process helps to ensure that the proper documentation is used for testing.

Step 3: Select the right way to do static testing. There are two ways to do static testing, either manually or automatically.

Step 4: Define the scope of the testing and what needs to be tested. It may be the Code Quality, security, or compliance issue that you need to detect.

Step 5: Choose the right tool for automatic static testing. There are many tools available in the market, and only a few are explained above. Choose the tool that will suit you and perform automatic static testing based on your project requirements.

Step 6: Any issues or bugs during the static testing process are reported and documented.

Step 7: The results collected during the static testing process should be analyzed to determine the quality of the software product. This is the final step in static testing.

Best Practices for Static Testing

Implementing static testing effectively requires following a set of best practices to maximize its benefits and ensure that Code Quality is maintained. Below are the best practices for static testing:

• Start Static Testing in the Early Phase: The earlier you catch defects, the cheaper they are to fix. Integrating static testing at the earliest stages of development (requirements, design, and coding) helps prevent major issues from propagating further in the software development lifecycle.
• Use Automatic Static Testing Tools: Integrate static analysis tools into the Continuous Integration (CI) pipeline (e.g., Jenkins, GitLab CI, Travis CI) to ensure every code change is automatically tested.
• Combine Manual Testing with Automated Testing: Automated tools excel at identifying syntactic errors and common patterns but may miss complex design flaws or architectural issues that require human insight.
• Prioritize Based on Impact: Focus on reviewing complex, critical, or security-sensitive code. High-risk areas may include code handling sensitive data, financial transactions, or complex business logic.
• Integrate with CI/CD Pipeline: Set up static analysis tools in your CI/CD pipeline so that each new code push triggers static testing. The pipeline should block merging code if critical issues are detected.
• Do Review and Improve: Regularly assess the effectiveness of static testing by gathering feedback from developers and analyzing defect trends. Identify areas for improvement and refine the static testing process accordingly.

Static vs Dynamic Testing

Here is a concise overview of the differences between static and dynamic testing.

Talk to an Expert

Challenges in Static Testing

Static testing is a valuable method for identifying defects early in the software development lifecycle, but it also presents several challenges. Here are some of the main challenges associated with static testing:

• Static testing requires manual effort to read through and analyze documents, designs, or code, which can be time-consuming, especially with large codebases or complex systems.
• Static testing is limited to finding certain types of defects, such as syntactical errors, logical inconsistencies, and standard violations. It may not catch runtime issues like memory leaks or performance bottlenecks.
• Unfortunately, static code-checking tools can introduce errors, which can take additional time and effort. False positives occur when code is flagged as problematic or incompatible when there is no problem. False negatives occur when the tool fails to identify a real problem in the code.
• Incorporating tools for static analysis (like linting tools, and code analyzers) into the development process can face resistance, and setting up these tools effectively can be challenging if not properly integrated into the workflow.
• Involving multiple stakeholders, such as developers, business analysts, and testers, can lead to coordination problems, especially if roles and responsibilities are not well-defined.

Conclusion

Despite the challenges, static testing is valuable in catching defects early, improving Code Quality, and reducing overall project costs when integrated effectively into the development process.

Static testing is a highly valuable process in software development, as it helps identify defects early in the life cycle before code is executed. BrowserStack Code Quality is a tool that makes static testing easy and effective. With this tool, you can integrate your code repository, get reviews, and continuously monitor it via the CI/CD pipeline to get recommendations on our code issues.

Also, Browserstack provides various other tools, such as Live, App Live, App Automate, Percy, etc., for web, mobile, and desktop applications, which can help you proceed with dynamic testing to deliver a bug-free software product.

Try BrowserStack Code Quality