Categorizing Tests in JUnit 5 and Running Different Categories with Gradle

Improving build time by categorising the tests and running them concurrently

In modern software development, organising tests into categories such as unit, integration-low, integration-medium, and integration-heavy is a best practice. It ensures clear separation between different types of tests, each having its own scope, purpose, and execution time. With JUnit 5, categorising tests is simple using custom tags, and with Gradle, we can configure tasks to run these categories separately for more efficient CI/CD pipelines.

Why Categorize Tests?

• Unit Tests: Fast, isolated tests that check individual units of code (e.g., a class or method).

• Integration-Low Tests: Simple integration tests that may involve interactions between a few components or lightweight services.

• Integration-Medium Tests: More involved tests that check interactions between multiple components, possibly including databases or external APIs.

• Integration-Heavy Tests: Full-fledged integration tests, often involving multiple services, databases, or complex environments, which take longer to run.

By tagging these tests, we can optimize the testing process, running lightweight tests frequently and heavier tests selectively to speed up the build process.

Categorizing Tests in JUnit 5

In JUnit 5, we can use the @Tag annotation to label tests according to their category:

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;

public class ExampleTests {

    @Test
    @Tag("unit")
    void unitTest() {
        // Unit test logic
    }

    @Test
    @Tag("integration-low")
    void integrationLowTest() {
        // Low complexity integration test logic
    }

    @Test
    @Tag("integration-medium")
    void integrationMediumTest() {
        // Medium complexity integration test logic
    }

    @Test
    @Tag("integration-heavy")
    void integrationHeavyTest() {
        // Heavy integration test logic
    }
}

You could annotate the same tag at the class level as well.

Configuring Gradle to Run Different Test Categories

You can configure Gradle to run tests based on their categories by defining custom test tasks in the build.gradle file. The JUnitPlatform allows you to filter tests based on tags.

Here’s how to configure different tasks in Gradle to run the four test categories:

plugins {
    id 'java'
}

repositories {
    mavenCentral()
}

dependencies {
    testImplementation 'org.junit.jupiter:junit-jupiter:5.10.0'
}

test {
    useJUnitPlatform()
    // Default to running unit tests
    includeTags 'unit'
}

// Define a custom task to run unit tests
task unitTest(type: Test) {
    useJUnitPlatform {
        includeTags 'unit'
    }
    shouldRunAfter test
}

// Define a custom task to run low-level integration tests
task integrationLowTest(type: Test) {
    useJUnitPlatform {
        includeTags 'integration-low'
    }
    shouldRunAfter unitTest
}

// Define a custom task to run medium-level integration tests
task integrationMediumTest(type: Test) {
    useJUnitPlatform {
        includeTags 'integration-medium'
    }
    shouldRunAfter integrationLowTest
}

// Define a custom task to run heavy integration tests
task integrationHeavyTest(type: Test) {
    useJUnitPlatform {
        includeTags 'integration-heavy'
    }
    shouldRunAfter integrationMediumTest
}

// Configure test task dependencies (optional)
check.dependsOn unitTest, integrationLowTest, integrationMediumTest, integrationHeavyTest

How It Works:

• unitTest Task: Runs tests with the unit tag.

• integrationLowTest Task: Runs tests tagged with integration-low.

• integrationMediumTest Task: Runs tests tagged with integration-medium.

• integrationHeavyTest Task: Runs tests tagged with integration-heavy.

Running the Tests

You can run these tests individually or together using Gradle commands:

• Run Unit Tests:

./gradlew unitTest

Run Low-Level Integration Tests:

./gradlew integrationLowTest

Run Medium-Level Integration Tests:

./gradlew integrationMediumTest

Run Heavy Integration Tests:

./gradlew integrationHeavyTest

Run All Tests: By running

./gradlew check

Gradle will run all the test tasks as they depend on each other.

Benefits of Categorizing Tests:

1. Selective Test Execution: Faster feedback by running unit tests more frequently and reserving heavier tests for nightly or pre-deployment runs.

2. Efficient CI/CD Pipelines: Integration tests can be triggered based on the type of changes (e.g., low integration tests for service updates, heavy tests for major releases).

3. Better Resource Utilisation: Running lightweight tests in parallel with heavier ones improves resource usage during the build process.

Where have I applied this?

I tried this at Wayfair to reduce this in our long running tests, and we improved the build time by significantly by 55%, here are the snapshots of the old and the new build

Slow build:

After making the categorisation change,

Conclusion

Categorising tests in JUnit 5 using tags and configuring Gradle to run these categories selectively is a powerful approach to streamline testing workflows. It not only optimises test execution but also improves the overall build process, making it easier to manage small, medium, and large-scale projects.

By adopting this practice, you can ensure faster feedback loops, more stable releases, and improved developer productivity.