Screenshot testing in Jetpack Compose

Designed by Freepik

In this article, I’ll introduce a useful tool for screenshot testing in Jetpack Compose. This tool was officially announced at the last Google I/O as part of a new library. Although it’s still in the early stages (version 0.0.1-alpha08), it can already be integrated into your projects with minimal configuration and code, allowing you to start testing your UI efficiently.

What is Screenshot Testing?

Screenshot testing involves comparing a reference image (a baseline) with the current state of your UI to detect visual discrepancies. Screenshots are taken with specific configurations such as:

• Screen size
• Dark or light theme
• Font scaling

This approach allows developers to validate UI designs with stakeholders (e.g., designers) by taking screenshots, reviewing them, and then using these validated screenshots as a safeguard to ensure future changes don’t “break” the approved UI. If intentional UI changes occur, new reference images must replace the outdated ones.

The best part? The process is automated — only new reference screenshots need manual validation.

Why Use This Tool?

1. Backed by Google

Having Google’s developer team maintaining and evolving this tool ensures it follows best practices and integrates seamlessly with the Jetpack ecosystem.

2. Integration with Compose Previews

One standout feature is its ability to leverage Compose Preview annotations. If you’re familiar with previews in Jetpack Compose, you know how useful they are for quickly visualizing UI components. With this screenshot testing tool, you can use custom preview annotations to define multiple configurations and test them effortlessly.

For example, you can create a single annotation to generate previews for:

• Four screen sizes in both light and dark themes, or
• Two screen sizes with five different font scales.

If you want to learn more about optimizing your previews, check out my post, where I share tips like creating your own preview annotations.

3. Dedicated Source Set for Screenshot Tests

All screenshot tests are stored in a special source set called screenshotTest. This structure keeps screenshot tests separate from other unit and integration tests, maintaining a clean and organized codebase.

4. Future Enhancements

At London Droidcon, Jose Alcérreca and Adarsh Fernando (in their talk on testing strategies, around the 19-minute mark) mentioned an upcoming feature: the ability to use screenshot tests as previews in the files where your composables are defined. This would eliminate the need to duplicate code — one function for the preview and another for the screenshot test. While this feature isn’t available in version 0.0.1-alpha08, it’s a promising addition to look forward to!

Let’s Get Started

Enough talk — let’s set up this tool in your project!
Follow along as we configure the tool step-by-step, so you can start testing and validating your Jetpack Compose UIs with screenshot tests.

Setting Up the Plugin

To get started with screenshot testing in Jetpack Compose, you’ll need to ensure your project meets the following prerequisites:

• Kotlin Version: At least 1.9.20 (or newer). For this guide, I’ll be using a more recent version for improved compatibility.
• Android Gradle Plugin: Version 8.5.0-beta01 or higher.

[versions]

agp = "8.6.1"
kotlin = "2.0.21"
composeScreenshot = "0.0.1-alpha08"

[plugins]
compose-screenshot = { id = "com.android.compose.screenshot", version.ref = "composeScreenshot"}

Step 1: Add the Plugin

Include the screenshot testing plugin in your module-level build.gradle.kts file:

plugins {
    ...
    alias(libs.plugins.compose.screenshot)
    ...
}

Step 2: Enable Experimental Properties

In your project’s gradle.properties file, enable the necessary experimental properties:

android.experimental.enableScreenshotTest=true

You’ll also need to set the experimental flag in your module-level build.gradle.kts file:

android{
  ...
  experimentalProperties["android.experimental.enableScreenshotTest"] = true
  ...
}

Step 3: Check the ui-tooling Dependency

Ensure the ui-tooling dependency is included in your version catalog (if you're using one) and in your module-level build.gradle.kts file. This dependency is essential for rendering and testing Compose UI elements. If you’re using the Compose BOM (Bill of Materials), the version should be managed automatically:

[libraries]
...
# compose
androidx-compose-bom = { group = "androidx.compose", name = "compose-bom", version.ref = "composeBom" }
androidx-ui-tooling = { group = "androidx.compose.ui", name = "ui-tooling" }
...

dependencies {
    ...
    // Compose
    screenshotTestImplementation(platform(libs.androidx.compose.bom)
    screenshotTestImplementation(libs.androidx.ui.tooling)
    ...
}

Creating Screenshot Tests

In the previous section, I mentioned an exciting upcoming feature: the ability to use preview functions from screenshot testing classes to preview composables directly within the file where they are developed. While this feature isn’t available yet, there’s a workaround to avoid duplicating code in the meantime.

Step 1: Mark Previews as Internal

For composables that already have preview functions, start by marking these preview functions as internal. This ensures they remain accessible within your testing source set while keeping them encapsulated.

Step 2: Reference Preview Functions in Testing Classes

Next, create a dedicated testing class for each composable within the screenshotTest source set. In these classes, reference the existing preview function instead of creating a duplicate.

Generating Reference Images

To generate reference images for your screenshot tests, use the following Gradle commands based on your operating system:

• Linux and macOS: ./gradlew updateDebugScreenshotTest (./gradlew {:module:}update{Variant}ScreenshotTest)
• Windows: gradlew updateDebugScreenshotTest (gradlew {:module:}update{Variant}ScreenshotTest)

In my case, I’m running macOS and have a single module named app. Therefore, I use the following command:

./gradlew :app:updateDebugScreenshotTest

Output Location

After running the command, the reference images are generated and stored in the following directory:

/app/src/debug/screenshotTest/reference/com/example/obook/ui/component/scaffold/

Inside this folder, I find a subfolder named after the testing class, OBookScaffoldScreenshots. This folder contains five reference image files.

Why Five Files?

This is because I used the @PreviewScreenSizes annotation in my test class, which generates previews for five different screen sizes.

Preview Example

For instance, if I open the file corresponding to the portrait phone screen size, I can see the captured screenshot representing the composable for that specific configuration.

This setup provides a straightforward way to validate UI designs across multiple configurations efficiently.

Validating the Test Report

With reference images generated, you can now validate your screenshot tests and inspect the results through a detailed report.

Command to Validate Screenshot Tests

Depending on your operating system, run the following commands:

• Linux and macOS: ./gradlew validateDebugScreenshotTest (./gradlew {:module:}validate{Variant}ScreenshotTest)
• Windows: gradlew validateDebugScreenshotTest (gradlew {:module:}validate{Variant}ScreenshotTest)

Example Usage

In my case, since I have a single module named app, the command is:

./gradlew :app:validateDebugScreenshotTest

Report Location

The test report is generated at the following path:

app/build/reports/screenshotTest/preview/debug/index.html  
# General format: {module}/build/reports/screenshotTest/preview/{variant}/index.html

Successful Test Report

Testing Changes and Understanding Errors

To simulate a failure, I made a change to the text in the component and ran the validation command again. This resulted in an error.

Test Report With Errors

By inspecting the report, I could analyze the issue:

• Left Panel: Displays the reference screenshot image.
• Middle Panel: Shows the current image (with the changes applied).
• Right Panel: Highlights the differences between the reference and the current image.

In the case of a real error, you have two options:

1. Verify that the issue is an actual error, fix it, and re-run the validation to ensure the test passes successfully.
2. Confirm that the new image is correct due to an intentional design update, update the reference image to reflect the change, and then re-validate to ensure consistency.

Use Git LFS (Large File Storage)

Git LFS is a tool to avoid having large files that are not code files in our repository. In our case the reference images.

Follow official documentation to install Git LFS. In my case, I will install it using Homebrew:

brew install git-lfs

then track screenshot images with Git LFS

git lfs track "app/src/debug/**/*.png"

Add .gitattributes to the repository

git add .gitattributes

Commit the changes

git commit -m "Track PNG files in app/src/debug/* subfolders with Git LFS"

Configuring CI/CD

To streamline the development team’s workflow with screenshot tests, we need to automate the process by creating a CI/CD workflow.

Steps to Configure

Install LFS and Pull Files
Configure the workflow to install Git LFS (Large File Storage) and pull the required files. This ensures that the reference images are accessible when running the tests.

- name: Install Git LFS
  run: |
    sudo apt-get install git-lfs
    git lfs install

- name: Pull LFS files
  run: git lfs pull

Set Read Permissions
Grant the necessary read permissions to the workflow so it can access the required resources, including reference images and other test-related files.

permissions:
  contents: read
  pull-requests: read

Run Tests, Generate Reports, and Host Artifacts
The workflow should automate the following steps:

• Execute the screenshot tests.
• Generate the test report.
• Store the report as an artifact, making it accessible for review.

- name: Run screenshot tests
  run: ./gradlew :app:validateDebugScreenshotTest

- name: Upload screenshot test report
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: screenshot-test-report
    path: app/build/reports/screenshotTest/

Image Difference Threshold

When running the validation in GitHub Actions, an error might occur because the images are not identical. However, the root cause isn’t a significant issue but rather a minor discrepancy in how colors are rendered on different platforms. For example, your local machine (Mac) and the GitHub Actions runner (Ubuntu server) may generate slightly different color values, leading to false positives in the comparison.

Screenshot Difference

To address the issue of minor image differences, we can adjust the Image Difference Threshold. To configure this, add the following to your module-level build.gradle.kts file:

android {
  ...
  testOptions {
        screenshotTests {
            imageDifferenceThreshold = 0.002f // 0.2%
        }
    }
  ...
}

Now, when you run the pipeline again, the job will succeed without errors.

In a real project scenario, it’s generally better to generate reference images on the server itself. This ensures that both the updates and validations occur on the same machine, eliminating the need for adjustments to the Image Difference Threshold configuration. This approach helps maintain consistency across environments and simplifies the testing process.

Closing

If you found this article helpful or interesting, please give it a clap and consider subscribing for more content! I’d love to hear your thoughts! Your feedback and insights are always welcome, as I’m eager to learn, collaborate, and grow with other developers in the community.

Have any questions? Feel free to reach out!

You can also follow me on Medium or LinkedIn for more insightful articles and updates. Let’s stay connected!