Stop using Gradle buildSrc. Use composite builds instead

You are invalidating the whole project for any change within buildSrc, usually without there being a valid reason. We can keep the sweetness of Kotlin whilst avoiding this.

Pic by Tania MT with ❤

What is buildSrc

buildSrc is a directory at the Gradle project root, which can contain our build logic. This allows us to use the Kotlin DSL to write our custom build code with very little configuration and share this logic across the whole project.

This has become very popular in recent years and it is a common practice to define project-specific tasks or storing dependencies in buildSrc. We can share a single reference to a library artifact, having one place to modify it in the case of an update. All of this allows us to use IDE auto-complete and have code that is testable. But not everything is that great.

Gradle docs: A change in buildSrc causes the whole project to become out-of-date.

It’s a trap

Gradle builds highly utilise incremental builds and task caching — every time you see UP-TO-DATE or FROM-CACHE, it means some time was saved. Both of these don’t work if any change happens within buildSrc.

The impact depends on your logic within buildSrc, severity of editing it and also on your clean build time — build of the project from scratch.

1. Dependency update rebuilds the whole project — Incrementing a version number triggers a clean build even if the dependency is used only within one module.
2. Iteration speed is slow — When working on code within buildSrc It means you are having a clean build for every single change.
3. Local build cache is invalidated — It is invalidated for each change in buildSrc you perform or pull from your version control system.
4. Remote Gradle cache is invalidated — If your project uses a shared remote cache, any change in buildSrc basically nukes the whole remote cache both for CI and engineers. A remote cache is typically used in large projects with long clean build times, therefore the impact can be huge.

Example 1 — Dependency update

We will show two identical :app:assembleBuild builds of an example project. Both builds will be updating com.google.firebase:firebase-messaging from version 20.1.2 to 20.1.7. We will be using a great Gradle scan feature to demonstrate the differences in build execution time.

A — Build updating buildSrc. Edit of the version number triggers complete rebuild of the project and takes 1m 11s, avoiding almost no tasks and everything is rebuilt.

Change in /buildSrc/…/Dependencies.kt — commit here.

Whole project is recompiled.

B — Build updating build.gradle. Update of this dependency results only in a rebuild of the module using the dependency and build quickly finishes in 26s with 194 avoided tasks.

Change in /feature/push/build.gradle — commit here.

Only affected :feature:push module is recompiled.

Whilst modifying buildSrc, the build time is almost 3 times higher:
1m 11s vs. 26s.

The difference would be even more noticeable on larger projects and it is not out of the ordinary to see clean builds of several minutes. Keep in mind that the same clean build is waiting for all your team members after they pull the change and any dependency update invalidates the remote cache for the whole project.

Example 2 — Custom plugin modification

We will be modifying a custom plugin executing instrumented tests in Firebase Test Lab. The modification will only add a simple log message.

A — Modifying plugin in buildSrc. Any modification triggers again a project rebuild and takes 1m 16s this time.

Add in /buildSrc/…/FirebaseTestLabPlugin.kt — commit here

Whole project is recompiled.

B — Using exactly the same code, but with Composite Builds. There is no need for any rebuilding and only non-cacheable tasks are executed. The whole build takes 12s.

Add in /firebasePlugin/…/FirebaseTestLabPlugin.kt — commit here

Only the plugin recompiled, 208 avoided tasks

While iterating on a custom Gradle plugin, the impact of buildSrc can be very high. In this case build takes 1m 16s vs. 12s.

Use Composite builds — includeBuild

The solution to the problems stated above might be easy — Gradle Composite builds. We can move our logic from buildSrc into a separate firebasePlugin module and add includeBuild(“firebasePlugin”) within settings.gradle.

Only moves and one new line — commit here.

For a custom plugin — this is all you need to do.

To achieve feature parity with buildSrc — we still want to reference dependency constants directly in build.gradle. The solution is creating a plugin for dependencies.

You can find the complete commit here.

When we include this plugin into our /push/build.gradle file, we can again achieve the dependency: api Dependencies.FIREBASE_MESSAGING.

Conclusion

Using buildSrc became very popular and it’s a great way to define your build logic in Kotlin and have this logic testable. It is important to have in mind the tradeoff, which might be surprisingly high.

You may be paying the price in long build time for minor changes. You can still get all the sweetness of Kotlin and auto-complete only by making small tweaks and adding one line of includeBuild(“plugins”) to your settings.gradle.

Happy coding!

Edit: As one reader pointed out, Gradle actually have plans on “Making the implicit buildSrc project an included build.” Issue to follow.