Optimising CI build times of a Kotlin Multiplatform project

Kotlin Multiplatform supports different targets (platforms), which require different host operating systems for compilation. There are targets that can be compiled on any host OS, and there are targets that can be compiled only on a specific host OS. For example, Android and JVM targets can be compiled on any host, however Apple targets (ios, watchos, etc.) require macOS host.

From my observations, most commonly used targets are Android and iOS, sometimes JVM and JS are also supported, as well as additional Apple targets like watchOS or tvOS. In such cases you may find it easier to use the macOS host for all compilations. There is no extra setup required, and you can easily compile and publish your project with just a single Gradle command. CI setup is also much easier for single host builds. You can just use Apple hardware with macOS installed. If you use e.g. GitHub Actions, the configuration will be also simpler.

However there is one drawback of this approach — Apple hardware is more expensive, so it may cost you more to use it. For example in the case of GitHub Actions, it costs ten times more to use macOS than Linux, and twice more to use Windows than Linux. As a result, you can run out of quota, especially if you use the free plan.

Fortunately, there is a solution to the problem. We can use different operating systems, and compile only required targets on expensive hosts. For all other targets we can use a cheap Linux host. It will require additional Gradle and CI setup, but I think it’s worth investing in. We will also get another benefit — faster builds, because the project is compiled in parallel in multiple machines.

Prerequisites

Let’s define a simple Kotlin Multiplatform project, with just one shared Gradle module and a few platforms: Android, JVM, JS, Linux, iOS, and watchOS. Our build.gradle.kts file should look similar to this:

At this point we have a shared Kotlin Multiplatform module, which can be compiled and published to a Maven repository. We have to use a macOS machine, because there are Apple targets configured. So let’s fix it and try to compile only Apple targets on macOS, and use Linux for the rest.

Splitting the targets

We will split targets only if a special argument is passed from the command line: -Dsplit_targets

Let’s define a special class for handling command line arguments:

Now we need to define a helper function, which will return a required host type for a specified target:

We also need a function which checks if the compilation of the specified target is allowed on the current host:

Compilations are always allowed for Kotlin metadata, or if -Dsplit_targets command line argument is not specified. Otherwise we allow compilations only on specific hosts, defined for every target.

With the functions above, we can easily disable unnecessary compilations in the following way:

All we need to do now is to call disableCompilationsIfNeeded for all defined targets:

Congratulations! We have successfully split Kotlin Multiplatform targets between Linux and macOS hosts. Now we can use different hosts on our CI and pass -Dsplit_targets command line argument.

Example of GitHub Actions jobs configuration for build

Splitting publications

So far we split only compilations. But if publication is required, it would be good to use different hosts here as well. I assume that the maven-publish plugin is used. However the approach demonstrated in this section should be easily adoptable for any other plugin.

First thing we need is to add another command line argument: -Dmetadata_only. When this argument is specified, we will enable only metadata publications.

As usual, we will need a helper function, which checks if a specified publication is allowed:

The function contains checks in the following order:

• If the publication is a metadata publication, then it is allowed if either the -Dmetadata_only argument is specified or the -Dsplit_targets argument is not specified.
• Otherwise the publication is a normal one (not metadata). It is disabled straightaway if the -Dmetadata_only argument is specified.
• If the publicationName property is set for the publication, then we can use it to find a corresponding Kotlin target. We find a first target with a name starting with publicationName sub-string. Once the target is found, we check if its compilation is allowed. The publication is allowed only if its target’s compilation is allowed.
• If the publicationName property is not set for the publication (e.g. it is the case for Android publications), then we can use the AbstractPublishToMaven.name property instead. We find a first target whose name is contained in the AbstractPublishToMaven.name string. Lastly, we again check if the target’s compilation is allowed.

Now we can actually disable publications for a Gradle project:

The function above gets all defined Kotlin targets first. Then it iterates over all defined Gradle publication tasks and disables those that are not allowed.

Now we just need to call the function in our shared module’s build.gradle.kts file, after the publication configuration code:

Example of GitHub Actions jobs configuration for publishing

Disabling unreachable tasks

In all examples above we were disabling various Gradle tasks. If now we execute ./gradlew publish -Dsplit_targets on a macOS host, we will have the following Gradle tasks disabled:

• :shared:compileDebugKotlinAndroid
• :shared:publishAndroidDebugPublicationToMavenRepository
• :shared:compileReleaseKotlinAndroid
• :shared:publishAndroidReleasePublicationToMavenRepository
• :shared:compileKotlinJs
• :shared:publishJsPublicationToMavenRepository
• :shared:compileKotlinJvm
• :shared:publishJvmPublicationToMavenRepository
• :shared:publishKotlinMultiplatformPublicationToMavenRepository
• :shared:compileKotlinLinuxX64
• :shared:publishLinuxX64PublicationToMavenRepository

However the execution will fail with the following error:

Execution failed for task ':shared:generateMetadataFileForLinuxX64Publication'.
> java.io.FileNotFoundException: /Users/arkadiiivanov/dev/split-targets-article/shared/build/classes/kotlin/linuxX64/main/klib/shared.klib (No such file or directory)

The task publishLinuxX64PublicationToMavenRepository is disabled and is not executed. But it depends on the generateMetadataFileForLinuxX64Publication task, which is enabled and is executed. The latter task transitively depends on the compileKotlinLinuxX64 task, which is again disabled. As a result the .klib file is not generated and the generateMetadataFileForIosArm64Publication task fails.

How can we fix that? We can recursively check all dependencies of already disabled tasks, and disable those tasks that are not accessible from any root task. A root task is a task that does not have any parent. A task is not accessible if it can not be reached from any root task while going only through enabled tasks.

The following diagram shows a very simple example of a Gradle task graph. Red tasks are already disabled, but we also need to disable grey tasks.

The following code snippet demonstrates a function, that can be used to disable all unnecessary tasks:

The code above does the following steps:

1. Collects root tasks — all tasks that are not listed as dependencies of any other tasks
2. Collects disabled tasks
3. For each disabled task, recursively checks dependencies
4. For each dependency task checks if it is accessible from any root task
5. Disables every task that is not accessible

The provided function should be called from the root build.gradle.kts file. After that we will have all irrelevant tasks properly disabled, and the build and the publication should work fine on all hosts.

Conclusion

Building and publishing a Kotlin Multiplatform project is easy. But there is always a room for optimisations, which require additional Gradle setup. In this article I described how we can reduce costs of using CI by moving some jobs from expensive macOS machines to much cheaper Linux machines. The proposed optimisations should also reduce build times, because of better parallelisation.

Also I think it would be good if JetBrains created an API for this kind of optimisations, so it is easier to use and is properly maintained.

I hope this article will help you with your project. Thanks for reading it and don’t forget to follow me on Twitter!