Automated migration of your projects to Bazel build system with Airin
Airin is a framework for automated migration of your projects to Bazel build system. In particular, it can help to migrate Android apps from Gradle to Bazel.
Bazel is an open-source build system developed by Google. It provides fast incremental builds by using local and remote caching. It can build various programming languages and platforms allowing to have a single build system, and a single team that supports it, for most of the projects in the organization including mobile, backend, and others. You can learn more about Bazel here.
But why would you need to automate the migration? Imagine having a project that consists of hundreds or even a thousand modules, a couple of millions of lines of code, and being developed by a hundred engineers. Even if you create build scripts for each module manually, it would be hard to keep up with the updates introduced by other engineers who contribute to the project.
By using a tool for migration automation it is possible to keep Bazel build configuration in sync with the initial build system until the project is ready to fully move to Bazel as a primary build system.
Airin is written in Kotlin language and consists of 2 components:
- Starlark template engine —a Starlark code generator that processes templates defined with Kotlin DSL that closely resembles Starlark language itself.
Starlark is a Python inspired language that is used by Bazel for build configurations.
- Migration component — a plugin to the build system of your project that collects the information about its build configuration and generates corresponding Bazel scripts based on provided Starlark code templates.
Airin currently targets Gradle build system and Android in particular but it is not limited with it. The support for Maven and other non-JVM build systems is in development.
All the source code, installation instructions, and examples could be found in the project repository.
Learn more about the project at BazelCon 2021 conference here.
How to migrate a Gradle project?
Let’s see how to migrate an Android Gradle project to Bazel.
To configure the migration we need to add some code that won’t be used by the application but will be present in the classpath of the Gradle build configuration itself. How we can do it? There are several ways to do it in Gradle but we will use the simplest one as an example.
We will use a buildSrc directory for this purpose. It should be created in the root directory of your project so that Gradle handles it in a special way. You can consider it as another module of your project, however, Gradle by default will add all the code from buildSrc to the classpath of build configuration scripts instead of your project source code.
The internal structure of a buildSrc is pretty much the same as any other Gradle module. Add the following code to the buildSrc/build.gradle file to configure it with Airin.
Step 1. Declare Starlark templates
Typically, your project would have a couple of types of modules. These can be Java/Kotlin library modules, Android library modules, etc. For each of these types, there should be created a Starlark code template, written with Kotlin DSL provided by Airin. Based on such templates, Airin will generate corresponding Bazel files in your project.
Each template holds a general structure of the Bazel file for every module type while allowing to dynamically specify the details such as target name, package name, list of dependencies, etc.
In addition, you can declare templates for Bazel files that are used only once per project such as WORKSPACE, root BUILD file and others.
All templates can be created in the buildSrc/src/main/kotlin/template directory. In the example below, you can see such a template and the corresponding Starlark code it generates.
One of the key goals of Airin’s Kotlin DSL is to resemble the Starlark language as close as possible in terms of syntax. This is specifically useful for cases when you copy the existing Starlark scripts to your Kotlin templates. The goal is to require as few changes as possible to make the code compilable in Kotlin. Of course, they would not be identical but still, Kotlin does a very good job in creating powerful DSLs. We will see more examples of Kotlin DSL for various Starlark language constructs later in this post.
Step 2. Map templates to the right Gradle modules
In order to make use of our Starlark templates and generate code based on them, we need to use components called TemplateProvider(s).
To map Starlark templates to the right Gradle modules we need to declare classes that extend a GradlePerModuleTemplateProvider.
The providers can be created under the buildSrc/src/main/kotlin directory. We do not add them to any package so that they could be used further without any import statements in Gradle build files.
The example below demonstrates the provider for Kotlin library modules that uses the template defined earlier.
Each provider must implement 2 functions:
canProvide(Project): Boolean
Determines if can provide templates for the given module. Usually, it is enough to check specific Gradle plugins applied to the module.
Theorg.gradle.api.Projectis a Gradle representation of a module.provide(Project, relativePath: String): List<StarlarkFile>
This is the main function where you need to provide the list of templates for the specific Gradle module. TherelativePathdefines the path to the module starting from the project root directory.
Another type of provider is GradleStandaloneTemplateProvider. It is required for templates that are used once per project. For example, this can be a WORKSPACE file, a root BUILD file, or others.
Let’s see the example of a provider for the WORKSPACE file.
Each TemplateProvider has access to a sharedData object that stores information available to all providers during the migration. In particular, it allows to retrieve the list of all external artifacts used in the project.
Step 3. Configure Airin Gradle plugin
The final step is to register newly created providers. To do this, configure the Airin Gradle plugin in the root build.gradle file and add all the created template providers.
The order of provider registration matters. For each module, Airin will go through all per module template providers until it finds the first one that can provide templates for the module. Each time it will start from the last registered provider. In the example above for each module, Airin will start from AndroidModuleBuild until KotlinModuleBuild (in case it did not find a suitable provider earlier). The Workspace will be processed in a separate round as it is a standalone provider.
After everything is configured, use the following command in the terminal to generate Bazel files for the project.
./gradlew migrateToBazelIf your templates are configured correctly you will be able to build your project with Bazel.
Continue reading to see complete and fully functional samples of migrating Android apps including Jetpack Compose.
Capabilities of a Starlark template engine
To use a standalone Starlark template engine add the following artifacts to your project.
Generating code for custom rules and macros
If you want to use the DSL for rules or macros that are not present in the standard Airin library, you have 2 options:
Option 1. Use dynamic notation directly in your templates:
The same approach might be used to add custom arguments to the DSL of the rules that are already present in the Airin standard library:
Option 2. Generate a type-safe Kotlin DSL for your rules using Kotlin symbol processor (KSP).
If you want to stick with option 2, update the build.gradle of a specific Gradle module with the following code.
Let’s see how various Starlark language components are represented in Kotlin DSL. All the features of the Starlark template engine are made with the type-safety in mind. Therefore, many of the potential code errors are prevented during the Kotlin compilation stage.
Variable assignments
Variable assignments are represented with Kotlin’s property delegation mechanism using by keyword.
List comprehensions
Starlark like Python has a list comprehensions feature. There is a corresponding Kotlin DSL construct for generating them as well. Make sure you’re using `in` with backticks.
Concatenations
Concatenations are implemented using `+` operator with backticks. The concatenation operation is type-safe. For example, you are not able to concatenate a list of strings with a list of integers if the former is required as a resulting type.
Slices
Similar to Python, Starlark allows using slice expressions. You can use the following DSL for generating them in your build scripts.
Load statements
In most cases, load statement declaration looks pretty straightforward.
However, If you need to load variables in a type-safe manner you can specify their types by yourself. In this case, loaded variables will further conform to specified types in Kotlin DSL.
Raw text injection
It is also possible to inject raw string statements into your templates so that they will be inserted into the generated file as-is. You can of course use the Kotlin string interpolation feature to pass arguments to the string.
Combining the above components
By combining the components above it is possible to generate more advanced build scripts.
The following example script takes the list of source files and does the modification of their contents by running a genrule on each of them. The resulting variable holds the list of modified files.
You can find more examples of advanced usages of the Starlark template engine:
Sample migration projects
There are various examples of migrating projects to Bazel available.
The simplest Android sample
You can start with the simplest example of migrating an Android application to Bazel that shows the basic migration concepts.
The project is available at this link.
Android Jetpack Compose sample
The more interesting example demonstrates the migration of a multimodule Android application that uses Jetpack Compose including Paging 3 and Navigation. In addition, the project uses Dagger with the component dependency mechanism and Room for data persistence.
The project uses a multimodule architecture that splits each feature module into 2 parts:
- API part — that defines a set of public interfaces that allow other application components to interact with the feature. No specific implementation logic is allowed here.
- Implementation part — that contains all the internal implementation details of the feature. Other features are not aware of this part which means that any update of this component shall not change the public application binary interface (ABI) of the feature. Therefore, it won’t force Gradle or Bazel to rebuild all modules that depend on the feature during incremental compilation.
The project is available at this link.
Conclusion
We have just seen how Airin can automate the migration process of Android projects to Bazel build system. By leveraging the capabilities of a Starlark template engine and a migration component, Airin can bring automation and flexibility to the migration process.
Therefore, to migrate an Android project to Bazel with Airin there are 3 main steps:
- Declare a set of Starlark code templates
- Map them to the right modules using template providers
- Register providers in the Airin Gradle plugin configuration.
Give it a try and let me know how it works for you. Airin is being actively developed currently, so any questions, comments, or contributions are welcome!
Check the project repository to find more details and examples.
Useful references
- Bazel Jetpack Compose example by Ben Lee
