Modular Architecture for faster Build Time
In big Android projects, project build duration is a very important metric that should be monitored frequently because big projects usually contain a lot of code with dependencies to many libraries, and some of these libraries may be huge like Guava, and usually a complex logic exist in certain areas of the project.
Enhancing build time is essential. Not only essential for engineering productivity but also essential for product time to market for getting products out of the door faster.
For big projects, solving this build problem may be very hard and very tricky, since it can require optimizations on the build management system (Gradle is assumed for this case), and may be requires huge refactoring on the current app implementation to reach to the desired build time target.
The other thing that we need to consider is what are we trying to optimize, there are two cases here to optimize for build time:
- Fully clean build optimization, which refers to cleaning and rebuilding a project from scratch. This is common for pull request verifications, where a full clean build is usually done in order to make sure that the new pull request’s code changes will NOT cause any build or unit test failures.
- Incremental build optimization, which refers to re-run a project after a minor source code change.
The purpose of this article is to show some ideas which can help to have better build times (mainly for Fully clean builds) and to offer some ideas toward having a modular android app.
Tuning
Tuning is always good, one of the greatest benefits about tuning, it usually does not require code changes! so any extra build improvement that you can gain from this step is a real quick win.
1. Analyzing
Fortunately, Gradle is a great build management system and there are numerous articles that discuss how to tune Gradle, one of the important things to be done first is to try to know why your build is slow since the parameters that you apply for a certain project do not always necessarily work for other projects.
Thanks to Gradle option --profile, you can start diagnosis your build time. This option will generate a report containing the duration of each subtasks of the task you are running.
Let’s see an example, here if you run --profile on your project clean full build as follows
You will find a report similar to the one below under build/reports/profile-[date-of-your-build].
So what does configuration, dependency resolution and task execution mean in this report.
1. Configuration; it refers to how sophisticated your Gradle script is. If there is no massive complications in your Gradle scripts, this configuration time should be small.
2. Dependency resolution; it refers to how much time taken for dependency resolution for your project. Excluding the first time ever building the project, this step should also be minimal since dependencies are cached locally.
3. Task Execution; this is the most important part, and usually the longest step. Task execution includes all the Gradle tasks that were performed mentioning the duration of every task. From this point, you should start, start on understanding what is going on and start enhancing the tasks that takes most of the build time, and re-iterate to see if your tuning makes a better difference.
2. Performing Tuning
This section gives some of the ideas to tune your slow build.
2.1. General Gradle Tuning (Over-the-counter tuning)
There are general and very known tuning parameters that should be done for Gradle based project, the most important ones are:
- Start Gradle daemon before running any Gradle command, which can be done simply by adding the following line to your gradle.properties file as follows.
org.gradle.daemon=true
2. Parallelize your build. This will be only effective if you have several modules in your project (modules will be discussed in the next sections).
org.gradle.parallel=true
3. Setting configure on demand to true to only run configuration for all the modules that participate in the build task (Usually has very minor enhancement impact).
org.gradle.configureondemand=true
Note: Setting app minSdkVersion version to 21 can also speedup your build process heavily, thanks to ART (Android Runtime) which is introduced in Android API 21 (Lollipop), one of the ways to implement this is to make minSdkVersion value parameterized, to give an option for the build script to override the default project minSdkVersion.
2.2. App Specific Gradle Tuning
Starting from the task execution step can tell you what is going on in order to perform project specific Gradle optimizations (note that every slow build usually has its own set of problems).
One example which you can see if your project has massive amount of unit tests (which is usually a good sign for having a good unit test coverage), you may find that your unit tests execution is big.
Unit test execution time is big? why this can happen if mocking is used? how this can happen?
There are many reasons which can lead to slow unit tests:
1. Leaning heavily on Robolectric for app unit tests (Robolectric is slow, I already blogged about this before in this post: https://medium.com/@hazems/lessons-learned-regarding-upgrading-to-robolectric-3-3-2-275ce715c246), so the action that can be taken here is to try to reduce Robolectric unit tests and replace them with pure Mockito and JUnit unit tests as much as you can.
2. Massive unit test reports which are generated every unit tests execution. Disabling these massive reports can be very important and can magically improve your test execution time and your whole build time.
2.2.1. Disabling Unit Test Reports
By default JUnit unit tests generate both HTML and XML reports every time you execute your app unit tests, in order to disable simply set these flags to false.
Do not be surprised if you found your build improves heavily after disabling test reports if you have a massive amount of unit tests.
2.2.2. Parallelize Running Unit Tests
If you have a powerful build server, which have several CPU cores, you can have an advantage from this feature.
Be careful before using this, since it can give unpredictable behavior if unit tests are sharing state or dependent on each others.
You can set maxParallelForks to a number <= the build server CPU cores to have the maximum build parallelization. For example, the following will set the maximum parallel forks to 8 if you have 8 cores on your test task for example.
You can get dynamically the number of available CPU cores using the following API Runtime.runtime.availableProcessors().
If after setting this parameter, you find no build enhancement, just revert back.
2.2.3. VM Forking for Unit Tests
Use this option only If you have unit tests that are consuming massive memory, in this case, a single VM may not be enough in this case.
Gradle has a very powerful option to fork a new test VM after a specific number of tests. The following will fork a new test VM after 200 unit tests.
Again, If after setting this parameter, you find no build enhancement, just revert back. Creating a VM is not a cheap operation, measurement is an important key here.
Modularization
Adding to tuning your Gradle build, sometimes, this is not really enough if you have a complex project with many build flavors and a lot shared logic. Therefore Modularization here becomes an essential surgery to have because you will need to build project code in parallel and to definitely have a faster build time.
Modularization does not only speed up your build time, it has a lot of other (and more be more important) benefits:
- Maximizing code share. Building your app in a modular way allows you to maximize code share with other apps in the same domain or with some common requirements.
- Having a smaller APK size. Having a single large shared module project between your domain apps is not efficient from APK size point of view. Since most of the time not all of your domain apps use 100% of the functionality offered by this shared module, so you will end up by having APKs with an extra un-needed size which affect your business users since business users tend to keep apps when they have smaller sizes as shown by the graph below that was demonstrated by Google in Google IO 2017. Designing your app in a modular way should avoid having large mega functionalities in a single module, functionalities or business areas should be divided properly and mapped properly into suitable modules
Pro-guard can help to cut down un-used methods but at the end, it comes with a cost on build time.
3. Flexibility. Having an option to version shared modules can give more flexibility for projects that need to perform updates on modules, without breaking other dependent projects. i.e. Doing versioning will give the ability to scale projects without worrying much about if a specific change in component (X) will break a dependent project (A) which is not currently tested since this change targets another dependent project (B) as part of a new feature that will be released soon.
Module Guiding principles
So, what is the definition of a module, and what are the guiding principles of creating a new module (the module boundaries).
A module is an independent unit of code that can be used to construct or used in a business use case.
To avoid having so many small modules (that are hard to maintain) with similar functionalities, there should be some guiding principles for having a new module, IMO, some of these guidelines (or boundaries) are:
- Every module should be covering an area of the business (such as analytics, authentication, content caching …etc).
- Two modules should NOT duplicate the same logic. Every module should have its DNA, should has its own logical boundary which separates it from other modules that target other business areas.
- Modules should allow their clients to extend or change their logic completely using extension points.
- Modules should be self-contained and having minimal inter-dependencies between each others.
Creating modules in a green field app (a new app) is definitely easier than transforming a non-modular app which did not have these guiding principles during its initial design. Given that every complex problem can be solved by breaking it down to smaller problems, transforming a non-modular system to a modular system can be performed but there are some questions that are needed to be answered first to determine the next actions to be taken. The next section discusses this.
Transforming non-modular to modular app (Top Ten Questions)
There are ten important high level questions that are needed to be answered to start transforming a non-modular app into a modular app:
- Is the app logic completely separated from the user interface classes?
- Does every app package represent the actual logical grouping for the classes under it?
- Are there no (or minimum) dependencies between concrete logic classes?
- Is the business logic NOT mixed with (or NOT scattered across) multiple unrelated classes?
- Are the app logic classes testable?
- Are there no (or minimum) dependencies on huge third-party libraries?
- Do you have effective unit tests with a good amount of test coverage?
- Do you have test coverage reports to monitor progress quality while adding or updating existing app features?
- Do you have automated UI tests (like Espresso tests) that cover app UI functionality which can help you to know as early as possible if something goes wrong while doing modularization refactoring? Are they integrated with CI night builds?
- Is there a team agreement to move to this direction?
If the answer to these questions is Yes, then the transformation will be smoother and your key to create your logical modules will be definitely the app packages.
If the answer to any of these questions is No, then you will need to create a plan or a strategy to facilitate this transformation.
Conclusion
Tuning Gradle build and modularizing your app can be your best friends in order to have a better build time. Enhancing app build time and modularization topics are really big and a single article cannot cover all of the ideas/aspects. This article tries to touch some of these ideas which helped me in improving a sophisticated system build time.
