Building Offline-First App using MVVM, RxJava, Room and Priority Job Queue

Kolkata, India street life

Earlier this year I attended Android Developers Meetup in NYC. The speaker, Instagram Developer, covered their efforts of making the Android app work offline to make it a great experience even for people with low/no network connectivity.

Below I will briefly describe Instagram’s approach and then provide a working example offline app using MVVM, Google’s new Lifecycle Architecture Components, Android Priority Job Queue library, Room, RxJava2, Dagger Android 2.11, Retrofit, ButterKnife.

Why offline mode?

80% of Instagram’s users are outside the US. Many of those live in the developing countries with limited network connectivity or limited data plan. These users primarily use Android devices, and that is why the team at Instagram targeted Android platform first in their efforts to make offline functionality available.

Offline read support

Instagram’s offline users are now able to see their newsfeed by serving previously loaded cached data. The idea is simple: each read request is identified by a Request Task (cache key). Each feed response gets cached on device by Response Store. Subsequently, when the user is offline and requests data for the same Request Task, the request is replayed and response for that cache key is served back by the local Response Store. The UI does not care that the data served is a cached version. When user tries to explicitly refresh their feed, a message informs them that they are currently offline.

Offline write support

Instagram’s offline users can leave comments, like content, follow others, save media, etc. Each of those use cases implements a PendingActionStore and all of the use cases are managed by a Manager that informs them of connectivity changes and lifecycle events. Basically, the idea is each PendingActionStore knows how to manage data locally and sync remotely and goes to work whenever the Manager directs it to do so.

Rolling my own

Let’s look at a sample Offline-First app I created. It may not cover all aspects of an offline app but it does cover the gist of it: save locally first, sync remotely next and reflect the changes in the UI during the process. The app enables users to seamlessly post comments while offline. Feel free to skip reading and jump into the source code directly at http://github.com/jshvarts/OfflineSampleApp

Comments are stored locally and then synced remotely

Workflow

The overall workflow can be summarized using this diagram:

• When a new comment is submitted, it gets stored in a local database first. The record is marked with syncPending=true. The comment text color in the UI is displayed in gray.
• If Internet connection is available, a background job will sync this record with remote repo.
• If no connection is available, a background sync job will be queued until the connection becomes available. During this time the user can close the app or even restart the device — the pending sync request is guaranteed to survive and wait for the connection to become available.
• Once the connection is restored, a background job will sync the record with remote datastore.
• If the record is successfully synced with remote repo, the corresponding local record is updated with syncPending=false. The comment text color in the UI will turn black.
• If the sync with remote repo failed, the local record gets deleted to provide data integrity. This scenario is an exception rather than the rule. In a real world application, you’d probably want to inform the user about this, possibly by showing a notification.

Your data should be stored locally first and synced remotely second.

Single Source of Truth

In our app, the local database is the single source of truth. Although data is synced with remote (cloud) database, data lookups always happen against the local database. Whenever sync with remote database fails, we delete the local record to guarantee data integrity between local and remote data repositories.

Offline design strategies

Unfortunately, there is no one size fits all solution when it comes to offline app design. My example app shows 2 possible ways to accomplish offline functionality (see below for details). Your milage may vary — it all depends on your particular app requirements.

1. Do you tell users when the app goes offline? If so, how?
2. Do you inform them when remote sync fails/succeeds? If so, how?
3. How do you tell users they are offline when they try to explicitly refresh content?
4. Do you keep track of the last scroll position when online in order to show only the content they have not seen now that they are offline?

These are just some of the questions you may face when designing offline functionality for your app.

App architecture

• MVVM (Model-View-ViewModel) pattern using Google’s new Lifecycle Architecture Components such as ViewModel,ViewModelFactory, LiveData, LifecycleObserver.
• Clean-ish Architecture with decent layer isolation and data repositories abstracted via interfaces. This should ease maintenance and testability. There is definitely some room for improvement around Job Queue integration.
• Background job for syncing comments with remote service is implemented using Android Priority Job Queue.
• RxJava 2, RxRelay are used to communicate between the layers asynchronously.
• Local Database is implemented using Google’s new Room Persistence Library.
• Remote API calls are implemented using Retrofit and fake remote datasource is done with help of JSONPlaceholder REST API.
• New Dagger Android 2.11 Injection API is used for injecting dependencies.
• As a bonus, the following quality checks are integrated into the build process: lint, checkstyle, pmd, findbugs.

Android Priority Job Queue

yigit/android-priority-jobqueue

This open-source library is the backbone of our offline app functionality. It is maintained by Googler Yigit Boyar whose name will inevitably come up when you research offline-first app design on Android. This mature library allows you to employ Job Scheduler API from Google on Lollipop and above and fall back on GcmNetworkManager to support API Level 9 and above.

Just like there are multiple ways to design offline apps, there are different ways of using Android Priority Job Queue. http://github.com/jshvarts/OfflineSampleApp contains 2 branches showcasing 2 approaches to using the library:

1. master: the app has to be open for the data to be synced with remote repo.
2. sync-in-background: the app does not need to be open for the background sync requests to execute. Android Priority Job Queue can wake up the app to perform the background sync jobs in the same process.

The latter approach is my favorite. It allows the application to be woken up on device boot when certain background job conditions are met (e.g. network access is available, WiFi is available, etc.).

Jobs in Android Priority Job Queue are highly configurable. Some of the common configuration parameters are:

• custom job priority
• network availability
• retry and job cancelation logic
• job persistence strategy

When Android Framework’s Job Scheduler is used (on Lollipop and above), job scheduling is managed by the OS which optimizes remote calls and conserves battery life. In addition, the scheduler complies with the Doze and Stand By restrictions.

The power of RxJava

While not applicable to offline apps specifically, note how RxJava helps us keep the code concise.

Here the comment is synced remotely upon successful local add comment call.

Once loaded initially, our View data always stays up-to-date by virtue of using Flowable. There are no explicit calls to refresh data. This makes the interaction between our data and UI truly Reactive.

Communicating sync events

Android Priority Job Queue library recommends using EventBus to communicate events from background jobs back to the UI. However, given that 1) the industry trend seems to be to replace EventBus with RxJava and 2) I already use RxJava extensively in the app, I decided to create a custom RxBus solution instead of EventBus:

Here is how our background job posts a sync response event to this bus:

Receiving event emissions from the RxBus is done differently depending on when you choose to process sync requests:

1. when the app is open
2. when the app is not open.

Let’s look at both scenarios below:

Syncing data when app is open

To receive event emissions when the app is open and when CommentsActivity is resumed, there is a SyncCommentLifecycleObserver. It is registered to observe lifecycle events from our View, CommentsActivity

The lifecycle event Lifecycle.Event.ON_RESUME subscribes to data emissions from our RxBus. This is roughly an equivalent to EventBus.getDefault().register(this)

The lifecycle event Lifecycle.Event.ON_PAUSE triggers unsubscribing (clearing Disposables) from our RxBus emissions. This is roughly an equivalent to EventBus.getDefault().unregister(this)

The above setup utilizes latest Architecture Components from Google and helps keep our code clean and manageable by adhering to Single Responsibility Principle.

To test this setup, try adding comments while offline. They will be stored locally only — comment text color will be gray. Then, re-enable network connection and observe your background jobs updating remote data store one comment at a time in the order they were added by the user. As comments are synced successfully, the text color will change to black.

Pro-tip: Use ‘elevator test’ to simulate flakey network connection while developing offline functionality. Use the app in an elevator and observe its behavior if the connection is lost and later re-established.

Syncing data when app is not open

To receive event emissions when the app is not open or backgrounded, follow these steps:

Add RECEIVE_BOOT_COMPLETED permission to AndroidManifest.xml so that your application can be woken up by the scheduler after the device is rebooted.

Set batch=false when creating job schedulers:

Make your background job persistent by adding.persist() job config param.

Note, the SyncCommentJob has a single instance variable, comment, which is a simple POJO. This is because by default, job persistence with Android Priority Job Queue is accomplished via Serialization so it’s easiest to construct Jobs with lightweight Serializable POJOs and expose other dependencies as Singletons outside of Dagger (as done with RemoteCommentService.getInstance()). Alternatively, you can a) use transient dependencies and some Dagger trickery or b) implement custom serialization using libraries such as GSON or Protobuf.

Sync comment response is then observed by SyncCommentResponseObserver which is initialized by our App class when the app is woken up.

To test this setup, try adding comments while offline. The comments will be stored locally as evident by the comment text color in the UI (gray). Try closing the app or restarting the device. Do not open the app yet but do re-enable network connection. Your jobs are now sent to remote data store to be synced. Open the app to confirm that the jobs were already synced as evident by the comment text color (black).

Conclusion

Let’s summarize benefits of offline apps again:

1. users can continue enjoying the app, even while offline.
2. users no longer get error messages due to network connection problems.
3. users benefit from more responsive apps
4. users benefit from conserving battery life.
5. the need for progress bars, etc. is diminished since users interact with fast local storage only. As a bonus, this simplifies the process of building the UI.

Given the above, apps that value user experience should take offline support seriously. Why make network calls for every UI interaction? I doubt your users enjoy waiting for network responses, seeing progress indicators and being told to ‘try again’ whenever network connection is lost. Let’s put user experience first and make our apps accessible to all users, not just those on blazing fast networks.

P.S. Thanks to AndroidWeekly for featuring this article in their issue #279

Recommended Resources

• Sample project to accompany this article can be found here: http://github.com/jshvarts/OfflineSampleApp
• Offline Mode at Instagram presented at New York Android Developers Meetup on January 2017 (fast-forward to 26:02)
• Enabling Trello Mobile Offline Mode
• Android application architecture: Get ready for the next billion! talk by Yigit Boyar.

Visit my Android blog to read about Jetpack Compose and other Android topics