Exploring Paging Library from Jetpack
In my quest to explore all that Android Jetpack has to offer, I took some time to understand capabilities of the new Paging library introduced at Google I/O ’18. This library can be used to implement endless scrolling (infinite scrolling) for your RecyclerView data. Let’s explore the various ways Paging library can be used by building a Note-taking app in Kotlin.
While paging is the main focus, we will use the following components of Android Jetpack:
- Paging Library
- Room Persistence Library
- Navigation library
- Lifecycle (ViewModel, LiveData)
Dagger Android 2.17 is used for dependency injection in most branches. Koin usage is also demonstrated. The app has a decent separation of layers (Clean Architecture) including mapping Model objects between Data and Domain layers.
You can check the source code at https://github.com/jshvarts/PagingRoom.
There are several branches:
- master branch that the rest of the repo is built upon. There is no paging implemented here.
- room-livedata branch that introduces the Paging library in its simplest form (
PositionalDataSourceis the default Paging implementation in Room). - room-rxjava branch that shows how you would use the Paging library with RxJava.
- livedata-custom-datasource branch where we will build a custom
ItemKeyedDataSourcewith Dagger. More details on various DataSource types are below. - livedata-custom-datasource-koin branch where we will build a custom
ItemKeyedDataSourcewith Koin. More details on various DataSource types are below.
Paging library overview
Above you see an overview of how Paging library fits into Clean Architecture.
Data layer: Contains implementation of the Domain layer abstractions such as Data Repository.
Domain layer: Contains your repository abstractions for the Data layer to implement. The green boxes here signify optional component — you don’t have to implement DataSource and DataSource.Factory if the built-in implementation already does what you need (in this case, Room comes with default paging implementation of type PositionalDataSource). It is a great example of how various Architecture Components work together.
Presentation layer: Here you have your ViewModel that produces LiveData<PagedList<T>>. Again, the green boxes signify optional APIs to implement. You can use LivePagedListBuilder or RxPagedListBuilder to produce your LiveData without using DataSource.Factory or having to customize your PagedList.Config.
UI layer: Here you observe your LiveData<PagedList<T>>. Once you have your PagedList, you submit it to the PagedListAdapter which binds your data and adds it to the RecyclerView.
PagedList is a central component to how Paging works. It loads data in pages asynchronously. It’s backed by DataSource which lazily delivers snapshots of data as the PagedList needs it (when user scrolls).
While scrolling, if placeholders are enabled, they are replaced by the actual data. Otherwise, the new PagedList simply gets appended to the currently displayed list based on the last visible item (it works similarly for scrolling up). See more on placeholders below.
Wrapped in LiveData, the PagedList then gets dispatched to the UI. If the data becomes stale (the data in the DataSource was changed while user was looking or scrolling the items), it gets invalidated and the DataSource dispatches new PagedList wrapped in LiveData. Paging library is able to support infinite scrolling while supporting data updates without ever calling the expensive onDataSetChanged().
DataSource is a wrapper around your sources of data (whether it is a local database such as Room, Realm, SQLite, etc.) or a remote API (usually accessed via Retrofit, Volley, etc.) or a combination of both local and remote sources.
Let’s look at 3 implementations of DataSource:
- PositionalDataSource offers position-based data loader for a fixed-size, countable data set. For instance, it allows scrolling through a list of contacts or jumping to a particular position in the list (i.e. jumping to contacts that start with a particular letter). This is how Room implements paging by default under the hood. The repo for this article contains examples of implementing PositionalDataSource with LiveData or PositionalDataSource with RxJava.
- ItemKeyedDataSource can be used when you can identify items before the start of the current list and after the end of the current list based on particular item key because the keys are ordered. The repo for this article contains an example of implementing ItemKeyedDataSource with LiveData.
- PageKeyedDataSource is often used for remote API calls to load data in chunks where the response returned will contain the data itself as well as pointers to next and previous pages of data (much like a LinkedList). It reduces the size of the data transfer for a given scroll. Since the total count is usually not available in remote APIs, placeholders should be disabled. More on that later.
Project setup
Let’s go over the code at https://github.com/jshvarts/PagingRoom/tree/room-livedata
We’ll start off with these main dependencies:
Implement the DAO. Note the return type of DataSource.Factory<Int, NoteEntity> where the key is an Int since we are using positions (remember by default Room provides PositionalDataSource). Why do we return a DataSource.Factory? A database is something that changes over time and each DataSource represents a snapshot of data at a given time. DataSource.Factory creates those snapshots for us.
Let’s define our database and pre-populate it with 100 items when it’s initially created:
It’s time to add our Repository interface in the Domain layer.
And implement it in the Data layer.
Note that the Domain layer “does not know” about any of the other layers (you can see that based on package imports). In turn, the Data layer implements the Domain layer abstractions. This will come in handy if we decide to swap our database implementation as well as for testability and modularizing our code.
Let’s create a Domain UseCase/Interactor so we can eventually put any business rules there. At the moment, the UseCase does not do anything special but it does hide details about our repository from the Presentation layer. The note listing presentation layer does not need to know all of the things (i.e. inserting or deleting notes) that the repository can do.
It’s time to set up our Presentation layer. Let’s add the ViewModel for loading notes.
The LivePagedListBuilder uses our default DataSource.Factory and PagedList.Config. The only value you must provide to configure your DataSource on the fly is the pageSize. For more flexibility, we could build a custom configuration object PagedList.Config and define a custom DataSource.Factory to pass in place of getNotesUseCase.allNotes(). You will see an example of using a custom DataSource.Factory later. For now let’s look at configuring the PagedList:
Placeholders
Placeholders provide visual indicators for items that have not been loaded yet but will be as soon as the user scrolls to these items. Besides better user experience, a nice benefit of the placeholders is that developers don’t need to implement a loading spinner at the bottom of the list since the users already see loading indicators (such as gray areas for icons) while the actual items are yet to be rendered. Note that placeholders can only be enabled if your DataSource provides total items count.
enablePlaceholders is set to true by default. If we don’t set initialLoadSizeHint, in a PositionalDataSource by default initialLoadSizeHint = PAGE_SIZE * 3.
Note that the inferred type of the noteList is LiveData<PagedList<Note>>. Let’s see how the UI layer (Fragment in this case) observes and uses this value.
Notice the line recyclerViewAdapter.submitList(pageNoteList). It’s a perfect time to look at our RecyclerView/Adapter/ViewHolder implementation.
Here is the ViewHolder:
Note that our ViewBinder does not just bind items that have been loaded but also placeholders for items yet to be loaded if user scrolls to them.
Let’s now look at the NotesAdapter:
The Google Paging sample repo has a great summary for the Adapter API: PagedListAdapter is a RecyclerView.Adapter base class which presents the contents of PagedLists in a RecyclerView. It requests new pages as the user scrolls, and handles new PagedLists by computing list differences on a background thread, and dispatching minimal, efficient updates to the RecyclerView to ensure minimal UI thread work.
Our NoteAdapter implements PagedListAdapter and defines a diffCallback which informs the PagedListAdapter how to compute list differences as PagedLists arrive. If data is ever invalidated, the diff is computed on the background thread and the Adapter seamlessly updates the RecyclerView.
Notice that we retrieve the Note item in onBindViewHolder() using getItem(position). The item is nullable since placeholders are enabled. We don’t attach the clickListener to item placeholders (null items) before they are loaded.
That’s it! Our RecyclerView is now able to load PagedLists as the user scrolls. If you want to debug/confirm that pagination works as expected, you can simply print out currentList inside your onBindViewHolder(). It is a getter of the PagedListAdapter.
If you aware of a better way to debug pagination (perhaps via some config setting?), please let me know!
Paging library with RxJava
We will now go over the code at https://github.com/jshvarts/PagingRoom/tree/room-rxjava
There is not a whole lot of difference between the above steps and using RxJava. Even the DAO method signatures stay the same (functions that support paging still return DataSource.Factory<Int, NoteEntity>. Much of the code is the same except for the NoteListViewModel and the NoteListFragment.
As far as the ViewModel goes, we just need to replace LivePagedListBuilder with RxPagedListBuilder!
You can define your noteList to be either Observable<PagedList<Note>> or Flowable<PagedList<Note>>. Consuming the stream in the Fragment is the biggest difference between Rx and LiveData implementations.
As you can see, there is slightly more code to write when RxJava is used since you now have to clear your CompositeDisposable to avoid memory leaks (something that LiveData gives you for free). Given that, I don’t really see a good enough reason to use RxPagedListBuilder when implementing pagination. I strive to write as little UI code as possible and keep it as “dumb” as possible (to avoid logic that would increase the need to write and maintain slow instrumentation tests for the UI).
Implementing custom ItemKeyedDataSource
Let’s go over the code at https://github.com/jshvarts/PagingRoom/tree/room-livedata-custom-datasource
We need to be able to load an initial PagedList and then load the next PageList based on item key. Let’s assume that our noteText is unique and can be used as our key (from business standpoint, it does not make a good key but we’ll just go with it so we don’t have to change too much of the code from above).
Let’s update our entity class so it’s clear that our NoteEntity has a unique index now.
Our DAO class will change to the following:
Note that the functions used for pagination no longer return DataSource.Factory. Instead, we will be creating our custom DataSource.Factory below and wrapping the results of the DAO calls ourselves.
Our NotesRepository interface will change to:
The interface is then implemented in the Data layer as follows:
At this point, I decided to drop the UseCase since the DataSource and DataSource.Factory to be created in the Domain layer can be used by the Presentation layer directly. This sort of goes against the principles I usually follow when implementing Clean Architecture but at the moment there is no business logic to include in the UseCase anyway and it makes the sample code easier to follow.
Let’s create our DataSource implementation:
Note how the key is defined as item.text and how we define what loadInitial and loadAfter mean.
Let’s define the Factory that utilizes that DataSource:
And that’s it! We are now ready to use it in the Presentation layer.
Note how we inject the NotesDataSourceFactory and then use it to create our LivePageListBuilder. There is still a decent separation of concerns since we are injecting the Domain layer component which itself gets created by injecting the Domain layer Repository interface.
And that’s it! There are no more code changes to make for this custom ItemKeyedDataSource.
Offline support with BoundaryCallback
I wrote about offline support before. The main concept is your local database is the single source of truth and you synchronize data with remote APIs whenever network connection is available. The Paging library introduces PageList.BoundaryCallback which will trigger remote data refresh when user scrolls through PageLists from local database and there is no more data to display. It will only do so if there is a network connection. There is a great example of doing so in the Paging codelab so I won’t cover it here. It’s an excellent feature of the Paging library that will greatly improve user experience as more apps start implementing it.
Conclusion
I really like the new Paging library. It gives you a lot of power, does a lot of work for you on the background thread and reconciles the PageLists that come in without ever having to call the costly onDataSetChanged(). Besides being efficient and not blocking the UI thread, it is lifecycle-aware — we don’t want any work to be done when the app is not in foreground.
In addition, I really like having a well-defined API for utilizing multiple data sources working together in tandem. All these features make developers’ lives easier and user experience significantly better. For these reasons, I am looking forward to seeing this library gain more popularity.
Resources
- Video: manage infinite lists with RecyclerView and Paging
- Paging library overview
- Paging codelab
- https://github.com/googlesamples/android-architecture-components/tree/master/PagingSample
Thanks to AndroidWeekly for featuring this article in their issue #329
Visit my Android blog to read about Jetpack Compose and other Android topics
