Infinite Scrolling with Android Paging Library and Flow API

Recently I was interviewed for an Android software development position, and I was asked to live code a simple app that demonstrates fetching a list of items from a network resource and display their information in an endless list.

I was given an API that supports paging and off I went coding away.
There was just one problem.

I had never implemented this behavior before, therefore, I was able to code an app that fetches results from the API, but I failed to implement the endless scrolling feature in the allowed time frame.

So, this means I now have a great topic for a blog post!.

What is endless scrolling?

You know when you scroll through Facebook or Instagram, and it seems as though the list of posts is so big you might never get to the end of it? Well, that’s because it’s true.
But loading huge amounts of data and feeding it to our RecyclerView would definitely be a problem.
It would clog the apps memory, or local storage, and will no doubt be detrimental to performance.

How can we make it better?

We can load a chunk of the data, then, when we have finished scrolling through it, we can load another chunk, right? Sounds good.

We would have to manage this though, make sure we don’t keep useless data, but also keep loaded data for reverse scrolling, we would also need to manage all those API calls and cancel them if needed, apply multithreading logic and keep the aliens from reaching the kill switch on Alpha Century…

Well, apart from that last thing about the aliens, there is a simple solution to manage all those issues

Presenting the Android paging library

Paging, or “Pagination” is something mobile developers have struggled with for years, but much like networking and concurrency, recent libraries have simplified the issue, allowing us developers to focus on our app-specific logic, rather than spend our time writing boilerplate code.

First, let’s keep in mind that to use the Android paging library we would need to work “Reactively”
Reactive programming is a concept that allows us to work asynchronously, and this blog post requires some knowledge of the reactive paradigm. There are a few ways to work reactively in Android, the main ones are:
RxJava, LiveData, and the new, still experimental, Flow API.

Android paging supports all three and, in this example, we will work with Flow.

Here is a diagram that represents the data flow from the endpoint (local or remote) all the way to the UI.

First, add the paging library dependency:

dependencies {
  def paging_version = "3.0.0-alpha07"

  implementation "androidx.paging:paging-runtime:$paging_version"

  // alternatively - without Android dependencies for tests
  testImplementation "androidx.paging:paging-common:$paging_version"

  // optional - RxJava2 support
  implementation "androidx.paging:paging-rxjava2:$paging_version"

  // optional - Guava ListenableFuture support
  implementation "androidx.paging:paging-guava:$paging_version"
}

To obtain data from a network resource, we usually have a dedicated interface that holds the backend API endpoints and retrieves data from it using OKHttp, retrofit, or a custom implementation, In my sample project, I had used the PixaBay API.

So the first thing we need to do is wrap our service with a PagingSource, the source will take in the service and any other parameter that is required for the data fetch (I.e. query parameters, tokens, id’s, etc.)

the source has one method to implement, the load() method, that returns a LoadResult.
LoadResult is a sealed class (similar to an enum with associated values) that has two data class types:

• Page. which contains one chunk of data from the source, and has 3 values:

data: the actual data.
prevKey: the index of the previous page or null if this is the first page
nextKey: the index of the next page or null if this is the last page.

• Error. which represents any type of error that occurred during the loading process.

Here is how we implement the pagingSource class.

So, as we can see, we can get the current key as a parameter of the load function for free! no need to manage that.
So our first line sets the page number we are at, and we called it “position”, if the params.key value is empty we will set a starting index based on the API index format.

val position = params.key ?: STARTING_PAGE_INDEX

Then we open a try-catch block to handle the network call and create a LoadResult from the response we get from the network,

return try {
    val response = service.getPics(position)
    val photos = response.photos
    
    LoadResult.Page(
        data = photos,
        prevKey = if (position == STARTING_PAGE_INDEX) null else position,
        nextKey = if (photos.isEmpty()) null else position + 1

    )
}

Important to note that this specific api supports paging, but if it does not, we need to manage that our selves

And lastly, catch the errors and return a LoadResult.Error with the exception.

catch (exception: IOException) {
    return LoadResult.Error(exception)
} catch (exception: HttpException) {
    return LoadResult.Error(exception)
}

Now that we have a paging source, we need to somehow create a flow of data from it.

Pager to the rescue!

Now we would go to our repository class.

The repository class is usually a layer that abstracts the data from form the services (network service or local database) to the viewModels, and in our case, it emits a flow of data.

class PixaBayRepository(private val service: PixaBayService)  {

    fun getPhotos(): PhotoList {
        return service.getPics(1)
   }
}

Now instead of exposing results from the service, we need to expose a flow of pagingData of the type of the results.

We also don’t directly call the service to fetch the data, we instead, instantiate a Pager instance. this class will take in a PageConfig and the service, and will supply us with a flow of pagingData of the type the service returns

Notice that if you don’t work with Flow or wish to work with LiveData or RxJava, you can convert the flow to observable, flowable, or live data at this point.

At this point, we have a flow of pagingData that holds the values our list should display, now what?
All we need to do is wrap our list adapter with a PagedListAdapter which exposes one important suspended method: submitList() which take a pagingData instance

class PhotosAdapter : PagingDataAdapter<Photo, RecyclerView.ViewHolder>(PHOTO_COMPARATOR) {
       // Your implementation
}

It takes in an implementation of the DiffUtill.ItemCallback class as a parameter, We can add this comparator as a companion object in the PhotosAdapter class

So we covered all the main components from the first diagram:

But where are we gonna put all those pieces?

Well, if you’ve followed the Android recommendations and Jetpack libraries, your design should look like this:

So the stream starts from the UI, the view would collect the photos flow from the ViewModel and submit it to the adapter.

viewModel.getPhotos().collectLatest {
     adapter.submitData(it)
}

Behind the scenes, the method sublimtList uses a coroutine to calculate the diff between the new data and the old data and update the list when the calculation is finished.

So we would launch this block in the lifecycleScope

lifecycleScope.launch {
       viewModel.getPhotos().collectLatest {
           adapter.submitData(it)
       }
   }

And that’s it!

Wasn't that fun?.
The paging library contains more great features like adding footer loading animations and list separators.

All the code for this example can be seen on my GitHub repository.