Android and App Engine with Kotlin

In the last Droidcon Vietnam/Droidcon Dubai I received positive feedback about the presentation “Kotlin on Client and Server using App Engine”. I had pending to write this post and push the code. If you prefer to check the repository directly and save 11 minutes of reading here it is:

cdsap/Kotlin-Client-Server

If you prefer reading it, this article covers:

• Simple Hello World in App Engine with Kotlin
• Developing Client and Server with Kotlin with a more advanced example
• Defining use cases from a third module which will be the same dependency for the Client and the Server

Motivation

Last week Google App Engine turned ten years, being one of the older services offered by Google:

Years ago I remember when we wanted to develop quick MVPs and we wanted to develop the backend side, we used App Engine. At that time you had only two language environments(Java/Python). The free daily limit of resources in quotas and the scaling infrastructure did in my opinion one the best solutions for the startups.

Today you have two different ways to run the instances:

• Standard Environment: is the traditional instance model with support for language environment of Java, Python, Go and PHP
• Flexible Environment: instances run within Docker containers on Google Compute Engine virtual machines. Supported languages are Python, Java, Node.js, Go, Ruby, PHP, and .NET.

Every instance contains the language environment, App Engine APIs, your application code and memory.

If you want to learn more about how is scaling App Engine and how instances are managed check these links:

An Overview of App Engine | App Engine standard environment for Java | Google Cloud

How Instances are Managed | App Engine standard environment for Java | Google Cloud

And what about Kotlin? On the website, there is no single reference to Kotlin as a language to develop within App Engine

At least, the theory says we will be able to develop in a Standard Environment targeting JVM, with the tasks of the Kotlin plugin, likecompileKotlin and compileTestKotlin, classes will be generated and we will have the same scenario as Java.

I found this great article written by Jon Imanol Durán posted last year:

Using Kotlin in Google App Engine

The article was written before Kotlin was announced as an official language in the Android platform. Jon explains how we can use Kotlin in App Engine. Thanks to this post I started to research the possibilities of Kotlin on App Engine and how we can work on the same project with Client/Server.

Hello World

One of the reasons I’m writing this article is to show how easy is to deploy Kotlin in App Engine. To start working, we need:

• Google Cloud SDK:

curl https://sdk.cloud.google.com | bash

It provides a set of tools for the products included in Google Cloud: Compute Engine, Kubernetes, Cloud Storage… If you are using Firebase Test Lab by command line, you are using the same SDK ( gcloud firebase test android models list)

• Google App Engine Component

gcloud components install app-engine-java

• Android Studio/IJ
• Gradle App Engine Plugin

GoogleCloudPlatform/app-gradle-plugin

This plugin provides tasks to build and deploy Google App Engine applications:

Once we have set up our local environment we can start developing the simple HelloWorld(actually is very similar to Jon’s example).

The requirement is: I want to write “Hello World” in the response to the request made on the local instance of App Engine

Let’s start with the configuration of the project, in the root build.gradle we have to include the classpath for App Engine Gradle plugin and of course Kotlin:

Thebuild.gradle of the project looks like this:

The war plugin extends the Java plugin to add support for assembling web application WAR files.

Regarding the code, we are going to keep it simple. We need a service that extends from HttpServlet and overrides the method doGet to write the response for the client request.

Additionally, we need to setup the WEB-INF/web.xml where we register the servlet and define how we want to map the requests:

Now we are ready to deploy locally with the GAE Gradle plugin

./gradlew appengineRun

This command generates a local instance of App Engine to test/debug our application, to access it only type in the browser:

And you have it, a piece of cake, right?

Here the complete example:

cdsap/HelloWorldAppEngine

Advanced Example

Once we have seen how easy is deploying a Kotlin App is time to move to an advanced example where we can share code between client and server.

The idea is to develop a straightforward API with Kotlin which will be consumed by the Android Client developed in Kotlin too, and finally, show how we can share some parts of the code.

Following the hype of the cryptocurrencies I have chosen to build an API for https://bx.in.th/, a popular exchange in Thailand, we will get data from Bx, and then we will offer this data to our clients. The diagram would be:

Client

From the client side we want to implement something like:

I’m not going to enter in details about presentation patterns or recommendations of that. I want to focus on how we layer the app to make it testable and easy to extend with future functionalities from the API.

The core of the app will be the domain layer where we define the uses cases of the app and the repository where we abstract the source/origin of the data required:

In the domain layer I’m going to define two uses cases:

• GetTrades: Getting the trades from the local repository
• SyncTrades: Getting the trades from the remote repository and inserting them in the local repository

The uses cases will be consumed by the presentation layer where you can put your favourite pattern, in the example is a simple MVP with coroutines to handle the asynchronous tasks

The detail of implementation for the local repository will be SqlLite using the abstraction Room. Related to the remote repository will be implemented with Retrofit to access the API defined by App Engine. Finally, to allow the synchronisation with the server side I will use JobScheduler that provides an API for scheduling jobs.

Here a snippet of the simple DI graph that will help us to understand how are built our dependencies:

And the implementation of the use case SyncTrades

The final picture with the details of implementation would be:

Server

Initially, we can be a little bit lost on the server side. Coming from Android development, we are not aware of current trends or frameworks.

Let’s keep things simple, concerning the architecture, we don’t have to be far from the approach of the client, at the end, we want to do the same: consume data from a remote repository, persist it and offer this data to an external client:

Before starting with the implementation, in terms of infrastructure, we need to introduce two services from Google Cloud that will help us to build and understand the server side together with App Engine.

Google Cloud DataStore

Based on Google BigTable, DataStore is a NoSql scheme Database integrated into Google Cloud.

If you are coming from relational databases these are key differences:

https://cloud.google.com/datastore/docs/concepts/overview

Like App Engine, is built for automatic scaling and offer some features like:

• Flexible storage and querying of data
• High availability of reads and writes
• Automatically encrypts all data

Google Cloud Endpoints

https://cloud.google.com/endpoints/

Cloud Endpoints is an excellent service that provides tools and libraries to build RESTful APIs on Google Cloud. Following the official site is:

An NGINX-based proxy and distributed architecture give unparalleled performance and scalability.

When you’re deploying a new version of App Engine, the proxy is deployed automatically with your application. It gives support for User Authentication with Firebase Auth, Google Auth and Auth0, and offer tools to monitor traffic, error rates and latency:

https://cloudplatform.googleblog.com/2016/09/manage-your-APIs-with-Google-Cloud-Endpoints.html

If we put the three products together we will have:

Where Cloud Endpoints provides API for our clients, App Engine hosts the web app and Cloud Datastore persists the data from our app.

Now we are ready to start the implementation on the server side. The domain layer looks very similar to the client side. We have two use cases:

• GetTrades : We will get the trades from the local repository
• SyncTrades: Will get the trades from the remote repository and will save it on the local repository

The local repository uses DataStore as persistence mechanism. Like in the client we are going to use one abstraction to facilitate the access to the data. In this case will be Objectify. The remote repository will be implemented by Retrofit. As Android developers, we know it very well, but as you can see in the description is not only for Android:

A type-safe HTTP client for Android and Java

Here is when we can notice the similarities between Client/Server, not just about the abstractions but the dependencies too(Dagger, Gson, Retrofit...)

Finally, to facilitate the synchronisation with Bx, we will use App Engine Cron Service that it allows you to configure regularly scheduled tasks that operate at defined times or regular intervals.

The diagram with the details of implementation is:

Let check some parts of the code:

Obviouslybuild.gradle has grown with more dependencies required by our implementation

Implementation of the use case SyncTrades:

Remote repository to get data from Bx:

Partial implementation of the module required to inject the dependencies:

And finally, I want to show how we are defining the API with Cloud Endpoints:

Check how easy Cloud Endpoints is defining the exposed API for the clients. For instance, if we want to access the endpoint getTrades, the URL generated is:

YOUR_URL/api/v1/trades/1

Additional steps to set the project will be:

• Set up Objectify entities in WEB-INF/web.xml
• Set up mapping and endpoints in WEB-INF/web.xml
• Set up the policy of the cron-job in WEB-INF/cron.xml (not working in local instances)

In summary, like we only were focus on providing the API and populate it, the implementation was very similar to the client and easy to develop :)

Problems

Of course, I would lie you if I tell everything was working from the first moment, here the problems I found during the implementation of the server

Retrofit

When you try to use Retrofit initially you are going to get this error :

java.lang.NoClassDefFoundError: java.net.ProxySelector is a restricted class. Please see the Google App Engine developer’s guide for more details.

You can find more details about the issue here:

Using Retrofit in Google App Engine · Issue #1149 · square/retrofit

Fortunately, it was resolved and now you can extend okhttp3.Call:

Check here to see the answer with the complete solution in Java

and use it in the declaration of Retrofit:

.callFactory(CallFactoryWrapper())

Cron

The documentation says you can’t call the Cloud Endpoints directly in a cron job:

Kindly they are providing the solution, and you have only to use a request to your endpoint in the service defined by the cron job.

Kotlin

I haven’t found any problem with Kotlin at all :)

Sharing between Client and Server

In terms of Android project if we want to implement same Logging implementation between different modules we can create another top module with the common parts like:

So how fits this approach in one project that has two modules of different platforms? We can define the interface of logging at the common module and implement it within the platform modules with the details of each platform. But is not the point where I want to go:

You have noticed that the approach for both platforms is identical, we have the domain layer implementing the uses cases driven by our requirements and a repository layer abstracting the source of the remote/local origin of data. Let’s compare regarding the types differences:

Quickly we notice the differences between the different implementations of the local storage, but we can see the similarities of the contracts and domain entities models in both platforms.

And is here where we can realise that we can drive the design of both platforms from the new module called Core. Concerning domain layer, the use case GetTrades is the same from both platforms although the implementation will be different of course. We can define the domain entities models for both platforms in the Core module avoiding future problems and encouraging the ownership of the domain understanding for the client/server app.

Here the approach for the Core module:

cdsap/Kotlin-Client-Server

Now server and client modules depend on Core, and they have to implement the Use Cases defined in the Core with common domain entities:

But of course this is a simple example, I would like to explore in the future more integrations from Repositories to Modules abstractions:

And this model brings new possibilities in our development/builds:

• Deploying new versions of API can be coupled with the Android Client, new changes on the API can trigger the pipeline on CI to validate the automation tests
• Generate the domain entities and use cases based on specs and generate endpoints and implementations on the client/server side

Final words

Thank you very much for reading until the end. I hope you enjoyed it.

From the Android developer perspective, it has been nice to explore other platforms using same tools and check how easy is deploy Kotlin projects in App Engine.

The complete repository is here:

cdsap/Kotlin-Client-Server

Feel free if you want to collaborate in the repo with the Frontend part( with Kotlin Js of course :))