Kotlin: do you need (another) HTTP client?

“clouds enveloping brown rockhill during daytime” by Sam S on Unsplash

It dawned on me while I was attending a local hackaton. Attendees were requested to get some results from a public API. And suddenly, people around me were arguing which HTTP client to use. Some went for RestTemplate, others for Apache HTTP Client, yet others tried something like JSoup.

It turns out most of the seasoned Java developers weren’t aware that Java standard library already has an HTTP client built it. It’s just called URL

To improve that situation, let’s take a simple task, like parsing response from GitHub API, without an external HTTP client (we’ll still use some library to parse JSON, though).

We’ll start with the data class, as it will help us figure out what we want to get:

data class Repo(val name: String,
                val url: String,
                val topics: List<String>,
                val updatedAt: LocalDateTime)

Since we’ll be using Kotlin, it’s only natural that the URL we’ll be getting will belong to JetBrains:

https://api.github.com/orgs/jetbrains/repos

Our initial code will look something like this:

Since GitHub API is paginated, we fetch our repos page by page, until there are no more pages left.

Of course there are other nicer ways to write the same logic, but that’s not the focus of this article.

Now we’ll add some code to print those results, in descending order, sorted by updatedAt field:

allRepos.sortedByDescending { 
    it.updatedAt 
}.forEach {
    println(it)
}

Now we’re all set to do some networking.

To get contents from a remote URL we can simply use:

URL(url).openStream().use { 
    it // InputStream
}

Calling use will automatically close the stream when we exit the block.

Common mistake related to use on streams is to try to return a reader from use block like that:

Everything you need to do before the stream is closed must be done inside the use block:

Now, suppose we have this input stream representing JSON, how can we parse it?

One option is to use Jackson:

compile group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.9.6'

The common option is to use ObjectMapper().readValue() method.

But since the response of this specific API is JSON array, parsing it becomes a bit cumbersome:

val repos: List<Repo> = ObjectMapper().readValue(it,
        (object : TypeReference<List<Repo>>() {}) )

Short, but ugly.

And turns out it doesn’t work:

InvalidDefinitionException: Cannot construct instance of `Repo` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)

One option to fix that is to add another dependency:

compile "com.fasterxml.jackson.module:jackson-module-kotlin:2.9.+"

And use the new jacksonObjectMapper() function:

val repos: List<Repo> = jacksonObjectMapper().readValue(it)

But that will create more problems that it will resolve.

What if there was another way to parse JSON, which would give us slightly more control?

Luckily, it exists in the form of readTree()

val result = ObjectMapper().readTree(it).map { node -> // JsonNode
    // Our parsing code comes here
}

Now we can iterate over each object in the JSON array returned by the API, and start parsing them.

Getting some properties is very trivial:

But since topics are nested JSON array, which is also optional, we use safe call and map the values:

Finally, we return our newly created data class:

Repo(name,
     htmlUrl,
     topics=topics,
     updatedAt=LocalDateTime.parse(updatedAt))

Does it work? Of course it doesn’t! We get the following exception:

DateTimeParseException: Text '2018-08-09T04:43:06Z' could not be parsed, unparsed text found at index 19

Seems that GitHub is using different format than what LocalDateTime uses by default (which is ISO_LOCAL_DATE_TIME).

Let’s fix that by using the correct date-time format for GitHub:

updatedAt=LocalDateTime.parse(updatedAt, DateTimeFormatter.ISO_OFFSET_DATE_TIME)

That’s much better. But there’s another problem. Our topics always return empty:

Repo(name=Chocolatey, url=https://github.com/JetBrains/Chocolatey, topics=[], updatedAt=2017-03-29T02:41:51)

After a quick skim through GitHub API documentation you may notice that to get the topics, you’re required to set a specific header:

Accept: application/vnd.github.mercy-preview+json

But how do you do that with URL? Do you need to forget about it and go back to the conventional HTTP clients?

Of course I didn’t bring you all the way here just to tell you that “sorry, but our princess is in another castle”. We’ll just have to use another, lower level API for that, called openConnection()

One doesn’t simply use openConnection(), of course:

URL(url).openConnection().use { ... } // Won't work

You’ll have to get the input stream from it:

URL(url).openConnection().
        getInputStream().use { ... }

That’s what openStream() method did for us before.

Now having that connection, we can set headers on it:

openConnection().setRequestProperty("Accept", "application/vnd.github.mercy-preview+json")

But the problem is that setRequestProperty() is not fluent. You cannot nicely chain it and call it like that:

Luckily, in Kotlin we have apply() to solve that:

And our topics parsed as expected now:

Repo(name=Chocolatey, url=https://github.com/JetBrains/Chocolatey, topics=[choco, chocolatey, chocolatey-packages, jetbrains], updatedAt=2017–03–29T02:41:51)

Final code looks like this:

This could be shortened

If you won’t want to concert yourself with streams and headers, and also (very) confident that your responses would fit into memory, there’s also a simpler way provided by Kotlin: readText() method:

Summary

So, this article demonstrates that it’s not mandatory to introduce another dependency for simple HTTP use cases in either Kotlin or Java.

Does that mean you should stop using HTTP clients you’re using now?

Probably not. HTTP client libraries have a lot of great features, like following redirects, caching responses, handling security, and much more.

But make sure that you consider carefully the use cases you have. And don’t bring the gorilla, if you only need the banana.