Testing Kotlin Scripts
Recently, I’ve taken it upon myself to do a bit of automation scripting for some of the more tedious day to day tasks I am faced with at work. One such task involves some simple git command parsing and string manipulation, and is very time-consuming. Through some deliberation with my colleagues, I’ve written the script now in several languages in an act to compare them internally. What’s more, one of those languages was Kotlin Script, and testing it was a bit of a bumpy road, but turned out to be quite simple in the end.
I should make a huge note here that I love Kotlin as a language, and am in no way looking to cast a shadow on just how great it really is. Moreover, I’m trying to give a current state of afairs (note the date given in the conclusion) and perhaps save some people a bit of headache, if they value unit testing their custom tools (which they should!).
Other options, and Kotlin
In today’s world, there are a myriad of scripting languages that one can choose to write custom tools for. Initially in fact, when I wrote this test script, it was in Python, which is excellent for such things. A colleague of mine then commented on while it looks like the script works, they had no idea as to what was going on! This is of course, because they’re an Android developer, and haven’t ever worked with Python.
To make mention of it, shell is also a dying alternative. The terse and complex nature of shell’s syntax gives it a sometimes unwieldy barrier of entry, and even some experienced programmers must refer back to syntax guides just to figure out exactly what’s going on. It is quite difficult to write a complex script in any acceptable amount of time, especially when compared to alternatives.
In Android projects, the most obvious candidate for a scripting language is Groovy. It is what the Gradle build system is configured in by default, and is easily testable via Spock. What’s more, there exists the Gradle TestKit which enables you to write unit tests on Gradle tasks in conjunction with Spock. This was very easy to set up and get running in almost no time at all, and I was able to simply follow my normal GIVEN//WHEN//THEN pattern with a test subject as I would in any other code.
Enter Kotlin Scripts. As a scripting language, Kotlin is quite immature. There is a very large draw to be able to utilize it as your scripting language, especially if you plan to move application code to Kotlin. Consistency. New developers only need to learn one single language, and suddenly they are empowered to work anywhere in the code base, be it on application code or in custom scripts. Furthermore, first class support and a strict type system allows for better autocompletion and tooling within Android Studio itself. However, issues exist. Some issues with it currently are that you cannot import code from other script files, and cannot run tests on your script directly in any sane manner. Thus, some alternative path is required.
Test Setup
Setting up unit tests is quite simple. The following is an example hierarchy for a script and its tests.
tools/MyScript.kts
tools/tests/MyScriptTest.kt
tools/tests/build.gradleOur build.gradle file is a normal Gradle file with the Kotlin plugin applied, as documented here. We also apply JUnit as a dependency. Lastly, we apply sourceSets.test.kotlin.srcDirs = ["."] which will set our source directory for tests to the current working directory.
We can now implement unit tests as normal, with a simple piece of code to run our command:
private fun runCommand(command: String) = Runtime.getRuntime()
.exec(arrayOf("/bin/sh", "-c", command))
.inputStream.bufferedReader().readText()This will run the command given and return it’s output as a String. To run our script, we could call:
runCommand("kotlinc -script ../myScript.kts arg1 arg2")And then analyze the output and make assertions as necessary. We can then run our tests via the normal means:
gradle testIssues
So what are the glaring issues with this? Well, for one, it would be nice to be able to utilize the normal testSubject pattern, where we create an instance of a class and then assert different functionality against it. We can get close, but the solution is currently very complicated. See this file for details. Ideally, this will get a lot better as time goes on.
Secondly, and very importantly, testing like this is very slow. Since you need to start up a separate process for each test, you could be waiting up to 5 seconds per test for execution. Mileage on this will of course vary per machine.
Conclusion
This really isn’t meant to shed negative light on Kotlin Scripts. I think it’s a fantastic idea. But what needs to be remembered is that this is all still experimental, so I wouldn’t rush out and rewrite your scripts into Kotlin just yet. The future is very bright however, and I believe that in 6 months or a year or so, we will have much better support.
My professional recommendation, if you want to write a large amount of scripts, maintain a fast test suite for them, and be able to share code among these scripts, then your best bet is to probably use Groovy. This language is already being used in your app for Gradle, unless you’re a bleeding edge kind of person and have already moved to Kotlin Scripts for Gradle. If you’re like the rest of us however, your best and most performant experience will be ascertained from Groovy.
This advice is, of course, predicated on you being an Android developer. If you were utilizing, say, the Django web framework, it would make sense that you would want to implement scripts in Python, to maintain that consistency.
But who knows, in a year, my opinion could be of a completely different colour.
Please note this body of work is based off of the state of Kotlin Scripting as of August, 2017.
Update: A Git Repository illustrating how to set this kind of testing up is located here.
Update 2: CamelCase is the recommended way to name script files, as you can then refer to the script’s instance utilizing this@MyScript within them without issues.
