Kotest Pro Tips
As developers we spend a considerable portion of our time writing tests; unit tests in particular. And if you’ve made the move to Kotlin you have a couple of frameworks available that use Kotlin’s ability to write powerful DSLs to enrich and simplify the testing procedure.
One of those frameworks is Kotest (I am a committer to the project), and in this article I’ll highlight some of the lesser known features of Kotest. This article assumes you are running Kotest 3.1.2 or above.
Parallel Test Execution
If you have a powerful machine, or have a bunch of tests that are IO bound, then there’s no point waiting around for your test suite to finish!
Set the system property kotest.parallelism to a value higher than 1, and separate test files will be executed inside a thread pool. The number of concurrent tests is controlled by the value passed to the property.
Alternatively, override the parallelism() function inside your project config to achieve the same thing programatically.
Randomised Test Order
If you are using a single instance of a test class for all the tests in a spec (the default) then you may wish to randomise the order. In certain circumstances this helps ensure the tests do not have unintended dependencies on state set by an earlier test.
To randomise the tests in a single Spec we can override the function testCaseOrder() to return TestCaseOrder.Random.
class RandomSpec : StringSpec() {
override fun testCaseOrder() = TestCaseOrder.Random
init {
// tests here
}
}Note: This property only affects top level tests. Any nested tests will always be executed in strictly sequential order.
Of course, you may deliberately structure your tests to rely on state set in a previous test in which case this feature is moot!
Arrow Matchers
New in Kotest 3.1.0 was the addition of a brilliant new add on module for arrow. If you’re new to arrow then it’s a set of datatypes, type classes and more that really help when writing code in a functional style. Kotest itself uses arrow for things like error handling.
Add the kotest-assertions-arrow module to your build and then you’ll need a raft of new assertions (matchers) available that can be used with the datatypes provided by Arrow.
For example, if we have a function that parses a string and returns either an Int or a Double depending on the input, we can use the Either assertions.
class EitherExample : StringSpec({
fun parse(string:String): Either<Int, Double> = ...
"parse should detect an integer" {
parse("1").shouldBeLeft(1)
}
"parse should detect a double" {
parse("2.3").shouldBeRight(2.3)
}
})Another example is the non-empty-list datatype. This allows code to statically provide a list that is guaranteed to contain at least one element. Testing against this requires custom matchers as it is not part of the regular collections library.
class NelExample : StringSpec({
fun emails(): NonEmptyList<String> = ...
"emails should contain no nulls" {
emails().shouldNotContainNull()
}
"emails should be unique" {
emails().shouldBeUnique()
}
})There are other assertions available — for the Validation, Option, and Try datatypes as well as for Either and NonEmptyList shown here.
Focusing Tests
Sometimes we want to temporarily disable some tests in of a test suite. Perhaps we’re experimenting with some API changes and don’t want to have to keep changing all the tests until we’re happy with the new API. Or perhaps we’re debugging and want to reduce the noise in the output.
Whatever the reason, Kotest supports selectively running a single top level test by preceding the test name with “f:”. Then only that test (and subtests of it) will be executed.
For example, in the following snippet only the middle test will be executed.
class FocusExampleTest : StringSpec({
"this will be ignored" {
// test here
}
"f:this is not ignored as it is focused" {
// test here
}
"this should be ignored too!" {
// test here
}
})Once we are finished, we can just remove the “f:” prefix.
The opposite of this is also supported, which is to prefix a test with an exclamation mark and then that test (and subtests of it) will be disabled.
For example, in the next snippet we’ve disabled the first test by adding the “!” prefix.
class FocusExample2Test : StringSpec({
"!this will be ignored" {
// test here
}
"this will run" {
// test here
}
"this will run too" {
// test here
} "!this is also ignored" {
// test here
}
})
Dynamically Disabling Tests at Runtime
Similar to the last section, perhaps we have whole groups of tests that we want to exclude at runtime. An example could be tests specific to a type of operating system, or tests that run against a particular version of Hadoop. When we’re on a different operating system, or not connected to a hadoop cluster, we want those tests to be skipped.
We can easily support this through the use of tags. The idea has been around for a while, in frameworks such as ScalaTest and TestNG. The typical approach is that environment variables are used to enable or disable tags. What’s nice about the Kotest approach is the additional ability to enable or disable tags dynamically at runtime using Kotlin code inside your codebase.
To do this we create an instance of the TagExtension interface and then provide whatever tags we want to when the function is invoked.
First, lets create a tag for tests that should only run on Windows and add it to a test.
object WindowsTag : Tag()
class WindowsOnlyTest : StringSpec({
"program files should exist".config(tags = setOf(WindowsTag)) {
// test here
}
})Then we create an extension that will exclude this tag when not on a windows machine.
object WindowsTagExtension : TagExtension {
fun isWindows() = ...
override fun tags(): Tags =
if (isWindows()) Tags.Empty else Tags.exclude(WindowsTag)
}Then we just need to add this extension to our project config as we would any other extension, and the test will be marked as skipped when running on unix or a mac.
A Fresh Instance Per Test
Inside your test class we can override a function isInstancePerTest() to instruct Kotest to create a new fresh instance of the test class for each test case that is executed.
This is the standard model in Junit, where a fresh instance is created for each test method. It’s slightly trickier when you allow arbitrarily nested test trees, but Kotest allows it.
Even better, when this feature is enabled, setup / teardown functions and parent scopes are executed for each test case, making it trivial to re-use test fixtures without resorting to tricks.
Take a look at this example. It’s contrived but shows how each individual test scope runs independently from the others.
class WordSpecOneInstanceTest : WordSpec() { override fun isInstancePerTest(): Boolean = true var string = "" init {
"outer 1" should {
string += "a"
"inner 1" {
string += "b"
string.shouldBe("ab")
}
"inner 2" {
string += "c"
string.shouldBe("ac")
}
}
"outer 2" should {
string += "d"
"inner 1" {
string += "e"
string.shouldBe("de")
}
"inner 2" {
string += "f"
string.shouldBe("df")
}
}
}
}
As you can see, each test scope has it’s own copy of the string variable. If the class was a single instance, then the string variable would accumulate at each step failing the later tests. We can see this by writing the same test but with this feature disabled (the default).
class WordSpecOneInstanceTest : WordSpec() { var string = "" init {
"outer 1" should {
string += "a"
"inner 1" {
string += "b"
string.shouldBe("ab")
}
"inner 2" {
string += "c"
string.shouldBe("abc")
}
}
"outer 2" should {
string += "d"
"inner 1" {
string += "e"
string.shouldBe("abcde")
}
"inner 2" {
string += "f"
string.shouldBe("abcdef")
}
}
}
}
Mix and match this feature to suit your needs!
I hope this gives you some pro-tips on using Kotest. In a future article I’ll introduce more hidden features as well as the latest on property testing.
