Parameterized tests with Kotlin’s Sealed Classes

Photo by CDC on Unsplash

Parameterized tests are a very powerful feature. Those are tests that are taking method parameters. This is very useful if we need multiple runs of the same test but with different input arguments.

JUnit5 has built-in support (similar can be achieved on JUnit4 and a specific Runner). The API provides various ways to supply the method parameters like a list of strings or the values of an enum.

Let’s look at the enum version. Let’s also assume that we have some code that allows various ways to log-in and those are summed up in this enum:

enum class Method {
    FACEBOOK, GOOGLE, EMAIL
}

We probably should have a test that verifies that we track these sign-ins correctly. Therefore we might write something like:

@Test
fun `tracks with Facebook`() {
    
    tested.submit(Method.FACEBOOK)

    verify(analytics).trackAnalyticsEvent(
        Event(
            "login",
            "method" to "FACEBOOK"
         )
    )
}

The problem we will run into: we would need to write three of these tests!
What if we add Apple-Sign-In later? And thinking further ahead, we can easily think of similar methods that need Method as an input, like sign-up. This would lead to a lot of duplicated test-code.

Parameterized tests are here to help:

@ParameterizedTest
@EnumSource(Method::class)
fun `tracks submitting`(method: Method) {
    
    tested.submit(method)

    verify(analytics).trackAnalyticsEvent(
        Event(
            "login",
            "method" to method.name
         )
    )
}

This is built-in and will give us each value of our enum.

What about Kotlin?

Many of us live in the Kotlin world, where we might often have cases where we prefer sealed classes to enums.

sealed class Method {
    object Facebook: Method() 
    object Google: Method()
    object Email: Method()
}

But then we can not use the EnumSource for our tests anymore! To keep our parametrized test, we would need to provide a custom factory method. This is fine here and there, but every single time? Wouldn’t it be nice to have our own SealedClassesSource?

Let’s build one!

Where to start

The first thing is to create a new JUnit5 annotation:

@Target(AnnotationTarget.TYPE, AnnotationTarget.FUNCTION)
@Retention(AnnotationRetention.RUNTIME)
@ArgumentsSource(SealedClassesArgumentsProvider::class)
annotation class SealedClassesSource(val value: KClass<*>)

Next, we need to implement a SealedClassesArgumentsProvider that provides us with the elements for every run. Thanks to the Kotlin reflection capabilities we can ask any class for its sealedSubClasses

Therefore we can start as simple as this:

class SealedClassesArgumentsProvider: ArgumentsProvider, AnnotationConsumer<SealedClassesSource> {

    private lateinit var source: SealedClassesSource

    override fun provideArguments(context: ExtensionContext?) = 
        source.value
          .sealedSubclasses
          .map { Arguments.of(it) }
          .stream()
    

    override fun accept(source: SealedClassesSource) {
        this.source = source
    }
}

All that is left is to update our test:

@ParameterizedTest
@SealedClassesSource(Method::class)
fun `tracks submitting`(method: KClass<Method>) {
... 
}

Not as nice (yet)

Unfortunately, we are now getting KClass<Method>, not an actual instance of Method. It would be better to have real instances as we had with enums:

@ParameterizedTest
@SealedClassesSource(Method::class)
fun `tracks submitting`(method: Method) {
... 
}

For simple sealed classes that use objector have empty constructors, we could handle this straight forward:

fun create(what: KClass<*>) = 
   what.objectInstance ?: what.constructors.first().create()

Our SealedClassesArgumentsProvider can then map the classes to instances:

class SealedClassesArgumentsProvider: ArgumentsProvider, AnnotationConsumer<SealedClassesSource> {

    private lateinit var source: SealedClassesSource
    override fun provideArguments(context: ExtensionContext?) = 
        source.value
          .sealedSubclasses
          .map { create(it) }
          .map{ Arguments.of(it) }
          .stream()
...
}

What about nesting?

Looking at our implementation we have another problem that prevents us from having a general solution for sealed classes. As I wrote in my previous post, sealed classes are very powerful and offer functionalities that enums do not possess, like expressing hierarchies:

sealed class Error {
   sealed class SignInError {
      sealed class EmailError : SignInError() {
         object Denied : EmailError()
         object Invalid : EmailError()
         object Existing : EmailError()
      }
      class IOError(val message: String): SignInError()
   } 
...
}

Our implementation was relying on the simple call to sealedClassesSource but this simply enumerates the direct children.
What we actually have here is a tree, that we would need to flatten with a bit of recursion:

fun <T: Any> Collection<KClass<out T>>.squashed() = 
   flatMap {
        listOf(it) + it.sealedSubclasses.squashed<T>()
    }

Applied to our SealedClassesArgumentsProvider:

override fun provideArguments(context: ExtensionContext?) = 
        source.value
          .sealedSubclasses
          .squashed()
          .map { create(it) }
          .map{ Arguments.of(it) }
          .stream()

And voila, we have a general-purpose source for parameterized tests with enums.

Sum up

As we saw, it’s very easy to build our provider to make the things we love. Kotlin and JUnit5 are a great fit, and with things like this, they work even better together.

Special thanks to Sam Brannan from JUnit5 team.

Code

You can find all the code here: https://github.com/dpreussler/junit5-kotlin

The version is based on the code shown but a bit more advanced.

You don’t need to pass in the type as it's determined via reflection from the test method (like latest versions of EnumSource can do):

@ParameterizedTest
@SealedClassesSource
fun `tracks submitting`(method: Method) {
... 
}

Also, it can instantiate sealed classes even if they have non-empty constructors, and provides a custom type factory that can be used for unsupported cases, or to simply mock those:

@Target(AnnotationTarget.TYPE, AnnotationTarget.FUNCTION)
@Retention(AnnotationRetention.RUNTIME)
@ArgumentsSource(SealedClassesArgumentsProvider::class)
annotation class SealedClassesSource(
  val factoryClass: KClass<out TypeFactory> = 
                                        DefaultTypeFactory::class
){
   interface TypeFactory {
     fun create(what: KClass<*>): Any
   }
}