Test everything

How we write consistent good covered tests

Test are important. Nobody argue that. You may have it in the project, or may not. But you know, that if you started to write them, they are important to you, since you adapt architecture, write new code (tests and utilities for them). Thus, you spend time on that. Time is the money, so tests must be important for everyone in the “team”.

I’ve seen a lot of tests and wrote couple of them :) And despite the fact that this wish and duty to write them is a good thing in long term development, some of them are tend to cover only small piece of logic without additional checks, which leads to leak of checking logic and your test become not healthy.

Let me demonstrate that:

interface One {
    val havePossibility: Boolean
    fun invoke()
}

interface Two {
    fun invoke()
}

class Sample(
        private val one: One,
        private val two: Two
) {
    fun execute() {
        if (one.havePossibility) {
            one.invoke()
        }
        two.invoke()
        two.invoke()
    }
}

class Test {
    private val one = mockk<One>(relaxUnitFun = true) {
        every { havePossibility } returns false
    }
    private val two = mockk<Two>(relaxUnitFun = true)

    private val testSample = Sample(one, two)

    @Test
    fun `when have no possibility invoke just two`() {
        testSample.execute()
        verify(exactly = 2) {
            two.invoke()
        }
    }

    @Test
    fun `when have possibility invoke one`() {
        every { one.havePossibility } returns true
        testSample.execute()
        verify {
            one.havePossibility
            one.invoke()
            two.invoke()
        }
    }
}

I wrote such tests, we wrote such tests. But now we decided to change, improve that.

Prerequisites

Note: Now we will talk only about tests with mocking interfaces and checking logic behind the test methods. We will not touch tests, in which we just asserting results

So, what’s wrong with that tests? Basically it seems fine, since you execute some methods and what to check, core things, that were executed, but your tests is not reliable. If tomorrow, someone will change code, the test will not fail. For example, those variations of execute will not break those tests:

//change the ordering
two.invoke()
if (one.havePossibility) {
    one.invoke()
}
two.invoke()
//move invoke out parentheses
if (one.havePossibility) {}
one.invoke()
two.invoke()
two.invoke()
//add new unit methods
two.invoke()
if (one.havePossibility) {
    one.invoke()
    one.invokeOther()
}
two.invoke()

You will be definitely right, if you will say, that we could use verifyOrder/verifySequence and other constructions for improving verification block. So we decided to do the same and add basic logic of writing tests, which we use everywhere for uniformity and consistency.

What we wanted to achieve, is:

• Track everything, what happens in the code base, relating to particular test. If something will change, test should fail
• Ability to write tests by special template to reduce the number of potential misleading tests
• Not increase the complexity of a test-around logic. Need to have some handy mechanism

And here we found an interesting solution with using kotlin’s property delegates. I’ve already wrote about couple of handy usages for this feature. You can check out this and this articles.

Use Kotlin Delegated Properties to register and verify all mocks

Our solution is applied to Mockk library, but you can use any other mocking library, you like. So the core solution for us was to use verifySequence, obviously, but:

• If you will try to use verifySequence without any mocks verified confirmation, you can face with that issue (we use first test from our example, it should fail, since we don’t check one.havePossibility property call):

@Test //this test passed
fun `when have no possibility invoke just two`() {
    testSample.execute()
    verifySequence {
        two.invoke()
        two.invoke()
    }
}

• If you will try to use only verifySequence, then you will quickly come to the need for use coVerifySequence, if your code contains usage of suspend functions.
• If you will use confirmVerified you will have to pass all mocks, that used in a test class in case if new functionality for your test case will appear. If you will use just couple of mocks, that test will pass with new functionality from other mocks. If you will include all mocks all the time, the test will fail. But it requires to write big amount of boiler plate code and to remember to add new mocks to every confirmVerified each time they appear

Enlightenment

So let’s wrap our main thoughts into single interface:

val NO_VERIFY: suspend MockKVerificationScope.() -> Unit = {}

interface MockkedTest {
    val registeredMocks: MutableList<Any>

    @After
    fun confirmAllMocksVerified() = confirmVerified(*registeredMocks.toTypedArray())

    fun coVerifyFlow(verifyBlock: suspend MockKVerificationScope.() -> Unit = NO_VERIFY) {
        if (verifyBlock !== NO_VERIFY) {
            coVerifySequence {
                repeat(times) { verifyBlock() }
            }
        }
        confirmVerified(*registeredMocks.toTypedArray())
    }
}

Now we have a single function for checking entire flow without missing anything, since we confirm all registered mocks in the end of each calling of coVerifyFlow. Also we use confirmAllMocksVerified just for those, who will not use coVerifyFlow(for example, they forgot to do that). :)

How to register mocks?

Now the question is, how to add mocks to registeredMocks list. I’m lazy and I don’t want to remember about that and do it every time, I add new mock. There is a solution for you, my friend. Kotlin delegated properties. Let’s add those methods to our interface

fun <T : Any> mockked(
        mockkClz: KClass<T>,
        name: String? = null,
        relaxed: Boolean = false,
        vararg moreInterfaces: KClass<*>,
        relaxUnitFun: Boolean = false,
        block: T.() -> Unit = {}
) = MockkDelegate(mockkClz, name, relaxed, *moreInterfaces,
        relaxUnitFun = relaxUnitFun,
        block = block,
        registeredMocks = registeredMocks
)

class MockkDelegate<T : Any>(
        mockkClz: KClass<T>,
        name: String? = null,
        relaxed: Boolean = false,
        vararg moreInterfaces: KClass<*>,
        relaxUnitFun: Boolean = false,
        block: T.() -> Unit = {},
        registeredMocks: MutableList<Any>
) : ReadOnlyProperty<Any, T> {
    private val field: T = MockK.useImpl {
        MockKGateway.implementation().mockFactory.mockk(
                mockkClz,
                name,
                relaxed,
                moreInterfaces,
                relaxUnitFun
        ).apply(block)
    }

    init {
        registeredMocks.add(field)
    }

    override fun getValue(thisRef: Any, property: KProperty<*>): T = field
}

So the first method just copies declaration of MockK.mockk method, for easy migrate to the new system (just replace = mockk to by moccked in the test code) and save all opportunities provided by Mockk.

The second is a delegation class that just calls, what is called inside Mock.mockk method and also add mock to registeredMocks list. We can not just invoke mockk method, since we can not pass generic param to reified generif param of mockk method.

Additionaly we can have a top level inline function (since you can not use them in interfaces):

inline fun <reified T : Any> MockkedTest.mockked(
        name: String? = null,
        relaxed: Boolean = false,
        vararg moreInterfaces: KClass<*>,
        relaxUnitFun: Boolean = false,
        noinline block: T.() -> Unit = {}
) = mockked(T::class, name, relaxed, *moreInterfaces, relaxUnitFun = relaxUnitFun, block = block)

Now let’s see, how our tests were changed:

class Test : MockkedTest {
    override val registeredMocks: MutableList<Any> = mutableListOf()

    private val one by mockked<One>(relaxUnitFun = true) {
        every { havePossibility } returns false
    }
    private val two by mockked<Two>(relaxUnitFun = true)

    private val testSample = Sample(one, two)

    @Test
    fun `when have no possibility invoke just two`() {
        testSample.execute()
        coVerifyFlow {
            one.havePossibility
            two.invoke()
            two.invoke()
        }
    }

    @Test
    fun `when have possibility invoke one`() {
        every { one.havePossibility } returns true
        testSample.execute()
        coVerifyFlow {
            one.havePossibility
            one.invoke()
            two.invoke()
            two.invoke()
        }
    }
}

Structure all test with providing a template method to call

Now everything is almost great, but we want just to segregate definition of conditions (declaration of every and some preparations) from action we going to test and from verification sequence.

To do that let’s present test function:

inline fun MockkedTest.test(
        action: () -> Unit,
        noinline verify: suspend MockKVerificationScope.() -> Unit = NO_VERIFY
) = test({}, action, verify)

inline fun MockkedTest.test(
        conditions: () -> Unit,
        action: () -> Unit,
        noinline verify: suspend MockKVerificationScope.() -> Unit = NO_VERIFY
) {
    conditions()
    resetRegisteredMocks()
    action()
    coVerifyFlow(1, verify)
}

So the algorithm is simple. We call all preparations through conditions invocation. That include calling every {} and some methods from tests class, that should not be verified.

To prevent verifying for preparation methods of test class, we use resetRegisteredMocks function, that places inside MockkedTest:

fun resetRegisteredMocks() {
    when (registeredMocks.size) {
        0 -> return
        1 -> clearMocks(registeredMocks.first(), answers = false)
        else -> clearMocks(registeredMocks.first(), *registeredMocks.toTypedArray(1), answers = false)
    }
}

Then we just call action and coVerifyFlow with provided verify argument

That’s basically all. (additionally you can track a return value, that action would return and provide a lambda argument for asserting that value).

Let’s see, how our tests were changed:

class Test : MockkedTest {
    override val registeredMocks: MutableList<Any> = mutableListOf()

    private val one by mockked<One>(relaxUnitFun = true) {
        every { havePossibility } returns false
    }
    private val two by mockked<Two>(relaxUnitFun = true)

    private val testSample = Sample(one, two)

    @Test
    fun `when have no possibility invoke just two`() = test(
            action = testSample::execute,
            verify = {
                one.havePossibility
                two.invoke()
                two.invoke()
            }
    )

    @Test
    fun `when have possibility invoke one`() = test(
            conditions = { every { one.havePossibility } returns true },
            action = testSample::execute,
            verify = {
                one.havePossibility
                one.invoke()
                two.invoke()
                two.invoke()
            }
    )
}

Afterwards

We used that little improvement in our tests and it helps us to track everything, what is happens inside the test methods. I think there are more elegant ways in this world. Possibly Spek gives more powerful tool for structuring you tests. But I think you agree, that those small changes, that are not really requires to rewrite anything, but make a huge jump in tests improvement, are worthy of consideration :)

If you liked the article, pls clap on it and follow me, there are much more stuff inside! :)