Complex test data prototyping with Shapeless and Monocle

4 min readNov 24, 2021

Introduction

This article describes how Shapeless and Monocle libraries, along with GoF patterns and type classes derivations can help generate complex data for unit tests in an easy way. I'd like to ask for some patience in advance: there is going to be plenty of code because the problem I'm going to present is pretty visible after certain code-base size and domain model complexity.

System under the test

For the sake of the article example, let’s consider Spark application for personal expenses reports calculations. The main goal of such an application is to provide actionable recommendations for the user on how he/she can save some money based on expenses history. For instance, check if a user spends too much money on lunch in restaurants during the workweek.

Let’s begin with domain model definition:

Despite the fact, that for now, our app will produce only a single financial recommendation, the domain model was designed keeping in mind system flexibility and covering other use cases, such as shopping online or abroad. So let’s consider the recommendation we are going to generate:

The application calculates the total amount of money spent on restaurants in the workweek during the current and previous month. If those values differ by more than 75%, we can suggest optimizing expenses in this category. For simplicity, let’s say we generate this report once per month.

Writing test

Plenty of business logic has been implemented in the previous section. As a next step, let’s try to write a unit test for our job. In this unit test, we will check the simplest scenario — a single user spend twice more money on restaurants in December 2021 compared to November 2021. A crucial part of this test is data. To test other parts of logic, among restaurants expenses, let’s add some grocery expenses and weekend expenses.

For unit test implementation Munit library was used.

And here we can spot the first problem: we have to write a lot of similar and straightforward code to instantiate data for our tests, such as Expense objects. Implementation and maintenance of such a codebase are tedious and problematic because each domain model change leads to many small fixes in the tests.

Along with this, we need to fill plenty of fields with values we really don’t care about in scope of tests, such as card field in Expense class. First thought would be to use default values, but this is not recommended way to go, because test and production code should be decoupled as much as possible. E.g. we can make card or address defaulting to some dummy values, but this would lead to unexpected mistakes in production code.

Test refactoring first iteration

I was inspired by GoF Prototype pattern to solve problem with data copy-pasting. Essentially, we can instantiate prototype of complex object (Null object pattern) and then make it copy with more specific values we care about. Luckily, this is pretty easy to do with Scala’s copy method for case classes.

After refactoring we get next result:

Looks better now, but still is not good enough. We can spot two other problems in this code:

Prototype instantiation remains tedious, while this is writing simple code, which can be generated for us.
Shapeless type classes derivation capabilities can help to solve this task.
Using copy is not composable: if we want to apply the same transformation but for different objects, we would need to repeat the same copy invocation.
Answer to this problem - optics, Monocle in particular.

Prototype instantiation

In order to instantiate any case class prototype, we need to instantiate it with some “default” values, such "" for string, 0 for integers, None for options, and so on. shapeless is a perfect tool to solve this. Generating such prototype instance sounds like similar deriving type class, which shapeless perfectly does. In our case, Prototype type class should return the prototype value.

Implementation looks next:

Let’s create also package object for convenient import:

package object prototype extends PrototypeSyntax

Let’s see small demo:

prototype[Amount] // creates Amount(0,0,) object instance

Perfect, first part is ready.

Prototype modifications

After we have prototype generation in place, we want to apply various composable (!) transformations to it. monocle ‘s Lens is another great tool that can help us with this. We can split different fields transformation to an individual lens to compose at the end. For instance:

Test final refactoring

Having all the tools and approaches we considered before, let’s put all the parts together in final test result:

Looks much cleaner now, isn’t it?

Conclusion

In conclusion, I want to share the pros and cons of this approach.

Pros:

Easy to create complex structures with only fields and values need for test purposes.
No copy-pasting and tedious refactoring for model changes.
Clean and readable code to show the intent of each data entry.

Cons:

Additional small self-written framework to support;

A complete example can be found on Github