Paying homage to the Compall-Michael pattern, and anticipating Scala 3 Opaque Types

In a nutshell, we want to have "compile-time wrapper types" that provide type-safety at compile-time but without runtime overhead (e.g. no extra memory allocations). This is more easily illustrated with an example (even a toy one), which I will borrow from the Opaque Types section of the Moving Forward from Scala 2 to Scala 3 course that a team of us at Lunatech, led by Eric Loots, prepared and released in 2020.

I will start by illustrating the example using well-known features of Scala 2 - Plain type aliases, Case classes, and Value classes - before moving on to show the "Compall-Michael" representation, and then finally finishing with a solution using Scala 3 Opaque Types.

(Note that I do not cover all the ways that we can solve this in Scala — e.g. using tagged types or even other approaches that make use of excellent libraries such as scala-newtype. Such approaches using third-party libraries are well worth considering in your own real-world use-cases.)

The example talks about Rockets, Boosters, Kilometers, and Miles and was inspired by the famous Mars Probe story of faulty conversion between metric and imperial units. (I feel the need to stress that if you are building an application and you need to work with dimensions and units of measure in a type-safe way then you would almost certainly want to use a proven library like Squants. The snippets below simply constitute toy examples to illustrate and compare different approaches.)

A typical Scala 2 approach is to use plain type aliases for readability. In the following code snippet, instead of using Double, we introduce two aliases Kilometers and Miles which help us to document when we are working in kilometres and when we are working in miles — e.g. we can document that the Booster class works with miles as a unit of measure.

However, this does not provide any type-safety because the aliased type is transparent. Consider the following code snippet where we use the other defined alias Kilometers to help us to document that the Rocket class works with kilometres as a unit. The fact that Miles is transparently a Double means that we can inadvertently add kilometres and miles together and the code will compile just fine - meaning that a bug has just slipped through.

Similarly, as these are just values of type Double, with just these plain type aliases we are unable to enforce certain constraints such as "distance cannot be negative". Ideally we want to make this kind of illegal state unrepresentable.

The simplest approach to achieve type-safety and catch the bug of mixing up miles and kilometres at compile-time would be to create distinct new "wrapper" types. In the following code snippet, we introduce two case classes Kilometers and Miles that each "wrap" their real underlying Double value. And because these are distinct types we are not able to just simply operate on miles when kilometres is the expected unit of measure. The case classes provide a way to create values of Kilometers and Miles and we provide two helper functions Units.add and Units.toKilometers for adding two values of Kilometers and for converting from Miles to Kilometers.

Furthermore, if we wanted to enforce the constraints about distance only being positive, we could take the "smart constructor" approach to restrict how we obtain values of type Kilometers and Miles. There are quite a number of ways to achieve this in Scala. For a good explanation of the various ways to implement smart constructors I invite you to check out this recent blog post. As I am using Scala 2.13.5 to compile these examples, I will take advantage of the newer cleaner approach of using a private[foo] access modifier that is possible with the -Xsource:3 compiler option. (If you are on a version of Scala pre-2.13.2, then I would personally recommend the approach of using sealed abstract case class, which is also explained in the linked blog post).

In the following code snippet, the private[Units] modifier means we can only access the Kilometers() and Miles() constructors from within the scope of object Units — externally the only way to obtain Kilometers and Miles values is through the kilometers and miles "smart constructors" that perform some validation.

So we have achieved what we wanted — which is that we prevent the bug at compile-time — but at the cost of some runtime overhead because we now have to allocate the wrapper Kilometers and Miles objects in memory. It is safe to bet that in practice for most applications this extra overhead is not critical and personally I reach for this simple approach most of the times. However, it is not unheard of to have some performance-critical use-cases where you want to avoid the overhead of unnecessarily allocating and garbage-collecting objects.

Extending the wrappers with AnyVal promises to eliminate the overhead of using a the case-class wrapper presented in the previous section. Classes that extend AnyVal, in addition to meeting a number of other criteria as explained here on this Scala Lang page, are known as value classes. The compiler can still prevent the bug of incorrectly mixing up values of Miles and Kilometers, but without the need to allocate wrapper objects.

However, in practice allocations can still happen in a number of situations. For example, imagine that we decide to allow the toKilometers method to accept values of a super-type Distance (of which Kilometers and Miles would be two sub-types), we could have code like the following.

This code is perfectly fine, but if you had chosen the AnyVal route to avoid allocations, then you would be disappointed to discover that now with the supertype-subtype relationship you will once again have allocations when you pass a value of Kilometers or Miles to the function toKilometers.

The various limitations of Value-classes are extensively discussed on the Scala Lang page on Value Classes and on the SIP-15 page there is recognition that improvements can be made for certain use-cases (e.g. numerical computing).

As a quick aside, one interesting limitation is that value classes cannot be nested. Although not strictly necessary for our particular use-case, imagine that we wanted to add some convenient syntax for invoking our helper functions add and toKilometers — e.g. using + infix operator instead of add and being able to invoke .toKm on a Miles value instead of toKilometers. One typical usage of Value-classes is in combination with implicit classes to provide allocation-free extension methods. However, if we attempt to define extension methods in this way, as in the following code snippet, then the code does not compile:

This gives the following compilation error:

So in this case, to get around the limitation of nested value-classes, we have to define the extension methods using just implicit class without the extends AnyVal (accepting that this will allocate new instances of these implicit classes when using the extension methods):

In contrast to the preceding well-known and widely-used techniques seen thus far in the post, the approach that I am calling here the "Compall-Michael" pattern is almost certainly less well known (even though in some respects it already has a very long heritage in Scala). This technique makes use of Scala’s ability to emulate the ML module system, which I first heard Martin Odersky discuss in his flatMap Oslo talk in 2014 (click here for the slides) and which is very well explained in this blog post from that same year entitled "Scala’s Modular Roots" by Dan James.

For an in-depth explanation of how the emulation of ML modules in Scala can be achieved, I invite you to read the blog post linked above (and if you want a crash course on ML modules then this blog post is a good quick read). Here I will just say that the main aspects that interest us are emulating ML signatures using sealed trait s and emulating ML structures by implementing the signature with new { …​ } and assigning it to a val. (In our use case we will not need to emulate ML functors which is the third important aspect of the ML module system).

The signature can be thought of as the public API of our module without any implementation. This is an API that not only consists of abstract functions (def declarations) but also unassigned values (val declarations) and abstract types (type declarations). In the code snippet below we have our signature UnitsModule that we define as a sealed trait (it could equally by a sealed abstract class). Inside the signature, we reproduce the same public API that we introduced already in the "Case class wrappers" section. Specifically, we have: - abstract type members Kilometers and Miles - abstract val declarations ZeroKm and ZeroMi that will represent "zero" values of our two types of unit - abstract def smart constructor declarations kilometers and miles that will allow us to create instances of our two types of unit - abstract def declarations that allow us to do useful things with our two types of unit, in this case a method add for adding two Kilometers values and a method toKilometers for converting a Miles value to a Kilometers value - and finally, if we want to, we can even define our extension methods from before, defined in terms of the add and toKilometers methods. (Note, however, that once again we cannot make use of "allocation-free extension methods" because of the limitation of only being able to define them at the top-level or enclosed in a statically accessible object.)

The structure can be thought of as the implementation of the signature. In the following code snippet, we instantiate an anonymous class that implements the UnitsModule trait and assign the instance to val Units. In the body of the anonymous class we provide concrete definitions of the two types Kilometers and Miles, defining them both as Double. Similarly, we provide implementations of all the val and def declarations, and in these implementations we can treat values of type Kilometers and Miles as Double values (meaning e.g. that we can directly perform arithmetic operations such as + and * on these values.

However, even though on the inside of the body of new UnitsModule { …​ } we know that both Kilometers and Miles are represented as Double values, from the outside the types Units.Kilometers and Units.Miles are completely different and cannot be used interchangeably. So the compiler can again prevent the bug of adding a value of type Miles to a value of type Kilometers. But in this case, there is absolutely no wrapping involved and the runtime representation of Miles and Kilometers values is indeed Double.

The one big gotcha of this approach is that the ascription val Units: UnitModule is crucial. If you omit that then you break the abstraction and allow the outside world to see that Kilometers and Miles are just aliases for Double and you are back to square one. Having the type annotation :UnitModule means that the outside world can only treat Kilometers and Miles as existential types (meaning, in a nutshell, that the outside world knows only that types of these two names exist, but nothing else). It is this subtle move that gives us the needed opacity around the real representation of Kilometers and Miles.

So consider the following code snippet where we have forgotten the annotation and we have simply val Units = new UnitsModule (or we could have the same effect with object Units extends UnitsModule), now were are back to the situation where Kilometers and Miles are transparently Double and we are back to being able to make the original mistake of adding miles to kilometres.

I highly recommend that you watch the talk and/or read the slides as it goes much further than the basic usage that I have described here. Particularly fascinating is the example on infinitely recursive types around minute 40 of the talk. I should also point out that the main motivation put forward in Stephen Compall’s talk is "improving abstraction" rather than any concerns about memory allocations or performance-critical use-cases.

Opaque types were originally proposed in SIP-35 (cf. the Motivation section). According to the Dotty docs, they aim to "provide type abstraction without any overhead”. Scala 3 introduces the opaque keyword that can be added in front of a plain type alias.

However, these type aliases by themselves are not very useful. That is because, outside of the scope of Units we only know the type names Kilometers and Miles but we cannot do anything useful. At a minimum we need to provide a way to introduce values of our opaque types and a public API for working with values of our opaque types. So here again we reproduce the public API that we first introduced in the "Case class wrappers" section, as well as the extension methods that provide convenient syntax for our Units.add and Units.toKilometers helper functions. However, for this we make use of the new Extension methods feature of Scala 3. The combination of Opaque Types and Extension methods go well together in Scala 3 and make for a cleaner final solution.

And now, even though inside the body of Units we can treat both Kilometers and Miles as Double values, on the outside, we cannot use these types interchangeably and we have to do the necessary conversion. Note that the extension methods make it a cleaner to add two Kilometers values and to convert from Miles to Kilometers.

Hopefully this blog post has made you as keen as I am to really get going with Scala 3 to make use of the new expressive capabilities, Opaque Types being just one of many. If you want to start exploring this very promising new evolution of Scala then the best place to start is the official Scala Lang Scala 3 page. If you want self-paced, hands-on practical exercises to get acquainted with some of these new expressive capabilities, I invite you to run through Lunatech’s Moving Forward from Scala 2 to Scala 3 course. However, if you are still likely to be using Scala 2 for the near to medium term, I hope this post was a gentle introduction to the interesting "Compall-Michael" technique for declaring "compile-time wrapper types" that makes use of existing language features.