Intro

A few months ago, I ran into a rather unexpected issue in a Spring-based Scala project:

Exception in thread "main" java.util.NoSuchElementException: head of empty list
  at scala.collection.immutable.Nil$.head(List.scala:663)
  at scala.collection.immutable.Nil$.head(List.scala:662)
  at scala.collection.immutable.List.map(List.scala:247)

Yes, I know — Spring and Scala is not the most common combination. That’s a story for another post. The problem appeared in a production environment, so the priority was clear: find the root cause as quickly as possible and ship a fix.

At first glance, the error looked like a classic case of calling head on an empty list (Nil). However, the stack trace suggested something far stranger: the exception occurred inside a list transformation, during the execution of map. Two scenarios seemed plausible:

  1. head was being invoked on an empty list somewhere inside the transformation logic,
  2. we had encountered a bug in the Scala standard library.

The full stack trace was longer and passed through several abstraction layers, so I prepared a minimal test case to reproduce the issue. Unfortunately — or perhaps fortunately — the test behaved correctly. The logic held up. This increased the likelihood of the second scenario, unlikely as it initially seemed.

At that point, we had no idea that the real cause was hiding much deeper — not in Scala itself, but in the JVM. Somewhere beneath all the declarative elegance was something capable of breaking even a fundamental assumption: that Nil is a singleton.

The Real Culprit

The issue was eventually traced back to the MongoDB support module provided by Spring. More specifically, to the part of Spring Data MongoDB responsible for deriving codecs using reflection.

This component attempts to inspect domain classes at runtime, discover their fields and constructors, and automatically build codecs for reading and writing documents. For plain Java classes, this works seamlessly — most Java types expose predictable constructors and follow patterns that fit well with reflective instantiation.

For Java’s own standard collections, Spring Data MongoDB contains special-case logic that knows how to instantiate them safely during codec generation. However, this logic does not extend to Scala collections or Scala’s singleton types.

In practice, this meant that if we wanted to continue using Spring’s default codec derivation, our database models would have to rely on Java arrays or Java collection types instead of Scala’s collection hierarchy.

To understand exactly why Spring’s reflection logic proved so destructive for Scala lists, we first need to understand how Scala compiles singleton objects to bytecode and what makes Nil so special.

Scala Objects: A Promise of Singularity

Scala, unlike Java, does not use the static keyword. That doesn’t mean it deprives us of static fields or methods — it simply takes a different approach. Consider the following example:

1
2
3
4
5
6
7
8

class Recipe(val steps: List[Step])

object Recipe {
  val empty: Recipe = new Recipe(List.empty)
  
  def merge(first: Recipe, second: Recipe): Recipe =
    new Recipe(first.steps ++ second.steps)
}

Here, object Recipe is the companion object for class Recipe. The empty value and the merge method are not invoked on any particular instance of Recipe. Let’s look at the equivalent Java code produced after decompilation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

public class Recipe {
    private final List<Step> steps;

    public static Recipe empty() {
        return Recipe$.MODULE$.empty();
    }

    public static Recipe merge(Recipe var0, Recipe var1) {
        return Recipe$.MODULE$.merge(var0, var1);
    }

    public Recipe(final List<Step> steps) {
        this.steps = steps;
    }

    public List<Step> steps() {
        return this.steps;
    }
}

public final class Recipe$ implements Serializable {
    private static final Recipe empty;
    public static final Recipe$ MODULE$ = new Recipe$();

    private Recipe$() {
    }

    static {
        // Some decompilers produce slightly odd output
        empty = new Recipe(.MODULE$.List().empty());
    }

    private Object writeReplace() {
        return new ModuleSerializationProxy(Recipe$.class);
    }

    public Recipe empty() {
        return empty;
    }

    public Recipe merge(final Recipe first, final Recipe second) {
        return new Recipe((List)first.steps().$plus$plus(second.steps()));
    }
}

Our Scala object has been translated into a separate class — Recipe$. It cannot be extended and has a single private constructor, which means no additional instances can be created. The singleton instance is created eagerly and stored in the static field MODULE$.

You can also see that our fields and methods appear twice: in the companion object class (Recipe$), and as static members on the Recipe class. This is how Scala emulates Java-style static access while still keeping the object model consistent.

This mechanism forms the foundation of singleton values in Scala’s standard library: Nil, None, and many others. From the JVM’s perspective, all of them are simply final classes with private constructors and one eagerly created instance.

Java Reflection: The Art of Ignoring Boundaries

Consider the problem of data serialization. When building a REST service, sooner or later we need a way to convert incoming requests into objects, and objects back into some wire format. Of course, we could manually implement a codec for every class. But this quickly becomes tedious, error-prone, and difficult to maintain as the project grows.

Is there a way to automate it?

To do so, we would need the ability to inspect classes at runtime. To see what fields they define, what constructors they expose, what annotations are attached, and how their structure maps to something like JSON. The JVM provides exactly such a mechanism — reflection.

Every object in the JVM carries a reference to a Class instance representing its type.
This metadata gives access to:

  • the fully qualified name,
  • all fields,
  • all methods,
  • all constructors,
  • annotations,
  • implemented interfaces and superclasses.

But the real power of reflection is that it doesn’t stop at introspection. If we obtain a reflective handle to a field, we can read its value — or even modify it. With a method handle, we can invoke the method. With a constructor reference, we can instantiate new objects, even if the constructor is private.

Using these primitives, we can build simple codecs for primitive types and collections, and then derive codecs for complex classes by walking over their fields. This is, in essence, how serialization libraries such as Jackson work under the hood.

And reflection goes further still.

It does not restrict itself to public members. It happily works with private fields, protected methods, and constructors that were never meant to be called directly. Once you obtain a reflective reference, access modifiers become… negotiable.

This is where the boundaries begin to blur.

Later in this article, I will show how the same mechanisms can be used to break one of Scala’s fundamental assumptions — and create an additional instance of Nil, a value that is supposed to be a singleton.

When Scala Meets Java Libraries

Is it possible to build a Scala application without relying on any Java libraries?

Of course. The Scala ecosystem is mature, and many problems can be solved using purely Scala-based tools.

But that doesn’t mean it is always sensible. Rewriting a database driver from scratch in Scala makes little practical sense when a high-quality Java implementation already exists. In cases like this, the typical solution is to provide a thin Scala wrapper around an existing Java API. Examples include:

  1. mongo-scala-driver
  2. neo4j-scala
  3. lucene4s

Besides these wrappers, there are also many Java libraries that can be used directly from Scala. Doing so requires the programmer to interact with Java’s semantics and conventions. Some of the most common friction points include:

  1. Exception handling – Scala often represents errors as values (e.g., Either, Try) instead of try / catch.
  2. Null handling – Java APIs commonly use null; Scala code typically prefers Option.None.
  3. Collections – the Scala collection hierarchy differs significantly from Java’s.
  4. Asynchrony – Scala and Java provide separate Future implementations.
  5. Reflection – Java reflection does not understand Scala’s assumptions or guarantees.

These are only high-level considerations. Designing a clean, idiomatic Scala API on top of Java requires a deeper understanding of both ecosystems.

Scala’s standard library helps with some of the gaps through the scala.jdk package, which provides collection converters, asynchronous bridging utilities, and stream adapters. Moving between the two worlds becomes mostly a matter of calling the appropriate conversion methods.

Reflection, however, is an entirely different story. There is no automatic way — and no mechanism in the JVM — to extend Java’s reflective logic so that it respects Scala’s abstractions. Reflection operates purely on JVM constructs: classes, fields, constructors, and methods. It has no notion of companion objects, value classes, case classes, sealed hierarchies, or singleton objects.

And that logic is entirely in the hands of whoever wrote the reflective code. It is easy to forget that Scala-specific rules simply do not exist at runtime.

In an ideal world, Java libraries would natively support every JVM language out of the box. Reality is less accommodating. Most library authors do not have the time, incentive, or even awareness necessary to account for Scala’s semantics.

It helps to look at this from a runtime perspective. The JVM does not know that a given class represents a Scala singleton object. It does not know that this object must never be instantiated more than once. It only sees a final class with a private constructor.

So can we say that Scala’s guarantees are illusory? A short answer: at compile time — no; at runtime — yes.

How to Break Scala’s Guarantees

Let’s now look at how Scala’s guarantees can be broken using Java reflection. The following examples are purely educational — you should not use them in production.

As mentioned earlier, access modifiers are not much of an obstacle for reflection. So let’s try to obtain the constructor of Nil and invoke it manually:

1
2
3
4

val cls = Nil.getClass
val constructor = cls.getDeclaredConstructors.head
constructor.setAccessible(true)
val instance: Nil.type = constructor.newInstance().asInstanceOf[Nil.type]

The key detail here is the use of getDeclaredConstructors instead of getConstructors.

  • getConstructors returns only public constructors.
  • getDeclaredConstructors returns all constructors, including private ones.

Since Nil’s constructor is private, we must also call setAccessible to bypass access checks.

At this point, we are holding a brand-new instance of Nil. Not the singleton — an entirely separate object. Calling map on this instance results in the familiar exception:

Exception in thread "main" java.util.NoSuchElementException: head of empty list
  at scala.collection.immutable.Nil$.head(List.scala:663)
  at scala.collection.immutable.Nil$.head(List.scala:662)
  at scala.collection.immutable.List.map(List.scala:247)

Why does this happen?

Because the implementation of most List methods relies on the identity check this eq Nil. For this condition to hold, both sides must reference the same instance. Scala assumes that Nil is a true singleton — the only empty list in the entire runtime.

By creating a second instance, we have violated this assumption. As a result, methods that rely on eq to distinguish Nil from non-empty lists (::) begin to misbehave. In other words: reflection didn’t just bypass a private constructor — it broke a fundamental invariant of Scala’s collection model.

Conclusion

Scala builds powerful abstractions on top of the JVM. Singleton objects, sealed hierarchies, exhaustive pattern matching — all of these features create a sense of structure and safety that Java itself does not provide.

But underneath those abstractions lies the same old JVM. A platform that knows nothing about Scala’s guarantees, and happily allows us to bypass them when reflection gets involved.

By forcing a second instance of Nil into existence, we didn’t just trigger a weird corner case. We demonstrated that some of Scala’s most fundamental assumptions — such as the idea that Nil is always a singleton — exist only as long as the JVM agrees to play along. Most of the time it does. Until it doesn’t.

In day-to-day development, this is rarely a practical concern. Libraries do not typically instantiate private constructors of standard library objects, and well-behaved applications stay far away from this kind of reflective code.

But the boundary is there — thin, invisible, and surprisingly easy to cross. It’s a useful reminder that:

  • Scala’s invariants are enforced at compile time,
  • but the JVM ultimately controls runtime behavior.

Understanding this difference makes us better engineers. It helps explain mysterious bugs, informs our design choices, and teaches us where the sharp edges of the platform truly are. And sometimes, as in the case of Nil, it reveals just how fragile even the strongest abstractions can be.

Further Reading

In case you enjoyed this article, here are a few more you might find useful: