Introduction to Custom Generators

Why use custom generators?

Until now, we have only used default generators in the properties that we have written. If we were to use these tools and techniques only, there is a chance that we could improve our tests by quite a bit.

In the previous chapter, we saw how to test an encoder and a decoder with symmetric properties. Say we suspected there was a bug whenever there are more than 255 elements in a map. 255 is a number that could hit the upper limits of what a byte can store.

To put it in other terms, the properties we’ve written so far have relied on fairly random generated data to act as inputs. The theory is that with enough random walks, we can eventually get into every nook and cranny of the code and find all the edge cases. But there’s no guarantee that this will ever happen. In fact, the opposite could be true. Because the runs are limited and edge cases are rare, random data could be ineffective at finding bugs.

For example, let’s imagine we have a system that has a subtle bug that only gets triggered when a key gets overwritten too many times in a database. If PropEr generates sequences of random keys for our properties, chances are that few of them will be duplicates. If the keys generated are almost never duplicated, our property is unlikely to exercise overwrites at all, and therefore unlikely to ever uncover the bug. We have to differentiate between a variety of data inputs and a variety of operations. This is where custom generators come into play, they can help debug applications in more detail.

The limitations of default generators

Default generators are essentially critical building blocks of our properties. They cover a large potential data space, which is useful in some types of tests. The properties that could be interesting to us may however require a very narrow focus on some limited edge cases. These generators can be useful to discover unexpected issues and problems, but less helpful in exploring tricky areas known to contain bugs in our programs.

Let’s say that we’re writing a list of key and value pairs to insert in a map, and we want to make sure that as long as we enter keys and values in there, all the keys will be found in the map. We may get a property that looks a bit like this: