The reduce Function

Introduction

In addition to map and filter, the reduce function in the Enum module is another frequently used tool when working with data. It is also the backbone of other functions in this module, including group_by, uniq_by, and take_sort. This section shows how to use Flow.reduce/3 to count the number of working airports in each country present in the dataset, in parallel. Then, we’ll move on to other convenience functions to group and sort data.

How reduce works

Before we begin, it’s useful to understand how reduce works in Flow. In reduce, we have something called an accumulator that holds the latest state. The accumulator could be of any data type. Each element in the list we want to process is given to a reducer function, followed by the accumulator value, often shortened to acc for brevity. Once all items in the list have been transformed using the reducer function, the last accumulator value results from the reduce operation. Here is the blueprint of Enum.reduce/2:

Enum.reduce(enumerable, fn item, acc -> 
  # Inside the reducer function we can 
  # use the `item` event, and
  # return the `acc` value.
  acc 
end)

Problems with Enum.reduce

Like with Flow.map/2, each reducer function runs concurrently by default. However, we’ll see unexpected results if we directly replace Enum.reduce/2 with Flow.reduce/3 in our code. Since each reducer function runs in its process, it does not know other accumulator values. The final outcome will be the combined list of all results, which contains duplicates. Let’s demonstrate the issue by adding Flow.reduce/3 to open_airports/0, just before Enum.to_list/1: