Windows and Triggers

Learn how windows and triggers are used to group events in a flow and take data out of a flow, respectively.

We'll cover the following

Introduction

Operations like Flow.reduce/3, group_by/2, and take_sort/3 require all events to complete before they produce a result. After all, we cannot group or sort a list without all the data in the list being available to us. However, what if the data source keeps producing events for a long time or keeps producing events forever? This means we’ll have to wait for a result for either a very long time or indefinitely. To avoid this, Flow introduces windows and triggers. This section will discuss how windows and triggers can help us get progress updates with long-running flows.

Windows

Windows group all events are going through Flow. We can think of windows as folders for events. They allow us to organize data conveniently. So far, we’ve used Flow.Window.global/0, which creates a global window for us by default. Global windows group all events together and close only when all events have been received. These are the other types of windows:

  • The Flow.Window.fixed/3 window groups events by a timestamp value on the event, using a specified time duration.

  • The Flow.Window.periodic/2 window is like fixed/3 but groups events by processing time.

  • The Flow.Window.count/1 window groups events when they reach the given count.

Windows also influence how reducer operations work. At the beginning of a window, the accumulator is reset to collect data from scratch. Remember that Flow.reduce/3 uses a function to create the initial accumulator value. This function is called at the beginning of each window.

Triggers

We can also specify a trigger for each window. Triggers work like checkpoints, and all trigger functions take a window as their first argument. When a trigger event occurs, we have the opportunity to take data out of the flow by specifying what events to emit. We can even change the accumulator value on the fly. These are available triggers:

  • Flow.Window.trigger_every/2 triggers when we reach the given count of events.

  • Flow.Window.trigger_periodically/3 triggers at the specified time duration.

  • Flow.Window.trigger/3 is used to implement custom triggers.

By default, every flow has an internal trigger for an event called :done, which occurs when no data is left to process.

Use different combinations of windows and triggers

Depending on the type of window and trigger we choose, we can get different insights into the flow. In a moment, we’ll see how to use trigger_every/2 with the default global window. We won’t cover all possible combinations in this course, but the good news is that they are well documented online, and we can use them. After completing this chapter, we encourage you to check out the official documentation for the Flow.Window module.

We’ve covered the basics of windows and triggers, but one important thing is left—detecting when a trigger occurs. Luckily, there is a function that makes this very easy.

Catch trigger events with on_trigger/2

Get hands-on with 1200+ tech skills courses.