Concurrent Data Processing in Elixir

INTERACTIVE COURSE

Concurrent Data Processing in Elixir

Intermediate

78 Lessons

11h 10min

Certificate of Completion

AI Explanations

AI Explanations

Concurrent Data Processing in Elixir

49 Playgrounds

7 Quizzes

36 Illustrations

Takeaway Skills

Understand how concurrency works in Elixir

Learn about Elixir supervisors and GenServers

Learn to use data processing pipelines

Learn to use the Flow module in Elixir

Learn to set up data ingestion pipelines

Course Overview

Most engineers don’t think that concurrency is distinct from programming. Data processing includes transforming information in some way, like reporting, data aggregation, or analytics. On the other hand, concurrency involves significantly improving performance. This course is designed to help you navigate the world of concurrency tools available in the Elixir ecosystem. You’ll explore popular modules and libraries like GenServer, GenStage, Flow, and Broadway. You’ll be able to ingest data, set up pipelines...Show More

Course Content

1

Getting Started

What Does This Course Cover?

Course Prerequisites and Intended Audience

2

Easy Concurrency with the Task Module

Introduction to Concurrency

An Introduction to the Task Module

Create the Playground Start Tasks and Retrieve Results Synchronous vs. Asynchronous Programs Retrieve a Task’s Results Use the await and yield Functions to Retrieve Results Stop a Task and Async Task Manage a Series of Tasks Parameters of the async_stream Function Link Processes Meet the Supervisor Use Task.Supervisor Understand the “Let It Crash” Philosophy Wrapping Up Quiz: Easy Concurrency with the Task Module

3

Long-Running Processes Using GenServer

Long-running Processes and Starting GenServer GenServer Callbacks: init GenServer Callbacks: handle_continue and handle_call Genserver Callbacks: handle_cast GenServer Callbacks: handle_info GenServer Callbacks: terminate Make Improvements to Our GenServer The blueprint of our job-processing system How to Use Anonymous Functions Introduce DynamicSupervisor Process Restart Values and Frequency Implement a Supervisor Supervisor Strategies Registry Processes Register New Processes and Query the Registry Limit Concurrency and Inspect the Supervisors Examine Running Processes and Stop the Supervisors Wrapping Up Quiz: Long-running Processes Using GenServer

4

Data Processing Pipelines with GenStage

Back-Pressure Introduce GenStage The Producer and Consumer Build the Data Processing Pipeline Understand Consumer Demand Revisiting the Producer Add More Consumers and Buffer Events Add Concurrency With ConsumerSupervisor Create Multi-stage Data Pipelines Scale Up a Stage with Extra Processes Choose the Right Dispatcher Wrapping Up Quiz: Data Processing Pipelines with GenStage

5

Process Collections with Flow

Introduction to the Flow Library Read Files Improve Performance with Streams Run Streams Concurrently and Understand Flows The reduce Function Route Events with Partitions Group and Sort Windows and Triggers Get Snapshots from Slow-running or Infinite Flows Add Flow to a GenStage Pipeline Wrapping Up Quiz: Process Collections with Flow

6

Data Ingestion Pipelines with Broadway

15 Lessons

7

Concluding the Course

1 Lesson

COURSE AUTHOR

How You'll Learn

You don’t get better at swimming by watching others. Coding is no different. Practice as you learn with live code environments inside your browser.

Videos are holding you back. Educative‘s interactive, text-based lessons accelerate learning — no setup, downloads, or alt-tabbing required.

Learn faster and smarter with adaptive AI tools embedded in every Educative course.

Built-in assessments let you test your skills. Completion certificates let you show them off.

FOR TEAMS

Interested in this course for your business or team?

Unlock this course (and 1,000+ more) for your entire org with DevPath