AI-powered learning
Save this course
Concurrent Data Processing in Elixir
Gain insights into data processing and concurrency in Elixir. Explore GenServer, GenStage, Flow, and Broadway to enhance performance, set up pipelines, and handle increased workloads efficiently.
78 Lessons
11h 10min
Join 2.9 million developers at
Join 2.9 million developers at
LEARNING OBJECTIVES
- Understand how concurrency works in Elixir
- Learn about Elixir supervisors and GenServers
- Learn to use data processing pipelines
- Learn to use the Flow module in Elixir
- Learn to set up data ingestion pipelines
Learning Roadmap
1.
Getting Started
Getting Started
Get familiar with concurrent data processing in Elixir and its prerequisites.
2.
Easy Concurrency with the Task Module
Easy Concurrency with the Task Module
Unpack the core of concurrent programming in Elixir with the Task module and supervision.
Introduction to ConcurrencyAn Introduction to the Task ModuleCreate the PlaygroundStart Tasks and Retrieve ResultsSynchronous vs. Asynchronous ProgramsRetrieve a Task’s ResultsUse the await and yield Functions to Retrieve ResultsStop a Task and Async TaskManage a Series of TasksParameters of the async_stream FunctionLink ProcessesMeet the SupervisorUse Task.SupervisorUnderstand the “Let It Crash” PhilosophyWrapping UpQuiz: Easy Concurrency with the Task Module
3.
Long-Running Processes Using GenServer
Long-Running Processes Using GenServer
19 Lessons
19 Lessons
Work your way through GenServer callbacks, process management, and optimizing concurrency in Elixir.
4.
Data Processing Pipelines with GenStage
Data Processing Pipelines with GenStage
13 Lessons
13 Lessons
Find out about GenStage's role in managing data pipelines, back-pressure, and concurrency.
5.
Process Collections with Flow
Process Collections with Flow
12 Lessons
12 Lessons
Take a closer look at concurrent data processing with Elixir's Flow library for large datasets.
6.
Data Ingestion Pipelines with Broadway
Data Ingestion Pipelines with Broadway
15 Lessons
15 Lessons
See how Broadway simplifies data-ingestion pipelines with concurrent processing, batching, and fault tolerance.
Certificate of Completion
Showcase your accomplishment by sharing your certificate of completion.
Complete more lessons to unlock your certificate
Developed by MAANG Engineers
ABOUT THIS COURSE
Most engineers don’t think that concurrency is distinct from programming. Data processing includes transforming information in some way, like reporting, data aggregation, or analytics. On the other hand, concurrency involves significantly improving performance.
This course is designed to help you navigate the world of concurrency tools available in the Elixir ecosystem. You’ll explore popular modules and libraries like GenServer, GenStage, Flow, and Broadway. You’ll be able to ingest data, set up pipelines, and run concurrent processes with Elixir.
By the end of the course, you’ll have a wide range of new techniques to simplify your product, improve code performance, and make your application more resilient to increased workloads.
ABOUT THE AUTHOR
The Pragmatic Programmers
We create timely, practical books and learning resources on classic and cutting-edge topics to help you practice your craft and accelerate your career.
Trusted by 2.9 million developers working at companies
A
Anthony Walker
@_webarchitect_
E
Evan Dunbar
ML Engineer
S
Software Developer
Carlos Matias La Borde
S
Souvik Kundu
Front-end Developer
V
Vinay Krishnaiah
Software Developer
Built for 10x Developers
No Passive Learning
Learn by building with project-based lessons and in-browser code editor
Personalized Roadmaps
The platform adapts to your strengths & skills gaps as you go
Future-proof Your Career
Get hands-on with in-demand skills
AI Code Mentor
Write better code with AI feedback, smart debugging, and "Ask AI"
MAANG+ Interview Prep
AI Mock Interviews simulate every technical loop at top companies
Free Resources
