Concurrent Data Processing in Elixir - AI-Powered Learning for Developers

Home>

Courses>

Concurrent Data Processing in Elixir

to do

Intermediate

78 Lessons

11h 10min

Certificate of Completion

to do

AI-POWERED

Explanations

AI-POWERED

Explanations

This course includes

49 Playgrounds

7 Quizzes

This course includes

49 Playgrounds

7 Quizzes

Course Overview

Most engineers don’t think that concurrency is distinct from programming. Data processing includes transforming information in some way, like reporting, data aggregation, or analytics. On the other hand, concurrency involves significantly improving performance. This course is designed to help you navigate the world of concurrency tools available in the Elixir ecosystem. You’ll explore popular modules and libraries like GenServer, GenStage, Flow, and Broadway. You’ll be able to ingest data, set up pipelines...Show More

What You'll Learn

Understand how concurrency works in Elixir

Learn about Elixir supervisors and GenServers

Learn to use data processing pipelines

Learn to use the Flow module in Elixir

Learn to set up data ingestion pipelines

What You'll Learn

Understand how concurrency works in Elixir

Course Content

Getting Started

What Does This Course Cover?

Course Prerequisites and Intended Audience

Easy Concurrency with the Task Module

Introduction to Concurrency

An Introduction to the Task Module

Create the Playground Start Tasks and Retrieve Results Synchronous vs. Asynchronous Programs Retrieve a Task’s Results Use the await and yield Functions to Retrieve Results Stop a Task and Async Task Manage a Series of Tasks Parameters of the async_stream Function Link Processes Meet the Supervisor Use Task.Supervisor Understand the “Let It Crash” Philosophy Wrapping Up Quiz: Easy Concurrency with the Task Module

Long-Running Processes Using GenServer

Long-running Processes and Starting GenServer GenServer Callbacks: init GenServer Callbacks: handle_continue and handle_call Genserver Callbacks: handle_cast GenServer Callbacks: handle_info GenServer Callbacks: terminate Make Improvements to Our GenServer The blueprint of our job-processing system How to Use Anonymous Functions Introduce DynamicSupervisor Process Restart Values and Frequency Implement a Supervisor Supervisor Strategies Registry Processes Register New Processes and Query the Registry Limit Concurrency and Inspect the Supervisors Examine Running Processes and Stop the Supervisors Wrapping Up Quiz: Long-running Processes Using GenServer

Data Processing Pipelines with GenStage

Back-Pressure Introduce GenStage The Producer and Consumer Build the Data Processing Pipeline Understand Consumer Demand Revisiting the Producer Add More Consumers and Buffer Events Add Concurrency With ConsumerSupervisor Create Multi-stage Data Pipelines Scale Up a Stage with Extra Processes Choose the Right Dispatcher Wrapping Up Quiz: Data Processing Pipelines with GenStage

Process Collections with Flow

Introduction to the Flow Library Read Files Improve Performance with Streams Run Streams Concurrently and Understand Flows The reduce Function Route Events with Partitions Group and Sort Windows and Triggers Get Snapshots from Slow-running or Infinite Flows Add Flow to a GenStage Pipeline Wrapping Up Quiz: Process Collections with Flow

Data Ingestion Pipelines with Broadway

15 Lessons

Concluding the Course

1 Lesson

Course Author

Trusted by 1.4 million developers working at companies

Anthony Walker

@_webarchitect_

Emma Bostian 🐞

@EmmaBostian

Evan Dunbar

ML Engineer

Carlos Matias La Borde

Software Developer

Souvik Kundu

Front-end Developer

Vinay Krishnaiah

Software Developer

Eric Downs

Musician/Entrepeneur

Kenan Eyvazov

DevOps Engineer

Souvik Kundu

Front-end Developer

Eric Downs

Musician/Entrepeneur

Anthony Walker

@_webarchitect_

Emma Bostian 🐞

@EmmaBostian

Hands-on Learning Powered by AI

See how Educative uses AI to make your learning more immersive than ever before.

Instant Code Feedback

Evaluate and debug your code with the click of a button. Get real-time feedback on test cases, including time and space complexity of your solutions.

AI-Powered Mock Interviews

Put your skills to the test in a simulated interview setting. Receive personalized feedback based on your performance. Available in Premium & Premium Plus plans.

Adaptive Learning

At various checkpoints throughout Educative courses, you will be prompted to take a quick assessment. Receive a condensed curriculum tailored to your strengths and skill gaps.

Explain with AI

Select any text within any Educative course, and get an instant explanation — without ever leaving your browser.

AI Code Mentor

AI Code Mentor helps you quickly identify errors in your code, learn from your mistakes, and nudge you in the right direction — just like a 1:1 tutor!

FOR TEAMS

Interested in this course for your business or team?

Unlock this course (and 1,000+ more) for your entire org with DevPath