Concurrent Data Processing in Elxir by The Pragmatic Programmer-01.png

myDokcer.tar.gz

ch1_1

ch_3

ch_3.1

ch1_1-task supervisor

ch_4.1

ch_5.1 RabbitMQ

Ch_5.2 RabbitMQ

Most engineers don’t think that concurrency is distinct from programming. Data processing includes transforming information in some way, like reporting, data aggregation, or analytics. On the other hand, concurrency involves significantly improving performance.

This course is designed to help you navigate the world of concurrency tools available in the Elixir ecosystem. You’ll explore popular modules and libraries like GenServer, GenStage, Flow, and Broadway. You’ll be able to ingest data, set up pipelines, and run concurrent processes with Elixir.

By the end of the course, you’ll have a wide range of new techniques to simplify your product, improve code performance, and make your application more resilient to increased workloads.

Concurrent Data Processing in Elixir

How many callbacks are available for us to implement in `Broadway`?

How many callbacks are available for us to implement in <code>Broadway</code>?

**(True or False)** `Broadway` is an Elixir library that makes working with message brokers easy.

(True or False) <code>Broadway</code> is an Elixir library that makes working with message brokers easy.

The _______ pipeline is generated by `Broadway`.

The _______ pipeline is generated by <code>Broadway</code>.

**(True or False)** The `:default` group of processors is the only one that’s allowed for now when configuring `:processors`.

(True or False) The <code>:default</code> group of processors is the only one that’s allowed for now when configuring <code>:processors</code>.

Incoming messages sent by the broker are processed by ________.

Incoming messages sent by the broker are processed by ________.

**(True or False)** The `amqp` client, which `broadway_rabbitmq` uses, could generate lots of debug messages.

(True or False) The <code>amqp</code> client, which <code>broadway_rabbitmq</code> uses, could generate lots of debug messages.

**(True or False)** If we use the `:reject_and_requeue` setting, messages will not be redelivered indefinitely.

(True or False) If we use the <code>:reject_and_requeue</code> setting, messages will not be redelivered indefinitely.

The ________ callback receives the current batcher and the list of messages in the batch.

The ________ callback receives the current batcher and the list of messages in the batch.

Broadway gives us the option to define batches at runtime using `:batch_key`. This is called ________.

Broadway gives us the option to define batches at runtime using <code>:batch_key</code>. This is called ________.

________ is an excellent choice when working with popular message brokers.

________ is an excellent choice when working with popular message brokers.

Quiz: Data Ingestion Pipelines with Broadway

Quiz: Data Ingestion Pipelines with Broadway

Take a quiz to check what we've learned from this chapter.

Getting Started

Easy Concurrency with the Task Module

Long-Running Processes Using GenServer

Data Processing Pipelines with GenStage

Process Collections with Flow

Data Ingestion Pipelines with Broadway

Concluding the Course

Quiz: Data Ingestion Pipelines with Broadway