Adjust Batch Size and Timeout

Use batch size

If we inspect batch_info.size in the handle_batch/4 callbacks, we notice that our batches contain no more than fifty messages. The maximum number of messages received is also known as batch size and is typically controlled by the :batch_size value for each batcher in start_link/1. We can see this here:

...
batchers: [
  ...
  default: [
    batch_size: 100
  ]
]

However, the :batch_size default value for each batcher is already 100. It may seem like something is going wrong since we only get half the expected amount.

This is actually by design and is specific to the broadway_rabbitmq producer. The BroadwayRabbitMQ.Producer module maintains an active connection to the RabbitMQ server, which allows it to receive messages as soon as they get in the queue. This is why it limits the number of messages coming from RabbitMQ to control the flow and handle back-pressure. This is documented in the official broadway_rabbitmq documentation page.

The good news is that we can easily increase this limit by passing some extra configuration to our producer_config: