Example: Topics & Partitions
Explore how Kafka manages topics and partitions for asynchronous microservices communication. Understand producer and consumer roles in a Spring Kafka environment, including configuration, record processing, consumer groups, and fault tolerance strategies to implement efficient event-driven microservices.
Technical parameters of the partitions and topics #
The topic order contains the order records. Docker Compose configures the Kafka Docker container based on the environment variable KAFKA_CREATE_TOPICS in the file docker-compose.yml in such a way as to create the topic order.
The topic order is divided into five partitions, as a greater number of
partitions allows for more concurrency. In the example scenario, it is
not important to have a high degree of concurrency. More partitions
require more file handles on the server and more memory on the client.
When a Kafka node fails, it might be necessary to choose a new leader
for each partition. This also takes longer when more
partitions exist.
This argues for a lower number of partitions as used in the example in order to save resources. The number of partitions in a topic can still be changed after creating a topic.
However, in that case, the mapping of records to partitions will change. This can cause problems because then the assignment of ...