Partition Allocation

Kafka follows an algorithm when assigning partitions of a topic to brokers. Kafka attempts to evenly divide the partitions among the brokers as much as possible. As an example, consider a topic with 5 partitions and a replication factor of 3, making for a total of 15 partition replicas to be allocated amongst 5 brokers. The general algorithm will proceed as follows:

• There should be 3 partition replicas per broker.

• Initially, we start with determining the locations for the partition replica leaders. In our example, the replica leader for partition 0 is placed on broker 1, the replica leader for partition 1 is placed on broker 2, the replica leader for partition 2 onto broker 3, and so on and so forth. A pictorial representation is shown below:

Broker 1	Broker 2	Broker 3	Broker 4	Broker 5
Leader Partition#0	Leader Partition#1	Leader Partition#2	Leader Partition#3	Leader Partition#4

Once the replica leaders have been assigned to the brokers, we start assigning the replica followers for each partition.

• We start with partition 0. Its leader is already assigned to broker 1 and we need to assign two more replicas for this partition. We’ll assign the remaining replicas of partition 0 on broker 2 and broker 3. For each partition, we place the replicas at an increasing offset from the leader. The goal is not to co-locate the replicas on the same broker as the replica leader or on the same broker together.

Broker 1	Broker 2	Broker 3	Broker 4	Broker 5
Leader Partition#0	Leader Partition#1	Leader Partition#2	Leader Partition#3	Leader Partition#4
	Replica Parition#0	Replica Parition#0

• Similarly, the replicas for partition 1 get placed on broker 3 and broker 4 since the leader resides on broker 2.

Broker 1	Broker 2	Broker 3	Broker 4	Broker 5
Leader Partition#0	Leader Partition#1	Leader Partition#2	Leader Partition#3	Leader Partition#4
	Replica Partition#0	Replica Partition#0	Replica Partition#1
		Replica Partition#1