Search⌘ K
AI Features
Home
Courses
Distributed Systems for Practitioners

Introduction to Kafka

Explore the core concepts of Apache Kafka, including its goal to deliver scalable, durable, and high-throughput messaging. Understand Kafka’s architecture with topics, partitions, brokers, replication, and leader election to grasp how it ensures data availability and fault tolerance.

We'll cover the following...

Apache Kafka is an open-source messaging system initially developed at Linkedin by Kreps et al.J. Kreps, N. Narkhede, and J. Rao, “Kafka: a Distributed Messaging System for Log Processing,” NetDB’ 11, June 2012, 2011, 2011. and Wang et al.G. Wang et al., “Building a Replicated Logging System with Apache Kafka,” NetDB’ 11, June 2012, 2011, 2011. and then donated to the Apache Software Foundation.

Goal of Kafka

The primary goal of Kafka is performance, scalability, durability, and availability.

Performance

The ability to exchange messages between systems with high throughput and low latency.

Scalability

The ability to incrementally scale to bigger volumes of data by adding more nodes to the system.

Durability & availability

The ability to provide durable and available data even in the ...