This device is not compatible.


Event Streaming Using Kafka and Live Visualization

In this project, we'll learn to connect the Kafka server with a consumer and producer using Python code. In the end, we'll stream your producer's CPU percentage to the consumer and plot a live graph on the consumer side using the animation method.

Event Streaming Using Kafka and Live Visualization

You will learn to:

Connect the Kafka server with Python

Create the consumer and producer using Kafka

Stream data through Kafka

Plot the data into a live graph using Python


Data Visualisation

Live Streaming


Basic knowledge of Python

Basic knowledge of the message broker architecture

Knowledge of Zookeeper and Kafka

Basic knowledge of plotting




Project Description

In this era of IoT devices, event streaming is a core and essential part of each IoT system. Apache Kafka is essential for event streaming and the monitoring of an IoT system. It is a stream-processing framework that implements a message bus. Its goal is to provide a single, high-throughput, low-latency platform for real-time data flow.

In this project, we’ll create a producer and a consumer using Python. After that, we’ll use Kafka to stream data from the producer to the consumer. We’ll plot a live graph of data on the consumer side.

Project Tasks


Create a Producer for Live Transmission

Task 0: Initial Setup

Task 1: Import the Kafka Producer

Task 2: Create the Producer in Python

Task 3: Send Sample Messages

Task 4: Send Live CPU Percentage


Create the Consumer in Python

Task 5: Import the Kafka Consumer

Task 6: Create a Consumer in Python

Task 7: Get Messages From Topic


Live Plot the CPU Percentage

Task 8: Import Modules

Task 9: Live Plot the CPU Percentage

Task 10: The Final Plotting of Code