Go through the different components of the Kafka tool involved in building data pipelines step by step.

Become a Big Data Professional

Apache Kafka is a mainstay platform for building highly scalable data pipelines. This module starts with a review of the fundamentals of Kafka. Moving ahead, we will cover the functioning of the major components of Kafka with practical examples. In the end, we will take a comprehensive look at the internal architecture of Kafka. By the end of this module, we will have the knowledge to create scalable data pipelines using Kafka.

Building Scalable Data Pipelines with Kafka

So far, we discussed the theory behind Zookeeper and have seen a Java example with Zookeeper's programmatic API. In this lesson, we'll exercise different commands offered by the <b>zkCli.sh</b> utility that comes with the Zookeeper download. We fire up the Zookeeper server and then invoke the command line client to connect to the server. The various commands are listed in the widget below with an explanation for each. You can connect to the terminal below and execute the commands to see their outcome.


Let's discuss the command line tool zkCli.sh that can be used to connect and interact with the Zookeeper server.

Basics

Kafka Producer

Kafka Consumer

Kafka Internals

Appendix

Conclusion

Zookeeper: Practical