Kafka Streams Topology Visualization
Explore how to expose and visualize Kafka Streams topologies using Spring Boot Actuator. Understand configuring endpoints to retrieve topology descriptions, applying meaningful processor names, and using visualization tools to create graphical representations for easier analysis and documentation.
We'll cover the following...
Spring Boot Actuator can be used to conveniently expose an endpoint returning a text-based representation of our Kafka Streams topology. We can use this text-based representation, or topology description, for documentation and use third-party open-source tools to visualize the topology by converting the text-based representation to an image.
Generating a topology text-based representation is not a Spring Boot feature; we can do it in non-Spring Boot applications as well. However, Spring Boot makes our lives easier when it comes to accessing it.
Below is a fully functional Spring Boot Kafka Streams application with the actuator configured to expose the metrics endpoint:
spring:
cloud:
stream:
kafka.streams.binder:
applicationId: foo
brokers: localhost:9092
deserializationExceptionHandler: logAndContinue
configuration:
cache.max.bytes.buffering: 0
bindings:
process-in-0.destination: input-topic
management:
endpoints.web.exposure.include:
- metrics