Kafka Streams for Software Developers/

...

/

Stateful Processing Basic Concepts

Stateful Processing Basic Concepts

Learn about state stores in Kafka Streams and the KTable abstraction.

We'll cover the following...

Advanced Kafka Streams applications can require operations such as aggregation or windowing. An example of an aggregation operation is count, which, as the name suggests, counts the total number of records of the same key in a topic. 

stream.peek((k, v) -> System.out.printf("Incoming record - Key: %s, Value: %s\n", k, v))

        .groupByKey()

        .count()

        .toStream()

        .peek((k, v) -> System.out.printf("Count - Key: %s, Value: %s\n", k, v));
A simple stateful topology using the count operator

This type of operator requires the application to remember previous events, or in other words, to keep a state, which is why Kafka Streams applications and topologies using at least one of these operators are called stateful.

Keeping and managing a state in distributed applications in a fault-tolerant manner on our own would have been a very hard and complex task. Fortunately, Kafka Streams does all of this for us.

State store

In order to keep and manage the state, Kafka Streams uses an abstraction called a state store. ...