Serialization and Deserialization
Explore how to implement serialization and deserialization in Kafka Streams by creating custom Serdes for JSON data using Jackson. Learn to convert byte arrays to objects and handle serialization errors to build robust stream processing applications.
Previously, we published unstructured strings to our new topology. In most real-life scenarios, we would be using a structured string with a defined format, like JSON with a specific schema.
In order to accept different data types, we need to provide Kafka Streams with something called Serde, which is short for serializer/deserializer. It is a class in charge of mapping the byte array used by Kafka to store and transfer records into a meaningful object that we can use in our application. Kafka Streams comes with Serdes for multiple data types, including primitives, String, UUID, and Void.
However, Kafka Streams does not come with Serdes for popular formats like JSON, protobuf, or Avro. The records in our input topic are in JSON format, so we would implement a JSON Serde for our new Track class:
package io.github.stavshamir.serdes;
public class TrackSerde {
}
Implementing a Track Serde
To implement a Serde, we need to ...