Failure Recovery in Flink
Explore how Flink efficiently recovers from failures by periodically checkpointing operator states using the Asynchronous Barrier Snapshotting algorithm. Learn how Flink manages state storage, coordinates checkpoints with external systems like Kafka, and ensures exactly-once processing guarantees to maintain stream processing reliability.
As mentioned previously, stream processing applications in Flink are supposed to be long-lived. So there must be an efficient way to recover from failures without repeating a lot of work. For this purpose, Flink periodically checkpoints the operators’ state and the position of the consumed stream to generate this state. In case of a failure, an application can be restarted from the latest checkpoint and continue processing from there.
All this is achieved via an algorithm similar to the Chandy-Lamport algorithm for distributed snapshots, called Asynchronous Barrier Snapshotting (ABS).
Asynchronous Barrier Snapshotting (ABS)
The ABS algorithm operates slightly differently for acyclic and cyclic graphs, so we will examine the first case here, which is a bit simpler.
Working
The algorithm works in the following way:
-
The Job Manager periodically injects some control records in the stream, referred to as stage barriers. These records are supposed to divide the stream into stages. At the end of a stage, the set of operator states reflects the whole execution history up to the associated barrier. Thus it can be used for a snapshot.
-
When a ...