DynamoDB Data Lifecycle and Change Data
Explore how DynamoDB extends beyond basic key-value access by managing data lifecycle with TTL for automatic expiry, Streams for near-real-time change capture, and Lambda to trigger event-driven workflows. Understand practical archive patterns using S3 and Firehose to retain historical data efficiently.
In the previous lesson, you learned how DynamoDB handles consistency and write coordination through read consistency modes, conditional expressions, and transactions. The next question is what happens after data is written to the table. Items' age, sessions' expiration, and downstream systems often need to react to item-level changes. This lesson shifts focus from write-time guarantees to the post-write lifecycle, covering three post-write capabilities that extend DynamoDB beyond basic key-value access and support event-driven workflows.
The first pillar is automatic expiry through time-to-live (TTL), which removes stale items without manual intervention. The second is DynamoDB Streams, a near-real-time change data capture mechanism that exposes every insert, update, and delete as a consumable record. The third combines Lambda triggers, archive pipelines, and export capabilities that let you react to changes, build materialized views, and preserve history in Amazon S3. Together, these features handle cleanup, change propagation, and long-term retention as a cohesive system rather than a collection of ad hoc scripts.
The core AWS services and features you will work with include DynamoDB TTL, DynamoDB Streams, Lambda event source mappings, Kinesis Data Firehose for continuous delivery, the DynamoDB export-to-S3 feature, and Amazon Athena for querying archived data. Each plays a distinct role in the data life cycle, and understanding their mechanics is essential for designing production-grade, event-driven workloads on AWS.
Time-to-live for automatic expiry
TTL lets you attach an expiration timestamp to individual items so DynamoDB’s background process removes them automatically. You enable TTL by designating a numeric attribute on the table. Each item that should expire stores a Unix epoch timestamp (in seconds) in that attribute. When the current time exceeds the stored value, DynamoDB marks the item for deletion.
The deletion is best-effort and asynchronous. Items typically disappear within minutes of expiry, but the delay can stretch to 48 hours under heavy load. Until the background scanner physically removes an expired item, it still appears in reads. Applications should therefore filter results by comparing the TTL attribute against the current time rather than assuming that expired items are already gone.
Attention: TTL is not a precise scheduler. If your use case demands deletion at an exact second, you need an external mechanism such as a scheduled Lambda function. TTL is designed for eventual cleanup, not time-critical enforcement.
A critical cost advantage makes TTL attractive for high-volume workloads. TTL deletions do not consume write capacity units. DynamoDB treats them as free system operations, which means you can expire ...