Building Event-Driven Architectures with Amazon DocumentDB Change Streams

This article introduces how to build event-driven architectures for real-time data synchronization by integrating DocumentDB's change data capture with Lambda triggers.

About 8 min readLast updated: 2026-10-31

How Change Streams Work

DocumentDB Change Streams is a feature that captures changes to a collection (inserts, updates, deletes, and replaces) in real time. It is compatible with the MongoDB Change Streams API, so you can use the code of existing MongoDB applications as is. Change Streams is built on the cluster's change log (oplog), and changes are delivered in the order in which they occurred. Each event includes operationType (insert, update, delete), documentKey (the ID of the changed document), and fullDocument (the entire document after the change).

Integration with Lambda Triggers

When you configure DocumentDB Change Streams as a Lambda event source mapping, change events are automatically delivered to a Lambda function. Inside the Lambda function, you run processing according to the type of event. Typical use cases are OpenSearch index updates (reflecting document changes in the search index immediately), ElastiCache cache invalidation (deleting the cache of changed documents), data synchronization to DynamoDB (maintaining read-optimized views), and SNS notifications (notifying changes that match specific conditions). By configuring the batch size and batch window and processing multiple events together, you can improve throughput.

Figure: The four stages from a change being written to the oplog to its arrival downstream

1. The change is recorded in the oplogAn insert / update / delete / replace on a collection is written to the cluster's change log (oplog). Change Streams is built on this oplog, and changes are delivered in the order in which they occurred.
2. The change event is assembledEach event contains operationType (insert, update, delete), documentKey (the ID of the changed document), and fullDocument (the entire document after the change). Because it is compatible with the MongoDB Change Streams API, you can use the code of existing applications as is.
3. The event is delivered to the Lambda event source mappingIf you configure it as an event source mapping, change events are automatically passed to the Lambda function. Grouping multiple events with the batch size and batch window lets you raise throughput.
4. The change is reflected downstream, and a resume token holds the restart pointThe function runs OpenSearch index updates, ElastiCache cache invalidation, synchronization to DynamoDB, SNS notifications, and so on. If you keep hold of the resume token contained in each event, you can reread from the event after the last processed one during failure recovery (automatic with event source mappings).

Failure Recovery and Operational Considerations

Each Change Streams event contains a resume token, and by saving this token, you can resume reading from the event after the last processed one during failure recovery. With Lambda event source mappings, management of the resume token is automated. Because the Change Streams change log is retained for up to 7 days, old events are lost if processing stops for 7 days or more. To prepare for long outages, a design that takes regular full snapshots separately from Change Streams is recommended. Because enabling Change Streams has a slight impact on cluster performance, you run performance tests in advance in production environments.

Change Streams Pricing Considerations

Change Streams itself incurs no additional charge, but retaining the change log consumes I/O and storage. The change log retention period is up to 7 days, and storage usage increases for collections with many writes. When you use Lambda triggers, charges for the number of Lambda invocations and execution time apply. For collections with a high change frequency, you set a large Lambda batch size to reduce the number of invocations and optimize cost. If you choose to forward to Kinesis Data Streams, shard-hour charges are added.

The Scope of Use for Change Data Capture

Change Streams serves as a foundation for linking various kinds of processing to data changes as the starting point. Typical uses are keeping a search engine's index in step with document changes and invalidating the cache of changed data so that the latest data is always returned. Uses such as maintaining a separate read-optimized view, or streaming changes into an analytics platform for aggregation, are also expanding. The advantage is that you can add downstream processing in a loosely coupled way without modifying the application on the write side, which increases flexibility when you extend a system step by step.

Filtering and Shaping Change Events

Change Streams uses a MongoDB-compatible aggregation pipeline, which lets you narrow down the changes you receive. Controls such as targeting only specific collections or operation types, or reacting only when fields of interest change, are possible. If you use the option to retrieve the entire document after the change in update events, you can pass the complete state, not just the difference, to subsequent processing. By passing only the events you need, you can reduce downstream processing load and cost. Rather than screening every event on the receiving side, a design that narrows them down at a stage close to the source is efficient.

Fan-Out to Kinesis and MSK

When you want to deliver change events to multiple processing systems, a configuration that forwards them to Kinesis Data Streams or MSK through Lambda is effective. Once you consolidate them in a streaming platform, multiple consumers can each consume the same changes for their own purposes, and you can run search updates / analytics / notifications and so on in parallel. For data where order matters, you preserve order by designing keys so that the data is consolidated in the same shard or partition. You can evolve from a simple configuration that processes events directly in Lambda to a highly extensible configuration with a streaming platform in between as your requirements grow.

Design Considerations and Best Practices

In event-driven designs with Change Streams, you keep a few established practices in mind. You design processing to be idempotent so that no problems occur even if the same event is redelivered. You make it possible to resume from the point of interruption with resume tokens, and you also take separate snapshots to prepare for the fact that old changes are lost if processing stops beyond the change log retention period. Because enabling the feature can affect cluster load, you verify performance before going into production. By monitoring latency and failures and acting early when backlogs grow, you can maintain highly reliable change integration.

Summary

DocumentDB Change Streams is a feature that captures document changes in real time to build event-driven architectures. It processes changes automatically with Lambda triggers and, through integration with Kinesis Data Streams, delivers data to downstream analytics pipelines. It manages the position for resuming processing with resume tokens, achieving highly reliable change data capture.

References (Official AWS Resources)

The primary sources for this page are the official AWS website and documentation. Check the official pages below for the latest specifications and pricing.

If this page and the official documentation disagree, treat the official documentation as authoritative.