AWS First Steps
Database

Building Event-Driven Architectures with Amazon DocumentDB Change Streams

Learn how to use DocumentDB's change data capture to integrate with Lambda triggers and build event-driven architectures for real-time data synchronization.

About 5 min readWritten and edited by Awashiro

How Change Streams Work

DocumentDB Change Streams is a feature that captures changes (inserts, updates, deletes, replacements) to a collection in real time. It is compatible with the MongoDB Change Streams API, so you can use existing MongoDB application code as-is. Change Streams are built on the cluster's change log (oplog) and deliver changes in the order they occurred. Each event includes the operationType (insert, update, delete), documentKey (the ID of the changed document), and fullDocument (the complete document after the change).

Integration with Lambda Triggers

By configuring DocumentDB Change Streams as a Lambda event source mapping, change events are automatically delivered to a Lambda function. Inside the Lambda function, you execute processing based on the event type. Typical use cases include updating OpenSearch indexes (immediately reflecting document changes in the search index), invalidating ElastiCache caches (deleting caches for changed documents), synchronizing data to DynamoDB (maintaining read-optimized views), and sending SNS notifications (alerting on changes that match specific conditions). You can improve throughput by configuring batch size and batch window to process multiple events together.

Figure: the four stages from a write landing in the oplog to the downstream update
  1. 1. The change is recorded in the oplogInserts, updates, deletes and replacements on a collection are written to the cluster change log (the oplog). Change Streams build on that log, so events are delivered in the order the changes occurred.
  2. 2. The change event is assembledEvery event carries operationType (insert, update, delete), documentKey (the id of the changed document) and fullDocument (the whole document after the change). The API is compatible with MongoDB Change Streams, so existing application code works unchanged.
  3. 3. The event is delivered to a Lambda event source mappingOnce configured as an event source mapping, change events flow into the Lambda function automatically. Batch size and batch window let you group several events together and lift throughput.
  4. 4. Downstream systems are updated and the resume token marks the restart pointTypical work here is refreshing an OpenSearch index, invalidating an ElastiCache entry, syncing to DynamoDB or publishing an SNS notification. Keeping the resume token carried by each event lets you restart right after the last processed event; the event source mapping handles that for you.

Disaster Recovery and Operational Considerations

Each Change Streams event includes a resume token. By saving this token, you can resume reading from the event after the last processed one during disaster recovery. When using Lambda event source mappings, resume token management is automated. The change log is retained for up to 7 days, so if processing is stopped for more than 7 days, older events will be lost. To prepare for extended outages, it is recommended to design periodic full snapshots in addition to Change Streams. Enabling Change Streams has a slight impact on cluster performance, so conduct performance testing before using it in production.

Change Streams Pricing Considerations

Change Streams itself incurs no additional charges, but the change log consumes I/O and storage. The change log retention period is up to 7 days, and storage usage increases for collections with heavy writes. When using Lambda triggers, Lambda invocation and execution time charges apply. For collections with high change frequency, set a larger Lambda batch size to reduce invocation counts and optimize costs. If you choose to forward to Kinesis Data Streams, shard-hour charges are added.

Summary

DocumentDB Change Streams is a feature that captures document changes in real time to build event-driven architectures. Lambda triggers automatically process changes, and integration with Kinesis Data Streams delivers data to downstream analytics pipelines. Resume tokens manage the restart position for processing, enabling reliable change data capture.

References (Official AWS Resources)

The primary sources for this page are the official AWS website and documentation. Check the official pages below for the latest specifications and pricing.

If this page and the official documentation disagree, treat the official documentation as authoritative.

Related Services

Amazon DocumentDB SpecializedA MongoDB-compatible managed document database serviceAWS Lambda EssentialA serverless compute service that runs code without server managementAmazon EventBridge PopularA serverless event bus that routes events from AWS services and SaaS applications

Related Articles

Running a MongoDB-Compatible Database with Amazon DocumentDB - Design Patterns and ScalingSelect the right instance class for MongoDB-compatible DocumentDB and ensure scalability with sharding through Elastic Clusters. This article also covers backup strategies.Event-Driven Architecture - Building Loosely Coupled Systems with Amazon EventBridgeLearn how to build event-driven architectures using Amazon EventBridge.Event Integration with Amazon EventBridge Pipes - Connection Patterns for Sources and TargetsLearn how to connect event sources like SQS, DynamoDB Streams, and Kinesis directly to targets, with filtering and enrichment steps in between.

More on This Topic

Multi-Region Architecture with Amazon Aurora Global Database - Designing for DR and Global ReadsAchieve sub-second RPO through storage-layer physical replication. This guide covers planned and unplanned failover procedures and how to leverage Write Forwarding for global read workloads.Cassandra-Compatible Database - Serverless Distributed Database with Amazon KeyspacesLearn how to design and operate distributed databases using Amazon Keyspaces (for Apache Cassandra) and DynamoDB.Achieving Microsecond Latency with Amazon DynamoDB Accelerator (DAX) - In-Memory Cache DesignLearn about read acceleration for DynamoDB with DAX, cache strategies, and cluster design.Document Database in Practice - Flexible Data Modeling with Amazon DocumentDB and DynamoDBLearn how to design and operate document databases using Amazon DocumentDB and DynamoDB.Running MongoDB Workloads as a Managed Service with Amazon DocumentDB - Document Model and Query DesignManage document data with a MongoDB-compatible API, and achieve read scaling and disaster recovery with up to 15 read replicas and global clusters. This article also covers sharding with Elastic Clusters.Running a MongoDB-Compatible Database with Amazon DocumentDB - Design Patterns and ScalingSelect the right instance class for MongoDB-compatible DocumentDB and ensure scalability with sharding through Elastic Clusters. This article also covers backup strategies.Amazon DynamoDB Table Design Patterns - Single-Table Design and GSI StrategiesLearn about DynamoDB partition key design, single-table design, and how to implement diverse access patterns using GSIs.Building Multi-Region Databases with DynamoDB Global Tables - Active-Active ReplicationAchieve low-latency reads and writes with multi-region active-active replication. Learn about conflict resolution mechanisms and DR design.

Similar Articles and Services

Amazon EventBridgeA serverless event bus that routes events from AWS services, SaaS applications, and custom applications, serving as the backbone of event-driven architecturesDocument Database in Practice - Flexible Data Modeling with Amazon DocumentDB and DynamoDBLearn how to design and operate document databases using Amazon DocumentDB and DynamoDB.Amazon DocumentDBA fully managed document database with MongoDB-compatible APIs that provides scalable JSON document storage, querying, and indexingAmazon EventBridge PipesA service that builds point-to-point integrations from source to target with built-in filtering and transformationEvent-Driven Architecture - Building Loosely Coupled Systems with Amazon EventBridgeLearn how to build event-driven architectures using Amazon EventBridge.