Data Governance with Amazon DataZone - Data Discovery, Sharing, and Access Control
Learn how to build a domain-based data catalog and implement data discovery, sharing, and access control through subscription workflows.
Analytics and Search
Articles on Athena, OpenSearch, Glue, data lake construction, and other AWS data analytics and search services
Record Matching with AWS Entity Resolution - Customer Data Deduplication and Integration
Learn about record matching across multiple data sources with Entity Resolution and how to design matching workflows.
Building a Log Analytics Platform with Amazon OpenSearch Service - Index Design and Dashboard Construction
Build a log analytics platform and optimize costs with index lifecycle management. Learn about OpenSearch Dashboards and Serverless mode.
Running Apache Spark on Amazon EMR - Cluster Design and Cost Optimization for Big Data
Learn how to build Spark clusters with EMR, choose between EMR Serverless, and optimize costs with spot instances.
Privacy-Preserving Data Collaboration with AWS Clean Rooms
Run joint analysis across multiple companies without sharing or copying data. Learn about aggregation rules for preventing individual identification and Cryptographic Computing for encrypted analysis.
Running Spark Jobs Serverlessly with Amazon EMR Serverless - Big Data Processing Without Cluster Management
Learn about running Spark and Hive jobs with EMR Serverless, job run design, and cost optimization strategies.
Building BI Dashboards with Amazon QuickSight - Serverless Analytics and Embedded Visualization
Achieve fast queries with the SPICE engine and integrate BI capabilities into your own applications using the embedding API. This article introduces Q's natural language queries and Reader session-based pricing.
Practical Guide to Amazon OpenSearch Serverless - OCU Design and Optimization Strategies by Collection Type
Amazon OpenSearch Serverless is a fully managed service that handles search and analytics workloads with auto-scaling while eliminating cluster management. This article explains the OCU billing model, collection type selection criteria, and index design best practices from a practical perspective.
Building a Real-Time Data Pipeline with Amazon Kinesis - Choosing Between Data Streams and Data Firehose
Ingest real-time data with Data Streams and automatically deliver it to S3, Redshift, and OpenSearch with Data Firehose. This article explains how to build a streaming pipeline with shard design and on-demand mode selection.
Digital Twins - Building 3D Digital Replicas of Physical Spaces with AWS IoT TwinMaker
Learn how to build digital twins with AWS IoT TwinMaker. This article covers 3D scene creation, IoT data integration, Grafana dashboard integration, and industrial equipment visualization.
Customer Identity Unification - Resolving Scattered Customer Data with AWS Entity Resolution
Learn how to perform entity resolution (record matching) on customer data using AWS Entity Resolution. This article covers ML-based matching, rule-based matching, privacy protection, and integration with Clean Rooms.
Privacy-Enhanced Data Analytics - Secure Data Collaboration with AWS Clean Rooms
Learn about privacy-enhanced data analytics with AWS Clean Rooms. Explore multi-organization data collaboration, access control through analysis rules, and differential privacy.
Building an Apache Kafka Streaming Platform with Amazon MSK - Cluster Design and Operations
Design a managed Apache Kafka cluster, learn when to choose MSK Serverless, and explore data integration patterns with MSK Connect.
BI Dashboard Visualization - Building a Data-Driven Decision Platform with Amazon QuickSight
Explains how to build interactive BI dashboards with Amazon QuickSight and a serverless data analytics platform with Athena integration. Covers high-speed visualization with the SPICE engine and practical methods for sharing insights across the organization.
Building a Cloud Data Warehouse with Amazon Redshift - Choosing Between Serverless and RA3
Learn the criteria for choosing between Serverless and RA3 provisioned clusters, and how to prevent data silos using data sharing and Spectrum for data lake integration.
Data Lake Governance - Centralized Access Control with AWS Lake Formation
Learn about building, access control, and governance for data lakes using AWS Lake Formation. This article covers fine-grained column-level and row-level permission management for S3-based data lakes, along with Glue and Athena integration.
Building ETL Pipelines with AWS Glue - Crawler and Job Design
Automatically detect schemas with crawlers and execute ETL processing with Glue jobs. This guide covers Data Catalog usage patterns and visual development with Glue Studio.
IoT Data Analytics - Structuring and Analyzing Device Data with AWS IoT Analytics
Learn how to build an IoT device data collection, preprocessing, and analytics pipeline with AWS IoT Analytics. Covers the four components - channels, pipelines, data stores, and datasets - plus QuickSight integration.
Industrial IoT Monitoring - Collecting and Visualizing Equipment Data with AWS IoT SiteWise
Learn how to collect, model, and visualize industrial equipment data with AWS IoT SiteWise. Covers OPC-UA gateways, asset models, and SiteWise Monitor dashboards.
Building a Data Lake with AWS Lake Formation - Fine-Grained Access Control and Data Catalog
Establish data lake governance with column-level and row-level fine-grained access control and tag-based management. This article covers integration with Glue Data Catalog and cross-account access.
Big Data Processing with Amazon EMR - Spark and Hive Execution Environments
Learn how to run Spark jobs and Hive queries on EMR clusters, choose between EMR Serverless, and optimize costs with managed scaling.
Amazon Redshift Performance Tuning - Optimizing Distribution Keys, Sort Keys, and WLM
Learn how to select distribution styles and sort keys, design workload management (WLM), and accelerate queries with AQUA in Amazon Redshift.
Building a Data Lake with Amazon S3 and Lake Formation - Design Patterns and Governance
Explore data lake design patterns using S3 as the storage foundation and Lake Formation for fine-grained access control. This article also covers ETL pipelines and cost optimization.
Real-Time Stream Processing with Amazon Managed Service for Apache Flink - Stateful Processing and Window Aggregation
Run real-time stream processing with SQL or Java/Python applications in a fully managed Apache Flink environment. This article covers design patterns for window aggregation, pattern detection, and Kinesis/MSK integration.
Vehicle Fleet Data Management - Efficiently Collecting Vehicle Telemetry with AWS IoT FleetWise
Learn how to collect vehicle telemetry data with AWS IoT FleetWise, including vehicle modeling, edge-side data filtering, and efficient data transfer to the cloud.
Getting Started with Event Streaming on Amazon MSK Serverless - Building a Zero-Ops Kafka Environment
Learn how to build a cluster-free Kafka environment with IAM authentication and simplify event streaming through integration with EventBridge Pipes.
Practical Use Cases for Amazon Quick - Department-Specific Scenarios and Workflow Automation Design Patterns
Explore concrete use cases for sales, IT, and finance departments, along with design patterns for notifications, approvals, and multi-step workflows using Quick Flows.
Data Search and Analytics in Practice - Building Full-Text Search and Visualization with OpenSearch
Learn about data search and analytics design with Amazon OpenSearch Service, including how to build an analytics foundation with full-text search, log analysis, and dashboard visualization.
Designing Streaming Data Processing - Building Real-Time Data Pipelines with Kinesis
Learn streaming data processing design techniques with Amazon Kinesis, including building real-time data pipelines with Data Streams, Data Firehose, and Lambda integration.
Building Blockchain Networks - Leveraging Distributed Ledgers with Amazon Managed Blockchain and QLDB
Explains how to build blockchain networks with Amazon Managed Blockchain and use Amazon QLDB as a verifiable ledger database. Covers practical use cases such as supply chain management and ensuring transparency in financial transactions.
Leveraging the Data Marketplace - Efficient Third-Party Data Acquisition and Utilization with AWS Data Exchange
Learn how to acquire and utilize third-party data with AWS Data Exchange. This article covers building data pipelines with S3 integration and publishing data as a data provider.
Real-Time Data Streaming - Instant Data Processing with Amazon Kinesis
Learn design patterns for real-time data pipelines combining Kinesis Data Streams and Data Firehose. This article covers shard design, buffering, Lambda transformations, and delivery to S3, Redshift, and OpenSearch.
Managed Kafka Streaming - Building Large-Scale Real-Time Data Pipelines with Amazon MSK
Learn how to build a fully managed Kafka cluster with Amazon MSK (Managed Streaming for Apache Kafka) and when to choose it over Kinesis. This article covers design patterns for large-scale real-time data streaming infrastructure.
Quantum Computing Service - Getting Started with Quantum Algorithm Development on Amazon Braket
Learn practical approaches to quantum computing with Amazon Braket. This article covers quantum circuit simulators, access to real quantum hardware, implementing hybrid quantum-classical algorithms, and integration patterns with Lambda.
Video Transcoding - Building a Scalable Media Conversion Pipeline with AWS Elemental MediaConvert
Learn how to build a video transcoding pipeline using AWS Elemental MediaConvert and S3. This article covers multi-format output, HDR support, and cost-efficient serverless media processing techniques.
Leveraging Third-Party Data with AWS Data Exchange - Data Procurement and Subscription Management
Procure third-party data products via Marketplace and build automated delivery pipelines to S3. This article also covers how to productize and monetize your own data.