Articles on Athena, OpenSearch, Glue, data lake construction, and other AWS data analytics and search services
This article presents concrete usage scenarios for each department, such as sales / IT / finance, and design patterns for notification / approval / multi-step workflows with Quick Flows.
An explanation of matching customer data records (entity resolution) with AWS Entity Resolution. Covers the three matching workflows (rule-based, ML-based, and provider service-based), ID mapping, pricing, privacy protection, and integration with Clean Rooms.
This article covers procuring third-party data products through Marketplace and building an automated delivery pipeline to S3. This article also introduces methods for productizing and monetizing your own data.
This article introduces data lake design patterns that use S3 as the storage foundation and achieve fine-grained access control with Lake Formation. It also explains ETL pipelines and cost optimization.
Learn how to build digital twins with AWS IoT TwinMaker. This article covers 3D scene creation, IoT data integration, Grafana dashboard integration, and industrial equipment visualization.
This article introduces running Spark jobs and Hive queries on EMR clusters, choosing between them and EMR Serverless, and cost optimization through managed scaling.
An explanation of collecting vehicle telemetry data with AWS IoT FleetWise. It introduces vehicle modeling, data filtering at the edge, and efficient data transfer to the cloud.
Explains data collection, modeling, and visualization for industrial equipment using AWS IoT SiteWise. Introduces OPC-UA compatible gateways, asset models, and SiteWise Monitor dashboards.
AWS IoT Analytics reached end of support on December 15, 2025. Explains the official migration targets (Kinesis Data Streams / Data Firehose, S3 / Timestream, Glue + Athena) and the 4-component architecture used while the service was offered.
Real-time data is ingested with Data Streams and delivered automatically to S3 / Redshift / OpenSearch with Data Firehose. This article explains how to build a streaming pipeline through shard design and choosing when to use on-demand mode.
Fine-grained column- and row-level access control and tag-based management establish data lake governance. This article introduces the integration with Glue Data Catalog and cross-account access.
Real-time stream processing with SQL or Java/Python applications runs in a fully managed Apache Flink environment. This article explains design patterns for window aggregation, pattern detection, and Kinesis/MSK integration.
This article covers designing managed Apache Kafka clusters, choosing between them and MSK Serverless, and data integration patterns with MSK Connect.
This article introduces a method for building a Kafka environment that needs no cluster management with IAM authentication, and simplifying event streaming through integration with EventBridge Pipes.
This article explains privacy-preserving data analytics using AWS Clean Rooms. It introduces data collaboration among multiple organizations, access control with analysis rules, and making use of differential privacy.
Explains design patterns for real-time data pipelines that combine Kinesis Data Streams and Data Firehose. Introduces shard design, buffering, Lambda transformations, and delivery to S3 / Redshift / OpenSearch.
Learn how to build a video transcoding pipeline using AWS Elemental MediaConvert and S3. This article covers multi-format output, HDR support, and cost-efficient serverless media processing techniques.
Explains how to build blockchain networks with Amazon Managed Blockchain, plus where Amazon QLDB (discontinued July 31, 2025) fits and its official migration path to Aurora PostgreSQL.
Automatically detect schemas with crawlers and execute ETL processing with Glue jobs. This guide covers Data Catalog usage patterns and visual development with Glue Studio.
Learn the criteria for choosing between Serverless and RA3 provisioned clusters, and how to prevent data silos using data sharing and Spectrum for data lake integration.
Learn how to select distribution styles and sort keys, design workload management (WLM), and accelerate queries with AQUA in Amazon Redshift.
Learn about running Spark and Hive jobs with EMR Serverless, job run design, and cost optimization strategies.
Learn how to build a domain-based data catalog and implement data discovery, sharing, and access control through subscription workflows.
Explains how to build interactive BI dashboards with Amazon Quick Sight (formerly Amazon QuickSight) and a serverless data analytics platform with Athena integration. Covers high-speed visualization with the SPICE engine and practical methods for sharing insights across the organization.
Learn how to acquire and utilize third-party data with AWS Data Exchange. This article covers building data pipelines with S3 integration and publishing data as a data provider.
Achieve fast queries with the SPICE engine and integrate BI capabilities into your own applications using the embedding API. This article introduces Q's natural language queries and Reader session-based pricing.
Learn streaming data processing design techniques with Amazon Kinesis, including building real-time data pipelines with Data Streams, Data Firehose, and Lambda integration.
Run joint analysis across multiple companies without sharing or copying data. Learn about aggregation rules for preventing individual identification and Cryptographic Computing for encrypted analysis.
Learn about building, access control, and governance for data lakes using AWS Lake Formation. This article covers fine-grained column-level and row-level permission management for S3-based data lakes, along with Glue and Athena integration.
Learn about data search and analytics design with Amazon OpenSearch Service, including how to build an analytics foundation with full-text search, log analysis, and dashboard visualization.
Learn how to build Spark clusters with EMR, choose between EMR Serverless, and optimize costs with spot instances.
Learn about record matching across multiple data sources with Entity Resolution and how to design matching workflows.
Learn how to build a fully managed Kafka cluster with Amazon MSK (Managed Streaming for Apache Kafka) and when to choose it over Kinesis. This article covers design patterns for large-scale real-time data streaming infrastructure.
Build a log analytics platform and optimize costs with index lifecycle management. Learn about OpenSearch Dashboards and Serverless mode.
Amazon OpenSearch Serverless is a fully managed service that handles search and analytics workloads with auto-scaling while eliminating cluster management. This article explains the OCU billing model, collection type selection criteria, and index design best practices from a practical perspective.
Learn practical approaches to quantum computing with Amazon Braket. This article covers quantum circuit simulators, access to real quantum hardware, implementing hybrid quantum-classical algorithms, and integration patterns with Lambda.