AWS First Steps
New feature

AWS Glue Data Quality makes ETL anomaly detection free and improves anomaly predictions

AWS Glue Data Quality introduces a new observation mode that reduces false anomaly detection and removes pricing for anomaly detection in ETL jobs, enabling more accurate monitoring of data quality across all Glue pipelines at no additional cost

AWS Glue Data Quality now offers improved anomaly detection with a new observation mode that reduces detection of false anomalies and removes pricing for anomaly detection in ETL jobs. This new capability handles irregular data arrival intervals gracefully by avoiding over-extrapolation of trends using a constant baseline instead of a linear trend, delivering more accurate alerts and reducing noise. The new observation mode is particularly useful for exploratory data analysis, datasets with flat or random patterns, workloads without predictable trends, and cases where data quality checks are run on varying schedules or in interactive environments like notebooks. Additionally, anomaly detection for AWS Glue ETL jobs is now available at no additional cost, allowing monitoring of data quality anomalies across all Glue pipelines without pricing concerns. These improvements are available in all AWS commercial regions and AWS GovCloud (US) regions. To get started, visit the AWS Glue Data Quality documentation. For pricing details, see the AWS Glue pricing page.

Read the original AWS announcement

Related articles and terms

Building ETL Pipelines with AWS Glue - Crawler and Job DesignAutomatically detect schemas with crawlers and execute ETL processing with Glue jobs. This guide covers Data Catalog usage patterns and visual development with Glue Studio.Cost Anomaly Detection - Catching Unexpected Spending Early with AWS Cost Anomaly DetectionLearn how to automatically detect cost anomalies with AWS Cost Anomaly Detection. Covers ML-based anomaly detection, monitor configuration, SNS/Slack notifications, and how it differs from Budgets.AWS GlueA serverless ETL service that automates data extraction, transformation, and loadingAWS Cost Anomaly DetectionA service that uses machine learning to automatically detect unusual spikes in your AWS spendingBuilding a Data Lake with Amazon S3 and Lake Formation - Design Patterns and GovernanceExplore data lake design patterns using S3 as the storage foundation and Lake Formation for fine-grained access control. This article also covers ETL pipelines and cost optimization.