AWS Glue Data Catalog now supports OAuth 2.0 bearer token authentication for Apache Iceberg REST Catalog APIs using AWS Sign-In, allowing engines like Apache Spark, Trino, and Apache Flink to authenticate with standard OAuth 2.0 bearer tokens.
Amazon EMR validates data quality natively within Apache Spark jobs, allowing data engineering teams to catch quality issues inline before bad data reaches downstream systems, using AWS Glue-powered rules without additional cost
AWS Glue Data Quality now offers an advanced mode using generative AI to detect data intent and propose business-relevant rules, accelerating setup for new datasets and large-scale data lake baseline checks.
AWS Glue zero-ETL captures archived Salesforce records in target tables, providing a complete replica of Salesforce data for analytics and reporting. Existing pipelines run uninterrupted while the isArchived field is accurately populated.
AWS Glue Data Catalog introduces preview support for exporting catalog metadata to S3 Tables, enabling querying of technical and business metadata using standard SQL.
AWS Glue Data Quality introduces a new observation mode that reduces false anomaly detection and removes pricing for anomaly detection in ETL jobs, enabling more accurate monitoring of data quality across all Glue pipelines at no additional cost
AWS Glue adds VPC connectivity, filter pushdown, and partition support to the REST API connector, enabling secure access to private endpoints, reduced data transfer costs, and faster ingestion through parallel reads
AWS Glue Data Quality adds a new Distribution Analyzer to generate frequency distribution profiles, integrating directly with DQDL rulesets for histograms and value distributions, stored in S3 for future querying.
AWS Glue Data Quality now supports anomaly detection and result storage in the AWS Glue Data Catalog for catalog-based data quality evaluations, providing consistent data quality across ETL jobs and catalog evaluations
AWS Glue SAP OData connector and zero-ETL integrations are now available in AWS GovCloud (US) regions, enabling data replication from SAP OData, Amazon DynamoDB, and Salesforce to Amazon Redshift and Amazon S3 without custom pipelines.
AWS Glue Interactive Sessions now supports Apache Spark Connect, enabling users to develop and run Spark applications from their preferred environments on AWS Glue's serverless infrastructure without managing clusters