AWS Glue Data Catalog now supports table optimization, statistics, and crawlers for Apache Iceberg V3

AWS Glue Data Catalog now supports table optimization, statistics, and crawlers for Apache Iceberg version 3 tables, enabling automatic management, query performance optimization, and discovery in Amazon S3. Optimization uses binpack, sort, and z-order strategies to compress V3 tables, improve query performance, and reduce storage costs by removing expired snapshots and orphaned files. These optimizations support V3 data types like variant, geospatial, and nanosecond-precision timestamps, and can generate NDV statistics for efficient query planning by analytics engines. Glue crawlers can detect V3 tables stored in Amazon S3 and register them in the Glue Data Catalog for querying with V3-enabled engines. These features are available in all AWS regions where Glue Data Catalog table optimization, statistics, and crawlers are provided.

Why it matters

Data engineers and data analysts can use this feature when they need to efficiently manage and optimize the performance of Apache Iceberg V3 tables