Amazon EMR Updates

Amazon EMR on EKS reduces EMR charge by 30 percent

Amazon EMR announces a 30 percent reduction in the EMR charge added to EKS when running EMR on EKS, applying to both per-vCPU-hour and per-GB-hour charges while EKS charges remain unchanged.

• Amazon EKS

Amazon EMR Serverless now supports IPv6 and dual-stack connectivity

Amazon EMR Serverless now supports IPv6 and dual-stack connectivity, expanding beyond IPv4-only connectivity and maintaining backward compatibility with existing IPv4 workloads.

Amazon EMR now supports native data quality checks in Apache Spark jobs

Amazon EMR validates data quality natively within Apache Spark jobs, allowing data engineering teams to catch quality issues inline before bad data reaches downstream systems, using AWS Glue-powered rules without additional cost

• AWS Glue

Apache Iceberg materialized views now support system-managed write protection

AWS now offers system-managed materialized views for Apache Iceberg, ensuring computed results remain unchanged and can be safely shared across users.

• AWS Glue

Amazon EMR Serverless now supports terabyte-scale shuffle

Amazon EMR Serverless increases per-job shuffle limit from 200 GB to 1TB, enabling terabyte-scale data processing.

Amazon EMR on EKS now supports IPv6 Amazon EKS clusters

Amazon EMR on EKS now supports running workloads on IPv6 Amazon EKS clusters, enabling large-scale Spark and Flink workloads to scale using the expanded IPv6 address space

Amazon EMR 7.14 is now available

Amazon EMR 7.14 delivers new features and application upgrades across EC2, EKS, and Serverless platforms, including performance improvements for Spark and Iceberg and increased storage limits for Spark jobs.

Amazon EMR introduces Long Term Support with Apache Spark 4.1

Amazon EMR introduces LTS releases, starting with emr-spark-8.1.0 and Apache Spark 4.1, providing 36 months of support for designated versions with critical fixes.

Amazon EMR on EKS now supports job run concurrency controls

Amazon EMR on EKS adds concurrency and queue limits for job runs on virtual clusters

Amazon EMR on EC2 now supports Spark Connect for interactive workloads

Amazon EMR on EC2 supports Spark Connect, enabling interactive Apache Spark sessions from SageMaker Unified Studio and IDEs for development and debugging.

Amazon EMR on EKS now supports Apache Spark troubleshooting agent

Amazon EMR on EKS provides natural language job failure diagnosis and automated root cause analysis. It also integrates with MCP-compatible AI coding agents.

Amazon EMR Serverless now supports larger worker sizes for compute and memory intensive workloads

Amazon EMR Serverless now offers larger worker configurations with 32 vCPUs and up to 244 GB of memory, enabling more compute and memory-intensive workloads

Amazon EMR Serverless now supports live configuration updates without application restarts

Amazon EMR Serverless allows updates to key application configurations such as maximum capacity and custom image settings without stopping and restarting the application.

Amazon EMR Serverless Now Supports Interactive Sessions with Spark Connect

Amazon EMR Serverless now supports interactive sessions with Spark Connect, enabling development and execution of Apache Spark applications from managed notebooks in SageMaker Unified Studio and other notebook environments or IDEs, with monitoring, debugging, and cost visibility for individual sessions