AWS Glue announces VPC support, filter pushdown, and partition support for the REST API connector
AWS Glue adds VPC connectivity, filter pushdown, and partition support to the REST API connector, enabling secure access to private endpoints, reduced data transfer costs, and faster ingestion through parallel reads
AWS Glue now supports VPC connections, filter pushdown, and partition support for the REST API connector. The REST API connector allows you to ingest data from any source exposing a REST-based API, including proprietary systems and emerging platforms without native AWS Glue connectors. With VPC support, you can access data sources hosted in private subnets or connected through VPNs or AWS PrivateLink without exposing traffic to the public internet. Filter pushdown translates query predicates into API-native parameters, ensuring only matching records leave the source, reducing data transfer costs and improving job performance. Partition support splits large datasets across multiple Spark workers using field-based or record-count strategies, providing parallel reads that reduce ingestion time for high-volume, paginated APIs. These capabilities are available in all AWS commercial regions where AWS Glue is offered.