Amazon ECS now supports fractional GPU scheduling with Amazon EC2 G6f instances
Amazon ECS enables GPU partitioning as small as one-eighth of an NVIDIA L4 GPU on G6f instances, ideal for small AI inference and model experimentation
Amazon Elastic Container Service (Amazon ECS) now supports fractional GPU scheduling with Amazon EC2 G6f instances, allowing you to run workloads on GPU partitions as small as one-eighth of an NVIDIA L4 Tensor Core GPU with 3 GB of GPU memory. Fractional GPUs provide the flexibility to right-size containers for small-model AI inference, model experimentation, graphics rendering, and other workloads that do not require a full GPU, helping reduce infrastructure costs compared to provisioning a full GPU instance. You can request a fractional GPU by setting GPU=0.125, GPU=0.25, or GPU=0.5 in the container definition of your Amazon ECS task definition. Amazon ECS then places the task on a G6f instance that satisfies the request. Fractional GPU configuration is supported on both Amazon ECS Managed Instances and Amazon ECS on EC2. With ECS Managed Instances, you get a fully managed experience where ECS automatically handles instance provisioning, scaling, patching, and lifecycle management, allowing you to focus on your GPU workloads rather than infrastructure operations. ECS Managed Instances also include capabilities built specifically for accelerated workloads, such as GPU metrics through Amazon CloudWatch Container Insights, and automatic health monitoring that detects GPU hardware failures and replaces unhealthy instances to minimize workload disruption. This capability is available in all AWS Regions where Amazon EC2 G6f instances are available.