Demand-Driven Infrastructure with AWS Auto Scaling - Designing and Optimizing Scaling Policies
Learn how to use target tracking, predictive, and scheduled scaling policies effectively, and optimize costs with mixed instances policies that leverage Spot Instances.
Overview of Auto Scaling
Auto Scaling is a service that automatically scales resources based on demand. It adds instances when traffic increases and removes them when traffic decreases. This prevents both cost waste from over-provisioning and performance degradation from under-provisioning. It offers three types of scaling policies - target tracking, step, and predictive - which you can use according to your workload characteristics. Auto Scaling applies not only to EC2 but also to ECS services, DynamoDB tables, Aurora replicas, and SageMaker endpoints, though this article focuses on the most common use case: EC2 Auto Scaling Groups.
Designing Scaling Policies
Target tracking scaling is the most recommended policy. Simply set a target value such as 70% CPU utilization or 1,000 ALB requests per minute, and Auto Scaling automatically adjusts capacity. Internally, it auto-generates two CloudWatch Alarms (one for scale-out, one for scale-in) and incrementally adjusts instances as metrics deviate from the target. Predictive scaling uses ML to analyze the past 14 days of traffic patterns and pre-provisions capacity based on predicted future demand. For a pattern where traffic surges every morning at 9 AM, it begins scaling out at 8:50 AM. Warm pools keep instances pre-initialized from an AMI with application startup completed, ready to be placed into service immediately when scale-out occurs. Step scaling lets you configure different scaling amounts based on the degree of metric deviation—for example, adding 1 instance above 70% CPU and 3 instances above 90%—making it well-suited for graduated responses.
Predictive Scaling and Scheduled Scaling
Predictive scaling uses machine learning to analyze the past 14 days of metric patterns, predicting future demand and executing scaling actions in advance. It complements the reaction delay of target tracking policies (several minutes from metric collection to instance startup completion), enabling response to sudden traffic spikes. Scheduled scaling pre-provisions capacity for predictable demand changes, such as before daily business hours begin or at sale start times. Combining predictive and scheduled scaling is effective: predictive scaling covers regular patterns while scheduled scaling handles event-driven demand. Predictive scaling offers a forecast-only mode, letting you verify prediction accuracy without actually scaling before enabling it in production. To gain a deeper understanding of scaling design and implementation, specialized books on Amazon are a useful resource.
Cost Optimization with Auto Scaling
Using Spot Instances with mixed instances policies in Auto Scaling groups can achieve up to 90% cost savings compared to On-Demand pricing. Specify multiple instance types and use the capacity-optimized allocation strategy to distribute Spot interruption risk. A configuration that secures minimum capacity with On-Demand and covers excess demand with Spot provides an excellent balance of stability and cost. Setting up a warm pool keeps pre-initialized instances in a pool, reducing startup time during scale-out. Use CloudWatch custom metrics (queue depth, active connections) in scaling policies to achieve more precise scaling that does not rely solely on CPU utilization.
Design Pitfalls and Anti-Patterns
Understanding common Auto Scaling design issues helps avoid production incidents. First, overly aggressive scale-in can terminate instances with in-flight requests. Address this by configuring the ALB deregistration delay (connection draining) appropriately from the default 300 seconds and implementing graceful shutdown via instance lifecycle hooks. Second, health check design errors are common. EC2 status checks alone cannot detect states where the OS is healthy but the application is frozen; enable ELB health checks with an application-level /health endpoint. Third, setting symmetrical scale-out and scale-in thresholds causes flapping when metrics oscillate near the threshold. Set the scale-in threshold substantially lower than scale-out (e.g., out at 70%, in at 40%) and use a longer scale-in cooldown (300 seconds or more). Fourth, scaling within a single AZ is fragile for availability. Always distribute across multiple AZs and enable AZ rebalancing.
Choosing Between Kubernetes HPA/Karpenter and EC2 Auto Scaling
For container workloads, EKS Horizontal Pod Autoscaler (HPA) combined with Karpenter (node autoscaler) is an alternative to EC2 Auto Scaling. HPA performs horizontal scaling at the Pod level, and Karpenter automatically provisions nodes with appropriate instance types based on Pod resource requests. EC2 Auto Scaling requires pre-defining a candidate instance type list, whereas Karpenter dynamically selects optimal instances from Pod requirements, reducing instance selection effort. On the other hand, EC2 Auto Scaling is better suited for non-container workloads (AMI-based applications, GPU workloads) and offers features unavailable in EKS such as warm pools and predictive scaling. Lambda is fully managed with no scaling design required, but has constraints including a 15-minute execution limit and VPC cold start latency, making EC2 Auto Scaling more appropriate for long-running or stateful workloads.
Summary
Auto Scaling builds demand-driven infrastructure using three types of scaling policies: target tracking, step, and predictive. Verify prediction accuracy with predictive scaling's forecast-only mode before production enablement, and leverage Spot Instances through mixed instances policies for cost optimization. Prevent flapping and interruptions through deregistration delay configuration, ELB health checks, and asymmetric threshold settings for stable operations.