How AWS API Throttling Works - The Token Bucket Algorithm and the Truth Behind 429 Errors
Learn how AWS API rate limiting is implemented using the token bucket algorithm, understand the concept of burst capacity, explore differences in throttling limits across services, and discover practical strategies to avoid throttling.
Why AWS Applies Rate Limits to Every API
Every AWS API has per-account, per-region rate limits (throttling). The response you receive when a limit is exceeded differs by service, and there are broadly three families. First, the EC2 API returns RequestLimitExceeded with HTTP 503 (Service Unavailable). Second, API Gateway returns HTTP 429 (Too Many Requests). Third, the APIs of many AWS services return ThrottlingException with HTTP 400 (Bad Request). In other words, if you simplify this to "throttling always shows up as a 429," you miss the families that come back as 503 or 400. It is safer to make retry decisions not only from the HTTP status but also from the error code (RequestLimitExceeded / ThrottlingException / TooManyRequestsException, and so on). Rate limiting serves two purposes. First, it ensures fairness in a multi-tenant environment. If one account makes massive API calls, it can affect the performance of other accounts sharing the same infrastructure. Rate limits are guardrails that prevent the "noisy neighbor problem." Second, it protects customers themselves. Application bugs can cause infinite loops that call APIs tens of thousands of times per second. Without rate limits, such runaway behavior would lead to enormous bills. Rate limits function as a safety net for detecting unintended runaway behavior early. Depending on the service, rate limit values are made visible in Service Quotas and you can apply for an increase through a quota increase request. However, not every service and not every limit can be viewed or raised in this uniform way. Some limits, like the EC2 API throttling values, take the form of requesting access in order to see the current value or to have it raised at all, so at design time think in two tiers: if it appears in Service Quotas, you can check it there; if it does not, confirm it through Support.
How the Token Bucket Algorithm Works
AWS API throttling is implemented using the token bucket algorithm. This algorithm works by replenishing tokens (permits) into a bucket (container) at a constant rate, with each API request consuming one token. When the bucket is empty, requests are rejected. Let's look at actual values (the figures below are those stated in the official documentation as of August 2026 and may differ by account or region). The EC2 API throttling documentation shows that the request token bucket for non-mutating actions that involve neither filters nor pagination (such as DescribeInstances) has a maximum capacity of 50 tokens and a refill rate of 10 tokens per second, while other standard non-mutating actions have a maximum capacity of 100 tokens and a refill rate of 20 tokens per second. Taking DescribeInstances as an example, from a full bucket you can send 50 requests instantaneously (burst). Once they are used up, you are bound by the refill rate and settle into a pace of 10 requests per second. The point to grasp here is the division of roles. The maximum bucket capacity determines how much you can send at once during a burst, and the refill rate determines the rate that is allowed in steady state. If you mix up this correspondence, where burst equals capacity and steady state equals refill rate, your estimates will be off by a factor of several. Burst capacity is a buffer that absorbs short-term spikes. In patterns where APIs are called all at once at application startup, this capacity is what takes effect. Conversely, in a batch job that keeps running for a long time, the capacity only helps for the very first instant, and the effective throughput is determined by the refill rate.
Throttling Granularity Varies by Service
Throttling granularity varies significantly across services. EC2 APIs have individual rate limits set per API action. DescribeInstances and RunInstances are managed in separate buckets, so throttling on DescribeInstances does not affect RunInstances. In addition, EC2 has a "resource rate limit" on a separate axis from the request rate limit. Actions that create or modify resources, such as RunInstances, consume a resource token bucket in addition to the bucket for the number of requests. For RunInstances, the resource token bucket has a maximum capacity of 1,000 tokens and a refill rate of 2 tokens per second (the figures stated in the official documentation as of August 2026). The important point is that this bucket is depleted in proportion to the number of resources being operated on, not the number of API calls. Launching 100 instances with a single RunInstances call consumes 100 resource tokens, so you can be throttled on the resource rate limit side even with only a few calls. In automation that launches or terminates instances in bulk, always include this second axis in your estimates. DynamoDB throttling, on the other hand, is applied at the table level. Requests exceeding a table's provisioned capacity (RCU/WCU) are throttled. This is a data access throughput limit, different from API-level throttling. Lambda's concurrent execution limit is also a form of throttling. The default quota for an account's concurrent executions is 1,000, but newly created accounts start with a reduced quota, which AWS raises automatically according to usage (as stated in the official documentation as of August 2026). If you therefore design on the assumption that even a new account can reach 1,000 from the very start, you will hit unexpected throttling during load testing. The reliable way to know the current value is to check Service Quotas. API Gateway has an account-level rate limit of 10,000 requests per second (default), with additional throttling settings configurable per API, per stage, and per method. This multi-layered throttling ensures that concentrated access to a specific API endpoint does not affect other endpoints.
Exponential Backoff and Jitter - The Retry Strategy SDKs Handle Automatically
The correct response to a throttling error is to retry with a combination of exponential backoff and jitter. Exponential backoff is a strategy that increases retry intervals exponentially: 1 second, 2 seconds, 4 seconds, 8 seconds, and so on. This gradually reduces request pressure on the throttled service. Jitter adds random variation to retry intervals. With exponential backoff alone, multiple clients throttled simultaneously would retry at the same time, causing throttling again in a "thundering herd" problem. Adding jitter distributes the timing of retries. AWS SDKs automatically implement this retry strategy internally. The default for the AWS SDK for JavaScript v3 is "a maximum of 3 attempts," which means 1 initial request plus at most 2 retries. If you misread this as "retry 3 times," you will estimate one more attempt than actually happens, so take care. There are exceptions: the DynamoDB and DynamoDB Streams clients come with a more aggressive setting of 4 attempts and a base backoff of 25 milliseconds (these are the values under the new retry scheme introduced in 2026, which is gradually moving from an opt-in setting to the default). Retry behavior is not a matter of any single language; three modes, standard / adaptive / legacy, are defined as a setting common to all AWS SDKs. The current generation of SDKs and the CLI default to standard, having moved away from the earlier legacy mode. adaptive is a mode in which the client learns its own send rate and throttles it down, an option for workloads where throttling has become the norm. Furthermore, standard and adaptive include an overall control called the "retry quota." This is a token bucket held internally by the SDK client: each retry consumes tokens, and they recover little by little as requests succeed. Once the tokens run out, retries are cut off. In other words, the token bucket that is the subject of this article is used not only for rate limiting on the service side but also as a mechanism on the client side to keep retries from being scattered indiscriminately. If you call APIs directly without using an SDK, you need to implement these retry mechanisms yourself.
Design Patterns to Proactively Avoid Throttling
Rather than retrying after throttling occurs, the ideal approach is to design systems that prevent throttling in the first place. The first pattern is reducing API calls. Instead of calling EC2's DescribeInstances every second to monitor instance state, you can use EventBridge events (EC2 Instance State-change Notification) to receive notifications only when state changes occur. Shifting from polling to event-driven architecture dramatically reduces API call volume. The second pattern is leveraging caching. Information that doesn't change frequently (account settings, region lists, etc.) can be cached locally to reduce API calls. The third pattern is using batch APIs. DynamoDB's BatchGetItem can retrieve up to 100 items in a single API call. Compared to calling GetItem 100 times individually, this reduces API call count by 99%. S3's ListObjectsV2 can also retrieve up to 1,000 objects per request using the MaxKeys parameter.
References (Official AWS Resources)
The primary sources for this page are the official AWS website and documentation. Check the official pages below for the latest specifications and pricing.
If this page and the official documentation disagree, treat the official documentation as authoritative.