How to use the rate limit bucket simulator
- Enter the bucket size, the refill rate in tokens per second and, if you want, the tokens at the start. Leave the start blank for a full bucket.
- List your request times in seconds, one per line. Write time:cost for a request that costs more than 1 token.
- Read the headline for how many requests were allowed, then the table for each request.
- Change the bucket size or the refill rate and see how bursts and the long run average change.
Formula
tokens before a request = min(bucket size, tokens left + seconds since the last request x refill rate)
allowed if tokens before >= the request cost; then tokens left = tokens before - cost
retry time for a rejected request = request time + (cost - tokens) / refill rate
A rejected request takes no tokens. The bucket size sets the largest burst and the refill rate sets the long run average. Times must not go backward.
A burst of requests
The bucket holds 5 tokens and refills at 1 token per second. Requests arrive at 0, 0.1, 0.2, 0.3 and 0.4 seconds, using up 5 tokens in all, so all five are allowed and the bucket holds 0.4 tokens at 0.4 seconds. A request at 0.5 seconds finds only 0.5 tokens and is rejected, with a retry time of 1.0 seconds.
At 1.5 seconds the bucket holds 1.5 tokens and the request is allowed, leaving 0.5. At 2.0 it holds 1.0 and is allowed, leaving 0. Requests at 2.1 and 2.2 are rejected, and one at 5.0 is allowed. In all, 8 of the 11 requests are allowed.
Burst and average
A token bucket lets a client send a burst up to the bucket size, then slows it to the refill rate. This is why many APIs use it: short spikes are fine, but a steady flood is held to the average. A larger bucket allows bigger bursts, and a higher rate raises the long run limit.
Where services differ
Real services vary in the details. Some refill in whole tokens at fixed ticks instead of continuously, some count rejected requests, some share one bucket across several servers, and some use other methods such as a fixed window or a leaky bucket. Use this page to understand the behavior of one common model, then read your service's documentation. For related tools, see the Exponential Backoff Schedule Generator, the Uptime and Downtime Calculator and the Cron Expression Generator.
Assumptions and limits
- Refill is continuous and a request is allowed when the bucket holds at least its whole cost.
- A rejected request does not use tokens and is not retried by the model.
- Up to 2,000 requests are handled, in time order.
- The simulation is a model, not a test of any real service.
Frequently asked questions
What is the difference between bucket size and refill rate?
The size is the most tokens that can be saved up, which sets the biggest burst. The refill rate is how fast tokens return, which sets the average rate over a long time.
Can a request cost more than one token?
Yes. Write the time, a colon and the cost, such as 2.5:3. A costly request is allowed only when the bucket holds at least that many tokens.
What does the retry time mean?
It is the earliest time at which the bucket would hold enough tokens for that request, if no other request took tokens in the meantime.
Why did my list fail?
Request times must be numbers and must not go backward. A line with a cost of zero or a negative time is also rejected, and the message names the entry.