Introduction

An API has finite capacity. Every request consumes resources such as CPU, memory, database connections, network bandwidth, downstream-service capacity, and operational budget.

Without request controls, one client, accidental retry loop, automated script, or traffic burst can consume enough capacity to reduce service quality for other clients.

Rate limiting controls how quickly a caller can consume an API. A complete rate-limit contract defines:

• Who or what is limited
• Which operations are included
• What unit is measured
• How much usage is allowed
• Which time window or replenishment model applies
• What happens when the limit is exceeded
• When the caller can retry
• Whether different plans receive different limits
• How batch, streaming, and asynchronous operations are charged
• How clients can observe remaining capacity

In your System Design curriculum, Rate-limit Semantics is Topic 4.9 and completes the API Design & Service Contracts module. It follows authentication versus authorization.

Prerequisites

What Is Rate Limiting?

Rate limiting restricts the amount of API usage permitted within a defined policy.

Caller sends request
        |
        v
Identify applicable rate-limit policy
        |
        v
Determine caller's current usage
        |
        +--> Capacity available:
        |       consume allowance
        |       process request
        |
        +--> Capacity exhausted:
                reject or delay request
                communicate retry guidance

A rate limit can measure requests, weighted operations, uploaded bytes, generated tokens, concurrent jobs, active connections, or another defined unit.

Rate Limit vs Quota vs Concurrency Limit

Why APIs Need Rate Limits

Rate limits can help:

• Protect service availability
• Prevent one caller from consuming all shared capacity
• Reduce accidental retry storms
• Control abusive automation
• Protect expensive downstream dependencies
• Provide predictable service tiers
• Control operational cost
• Encourage efficient client behaviour
• Protect authentication and recovery endpoints
• Maintain fairness between tenants

Rate-limit Dimensions

A policy must identify the dimension or partition to which consumption is charged.

Multiple Limits Can Apply

Incoming request
      |
      +--> Per-IP limit
      |
      +--> Per-user limit
      |
      +--> Per-tenant limit
      |
      +--> Per-endpoint limit
      |
      +--> Global capacity limit
      |
      v
Request proceeds only when
all required policies allow it.

A request can pass one budget and fail another. The response contract should give clients useful guidance without exposing sensitive internal capacity information.

HTTP 429 Too Many Requests

The HTTP 429 Too Many Requests status indicates that the client has sent too many requests within the applicable amount of time.

HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 30

{
  "type": "rate-limit-exceeded",
  "title": "Request rate exceeded",
  "status": 429,
  "detail": "Retry after the indicated delay.",
  "limitScope": "orders.write",
  "traceId": "trace-8f21"
}

The error representation should explain the condition safely. The response can include Retry-After to indicate how long the client should wait before making a follow-up request.

Retry-After

The Retry-After response field communicates when the client can attempt a follow-up request.

Delay in Seconds

Retry-After: 30

This form tells the client to wait the specified number of seconds.

HTTP Date

Retry-After: Wed, 23 Sep 2026 08:30:00 GMT

This form communicates an absolute HTTP date. Clients need to account for clock differences when interpreting an absolute date.

Rate-limit Response Contract

A useful contract can communicate:

• The HTTP status
• A stable machine-readable error type
• A safe human-readable explanation
• The retry delay when known
• The general limit scope
• A trace identifier
• Documented quota metadata where supported

{
  "type": "rate-limit-exceeded",
  "title": "Request rate exceeded",
  "status": 429,
  "detail": "The write-operation allowance is temporarily exhausted.",
  "retryAfterSeconds": 30,
  "limitScope": "orders.write",
  "traceId": "trace-8f21"
}

The response should not include internal server topology, private customer usage, or security-sensitive traffic thresholds.

Rate-limit Metadata Fields

APIs in production use different families of response fields. Some use provider-specific fields, while current IETF work defines structured RateLimit and RateLimit-Policy fields.

Illustrative Provider-specific Fields

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 12
X-RateLimit-Reset: 1758616200

These field names are widely recognizable but are not consistent across all providers. The reset value can also use different units or meanings, so the provider contract must be followed.

Illustrative Structured Policy

RateLimit-Policy: "standard";q=100;w=60
RateLimit: "standard";r=12;t=30

When using a draft or provider-specific rate-limit field family, document the exact syntax, version, quota unit, window, remaining budget, and reset semantics used by the API.

Request-based Limits

A simple policy charges one unit for each accepted request.

Policy:

100 requests per minute


Each request cost:

1 unit

This model is easy to understand but treats inexpensive and expensive operations as if they consume equal capacity.

Weighted Limits

A weighted policy charges different costs for different operations.

Weighted costs are application policy examples. Production values should be selected from measured resource consumption and documented client needs.

Batch-request Charging

A batch endpoint can contain many operations in one HTTP request. Charging only one unit for the outer request can allow callers to bypass the intended budget.

{
  "operations": [
    {
      "type": "getOrder",
      "orderId": "ord_1001"
    },
    {
      "type": "getOrder",
      "orderId": "ord_1002"
    },
    {
      "type": "getOrder",
      "orderId": "ord_1003"
    }
  ]
}

A batch contract should define whether it is charged by:

• Outer HTTP request
• Inner operation
• Returned item
• Weighted operation cost
• Request and response size

Fixed-window Limiting

A fixed-window policy counts usage within discrete time intervals.

Window 1:

10:00:00 through 10:00:59


Window 2:

10:01:00 through 10:01:59

Advantages

• Simple to understand
• Simple counter structure
• Easy expiration by window

Limitations

• Traffic can burst around a window boundary
• Clients can consume one full window immediately before reset and another immediately after reset
• Global synchronization can create reset-time traffic spikes

Sliding-window Log

A sliding-window log records request times and counts events within the continuously moving period preceding the current request.

Current time:

10:05:30


Window:

Requests after 10:04:30


Process:

Remove older timestamps
Count timestamps in active window
Allow or reject current request

Benefit

It can represent the rolling time-window policy accurately.

Consideration

Storing individual request timestamps can consume substantial memory for high-volume callers.

Sliding-window Counter

A sliding-window counter approximates rolling-window usage by combining counts from time segments.

Previous interval count
        |
        | weighted by overlap
        v
Estimated active usage
        ^
        |
Current interval count

This approach can reduce storage compared with recording every request. Exact behaviour and approximation error depend on the selected algorithm.

Token-bucket Algorithm

A token bucket contains a bounded number of tokens. Tokens are replenished over time, and a request consumes one or more tokens.

Bucket capacity:

10 tokens


Replenishment:

1 token per second


Request cost:

1 token


When token is available:

Consume token
Allow request


When no token is available:

Reject or delay request

Token buckets allow controlled bursts up to the bucket capacity while enforcing an average replenishment rate.

Conceptual Token Calculation

\[ AvailableTokens = \min \left( Capacity, PreviousTokens + ElapsedTime \times RefillRate \right) \]

The request is accepted when the available token count is at least the request cost.

Leaky-bucket Algorithm

A leaky-bucket design places accepted work into a bounded queue and releases work at a controlled rate.

Incoming requests
        |
        v
Bounded queue
        |
        v
Controlled release rate
        |
        v
Backend service


Queue full:

Reject new request

This design smooths bursts but can add queueing latency. The complete contract should define maximum queue depth, waiting time, and rejection behaviour.

Algorithm Comparison

Distributed Rate Limiting

Client requests
      |
      v
Load Balancer
      |
      +--> API Instance A
      |
      +--> API Instance B
      |
      +--> API Instance C
              |
              v
       Shared Rate-limit Store

When several instances handle requests for the same caller, independent in-memory counters can allow the caller to receive a separate allowance from each instance.

A distributed design should consider:

• Atomic counter updates
• Counter expiration
• Clock behaviour
• Store latency
• Store availability
• Regional coordination
• Counter cardinality
• Fail-open or fail-closed policy

Local vs Shared Limits

Multi-region Rate Limiting

A globally strict limit can require cross-region coordination, increasing request latency and creating a shared dependency.

Global allowance:

1,000 units


Possible regional allocation:

Region A receives a defined share.
Region B receives a defined share.
Region C receives a defined share.


Periodic process:

Rebalance regional budgets
according to measured demand.

Alternatively, the service can accept bounded approximation and reconcile usage asynchronously. The appropriate choice depends on whether the limit is for fairness, cost control, security protection, or a strict contractual quota.

Rate Limiter Failure Policy

A service must define what happens when the rate-limit store or policy service is unavailable.

Sensitive login, recovery, financial, or abuse-prevention limits can require a different failure policy from ordinary read-traffic fairness limits.

Security Rate Limits

Some rate limits specifically reduce credential guessing, enumeration, or automated abuse.

Examples include limits on:

• Login attempts
• Password-reset requests
• One-time-code verification
• Account-recovery attempts
• Registration attempts
• Invitation creation
• Search and enumeration endpoints
• Token issuance

Authentication and Rate-limit Identity

For authenticated requests, the limit should normally be associated with a trusted server-derived user, tenant, client, or workload identity.

Rate-limit identifier:

X-User-ID request header supplied by client

Rate-limit identifier:

Tenant and subject extracted from
a validated security context

Otherwise, a caller can alter the supplied identity to receive additional allowance or consume another caller's budget.

Service Plans and Entitlements

Different plans can receive different usage policies.

The actual values should come from measured capacity and approved service commitments. The policy must not rely on an editable plan value supplied by the caller.

Fairness within a Tenant

A tenant-wide limit can allow one internal workload to consume all of the organization's capacity.

Tenant budget
      |
      +--> Interactive user traffic
      |
      +--> Background synchronization
      |
      +--> Reporting jobs
      |
      +--> Administrative operations

The service can introduce sub-budgets, priorities, or workload classes so background activity does not starve interactive operations.

Priority and Admission Control

During overload, a system can admit requests based on workload priority and available capacity.

Incoming request
      |
      v
Classify workload
      |
      +--> Critical control operation
      |
      +--> Interactive request
      |
      +--> Background synchronization
      |
      +--> Bulk export
      |
      v
Apply class-specific budget
and admission policy

Priority should be determined by trusted server-side policy. A caller should not gain preferred treatment merely by setting an unverified priority field.

Conceptual Fixed-window Table

CREATE TABLE rate_limit_windows
(
    partition_key VARCHAR(200) NOT NULL,
    policy_name VARCHAR(100) NOT NULL,
    window_start DATETIME NOT NULL,
    consumed_units INT NOT NULL,
    expires_at DATETIME NOT NULL,

    PRIMARY KEY
    (
        partition_key,
        policy_name,
        window_start
    ),

    INDEX ix_rate_limit_expiry
    (
        expires_at
    )
);

A high-volume distributed implementation commonly requires a purpose-built shared counter mechanism rather than a normal relational row update for every request. The storage choice should be validated against required throughput, atomicity, latency, and availability.

PHP Fixed-window Example

<?php

declare(strict_types=1);

final class RateLimitDecision
{
    public function __construct(
        public readonly bool $allowed,
        public readonly int $limit,
        public readonly int $remaining,
        public readonly int $retryAfterSeconds
    ) {
    }
}

function checkRateLimit(
    PDO $pdo,
    string $partitionKey,
    string $policyName,
    int $limit,
    int $windowSeconds
): RateLimitDecision {
    if ($limit < 1 ||
        $windowSeconds < 1) {

        throw new InvalidArgumentException(
            'Invalid rate-limit policy.'
        );
    }

    $now =
        time();

    $windowStart =
        intdiv(
            $now,
            $windowSeconds
        ) * $windowSeconds;

    $windowStartText =
        gmdate(
            'Y-m-d H:i:s',
            $windowStart
        );

    $expiresAtText =
        gmdate(
            'Y-m-d H:i:s',
            $windowStart +
            $windowSeconds +
            60
        );

    $pdo->beginTransaction();

    try {
        $insert =
            $pdo->prepare(
                '
                INSERT INTO rate_limit_windows
                (
                    partition_key,
                    policy_name,
                    window_start,
                    consumed_units,
                    expires_at
                )
                VALUES
                (
                    :partition_key,
                    :policy_name,
                    :window_start,
                    1,
                    :expires_at
                )
                ON DUPLICATE KEY UPDATE
                    consumed_units =
                        consumed_units + 1
                '
            );

        $insert->execute([
            'partition_key' =>
                $partitionKey,
            'policy_name' =>
                $policyName,
            'window_start' =>
                $windowStartText,
            'expires_at' =>
                $expiresAtText
        ]);

        $read =
            $pdo->prepare(
                '
                SELECT
                    consumed_units
                FROM rate_limit_windows
                WHERE
                    partition_key =
                        :partition_key
                    AND policy_name =
                        :policy_name
                    AND window_start =
                        :window_start
                FOR UPDATE
                '
            );

        $read->execute([
            'partition_key' =>
                $partitionKey,
            'policy_name' =>
                $policyName,
            'window_start' =>
                $windowStartText
        ]);

        $consumed =
            (int)$read->fetchColumn();

        $pdo->commit();

        $remaining =
            max(
                0,
                $limit - $consumed
            );

        $retryAfter =
            max(
                1,
                ($windowStart +
                    $windowSeconds) -
                $now
            );

        return new RateLimitDecision(
            allowed:
                $consumed <= $limit,
            limit:
                $limit,
            remaining:
                $remaining,
            retryAfterSeconds:
                $retryAfter
        );
    } catch (Throwable $exception) {
        if ($pdo->inTransaction()) {
            $pdo->rollBack();
        }

        throw $exception;
    }
}

This example demonstrates contract logic rather than a recommended high-scale architecture. Database-specific upsert and locking behaviour must be verified for the selected platform.

PHP 429 Response

<?php

declare(strict_types=1);

function enforceRateLimit(
    RateLimitDecision $decision,
    string $limitScope
): void {
    header(
        'X-RateLimit-Limit: ' .
        $decision->limit
    );

    header(
        'X-RateLimit-Remaining: ' .
        $decision->remaining
    );

    if ($decision->allowed) {
        return;
    }

    http_response_code(429);

    header(
        'Content-Type: application/problem+json'
    );

    header(
        'Retry-After: ' .
        $decision->retryAfterSeconds
    );

    echo json_encode(
        [
            'type' =>
                'rate-limit-exceeded',
            'title' =>
                'Request rate exceeded',
            'status' => 429,
            'detail' =>
                'Retry after the indicated delay.',
            'limitScope' =>
                $limitScope
        ],
        JSON_THROW_ON_ERROR
    );

    exit;
}

The response-field family should match the published contract. Do not mix fields with incompatible meanings or undocumented reset units.

Client Handling of 429

Receive HTTP 429
      |
      v
Read Retry-After
      |
      +--> Valid value:
      |       wait for indicated delay
      |
      +--> Missing or invalid:
              use bounded exponential backoff
              with randomized jitter
      |
      v
Check overall operation deadline
      |
      +--> Deadline exceeded:
      |       stop retrying
      |
      +--> Deadline remains:
              retry only when operation is safe

A client should not place a 429 response into an immediate retry loop. Immediate retries consume more network and server capacity while the limit remains exhausted.

Exponential Backoff with Jitter

A conceptual retry delay is:

\[ Delay_n = \min \left( MaximumDelay, BaseDelay \times 2^n \right) + Jitter \]

When a valid Retry-After value is present, the client should respect that server guidance according to the API contract.

Retry Safety and Idempotency

A client must also determine whether the rejected or uncertain operation is safe to repeat.

GET request receives 429:

Wait and retry according to policy.


POST payment request receives 429
before processing:

Retry only according to the documented
idempotency-key contract.


POST request times out:

Do not infer from the timeout that
the operation was not completed.

Rate-limit handling and idempotency should be designed together for state-changing operations.

Prevent the Thundering Herd

Service becomes available
at one reset time
        |
        v
Thousands of clients retry simultaneously
        |
        v
Immediate traffic spike
        |
        v
Service is overloaded again

Controls can include:

• Randomized retry jitter
• Distributed reset times
• Token replenishment rather than hard resets
• Client-side pacing
• Queue-based admission
• Gradual recovery of allowed capacity

GraphQL Rate Limiting

Counting every GraphQL request as one unit can be misleading because operations can have very different resolver and data-source costs.

query ExpensiveOperation {
  customers(first: 100) {
    nodes {
      orders(first: 100) {
        nodes {
          lines(first: 100) {
            nodes {
              product {
                supplier {
                  id
                  name
                }
              }
            }
          }
        }
      }
    }
  }
}

A GraphQL protection model can combine:

• Request-rate limits
• Operation-complexity limits
• Maximum depth
• Maximum aliases
• Maximum page sizes
• Resolver deadlines
• Concurrent-operation limits
• Persisted-operation budgets

Weighted GraphQL Cost

Total operation cost
=
Selected scalar-field costs
+
Object-field costs
+
List-size multipliers
+
Expensive resolver weights

A shallow operation can still be expensive when it requests large lists or fields backed by costly downstream calls.

gRPC Rate Limiting

gRPC services can limit unary calls, stream creation, messages per stream, active streams, or weighted method cost.

message RateLimitInfo {
  string policy = 1;
  int64 limit = 2;
  int64 remaining = 3;
  int64 retry_after_seconds = 4;
}

A gRPC contract can communicate resource exhaustion through an appropriate status and defined error details.

Unary RPC:

Charge per method invocation.


Client stream:

Charge per stream,
per message,
per byte,
or weighted combination.


Server stream:

Limit stream creation,
active duration,
and delivered event volume.


Bidirectional stream:

Limit connections,
messages,
and concurrent subscriptions.

Streaming and Connection Limits

A client can open one persistent connection while consuming substantial resources through subscriptions or messages.

Streaming limits can include:

• Connections per user or tenant
• Subscriptions per connection
• Messages per second
• Bytes per second
• Queued outbound bytes
• Maximum stream lifetime
• Concurrent streams
• Reconnect frequency

Asynchronous Job Limits

Long-running operations should often use both a creation rate and an active concurrency limit.

Report-generation policy:

Creation limit:
Controls how quickly jobs are submitted.

Concurrency limit:
Controls how many jobs run simultaneously.

Daily quota:
Controls total resource consumption.

Payload limit:
Controls the size of each report request.

A request can be below the submission rate while still exceeding the permitted number of active jobs.

Soft and Hard Limits

A service can combine a burst limit and a sustained limit to support normal interactive traffic while preventing continuous high-volume consumption.

Adaptive Rate Limiting

A static limit does not account for changing service health. An adaptive admission policy can reduce allowed load when dependencies slow down or system saturation increases.

Observe service health:

- Latency
- Error rate
- Queue depth
- CPU
- Database saturation


Adjust admission:

Healthy:
Normal budget

Degraded:
Reduced budget

Critical:
Protect essential operations only

Adaptive policies need stable bounds, testing, observability, and safeguards against rapid oscillation.

Protect Downstream Dependencies

The public API can appear healthy while an internal database or downstream service approaches its capacity limit.

Public request allowance
        |
        v
Application service
        |
        +--> Database connection pool
        |
        +--> Payment provider
        |
        +--> Search service
        |
        +--> Messaging system

Admission policies should consider the most constrained dependency in the request path. Otherwise, the API can accept more work than its downstream systems can complete.

Rate Limiting Is Not Complete Abuse Prevention

Rate limiting is one security and reliability control. It does not replace:

• Authentication
• Authorization
• Input validation
• Fraud detection
• Bot detection
• Account-lockout safeguards
• Network-level protection
• Resource-specific business rules
• Monitoring and incident response

Rate-limit Observability

Useful metrics include:

• Allowed requests by policy
• Rejected requests by policy
• 429 response rate
• Budget consumption by tenant or client
• Remaining-budget distribution
• Retry volume after 429
• Requests arriving before Retry-After expires
• Rate-limit store latency
• Rate-limit decision errors
• Fail-open or fallback activations
• Concurrent-operation count
• Queue depth
• Top expensive operations
• False-positive support incidents

High-cardinality caller identifiers should be handled carefully in metrics. Detailed caller-level evidence can remain in protected logs rather than becoming an unbounded metric dimension.

Rate-limit Audit Logs

A rate-limit decision log can include:

• Policy name
• Trusted partition identifier
• Operation classification
• Request cost
• Decision
• Remaining allowance
• Retry delay
• Fallback mode
• Trace identifier

Logs should not expose raw access tokens, passwords, session identifiers, or unnecessary personal data.

Test Rate Limits with curl

Controlled Request Loop

for request_number in 1 2 3 4 5
do
  echo "Request ${request_number}"

  curl -i \
    -H 'Accept: application/json' \
    https://api.example.com/v1/orders

  echo
done

Inspect Response Fields

curl -sS \
  -D - \
  -o /dev/null \
  https://api.example.com/v1/orders

Perform load and throttling tests only against approved test environments. Uncontrolled rate-limit tests can disrupt other users or trigger security controls.

Troubleshooting Workflow

1. Capture one complete failing request and response safely.
2. Confirm the endpoint and final path after redirects.
3. Confirm the trusted caller, tenant, application, or IP partition.
4. Identify which rate policy was exceeded.
5. Identify the measured unit and request cost.
6. Inspect Retry-After and documented metadata fields.
7. Confirm whether reset values are delays, dates, or provider-specific timestamps.
8. Inspect whether the client retried immediately.
9. Check for shared retry loops across application instances.
10. Check concurrency, queue depth, and downstream capacity.
11. Inspect rate-limit store latency and errors.
12. Compare runtime behaviour with the published contract.

Common Rate-limit Mistakes

Recommended Test Cases

Rate-limit Best Practices

Practice Exercise

Add rate-limit semantics to the versioned order and payment API developed in the earlier lessons.

Requirements

1. Create a per-tenant read limit.
2. Create a stricter per-user write limit.
3. Create a separate payment-operation limit.
4. Use trusted authenticated identities for partitions.
5. Implement a token-bucket or fixed-window policy.
6. Return a documented 429 error representation.
7. Include valid Retry-After guidance.
8. Document all rate-limit metadata fields.
9. Add a maximum active-job limit for report generation.
10. Charge batch operations by inner request count or weight.
11. Use idempotency keys for retried payment requests.
12. Coordinate limits across multiple API instances.
13. Define limiter-store failure behaviour.
14. Add client backoff with randomized jitter.
15. Measure allowed, rejected, replayed, and retried requests.

Policy Design Template

Frequently Asked Questions