Skip to content

Rate limiting

Rate limiting is a Redis-backed check in RateLimitMiddleware. A signed-in request is counted against its user, and any other request against its client IP. The middleware runs before the route's authentication, so it verifies the access token itself.

flowchart LR
    REQ[Request] --> EX{Excluded path?}
    EX -->|yes| APP[Route]
    EX -->|no| KEY[Key by user id or client IP]
    KEY --> CFG[Load config from Redis]
    CFG --> RULE{First matching rule}
    RULE -->|none| APP
    RULE -->|found| SW[Sliding window in a sorted set]
    SW -->|allowed| HDR[Add X-RateLimit headers] --> APP
    SW -->|over limit| REJ[429]

Which requests are checked

Every HTTP request goes through the middleware except these:

    EXCLUDED_PATHS = frozenset(
        {
            "/api/v1/auth/login",  # Auth endpoints handle their own limits
            "/api/v1/auth/register",
            "/api/v1/auth/logout",
        }
    )
    EXCLUDED_PREFIXES = ("/api/v1/health/",)

Login, register and logout are skipped by exact path, and the health prefix covers /api/v1/health/live. Setting RATE_LIMIT_ENABLED = false turns the whole check off, which config.test.toml does.

The bucket key is the user id from a valid access token, or the client IP when there's no token or it doesn't verify:

    @staticmethod
    def _extract_user_id(request: Request, security_service: SecurityService) -> str:
        """Key a signed-in request by its user id and any other by client IP.

        Only a token whose signature and expiry check out gives a user id, so a forged
        cookie can't pick another user's bucket or limits.
        """
        try:
            return security_service.decode_user_id(request.cookies.get("access_token", ""))
        except InvalidCredentialsError:
            return f"ip:{get_client_ip(request)}"

decode_user_id checks the signature and expiry and reads the token's user_id claim, which login sets. A forged or expired cookie fails that check, so it can't pick someone else's bucket. Tokens issued before the claim existed fall back to the IP until they expire. get_client_ip uses the socket peer address. It reads X-Forwarded-For (first entry) or X-Real-IP only when the peer is a loopback or private address, which is the case behind the frontend nginx. nginx sets both to the visitor's address and drops any X-Forwarded-For the client sent. Behind Cloudflare it takes that address from CF-Connecting-IP, and only on connections from Cloudflare's ranges (Nginx configuration).

Rules

RateLimitService loads a RateLimitConfig from the Redis key rate_limit:config, which has no expiry. If the key is missing, it uses RateLimitConfig.get_default_config() without storing it. Rules are sorted by priority, highest first, and the first pattern that matches the path wins:

Pattern Group Limit Priority
^/api/v1/execute execution 10 per 60s 10
^/api/v1/auth/.* auth 20 per 60s 7
^/api/v1/admin/.* admin 100 per 60s 5
^/api/v1/events/.* sse 5 per 60s 3
^/api/v1/ws websocket 5 per 60s 3
^/api/v1/.* api 60 per 60s 1

All six use the sliding window. The middleware matches the raw path before routing, so a rule also counts requests to URLs that don't exist. Since login, register and logout are excluded, /api/v1/auth/me is the only real route the auth rule covers. No route lives under /api/v1/ws, so the websocket rule only counts requests that end in 404.

The counter key is rate_limit:sw:<user id>:<path>, or rate_limit:sw:ip:<addr>:<path> for anonymous requests, with the raw request path. A limit therefore applies to each URL separately, not to the group: two different execution IDs under /api/v1/events/executions/ get two separate budgets of 5.

Algorithms

The sliding window keeps one sorted-set entry per request, scored by its timestamp. The member is the timestamp plus a random suffix, so two requests in the same instant still count twice. Each check drops entries older than the window, adds the current one, counts, and refreshes the TTL to twice the window:

        pipe = self.redis.pipeline()
        pipe.zremrangebyscore(key, 0, window_start)
        pipe.zadd(key, {f"{now}:{uuid4().hex}": now})
        pipe.zcard(key)
        pipe.expire(key, window_seconds * 2)
        results = await pipe.execute()

If the count is over the limit, the request is rejected and retry_after is computed from the oldest entry still in the window.

The token bucket runs as a Lua script that refills limit / window tokens per second up to limit * burst_multiplier and stores the state as JSON:

        result = await self.redis.eval(
            self._TOKEN_BUCKET_LUA,
            1,
            key,
            max_tokens,
            refill_rate,
            now,
            window_seconds * 2,
        )
        allowed = bool(result[0])
        tokens = float(result[1])

No default rule uses it, so only a per-user override with a token bucket rule reaches this path.

Responses

Allowed requests that go through the check get three headers. Requests where no rule matched get them too, with a limit of 999999. Excluded paths skip the check and get none, and so does every request while RATE_LIMIT_ENABLED is off.

Header Value
X-RateLimit-Limit limit of the matched rule
X-RateLimit-Remaining requests left in the window
X-RateLimit-Reset Unix timestamp of the window end
Retry-After seconds to wait, on 429 only (60 if unknown)

A rejected request gets 429 with the API's error body, and Retry-After and X-RateLimit-Reset say when to try again:

{"status": 429, "detail": "Rate limit exceeded"}

The service also counts rate_limit.requests.total, rate_limit.allowed.total, rate_limit.rejected.total and rate_limit.bypass.total, labelled with the path after IDs are replaced by *. The algorithm label on rate_limit.requests.total also takes bypassed, no_limit and redis_error.

Per-user overrides

The admin API (/api/v1/admin/rate-limits/{user_id}) stores a UserRateLimit in RateLimitConfig.user_overrides, keyed by the real user ID:

@dataclass
class UserRateLimit:
    user_id: str
    rules: list[RateLimitRule] = field(default_factory=list)
    global_multiplier: float = 1.0
    bypass_rate_limit: bool = False
    created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
    updated_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
    notes: str = ""

A user without an override gets the neutral defaults: no bypass, a multiplier of 1 and no own rules. The check applies the override of the user the request is keyed by, and the usage stats in the same admin view read the counters indexed under rate_limit:index:{user_id}, which the check writes for signed-in requests.

Failure mode

Rate limiting fails open. When a Redis call fails, the service logs the error, counts the request with algorithm="redis_error" and lets it through, so a Redis outage doesn't take the API down with it.

Settings

Two settings apply. RATE_LIMIT_ENABLED switches the middleware on or off, and RATE_LIMIT_REDIS_PREFIX (default rate_limit:) prefixes every key. The limits come from get_default_config().