Multi-tier distributed rate limiting with three atomic algorithms, weighted costs, bans, and dry-run rollout

Rate Limiter

Working example → Multi-tier limits, weighted costs, penalty bans, token-bucket, VIP overrides, and middleware: apps/backend/src/services/rate-limiters.ts

RateLimit() defines a distributed rate limiter. Every counter runs as an atomic script on the hot backend (a single Lua round-trip on Redis, in-process in memory mode) — checks are race-free across every node, with no read-modify-write windows.

The rate limiter is a pure data-plane module: counters, bans, and overrides live on the hot backend; the durable plane stores only the limiter definition and pause state. It creates no jobs and no runs.

Quick start

limits/api.ts

import { RateLimit } from "oqronkit";

type Ctx = { userId: string; ip: string };

export const apiLimit = RateLimit<Ctx>({
  name: "api",
  algorithm: "sliding-window", // 'fixed-window' | 'sliding-window' | 'token-bucket'
  tiers: [
    { name: "per-user", key: (c) => c.userId, max: 100, windowMs: 60_000 },
    { name: "per-ip", key: (c) => c.ip, max: 300, windowMs: 60_000 },
  ],
});

middleware.ts

const result = await apiLimit.check({ userId, ip });

if (!result.allowed) {
  res.setHeader("Retry-After", Math.ceil((result.resetMs - Date.now()) / 1000));
  return res.status(429).json({ error: "rate limited", tier: result.tier });
}

check() returns a RateLimitResult:

{
  allowed: boolean
  remaining: number   // tokens left on the most-constrained tier
  resetMs: number     // absolute epoch ms when the blocking window resets
  tier?: string       // which tier blocked (absent when allowed)
  banned: boolean     // true when the key is currently banned
  dryRun?: boolean    // present when dryRun mode would have blocked
}

Tiers

Tiers are evaluated in order — the first tier that blocks wins and is reported in result.tier. Each tier derives its own key from the check context, so one limiter enforces per-user, per-IP, and global limits simultaneously:

tiers: [
  { name: "burst", key: (c) => c.userId, max: 10, windowMs: 1_000 },
  { name: "hourly", key: (c) => c.userId, max: 1_000, windowMs: 3_600_000 },
  { name: "global", key: () => "all", max: 50_000, windowMs: 60_000 },
];

Algorithms

All three run atomically on the backend and share the same result shape. Memory and Redis implementations are contract-tested for identical behavior.

Algorithm	Behavior	Choose when
`fixed-window` (default)	N per window; resets at a fixed boundary	Simplest; boundary bursts acceptable
`sliding-window`	Two-bucket weighted estimate — smooths boundary bursts	General-purpose API limiting
`token-bucket`	Capacity `max`, refilled `refillRate` tokens every `refillIntervalMs`	Bursty traffic with a sustained average rate

token bucket

RateLimit<Ctx>({
  name: "exports",
  algorithm: "token-bucket",
  tiers: [
    {
      name: "per-user",
      key: (c) => c.userId,
      max: 20, // bucket capacity (burst)
      refillRate: 5, // +5 tokens…
      refillIntervalMs: 60_000, // …every minute (sustained rate)
    },
  ],
});

Weighted costs

A check consumes 1 token by default. Heavy operations can consume more — per call or derived from context:

// Per call
await apiLimit.check(ctx, { cost: 5 });

// Derived automatically
RateLimit<Ctx>({
  name: "search",
  costEstimator: (c) => (c.deep ? 10 : 1),
  tiers: [{ name: "t", key: (c) => c.userId, max: 100, windowMs: 60_000 }],
});

Explicit opts.cost wins over costEstimator.

Reading usage without consuming

getUsage() peeks — internally cost: 0, which the atomic scripts treat as "report state, consume nothing":

const usage = await apiLimit.getUsage("user-123");
// [{ tier: 'per-user', remaining: 84, resetMs: 1751673660000 }, ...]

const status = await apiLimit.getStatus("user-123");
// { key, usage, override?, banned, banExpiresAt? }

Overrides (VIP limits)

Per-key overrides replace the tier's max for that key. Stored on the hot backend — effective on the next check across all nodes:

await apiLimit.setOverride("enterprise-user", { max: 10_000 });
await apiLimit.clearOverride("enterprise-user");

Penalty escalation & bans

Repeatedly-blocked keys can be banned automatically. Bans live on the hot backend with a native TTL — they expire without timers or cleanup jobs:

RateLimit<Ctx>({
  name: "login",
  tiers: [{ name: "attempts", key: (c) => c.ip, max: 5, windowMs: 60_000 }],
  penalty: {
    threshold: 10, // 10 blocked checks…
    violationWindowMs: 60_000, // …within one minute…
    banDurationMs: 900_000, // …bans the key for 15 minutes
    onBan: (key, tier) => alerting.notify(`banned ${key} (${tier})`),
    onUnban: (key) => alerting.notify(`unbanned ${key}`),
  },
});

Manual controls:

await apiLimit.ban("abusive-key", 3_600_000);
await apiLimit.unban("abusive-key");
await apiLimit.reset("key"); // clears ban + override + violation counter

reset() clears bans, overrides, and violation counters. In-flight window counters expire naturally with their window — they cannot be cleared mid-window.

Dry-run rollout

Roll a limiter out safely: dryRun: true evaluates everything — counters consume, events emit, hooks fire, bans record — but every check returns allowed: true, with dryRun: true set whenever it would have blocked. Watch stats() and the logs, tune your tiers, then flip the flag:

RateLimit<Ctx>({ name: "api", dryRun: true, tiers: [/* ... */] });

Skip conditions, hooks, failOpen

RateLimit<Ctx>({
  name: "api",
  // Bypass entirely (no tokens consumed) for internal traffic
  skip: (c) => c.userId.startsWith("svc-"),

  hooks: {
    onLimit: (ctx, result) =>
      metrics.increment("rl.blocked", { tier: result.tier }),
    onPass: (ctx, result) => {},
  },

  // Backend unreachable → allow (default true). Set false to block instead.
  failOpen: true,
  tiers: [/* ... */],
});

failOpen: true (the default) means a Redis outage lets traffic through rather than taking your API down. Set failOpen: false only where blocking on backend failure is genuinely safer than allowing.

Observability

Events (always emitted on oqron.eventBus): ratelimit:blocked, ratelimit:banned, ratelimit:unbanned, ratelimit:override, ratelimit:instance:enabled/disabled.
Stats — this node's counters: await apiLimit.stats() → { allowed, blocked, banned }.
Persisted logs (opt-in) — see Observability & Control:

RateLimit<Ctx>({
  name: "api",
  logs: { level: "info", maxEntries: 1000, maxWritesPerSec: 50 },
  tiers: [/* ... */],
});

Control & environments

// Only enforce in production (definition is inert everywhere else)
RateLimit<Ctx>({
  name: "api",
  environments: ["production"],
  tiers: [/* ... */],
});

The engine exposes the standard admin surface — list(), get(name), pauseInstance(name), resumeInstance(name). A paused limiter allows everything — a disabled protection mechanism must not take down traffic.

Semantics summary

Situation	Result
All tiers pass	`allowed: true`, `remaining` = most-constrained tier
A tier blocks	`allowed: false`, `tier` set
Key banned	`allowed: false`, `banned: true`, `resetMs` = ban expiry
`dryRun: true`	evaluation identical, result forced `allowed: true` + `dryRun` flag
`skip(ctx)` true	`allowed: true`, nothing consumed
Limiter paused	`allowed: true`, nothing consumed
Backend error	`failOpen` decides (`true` → allow, `false` → block)

RateLimit module vs queue `throttle`

Queues and workers have their own inline throughput controls — different tools for different jobs:

	Queue `throttle`	`RateLimit()` module
Scope	Per process — each node independently	Cluster-wide — shared atomic counters
Where	Pre-claim, inside the queue engine	Anywhere you call `check()`
Overhead	Zero broker round-trips	One atomic hot-backend call
Use for	"This worker should dispatch ≤ 20 jobs per 2s"	"This user gets 100 requests/min across all nodes"

// Per-process dispatch cap on a queue
Queue({ name: "emails", throttle: { max: 20, duration: 2_000 }, handler });

// Cluster-wide limit inside any handler
Queue({
  name: "emails",
  handler: async (job) => {
    const r = await providerLimit.check({ provider: "ses" });
    if (!r.allowed) {
      job.requeue(r.resetMs - Date.now());
      return;
    }
    await send(job.data);
  },
});

Rate Limiter

On this page