Run history, typed events, opt-in module logs, metrics, and the runtime admin surface

Observability & Control

Everything OqronKit does is observable through four channels — run history (what happened), events (what's happening now), module logs (opt-in persisted detail), and metrics/stats — plus a control surface for pausing, triggering, and cancelling at runtime.

Run history

Every execution (cron fire, schedule fire, queue/worker job attempt, webhook delivery, pub/sub group delivery) produces a run: input, output, error, per-run logs, a status timeline, progress, duration, and the worker that ran it.

const runs = oqron.persistence.runs;

// Filter + paginate
await runs.list(
  { name: "emails", status: "failed" }, // also: module: 'cron' | 'queue' | ...
  {
    limit: 50,
    orderBy: { field: "startedAt", direction: "desc", type: "number" },
  },
);

// Full record — logs, timeline, progress, output
await runs.get(runId);

// Prune manually (retention usually handles this)
await runs.delete(runId);

Recording is non-blocking: runs are created in memory + written through to the hot plane instantly, and flushed to Postgres in the background — history never slows the hot path. Retention is per definition (keepHistory, keepFailedHistory, removeOnComplete, removeOnFail).

Dashboards can also query runs directly with SQL.

Typed events

Every node emits a typed event stream on its instance bus — zero configuration, always on:

oqron.eventBus.on("job:success", (queueName, jobId) => {});
oqron.eventBus.on("job:fail", (queueName, jobId, error) => {});
oqron.eventBus.on(
  "schedule:fire:complete",
  (name, runId, status, durationMs) => {},
);
oqron.eventBus.on("queue:job:completed", (queueName, jobId, durationMs) => {});
oqron.eventBus.on("ratelimit:banned", (limiter, tier, key, banMs) => {});
oqron.eventBus.on("cache:miss", (cacheName, key) => {});
oqron.eventBus.on("webhook:paused", (dispatcherName) => {});
oqron.eventBus.on("pubsub:delivery:dead", (topicName, deliveryId) => {});

Event families: job:*, schedule:*, queue:*, worker:*, webhook:*, ratelimit:*, cache:*, pubsub:*, module:*, system:*. All names and argument tuples are TypeScript-checked — a typo won't compile.

Events are per-node and in-process — perfect for wiring metrics (Prometheus/StatsD) and alerting. For durable, queryable history use runs and module logs.

Module logs (opt-in)

Cache and RateLimit don't create runs (they're data-plane modules) — instead they support persisted logs, off by default, enabled per definition:

Cache({ name: "products", logs: true }); // defaults: level 'warn', 1000 entries, 50 writes/s

RateLimit({
  name: "api",
  logs: {
    level: "info", // error | warn | info | debug
    maxEntries: 2_000, // oldest trimmed beyond this
    ttlMs: 7 * 86_400_000, // optional age-based expiry
    maxWritesPerSec: 50, // hard cap on durable writes
  },
  tiers: [/* ... */],
});

How it stays safe on hot paths:

log() is synchronous and buffered — it never blocks or throws into your request path.
Writes beyond maxWritesPerSec are dropped and summarized (logs:dropped entry with the count) — debug level on a cache doing 10k ops/s cannot flood your database.
Entries land in the durable module_logs:<module>:<name> namespace (record_kind: 'log') — queryable like everything else:

SELECT value FROM oqron_records
WHERE project = 'my-app' AND environment = 'production'
  AND namespace = 'module_logs:ratelimit:api'
ORDER BY inserted_at DESC LIMIT 100;

What gets logged per level: error (fetcher/backend failures) · warn (bans, stampede-lock timeouts, drop summaries) · info (blocked checks, invalidations) · debug (every hit/miss/set/delete — rate-capped).

Metrics & stats

// Queue / worker / webhook — per-endpoint processing metrics
const queueEngine = oqron.getModule("queue") as QueueEngine;
queueEngine.getMetric("emails");
// { name, claimed, completed, failed, duration: { min, max, avg, last } }

// Scheduler metrics
const cron = oqron.getModule("cron") as CronEngine;
cron.getMetricsForSchedule("nightly-report");

// Cache / RateLimit — per-definition counters (this node)
await products.stats(); // { hits, misses, sets, deletes, fetches, fetchErrors, invalidations }
await apiLimit.stats(); // { allowed, blocked, banned }

// Backend health
await oqron.persistence.health(); // { hot: {...}, durable: {...} }

The control surface

Every module engine exposes the same runtime admin operations:

Operation	Cron/Schedule	Queue	Worker	Webhook	RateLimit	Cache
`list()` / `get(name)`	✅	via handle	—	endpoints CRUD	✅	✅
`pauseInstance` / `resumeInstance`	✅	`pause()`/`resume()` on handle	✅	`pause()`/`resume()`	✅	✅
Manual trigger	`triggerManual(name)`	`add()`	—	`fire()`	—	—
Cancel running	`cancelActiveJob(runId)`	✅	✅	✅	—	—
Module on/off	`enable()` / `disable()` on every engine

examples

const cron = oqron.getModule("cron") as CronEngine;
await cron.pauseInstance("nightly-report"); // disabledBehavior decides: hold | skip | reject
await cron.triggerManual("nightly-report"); // fire now
await cron.resumeInstance("nightly-report");

// Cancel any running job (searches the right module)
for (const m of ["queue", "worker", "webhook", "cron", "schedule"]) {
  if (await oqron.getModule(m)?.cancelActiveJob?.(jobId)) break;
}

// Dead letters: inspect + re-deliver
await oqron.persistence.records("queue_dlq:emails").list();
await orderHooks.resend(deadJobId); // webhooks: clone a dead delivery

Pub/Sub's control surface is per-group, not per-definition — a topic has many independent consumer groups, so the pause/resume/replay unit is the group: topic.pauseGroup(name) / resumeGroup(name) / listGroups() / replay(group, {from}). See Pub/Sub.

Complete working example — an HTTP admin API covering every surface on this page (crons, schedules, queues, DLQ, webhooks, runs, metrics, health): apps/backend/admin.ts

What to reach for, when

You want to…	Use
See why last night's job failed	`runs.list({ name, status: 'failed' })` → `runs.get(id)` — logs + timeline
Alert on failures in real time	`eventBus.on('job:fail', …)` → your metrics/alerting
Audit who got rate-limited this week	`logs: { level: 'info' }` on the limiter → query `module_logs:ratelimit:api`
Build a dashboard	SQL over `oqron_runs` / `oqron_jobs` / `oqron_records` (storage model)
Stop a runaway cron right now	`cron.pauseInstance(name)` / `cancelActiveJob(runId)`
Re-deliver a dead webhook	`dispatcher.resend(jobId)`
Re-deliver a range of pub/sub messages to one group	`topic.replay(group, { from })`

Observability & Control

Observability & Control

Run history

Typed events

Module logs (opt-in)

Metrics & stats

The control surface

What to reach for, when

On this page