Introduction to OqronKit

OqronKit is an industry-grade, crash-safe, and framework-agnostic backend orchestration and distributed processing engine for Node.js.

The Problem

Modern distributed systems require robust mechanisms for background computation. Developers often face challenges with:

• Scaling: Moving from a monolithic application to a decoupled microservice architecture often requires significant code rewrites.
• Reliability: Ensuring jobs are not lost when worker processes crash (e.g., OOM errors, SIGKILL).
• Consistency: Maintaining state and strict idempotency across distributed nodes.
• Tooling Sprawl: Piecing together disparate libraries for queuing, scheduling, rate limiting, caching, and webhooks.

The OqronKit Solution

One package, one runtime (new Oqron()), eight production modules — sharing a single crash-safety core, a queryable storage model, and a unified control surface.

Core Architectural Philosophy

1. Native Horizontal Scaling & Microservices: Every module is designed for horizontal scaling. Memory mode runs cleanly as a monolith in one process; connecting Redis (or Redis + Postgres) transforms the same code into a distributed, multi-node deployment without changing business logic. Services cooperate through a shared project, and per-definition environment gating controls exactly what runs where.
2. Server Independence (Producers vs Consumers): API nodes and worker nodes are strictly decoupled. A Queue() without a handler is a pure producer; a Worker() is a pure consumer.
3. Two-Plane Storage: A hot plane (Redis) for coordination — broker queues, locks, atomic counters, cache — and a durable plane (Postgres) for truth — definitions, jobs, run history — split into three board-queryable tables with typed record kinds.
4. Crash-Safety by Construction: Durable-first job writes, atomic claims with heartbeat extension, owner-fenced acks, boot reconciliation, and dead-letter queues give queues, workers, and webhooks at-least-once semantics. Scheduled work can opt into the same guarantees with durable: true.
5. Used = Enabled: Defining a Cron(), Queue(), or Cache() is all it takes — the runtime discovers and boots exactly the modules your code uses.

The 8 Core Modules

1. Scheduler (Cron + Schedule): cron expressions, intervals, one-shots, recurring calendars, iCal rules — with missed-fire recovery, leader election, and optional crash-safe durable fires.
2. Task Queue: unified queue with concurrency, retries with backoff, throttling, batch processing, and dead-letter queues.
3. Distributed Worker: decoupled publisher/consumer split for isolated background processing across services.
4. Pub/Sub: durable fan-out to independent consumer groups on the same job pipeline, with a retained message log for backfill and replay.
5. Webhook: fan-out event delivery with HMAC signing, glob event matching, per-endpoint circuit breakers, rate limits, and resend.
6. Rate Limiter: multi-tier distributed limits with three atomic algorithms, weighted costs, VIP overrides, auto-bans, and dry-run rollout.
7. Cache: stampede-protected two-tier caching (L1 memory + L2 distributed) with tag invalidation and O(1) full invalidation.
8. Observability & Control (built-in, not opt-in): run history, typed events, persisted module logs, metrics, and a runtime admin surface across every module.

Where to go next

• Installation → Quickstart — running in five minutes
• Architecture — how the runtime, modules, and planes fit together
• Storage Model — jobs vs runs, the three tables, dashboard SQL
• Environments & Microservices — isolation, gating, and service placement
• Crash Safety — the guarantees, mechanism by mechanism