Multi-node (Postgres, Redis)
View as MarkdownOne hub node is enough for many deployments. When you need more (availability, rolling deploys, more connections), run the same hub on several nodes behind a load balancer. No sticky sessions are needed.
FaaS mode is multi-node by nature: every function instance is a hub node, instances start and stop at any time, and any of them may serve any request. Everything on this page applies there too; the deploy pages (Vercel, Workers) already use postgresHubStore + postgresCluster.
What changes
Section titled “What changes”A computer’s WebSocket lives on one node (A). A browser request can land on any node (B). Two shared pieces make that work:
| One node (default) | Several nodes | |
|---|---|---|
store (HubStore) |
pgliteHubStore({ dataDir }): embedded Postgres, one process per data folder |
postgresHubStore({ connectionString }) or ({ pool }). Needs pg. |
cluster (ClusterAdapter) |
memoryCluster(): in-process |
redisCluster({ url }) or ({ client }) (needs ioredis), or postgresCluster({ pool }) / ({ connectionString }) on the same Postgres (no Redis). |
flowchart LR
B["Browser"] --> LB["Load balancer"]
M["User's computer"] -- "WSS" --> LB
LB --> N1["Hub node A<br/>holds the computer's socket"]
LB --> N2["Hub node B<br/>serves the browser request"]
N1 <-- "presence, commands,<br/>results, events" --> R[("Redis")]
N2 <--> R
N1 --> P[("Postgres")]
N2 --> P
- B looks up which node holds the computer (presence in the cluster), publishes the command on A’s channel; A forwards it to the computer and publishes the result back. A computer on HTTP streaming is not tied to a node: B writes the command to the computer’s outbox in the store and wakes its stream through the cluster.
- Session events fan out to every node over the cluster, so a browser streaming from any node sees them.
- Site tool calls run on the node that receives them: A for a WebSocket, whichever node gets the agent’s POST for HTTP streaming.
- An approval decided on B reaches the computer exactly once. The computer is the single judge: if two decisions race, the second gets
409.
Set it up
Section titled “Set it up”npm i pg ioredisimport { createHub, postgresHubStore, redisCluster, type HubUser } from '@uncage/hub';
declare function getUser(req: Request): Promise<HubUser | null>;
const store = await postgresHubStore({ connectionString: process.env.DATABASE_URL! });const cluster = redisCluster({ url: process.env.REDIS_URL! });
export const hub = await createHub({ publicUrl: 'https://app.example.com', // the same on every node getUser, store, cluster, sessionStorage: 'site',});
/** On shutdown: the hub closes only what it created, so close the store and cluster after it. */export async function closeHub(): Promise<void> { await hub.close(); await cluster.close(); await store.close();}Mount it in your server exactly as for one node (Mount the hub).
- Migrations run when the store starts. With Postgres they run in one transaction under a transaction-scoped advisory lock, so nodes can start together, also behind a transaction-mode pooler (PgBouncer, Neon
-pooler). - Every node must have the same configuration:
publicUrl,basePath,site,tools,skills,selectManifest,sessionStorage. A tool call runs on whichever node holds the computer’s socket, so every node needs every tool. - Share a Redis between deployments (staging and production, say) with different
keyPrefixvalues:redisCluster({ url, keyPrefix: 'myapp-staging:' }). The default isuncage:. It prefixes both the presence keys and the pub/sub channels.
Your own pool or client
Section titled “Your own pool or client”import { postgresHubStore, redisCluster } from '@uncage/hub';import { Redis } from 'ioredis';import pg from 'pg';
const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL, max: 10 });pool.on('error', (err) => console.error('postgres pool error', err)); // required: see belowconst redis = new Redis(process.env.REDIS_URL!);redis.on('error', (err) => console.error('redis error', err));
export const store = await postgresHubStore({ pool });export const cluster = redisCluster({ client: redis, keyPrefix: 'myapp:' });node-postgres emits 'error' on a pool when an idle connection dies (a database restart or failover). With no listener, that crashes the process. A pool or client the hub creates itself ({ connectionString }, { url }) gets a listener that logs (the onError option, default console.error). If you pass your own pool or client, attach your own 'error' listener.
Without Redis: postgresCluster
Section titled “Without Redis: postgresCluster”postgresCluster carries the cluster messages through a table in the same Postgres, so one database is all you run:
import { createHub, postgresCluster, postgresHubStore, type HubUser } from '@uncage/hub';import pg from 'pg';
declare function getUser(req: Request): Promise<HubUser | null>;
const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL, max: 10 });pool.on('error', (err) => console.error('postgres pool error', err));
export const hub = await createHub({ publicUrl: 'https://app.example.com', getUser, store: await postgresHubStore({ pool }), cluster: postgresCluster({ pool }), // one pool for both});How it behaves, so you can choose:
- Polling, not push. Each node polls the messages table every 300 ms while there is traffic and backs off to every 2 s when quiet (
activePollMs,idlePollMs,activeWindowMs). It needs noLISTEN/NOTIFY, so it works through any pooler and serverless driver. Expect up to about 2 s extra before an idle node reacts; Redis delivers at once. - At most once, with a bounded gap. A message whose transaction commits late is waited for up to 2 s (
gapTimeoutMs) before a reader moves past it. A message lost that way is never delivered twice: a command then fails with a timeout on the website (the user retries), and browser event streams backfill from the store. - Your database stays awake. A long-lived node polls at least every 2 s, forever. With a scale-to-zero database (Neon, for example) that means it never suspends while a node runs. Use Redis if that matters.
- Retention. Messages older than 60 s (
retentionMs) are deleted by any node every 30 s.
Rule of thumb: postgresCluster for FaaS mode and small deployments (one less service); redisCluster for many nodes, low latency, or a database that should be able to sleep.
Load balancer
Section titled “Load balancer”- WebSockets on
<basePath>/agentmust be forwarded (HTTP/1.1Upgrade), with idle timeouts above the 10 s heartbeat. No stickiness needed. - Streams: SSE on
<basePath>/sessions/:id/eventsand NDJSON on<basePath>/agent/streammust not be buffered, and need idle timeouts above 15 s (the heartbeats) and request timeouts abovestream.maxDurationMs(250 s). - Host header preserved, and client IP passed on for rate limits: see behind a proxy.
- Draining: on deploy, stop sending new requests to a node, then
close()it. Its computers reconnect to another node with exponential backoff (1 s, growing to at most 30 s); their runs keep going locally meanwhile, and events backfill on reconnect.
Limits to know
Section titled “Limits to know”- Rate limits (device codes per IP, failed code guesses per user, failed machine tokens per IP) are counted in the store, so they are shared by every node. The per-IP limits need the real client IP (see the caution above).
pgliteHubStorecannot be shared between processes: it takes a lock on its data folder. Several nodes must use Postgres.
Try it locally
Section titled “Try it locally”The reference demo has a two-node mode with a small round-robin proxy in front (no stickiness), so the computer and the browser usually land on different nodes:
cd examples/connected-demodocker compose up -d # redis:7-alpine on 127.0.0.1:6380, postgres:16-alpine on 127.0.0.1:5433pnpm buildHUB_NODES=2 REDIS_URL=redis://127.0.0.1:6380 \ DATABASE_URL=postgres://uncage:uncage@127.0.0.1:5433/uncage pnpm multinode# proxy on :3200, nodes on :3201 and :3202docker compose down # when doneThen connect a computer and run a task as in the quickstart, against http://localhost:3200. The E2E suite runs this setup too (UNCAGE_DOCKER_TESTS=1 pnpm exec playwright test --project=connected-multinode from e2e/), and checks that both nodes answered.
The hub’s own tests run the same two-node scenario in one process with ioredis-mock (pnpm --filter @uncage/hub test), and against real Redis and Postgres containers with UNCAGE_DOCKER_TESTS=1.