If your Node.js WebSocket service, which keeps two-way connections open for hours, has outgrown one machine, this article covers the move to several. One box holds tens of thousands of long-lived connections, but adding a second box breaks three things at once. Clients must keep landing on the box that owns their session, broadcasts must reach every box, and presence (who is online) must agree everywhere. Nginx solves stickiness with ip_hash routing, and the Socket.IO Redis adapter relays each event through Redis publish/subscribe. Presence lives in a Redis sorted set, a score-ordered list keyed by user and refreshed on every heartbeat.
How to verify
Confirm the current connection count and where they are distributed:
ss -ant | awk '/ESTAB/ && /:3000/' | wc -l # active connections per worker
curl -s http://127.0.0.1:3000/_metrics | grep socket_connections
redis-cli pubsub numsub socket.io#/# # active subscribers per channel
journalctl -u myapp --since "10 min ago" | grep -i 'socket\|disconnect'
# upstream balance — should be roughly equal per backend
for n in 3001 3002 3003; do curl -s http://127.0.0.1:$n/_metrics | grep ws_conn; done
If one backend has 8000 connections and another has 200, your sticky load balancer is broken — every client is hashing to the same backend. Check the ip_hash directive and that trust proxy is on.
What’s happening
WebSocket is HTTP that upgrades to a persistent TCP connection. Each connection holds a slot in the Node event loop and consumes a small amount of memory (typically 5-50 KiB per connection depending on buffering). A single Node process running on a modern machine handles 30-100k connections comfortably; beyond that, you need horizontal scaling.
Horizontal scaling adds three problems. (1) Stickiness: the load balancer must always send a returning client to the backend that has its session. WebSocket handshake is a long-poll fallback in some libraries — that has to also hit the same backend. (2) Cross-node broadcasting: when a user on box A emits an event to “room:42”, clients connected to box B must also receive it. The pub/sub adapter relays events through Redis. (3) Presence: knowing “who is online globally” needs a shared store; the adapter handles room membership, but presence lists usually need their own Redis hash with TTL refresh.
The procedure
-
Run multiple Node workers on each box (one per CPU) and multiple boxes behind a load balancer. Each worker listens on its own port:
// server.js const port = 3001 + cluster.worker?.id ?? 3000; io.listen(port); -
Nginx in front with
ip_hashso the same client always lands on the same backend:upstream socket_backend { ip_hash; server box-a.example.com:3001 max_fails=3 fail_timeout=10s; server box-a.example.com:3002 max_fails=3 fail_timeout=10s; server box-b.example.com:3001 max_fails=3 fail_timeout=10s; server box-b.example.com:3002 max_fails=3 fail_timeout=10s; } server { listen 443 ssl http2; location /socket.io/ { proxy_pass http://socket_backend; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_read_timeout 86400; # long-lived connections proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } } -
Install the Socket.IO Redis adapter (or the ws/uWebSockets.js equivalent) and wire it on every worker:
npm install @socket.io/redis-adapter ioredisimport { Server } from 'socket.io'; import { createAdapter } from '@socket.io/redis-adapter'; import { Redis } from 'ioredis'; const io = new Server(httpServer, { cors: { origin: '*' } }); const pubClient = new Redis(process.env.REDIS_URL); const subClient = pubClient.duplicate(); io.adapter(createAdapter(pubClient, subClient)); -
Now
io.to('room:42').emit('event', payload)reaches every connected client globally — the local Node delivers to its sockets, the adapter publishes to Redis, and other Node workers receive and deliver locally. -
For presence (who is online globally), maintain a Redis sorted set keyed by user with a TTL refresh on each heartbeat:
io.on('connection', socket => { const userId = socket.handshake.auth.userId; pubClient.zadd('presence', Date.now(), userId); const beat = setInterval(() => pubClient.zadd('presence', Date.now(), userId), 30_000); socket.on('disconnect', async () => { clearInterval(beat); await pubClient.zrem('presence', userId); }); });To list everyone online in the last 60 seconds:
const cutoff = Date.now() - 60_000; const onlineUsers = await pubClient.zrangebyscore('presence', cutoff, '+inf'); -
Tune the OS — default limits trip well before Node does. On each box:
echo "fs.file-max = 2000000" | sudo tee -a /etc/sysctl.conf echo "net.core.somaxconn = 65535" | sudo tee -a /etc/sysctl.conf sudo sysctl -p # systemd unit for the node service # LimitNOFILE=1048576
Common pitfalls
ip_hashis bypassed when many clients share an IP (corporate NAT, mobile carriers). For that case, switch to a stickiness cookie at the load balancer — Nginx Plus supportssticky cookie, or use HAProxy withcookie SRV insert.- The Redis adapter relays every emitted event through Redis. Emitting at 50k events/sec saturates a small Redis. Split rooms across multiple Redis instances or use the sharded adapter (
@socket.io/cluster-adapter). - Long-polling fallback (Socket.IO does this by default) is HTTP not WebSocket and gets routed by the LB without the WebSocket Upgrade. Without sticky, two polls land on different backends and the session breaks.
transports: ['websocket']disables polling and side-steps the issue if your clients support WS exclusively. proxy_read_timeoutdefaults to 60s on Nginx — a connection idle longer than that is killed. For long-lived chat, set it to 24h and rely on ping/pong from the application to detect dead connections.- Closing a connection from the server side:
socket.disconnect(true)closes the TCP connection.socket.disconnect()(no argument) only disconnects the namespace; the underlying socket persists.
For the single-box version of cluster mode with sticky routing, see Node.js cluster mode with sticky load balancing. In the engagements Stack Harbor runs, WebSocket services use the Nginx ip_hash + Redis adapter pattern by default, presence is in Redis sorted sets with TTL, and connection counts + p95 emit latency are wired to alerts as part of managed operations so capacity is known before it’s exceeded.