When a Node.js service misbehaves in production — slow handlers, growing heap, suspicious memory pattern — you usually cannot reproduce it on your laptop. Attaching Chrome DevTools to the live process is the fastest way to look without a restart. This runbook covers safely enabling the inspector on a running service, tunnelling over SSH, and the three views you actually need.
How to verify
Check whether the inspector is currently enabled and reachable:
sudo lsof -iTCP:9229 -sTCP:LISTEN
ps -ef | grep -E 'node.*--inspect'
curl -s http://127.0.0.1:9229/json/version | jq .
ss -tnlp | grep 9229
journalctl -u myapp --since "1 hour ago" | grep -i 'inspector\|debugger'
If port 9229 is listening on 0.0.0.0 you have a security problem — the inspector gives full RCE to anyone who can reach it. Bind it to localhost and reach it over an SSH tunnel.
What’s happening
Node’s V8 inspector exposes the Chrome DevTools Protocol over a WebSocket. Enabling it gives you live access to the JS heap, CPU profiles, the source code as V8 sees it, and a REPL evaluating in any execution context. The inspector adds essentially zero overhead until you actually attach — V8 emits events only when a client is listening — which is what makes it safe to enable in production.
You can enable the inspector at startup (node --inspect) or signal a running process to enable it on the fly (kill -SIGUSR1 <pid>). Production setups almost always use the signal approach because it avoids restarting a service to investigate, and because the port can be opened only when needed.
The procedure
-
Identify the process and signal it to open the inspector. SIGUSR1 tells V8 to bind to 127.0.0.1:9229 (default):
pgrep -f 'node.*server.js' sudo kill -SIGUSR1 <pid> sudo lsof -iTCP:9229 -sTCP:LISTEN # should show node listening on 127.0.0.1 -
Open an SSH tunnel from your laptop. Bind the remote port to your loopback only — never let DevTools reach over the public internet:
ssh -N -L 9229:127.0.0.1:9229 deploy@app-01.example.com -
In Chrome, navigate to
chrome://inspect. Under “Remote Target”, you should see your Node target with an “inspect” link. Click it. DevTools opens connected to the production process. -
The three tabs that matter in prod:
- Memory — “Take heap snapshot”. Wait 60-120 seconds, take another. Diff them in the “Comparison” view to find objects allocated and not released between snapshots. This is the leak-hunting workflow.
- Profiler — “Start” → wait 30 s → “Stop”. V8 captures a CPU sample profile; the bottom-up view shows where wall-clock time is being spent. Hot functions you didn’t expect are the win.
- Console — evaluate live JS in the running process.
process.memoryUsage(),require.cache,global.someLeakySingleton.size— useful for confirming hypotheses without redeploying.
-
When done, close DevTools and detach. The inspector stays open until you signal SIGUSR1 again (toggle) or restart the process. To force close from outside, send a second SIGUSR1 — V8 toggles the inspector on each signal.
sudo kill -SIGUSR1 <pid> # closes the inspector sudo lsof -iTCP:9229 -sTCP:LISTEN # should show nothing
Common pitfalls
- Heap snapshots pause the event loop while they serialize. A 2 GiB heap takes several seconds — during which the process serves zero requests. Take snapshots during a maintenance window or on a node drained from the load balancer.
- The CPU profiler samples at 1 ms granularity and adds about 5% overhead. Safe to use on production, but the wall-clock attribution is approximate — interpret hot functions, not absolute percentages.
- DevTools “Live Edit” lets you change source in the browser and push it to the process. Disable this mentally — production source must come from git, not a debugger.
- If you start the process with
--inspect-brk, it pauses at the first line waiting for a debugger. Never use this flag at boot in production — only--inspector the SIGUSR1 pattern. - Containers running with a PID namespace and read-only filesystem may not have a writable
/tmpfor the inspector’s intermediate files. Mount a tmpfs at/tmpor run with--inspect=0.0.0.0:9229bound to a docker network the host can reach via tunnel.
For automated profiling instead of interactive debugging, see V8 profiling with pprof. In the engagements Stack Harbor runs, the SIGUSR1-on-demand pattern is the default — the inspector is never bound at boot, and we maintain a tested runbook for tunnelling in safely as part of managed operations so an on-call engineer doesn’t fumble syntax during an incident.