Project detail
ChatScale: Why Real-Time Apps Break When You Scale Them
A Discord-style chat app, the continuation of my earlier real-time chat project, built to study what happens to WebSocket applications when you run more than one copy: reproduce the failure, fix it with Redis, and measure which signal should trigger autoscaling.
Problem
Run a Socket.IO chat app as multiple backend copies and messages silently disappear, because each server only knows about its own connections. CPU, the default autoscaling signal, also says little about WebSocket load.
Solution
Reproduced the message-loss bug, then fixed it with a Redis pub/sub adapter in a 57-line change switched on by one environment variable. Containerised the app (multi-stage, non-root images, health probes, graceful shutdown), deployed it to a 3-node local Kubernetes cluster (k3d) with Kustomize, and instrumented it with Prometheus and Grafana. Also deployed the full stack to the Red Hat OpenShift Developer Sandbox from a single 17-object YAML file.
Impact
Measured on the local k3d cluster: a CPU-based autoscaler never scaled (36% against a 60% target), while one on a custom connection-count metric added a pod in about 50 seconds. In k6 load tests, p95 message latency fell from 110 ms to 34 ms going from 1 to 4 pods, with no message loss.
What I found
- Scaling out silently drops messages across servers until they share state through Redis.
- CPU never triggered the autoscaler, but connection count did within about 50 seconds (custom metric via Prometheus and prometheus-adapter).
- Four pods cut p95 latency about 3× compared with one, with zero lost messages.
The base app
ChatScale continues my earlier Discord-style chat application: a full-stack, real-time messaging platform across 5 screens with server-based channels, member directories and sub-second synchronized delivery over Socket.IO WebSockets, on a Node.js and MySQL backend handling authentication, sessions and message persistence across 6 normalized tables, with a responsive React front end.
Deployment
Kubernetes results come from a local 3-node k3d cluster using Kustomize base and overlays. Pod-kill tests showed a 7 s replacement and under 1.5 s client reconnect. The Red Hat OpenShift Developer Sandbox deployment used in-cluster builds from GitHub, a TLS Route and OpenShift's restricted security model; the connection-count autoscaler was not run there.
Load testing
k6 driving the raw Socket.IO protocol with 50 to 100 concurrent users. Profiling latency also led to gzip, which cut the JavaScript payload by 70%. These were single runs, so treat the numbers as measurements, not proof.
Observability and auth
Backend instrumented with Prometheus metrics and a provisioned Grafana dashboard (connections per pod, messages/sec, latency, pod count). TOTP two-factor login with QR enrolment, recovery codes and AES-256-GCM encrypted secrets, with attempt counters kept in Redis so limits hold across backend copies.