niklasfrick / spark-dashboard

What is spark-dashboard?

Spark-dashboard is a real-time dashboard for monitoring hardware and LLM inference on Linux systems with NVIDIA GPUs. It polls GPU utilization, temperature, power, CPU cores, memory (including unified pools), disk, and network stats every second, while scraping vLLM metrics like tokens per second, TTFT, queue times, and KV cache usage via Prometheus. Built in Rust with a React frontend, it streams everything over WebSocket to a responsive UI with gauges, charts, and auto-rotating multi-engine tabs—install via `cargo install` and run as a systemd service on port 3000.

Why is it gaining traction?

In a sea of generic real time hardware monitors, this stands out with vLLM-specific smarts: auto-detects engines via process scan and Docker, aggregates across multiple instances in a "global" view, and flags GPU events like thermal throttling. The dev setup proxies to remote hosts seamlessly, and charts overlay inference requests on hardware traces for spotting bottlenecks. Early adopters on DGX Spark praise the glanceable metrics and 15-minute history without setup hassle.

Who should use this?

AI engineers tuning vLLM inference on NVIDIA workstations, DGX boxes, or cloud VMs—especially those juggling multi-engine setups for A/B testing models. It's ideal for real-time hardware in the loop debugging, like correlating batch sizes with power brakes during long runs. Skip if you're not on Linux/NVIDIA or need enterprise alerting.

Verdict

Grab it if vLLM monitoring is your pain point—solid for its niche despite 14 stars and 1.0% credibility score signaling early days. Docs and systemd integration are polished, but expect tweaks for production scale. Worth a spin over Grafana for quick local insights.