Spark-Stack: A Local LLM Inference Monitoring Dashboard for NVIDIA DGX Spark

Spark-Stack is an open-source monitoring tool designed specifically for local large language model (LLM) inference scenarios on NVIDIA DGX Spark. It integrates system metrics, vLLM inference observability, and persistent token tracking, adopting a "history-first" philosophy to provide a long-term activity tracking experience similar to WakaTime, filling the gap of dedicated monitoring solutions in the DGX Spark ecosystem. The project is developed and maintained by Sahil Kapoor (@kapoorsahil), with code hosted on GitHub (link: https://github.com/kapoorsahil/spark-stack) and released in May 2025.

Most monitoring tools on the market only focus on real-time metrics. Spark-Stack addresses the lack of dedicated monitoring solutions in the DGX Spark ecosystem by seamlessly integrating system-level monitoring with observability metrics unique to LLM inference, helping developers clearly understand model usage patterns, resource consumption trends, and the evolution of inference performance.

Spark-Stack includes three core functions:

1. System Metrics Monitoring: Covers GPU status (utilization, temperature, power consumption, etc.), per-core CPU load, unified memory usage, and system health metrics;
2. vLLM Inference Observability: KV Cache analysis, request tracking (token count, latency), batch processing monitoring, and throughput statistics;
3. Persistent Token Tracking: Daily/weekly/monthly total token consumption, model/endpoint usage distribution, peak time identification, and cost estimation.

Spark-Stack adopts a lightweight architecture with main components including:

• Data Collection Layer: Acquires data via the NVIDIA NVML library and vLLM Prometheus metric endpoints;
• Storage Layer: Uses local SQLite or optional PostgreSQL to store time-series data;
• Presentation Layer: A responsive web dashboard supporting desktop and mobile devices;
• Configuration System: Flexible JSON files supporting custom monitoring thresholds and alert rules. Deployment is straightforward, providing a systemd service file for auto-start on boot and a one-click installation script. DGX Spark users can complete setup within minutes.

Spark-Stack is suitable for multiple scenarios:

1. Individual Developers: Track token consumption, GPU utilization, inference latency, and optimization opportunities;
2. Small Teams: Fairly track resource allocation, identify scaling and optimization periods, and establish performance baselines;
3. Model Tuning: Compare the impact of quantization strategies, validate the effectiveness of KV Cache management, and determine optimal concurrent request configurations.

Spark-Stack is designed with compatibility in mind:

• Natively supports vLLM metric output formats;
• Optional Prometheus remote write integration for easy access to existing monitoring stacks;
• Provides an official Grafana dashboard template;
• Automatically uses NVIDIA DCGM to obtain more detailed GPU metrics (if available).

Spark-Stack is open-source under the MIT license, with code hosted on GitHub. The project has a clear structure, including complete documentation, example configuration files, and automated tests, allowing community contributors to easily extend new data sources or custom visualization components. It fills the gap in monitoring tools for the DGX Spark user community and is expected to become one of the platform's standard tools.

Spark-Stack represents the trend of local AI development tools moving towards specialization and refinement. It is not just a monitoring dashboard but also a tool that helps developers gain deep insight into local LLM inference workflows. It is recommended for developers who take local model inference seriously to try it out; its simple deployment, intuitive interface, and in-depth metric coverage make it an indispensable part of the DGX Spark ecosystem.