InferenceX Dashboard: An Open-Source Visual Analysis Platform for Continuous Inference Benchmarking

This article introduces InferenceX Dashboard, an LLM inference performance benchmarking visualization platform built with Next.js. Through nightly automated tests, the platform conducts comprehensive performance scans of popular models on mainstream hardware, providing a complete analytical view of throughput and latency. It addresses issues like outdated results and unrealistic configurations in traditional benchmarking, helping developers and enterprises make informed inference deployment decisions.

LLM inference performance analysis is at the core of AI services, but accurate evaluation faces many challenges:

1. Fast Software Iteration: Inference frameworks (e.g., vLLM, TensorRT-LLM) and model versions update rapidly, making static test results easily outdated;
2. Gamified Configurations: Public test results often come from specialized setups that are hard to reproduce in production environments;
3. Lack of Comprehensive Perspective: Traditional tests focus only on single metrics, ignoring the tradeoff between throughput and latency and the impact of multi-dimensional configurations. InferenceX attempts to address these issues through continuous automated testing, multi-dimensional scanning, and open-source data.

InferenceX follows five design principles:

• Continuous Updates: Run tests every night using the latest software and model versions;
• Comprehensive Scanning: For each model-hardware combination, scan different tensor parallelism degrees and numbers of concurrent requests;
• Realistic Scenarios: Configurations ensure universal applicability in production environments, with no optimizations targeted at specific tests;
• Open-Source Transparency: Code and data are fully open-source, and community validation is welcome;
• Throughput-Latency Panorama: Provide complete relationship graphs instead of isolated metric points.

Frontend Tech Stack

Framework: Next.js16 (App Router), TypeScript, Tailwind CSS4, shadcn/ui, D3.js, React Query

Backend & Data Layer

Database: Neon PostgreSQL (read-write separation), API: Next.js API Routes, Deployment: Vercel, Testing: Cypress+Vitest

Data Flow

Neon PostgreSQL → API Routes → React Query → Context Providers → D3.js Charts

Monorepo Structure

The packages/ directory includes modules like app (frontend), constants (shared constants), db (database layer), etc.

Test Frequency & Coverage

• Run automatically every night; covers mainstream GPUs like NVIDIA and AMD; tests popular models like DeepSeek and Llama; uses the latest framework versions.

Multi-dimensional Parameter Scanning

For each model-hardware combination, scan tensor parallelism degrees and maximum concurrent requests to generate complete throughput vs. latency curves.

Configuration Universality

Ensure test configurations are universally applicable in production environments to avoid the gap between lab data and real-world performance.

InferenceX provides rich interactive visualizations via D3.js:

1. Throughput-Latency Curves: Show performance tradeoffs under different concurrent loads;
2. Hardware Comparison: Intuitively compare the performance of the same model on different GPUs;
3. Model Comparison: Parallel comparison of performance and capabilities across multiple models;
4. Configuration Optimization Recommendations: Recommend optimal tensor parallelism degrees, concurrency counts, etc., based on data.

InferenceX provides value for the following scenarios:

1. Hardware Selection: Compare cost-effectiveness of different GPUs to support procurement decisions;
2. Model Deployment Optimization: Quickly find configurations that meet latency/throughput requirements;
3. Performance Trend Tracking: Record the evolution trajectory of framework and model performance;
4. Framework Selection: Reference cross-framework comparison data to understand scenario-specific pros and cons.

Open-Source Ecosystem

InferenceX is fully open-source, including dashboard code, benchmarking framework, and historical data. The community can contribute by submitting test configurations, improving visualizations, reporting anomalies, and sharing analyses.

Summary

Through continuous automated testing and open, transparent data, InferenceX addresses traditional benchmarking issues, helps developers and enterprises make informed deployment decisions, and is expected to become an authoritative performance reference in the LLM inference field.