LLM GPU VRAM 计算器：大模型部署显存与性能估算工具

LLM GPU VRAM Calculator: Overview This is an interactive web tool for estimating VRAM requirements, KV cache pressure, and throughput performance when running large language models (LLMs) on different GPU configurations.

Key Details:

• Original author/maintainer: jryaonj
• Source: GitHub project llm-gpu-vram-calculator (link: https://github.com/jryaonj/llm-gpu-vram-calculator)
• Online demo: https://jryaonj.github.io/llm-gpu-vram-calculator
• Release date: 2026-05-25
• License: MIT

Core Purpose: To help engineers plan LLM deployment by answering questions like: Can a model run on target hardware? How many GPUs are needed? What's the impact of quantization on VRAM and speed?

Background: Challenges in LLM Deployment When deploying LLMs, engineers face critical questions:

• Can a specific model run on the target GPU hardware?
• How many GPUs are required for the desired performance?
• How do quantization strategies affect VRAM usage and inference speed?

These questions need accurate estimates before actual deployment to avoid resource waste or failure. The LLM GPU VRAM Calculator addresses these gaps by providing a user-friendly way to compute these metrics.

Core Features of the Calculator

1. Guided Configuration: Covers model selection, GPU hardware, and runtime parameters (quantization, context length, concurrent requests).
2. Model Directory: Includes popular open-source models like Qwen3/3.5/3.6 (Dense/MoE), DeepSeek V3/R1 (MLA KV cache support), Gemma3/4 (Hybrid attention).
3. GPU Hardware Library: Contains key specs (VRAM, bandwidth, compute capacity) from various vendors.
4. Quantization Support: Estimates VRAM for weight (FP16, FP8, INT8, INT4) and KV cache (FP8, INT8) quantization.
5. Formula Panel: Explains the theoretical basis of calculations.
6. Data Export: Exports model metadata, GPU specs, and estimation results as CSV.
7. Internationalization: Supports English (en_US) and Chinese (zh_CN) interfaces.

Calculation Principles

• VRAM Estimation:

  • Weight VRAM: weight_vram_gb = total_params_b × (bytes_per_param + quant_overhead) (INT4 has extra overhead: 3/awq_group_size).
  • KV Cache: kv_cache_gb = layers × kv_heads × head_dim ×2 × context_tokens × kv_bytes /2^30 (linear with context length and concurrent requests).

• Available VRAM: usable_vram_gb = gpu_vram_gb × gpu_count - max(total_vram_gb × (1-utilization), reserve_gb) (reserve for memory fragments, CUDA graphs, etc.).

• Performance Estimation:

  • Prompt Pre-fill: prompt_tok_s = fp16_tflops ×1000 × gpu_count^0.6 / (total_params_b × sqrt(2)) (computation-intensive).
  • Token Generation: gen_tok_s = bandwidth_gbs × gpu_count^0.8 / (active_params_b × weight_bytes) (bandwidth-intensive).

Technical Implementation

• Tech Stack: TypeScript + React (frontend), Vite (build), ESLint (code规范), GitHub Pages (deployment).
• Project Structure:
  • src/data/modelDefs.ts: Model parameters, context length, metadata.
  • src/data/gpuCards.ts: GPU specs (VRAM, bandwidth, etc.).
  • src/utils/formulas.ts: Shared calculation functions.
• Data Sources:
  • Models: Hugging Face model cards/configs.
  • GPUs: Official vendor pages (supplemented by TechPowerUp).

Use Cases & Value

1. Deployment Planning: Evaluate model feasibility on existing hardware before purchasing/cloud resource申请.
2. Quantization Comparison: Compare FP16/INT8/INT4 to find optimal balance between VRAM and performance.
3. Long Context Evaluation: Understand KV cache's linear impact on VRAM for long text scenarios (document analysis, code generation).
4. Multi-Card Prediction: Estimate performance scaling with multiple GPUs.
5. Teaching Tool: Help learn LLM inference optimization (VRAM composition, bottlenecks, Roofline model).

Calibration & Usage Suggestions To get accurate results:

1. Run small benchmarks on target runtime/model.
2. Compare实测 throughput (pre-fill/generation) with tool estimates.
3. Adjust scaling indices or effective TFLOPS/bandwidth based on results.
4. Prioritize strict capacity planning (OOM is critical, speed issues are manageable).

Conclusion The LLM GPU VRAM Calculator bridges the gap between theoretical model specs and practical hardware deployment. It helps teams make data-driven decisions: choosing the right model, quantization strategy, and hardware combo to balance cost and performance. This tool is valuable for engineers, developers, and teams deploying LLMs in production.