Reading
Researchers propose the DuQuant++ method to address the activation outlier problem in the MXFP4 format. By using single-round outlier-aware rotation, it achieves more efficient W4A4 quantization and reaches SOTA performance on the LLaMA-3 model.
Section 01
Researchers propose the DuQuant++ method to address the activation outlier problem in the MXFP4 format. Using single-round outlier-aware rotation, it achieves more efficient W4A4 quantization, reaches SOTA performance on the LLaMA-3 model, halves online computation cost, and is compatible with the NVIDIA Blackwell architecture.
Section 02
Large model deployment faces storage and computation pressures, and quantization is a key technology. However, the MXFP4 format (32-element blocks share a scaling factor, natively supported by Blackwell) has an activation outlier problem: a single outlier forces the block scaling factor to increase, squeezing the dynamic range of other elements.
Section 03
Existing rotation methods have flaws: random Hadamard rotation lacks data specificity, leading to limited effectiveness; learnable rotation requires additional training and has questionable generalization. Neither of them utilizes outlier distribution information.
Section 04
Section 05
Single-round rotation halves online computation cost; under LLaMA-3 W4A4 quantization, DuQuant++ outperforms baselines in multiple tasks such as commonsense reasoning and code generation, reaching SOTA levels.
Section 06
Compatible with the NVIDIA Blackwell architecture (natively supports MXFP4); Practical suggestions: MXFP4 is suitable for Blackwell hardware, outlier handling is key to quantization, and algorithms need to align with the format's grouping structure.
Section 07
Extend to other low-precision formats, combine techniques like smoothing/cropping, explore aggressive configurations such as W2A2/W3A3, and develop hardware-friendly rotation implementations.
Keep going with more reads from the same topic.
Splinter is a minimalist, high-performance key-value (KV) and vector storage system enabling zero-latency inter-process communication via shared memory and atomic operations. With only 766 lines of core code, it supports millions of operations per second and 768-dimensional vector storage, offering a new architectural approach for local LLM inference and data-intensive applications.
Folkering OS is the world's first AI-native bare-metal operating system, entirely written in Rust no_std without relying on Linux, POSIX, or libc. It can generate commands from scratch, compile them into WASM, and run them in 10 seconds, achieving true self-evolution.
This project explores how to use Large Language Models (LLMs) to detect logical vulnerabilities in Ethereum smart contracts, especially business logic flaws that are difficult for traditional static analysis tools to capture, such as economic manipulation and execution order errors.
Cross-generational hardware testing based on NVIDIA A10G and H100 GPUs, comparing and analyzing the throughput, latency, and concurrent scalability of three mainstream LLM inference frameworks. SGLang achieves a 3.4x performance advantage over vLLM on H100, while Ollama faces architectural bottlenecks in high-concurrency scenarios.