Intel Arc Pro B70 Hands-On: A New Option for Consumer-Grade Large Model Inference

This article provides a detailed hands-on test of the Intel Arc Pro B70 GPU for large model inference, covering single-card/dual-card configurations, multiple quantization schemes, cross-platform comparisons with NVIDIA graphics cards, and an analysis of the energy efficiency advantages of the MoE architecture. Based on the Battlemage architecture, this graphics card is priced at $949 and equipped with 32GB GDDR6 ECC memory, offering a new option for the consumer-grade AI inference market.

NVIDIA has long dominated the large model inference field, and the release of the Intel Arc Pro B70 (Xe2/Battlemage architecture) brings new competition. In terms of hardware, the B70 is based on the full BMG-G31 core, with a single card having 32GB GDDR6 ECC memory (bandwidth 608GB/s). A dual-card configuration can provide 64GB memory with a total cost of less than $2000, capable of running 70B dense or 80B MoE models. The test platform was AMD Ryzen5 9600X + Ubuntu 26.04 + xe driver + oneAPI 2025.3.3.

All tests were real runs using the SYCL backend of llama.cpp (optimized for Intel GPUs), recording power consumption to calculate tokens-per-joule. It was found that the upstream llama.cpp did not enable the NDEBUG flag by default, leading to slow pre-filling. After fixing this, the speed increased by about 2x, and a PR has been submitted to contribute to the community.

Hands-on tests show that choosing the SYCL backend for Intel GPUs is better, with generation speed being 2.2x that of Vulkan (e.g., Qwen1.5B Q4_K_M: 229t/s vs 102t/s). The MMVQ+reorder path of SYCL has obvious advantages in the decoding phase, and the correct backend selection can bring substantial performance gains.

The MoE architecture performs excellently on the B70, activating only 3-4B parameters per forward pass, achieving large model quality at the cost of a small model. For example, the Qwen3.6-35B-A3B single-card generation speed is 54.7t/s with a power consumption of 114W; its tokens-per-joule is 3-4x that of large dense models, resulting in lower inference costs.

Quantization scheme tests covered Q4_K_M/Q8_0/F16. After an upstream PR fixed the Q8_0 performance issue, the Qwen27B Q8_0 speed increased from 4.88t/s to 15.3t/s. The dual-card configuration mainly increases memory capacity (not speed), allowing the running of models exceeding single-card capacity (e.g.,70B dense,80B MoE), and is also suitable for running two independent models simultaneously.

Compared with NVIDIA RTX3090/3080Ti and others, the B70 is competitive in memory capacity and energy efficiency ratio, with high cost performance. Video generation tasks (LTX-Video, Wan series models) were also tested, recording resolution/duration performance and memory overflow thresholds to provide references for multimedia developers.

With its large memory capacity, excellent energy efficiency ratio, and improved software stack, the B70 has become an attractive alternative for consumer-grade AI inference. It is recommended for users who mainly run MoE models, focus on energy efficiency, or have limited budgets but need large memory to consider the B70. With SYCL optimizations and upstream improvements, the performance of Intel GPUs will continue to improve. The test team also submitted multiple PRs to llama.cpp to fix issues, contributing to the open-source ecosystem.