Deploying Qwen-VL Multimodal Model on Rockchip Devices: A New Edge AI Vision-Language Solution

The qwen-vl-rknn project released by tristanpenman on GitHub is a CMake-based starter project that demonstrates how to run the Qwen-VL vision-language model via RKNN/RKLLM on Rockchip RK3588 and other NPU devices, enabling localized image understanding and text generation, and providing a new solution for edge AI multimodal applications. The project supports Linux and Android platforms, and features a modular architecture and containerized build.

With the rapid development of Large Language Models (LLMs) and Vision-Language Models (VLMs), the demand for running these models on edge devices has increased. However, cloud deployment has issues such as high latency, high privacy risks, and strong network dependency. Rockchip RK3588 and other NPU chips have powerful AI acceleration capabilities, but deploying mainstream multimodal models on these devices is a challenge for developers, and this project was created to address this problem.

Qwen is an open-source LLM family from Alibaba's Tongyi Laboratory. Qwen-VL is its extended vision-language model, supporting image and text (sometimes video) inputs and outputting text. Typical applications include image description, OCR, UI parsing, etc. Model evolution path: Qwen → Qwen-VL → Qwen2-VL → Qwen2.5-VL → Qwen3-VL.

The project adopts a dual-engine architecture of RKNN and RKLLM: RKLLM handles language model inference, while RKNN accelerates the visual encoder, making full use of Rockchip NPU capabilities. The project has a clear structure, including directories like cmake, cpp/src, scripts, thirdparty, etc., and its modular design facilitates expansion.

The target hardware is Linux/Android devices with Rockchip RK3588 chips (6 TOPS computing power). Recommended entry-level model: Qwen2-VL-2B (2 billion parameters, ~4.5GB, requires a device with 16GB memory, download link: https://huggingface.co/3ib0n/Qwen2-VL-2B-rkllm); advanced option: Qwen2-VL-7B (7 billion parameters, ~9.6GB, download link: https://huggingface.co/3ib0n/Qwen2-VL-7B-rkllm).

Native Linux build can be done via Docker command: docker compose run --rm native ./scripts/build-native.sh Release; Android 14 build: docker compose run --rm android ./scripts/build-android.sh Release. Custom CMake configurations are supported, such as paths for RKNN/RKLLM, OpenCV version and modules, etc.

Technical highlights include edge-side multimodal inference (tasks completed locally, privacy-friendly), modular architecture, cross-platform support (Linux/Android), and containerized build (reproducible). Application scenarios cover fields like intelligent monitoring, industrial quality inspection, smart retail, document processing, and assistive devices.

Limitations: Due to RK3588's computing power constraints, currently only 2B/7B models are supported; quantization leads to precision loss; Rockchip's AI ecosystem is still evolving. Future outlook: Deployment of larger models, lower latency, support for more models, and improved toolchain. Summary: This project provides an excellent starting point for edge AI multimodal applications, offers references for developers on CMake builds and RKNN/RKLLM integration, and promotes AI implementation.