Reading
Tests of Qwen 2.5 1.5B on Raspberry Pi NPU, Samsung S24 Ultra, iPhone 16 Pro, and RTX 4050 show that the iPhone's throughput halved after two iterations, the S24 encountered system-enforced frequency reduction, and the Hailo-10H NPU achieved an energy efficiency ratio comparable to RTX 4050 with power consumption below 2W
Section 01
Tests of Qwen 2.5 1.5B on Raspberry Pi NPU, Samsung S24 Ultra, iPhone 16 Pro, and RTX 4050 show that the iPhone's throughput halved after two iterations, the S24 encountered system-enforced frequency reduction, and the Hailo-10H NPU achieved an energy efficiency ratio comparable to RTX 4050 with power consumption below 2W
Section 02
Model: Qwen 2.5 1.5B (4-bit quantization) Platforms:
Test Conditions: 258-token prompt, 20 rounds of thermal iteration
Section 03
Section 04
| Platform | Throughput | Power Consumption | Features |
|---|---|---|---|
| RTX 4050 | 131.7 tok/s | 34.1 W | Limited by battery power upper limit |
| Hailo-10H | 6.9 tok/s | <2 W | Limited by module memory bandwidth, near-zero variance |
Section 05
Hailo-10H NPU performed brilliantly:
Section 06
For always-on personal assistant scenarios:
Keep going with more reads from the same topic.
Splinter is a minimalist, high-performance key-value (KV) and vector storage system enabling zero-latency inter-process communication via shared memory and atomic operations. With only 766 lines of core code, it supports millions of operations per second and 768-dimensional vector storage, offering a new architectural approach for local LLM inference and data-intensive applications.
LiteMind is a Python framework that provides developers with a unified API to integrate mainstream LLM providers such as OpenAI, Anthropic, Google Gemini, and Ollama. It supports multimodal input/output, tool calling, RAG enhancement, and agent construction.
OmniRoute is an open-source AI gateway that integrates over 67 large model providers through a single OpenAI-compatible endpoint. It offers intelligent routing, load balancing, automatic degradation, and quota management, helping developers get the most available AI services at the lowest cost.
This article introduces an open-source multimodal RAG framework based on Google Gemini Embedding 2, which can uniformly handle embedding and retrieval of four media types: text, images, videos, and audio. Combined with Supabase pgvector vector database and OpenRouter large language models, it provides a complete production-level retrieval-augmented generation pipeline.