LAPAI: Localized Personal AI Runtime Framework, an Open Source Project Enabling Lightweight Models with Persistent Memory and Learning Capabilities

LAPAI is an AI runtime framework focused on local offline operation. Through a dual-track memory system combining SQLite FTS5 and FAISS vector retrieval, it equips lightweight language models with long-term memory, knowledge learning, and personalized adaptation capabilities. It supports dual backends (LemonadeServer and Ollama), provides a complete local AI solution for game development, IoT, and personalized assistants, and emphasizes data privacy and local sovereignty.

With the development of large models, the demand for localized deployment has grown. However, existing solutions have issues such as privacy risks due to cloud dependence, high hardware requirements for large models, and lack of personalized memory mechanisms. LAPAI takes "lightweight yet powerful" as its core concept; through architectural optimization, it enables small models with around 3 billion parameters to perform excellently in specific tasks, and all data is saved locally, so conversations and knowledge accumulation are not lost.

LAPAI adopts a hybrid retrieval strategy, combining SQLite FTS5 full-text search (keyword exact matching) and FAISS vector embedding (semantic similarity retrieval). It includes a text retrieval layer, a FAISS vector index layer (supporting GPU acceleration and DirectML backend), and a multi-dimensional scoring mechanism (timeliness, importance, role weight). It has session management and automatic summarization functions, forming a learning loop of original conversation → summary → knowledge extraction.

Supports session continuity maintenance (resume from interruption) and personal knowledge management (automatically extract and save user information). It has a built-in learning pipeline that extracts thoughts and knowledge points from conversations to build a personalized knowledge base; users can customize the AI personality via PersonaAI.txt. It integrates the Coqui XTTS-v2 local TTS engine and provides an OpenAI-compatible API to lower the integration threshold.

Dual backend architecture: For AMD Ryzen AI devices, LemonadeServer is recommended (utilizing NPU acceleration); for other configurations, Ollama is used (compatible with models like Llama and Mistral). The minimum configuration requires 24GB of memory; it supports Ryzen AI/Intel Ultra processors and RTX2060+ graphics cards. When there is no dedicated graphics card, integrated graphics can also provide basic performance.

Game development: Unity integration example to create intelligent NPCs, controlling character actions via JSON responses. IoT: Arduino integration example to realize natural language control of LED/OLED peripherals. Minecraft: Integration with the Touhou Little Maid mod to provide immersive virtual character interaction.

All conversations, memories, and knowledge bases are stored locally, and users have full control over their data. Personal data is stored separately for easy backup and migration. It uses the MIT open-source license; core code is transparent, and third-party component licenses are clear, eliminating concerns about data leakage.

LAPAI proves that lightweight models can achieve interactive experiences close to large models through exquisite design, providing complete solutions for scenarios like games and IoT. In the future, it may add features such as online learning and multi-modal interaction, making it a noteworthy open-source project for local personalized AI.