NeoMind: A Rust-based Edge AI Platform Enabling Autonomous Decision-Making for IoT Devices via Large Language Models

NeoMind is an edge AI platform built with Rust that enables autonomous management and automated decision-making for IoT devices using Large Language Models (LLMs). It supports full features including multi-backend LLMs, BLE network configuration, MQTT protocol, AI Agent tool calling, real-time visual analysis, and offers two deployment modes: desktop application and server deployment. It aims to address challenges in edge AI and IoT integration such as protocol heterogeneity, real-time requirements, and security isolation.

With the rapid evolution of Large Language Model (LLM) capabilities, deploying them on edge devices to achieve localized, low-latency intelligent decision-making has become an industry hotspot. However, deep integration of LLMs with the Internet of Things (IoT) ecosystem faces many challenges: device protocol heterogeneity, real-time requirements, security isolation, multi-modal data processing, etc. NeoMind emerged as a solution, using Rust as the core service implementation language, combined with React/TypeScript frontend and Tauri desktop framework, providing an integrated solution from edge to cloud.

NeoMind adopts a modular, event-driven architecture design where components communicate decoupled via an Event Bus. The core layer includes neomind-core (defines core traits and type system), neomind-api (Axum-based Web API service), neomind-agent (AI Agent module), neomind-storage (redb-based embedded storage); the device and automation layer includes neomind-devices (MQTT device management, BLE network configuration), neomind-rules (rule engine), neomind-messages (message system); the extension layer provides an SDK and a process-isolated extension execution environment. The frontend is based on React18+TypeScript+Tailwind CSS, packaged into a cross-platform desktop application via Tauri 2.x.

NeoMind supports multiple LLM backends (Ollama, OpenAI, Anthropic, Google, xAI, Tongyi Qianwen, etc.), allowing users to choose flexibly. The AI Agent system has tool calling capabilities, divided into Focus Mode (single-round fast response, suitable for monitoring tasks) and Free Mode (multi-round open reasoning, suitable for fault diagnosis). Built-in tools include device query and control, Shell tools, AI metric tools, and a custom skill system (YAML+Markdown format).

Device access methods: BLE network configuration (zero-touch setup, supports multiple network types), MQTT protocol (built-in broker, mTLS authentication), automatic discovery and registration (LAN detection, AI-assisted type recognition). Real-time visual analysis transmits video frames via WebSocket streaming, can configure any LLM backend as a visual model, and the results are bound to data sources and displayed on the dashboard for scene understanding (e.g., person detection, device anomalies).

Storage architecture: Time-series storage (redb, efficient range query and aggregation), LLM classified memory (Profile user preferences, Knowledge domain knowledge, Tasks task history, Evolution system evolution insights), vector search (semantic retrieval). For security: The extension system uses a capability model for access control (e.g., device_control, agent_invoke, etc.), supports native dynamic libraries and WASM format, runs in process isolation, and automatically restarts on crash.

Deployment options: Desktop application (cross-platform, built-in server and Web UI), server binary (headless deployment, with built-in static service), one-click installation script (curl -fsSL https://raw.githubusercontent.com/camthink-ai/NeoMind/main/scripts/install.sh | sh). Typical scenarios include smart home hub, industrial equipment monitoring, laboratory automation, edge AI inference node, etc.

NeoMind demonstrates a modern edge AI system design paradigm: Rust's memory safety and asynchronous performance, native LLM support, deep IoT integration, security isolation mechanism, and developer-friendliness (480+ unit tests, complete CLI, extension SDK). As an open-source project, it provides a complete reference implementation for the next generation of intelligent edge devices, with a modular design that facilitates customization and expansion.