SharkEye: A Local Large Model-Powered Network Intrusion Detection System Running on Raspberry Pi 5

SharkEye is a self-hosted Network Intrusion Detection System (NIDS) that innovatively integrates deep packet inspection technology with local Large Language Model (LLM) inference capabilities. It can run fully offline on Raspberry Pi 5. It addresses the problem that traditional NIDS rely on predefined rules and struggle to handle new types of attacks, enabling intelligent detection and reporting of malicious network activities. It also offers advantages such as privacy protection, low latency, and cost-effectiveness.

Traditional network intrusion detection systems usually rely on predefined rules and signatures to identify malicious traffic, and they are ineffective against new and complex attacks such as zero-day attacks and Advanced Persistent Threats (APT). SharkEye's core innovation lies in introducing locally running LLMs, which can understand the semantic content of network traffic and identify complex attack patterns that are difficult to detect with traditional methods.

SharkEye seamlessly integrates network traffic capture, protocol parsing, and LLM inference: first, it extracts traffic features and payloads through deep packet inspection, then formats the input for LLM to perform semantic analysis. Optimized for Raspberry Pi 5 (4-core ARM Cortex-A​76 + optional 8GB memory), it uses technologies like model quantization (FP32​→INT8/INT4) and ARM-optimized inference frameworks (e.g., llama.cpp) to achieve smooth operation. Edge deployment advantages include: fully offline privacy protection, low-latency real-time detection, low cost with no cloud fees, and distributed scalability.

1. Data Capture and Preprocessing: Capture traffic using packet capture libraries, parse protocol information such as HTTP/DNS/TLS, and clean structured data (remove sensitive information while retaining threat features);
2. LLM Inference and Decision-Making: Input formatted data, output threat levels (normal/suspicious/malicious), natural language reports, and JSON-structured alerts;
3. Alert and Response: Multi-channel notifications (logs/Webhook/emails), support for custom scripts for automated mitigation (block IPs, adjust firewalls).

• Home Networks: Deployed on routers/Raspberry Pi to monitor IoT device anomalies, malware communications, and phishing attacks;
• Small Enterprises: Cost-effective basic intrusion detection solution without the need for a professional team;
• Edge/Industrial IoT: Adapt to network-constrained/unstable scenarios and strictly protect data privacy.

Resource Constraints: Reduce memory usage through model quantization, improve efficiency with ARM-optimized inference engines, and reduce redundant computations via batch processing and caching; Real-Time Requirements: Use stream processing to avoid delays, prioritize high-risk traffic with priority queues, and decouple capture and inference with an asynchronous architecture.

Possible future development directions for SharkEye include: multi-modal detection (combining traffic, system logs, and file behavior), federated learning (enhancing model capabilities under privacy protection), automatic rule generation (LLM automatically updates detection rules), and threat intelligence integration (importing external intelligence to enhance detection).

SharkEye demonstrates the practical value of LLMs on resource-constrained edge devices (such as Raspberry Pi 5), creating a new paradigm for threat detection that combines traditional security technologies with cutting-edge AI. For developers, it shows optimized deployment methods for LLM applications in specific scenarios; for security practitioners, it provides a low-cost and high-privacy solution. With the development of edge computing and AI, more such innovative applications will emerge.