Local AI Pose Analyzer: A Solution Combining Large Language Models and Pose Estimation Running on CPU

The open-source project pose-ai-processor developed by HJKMCS is a lightweight desktop application that combines MediaPipe pose detection with locally deployed large language models via LM Studio to achieve fully offline video pose analysis. This solution prioritizes privacy protection, supports CPU-optimized operation, requires no cloud APIs or high-performance GPUs, and is suitable for various scenarios such as fitness and rehabilitation medicine.

Current pose estimation and video analysis often rely on cloud APIs or high-performance GPUs, which pose privacy risks and limit usage on resource-constrained devices. This project aims to provide a fully local, CPU-optimized solution to address data leakage issues and hardware investment barriers.

Modular Architecture

Adopts a 10-file modular design, follows the principle of separation of concerns to enhance stability and maintainability. Key modules include main program entry, pose engine, AI client, etc.

Pose Detection Layer

Uses the lightweight MediaPipe framework, supports real-time CPU performance, and is compatible with low-power devices.

LLM Integration

Achieves offline intelligent analysis via LM Studio's local API, supporting dynamic adjustment of parameters such as temperature and Top-K.

Data Output

Generates timestamped JSON files containing frame-by-frame pose data, video metadata, LLM configuration snapshots, and analysis conclusions.

Fully Localized

All processing is done on the user's device; video data is not uploaded to external servers, making it suitable for sensitive content (e.g., medical rehabilitation, personal fitness).

Zero API Cost

Uses local LM Studio with no API call fees; a one-time hardware investment supports unlimited analysis, providing significant cost advantages in high-frequency scenarios.

• Fitness and Sports Science: Analyze the standardization of trainees' movements, identify posture issues, and provide correction suggestions.
• Rehabilitation Medicine: Track patients' rehabilitation progress and quantitatively evaluate treatment effects.
• Ergonomic Assessment: Detect sitting posture and remind users to adjust bad postures.
• Dance and Performing Arts: Analyze movement details and identify posture deviations.

1. Install LM Studio and start the local server (default port 1234);
2. Clone the repository: git clone https://github.com/HJKMCS/pose-ai-processor.git;
3. Install Python dependencies according to requirements.txt;
4. Run the main program, load the video file, and start analysis.

Current Limitations

• Relies on local LM Studio deployment, which has hardware memory requirements;
• Longer processing time for high-resolution videos;
• Pose detection accuracy is affected by video quality and lighting.

Improvement Directions

• Support more pose detection models (e.g., YOLO-Pose);
• Add real-time camera input;
• Introduce multi-person pose analysis;
• Develop a plugin system and time-series visualization.

pose-ai-processor demonstrates the possibility of local applications combining computer vision and large language models. Its modular architecture and privacy-first design provide a reference for similar projects. For users concerned about data privacy and wanting to run AI analysis locally, it is an open-source solution worth trying.