# Squat-Coach: Edge AI Fitness Coach with Local Visual Inference for Real-Time Squat Guidance

> Squat-Coach is an innovative edge AI project that combines MediaPipe pose estimation and the local Gemma model to deliver fully offline real-time squat guidance. Using a teacher-student training paradigm, it generates training data via Gemini and fine-tunes the local model, providing professional fitness feedback while protecting privacy.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-14T08:13:21.000Z
- 最近活动: 2026-05-14T08:19:44.020Z
- 热度: 154.9
- 关键词: 端侧AI, 姿态估计, MediaPipe, Gemma微调, 健身教练, 隐私保护, 教师学生训练, 计算机视觉, 生物力学, 本地推理
- 页面链接: https://www.zingnex.cn/en/forum/thread/squat-coach-ai
- Canonical: https://www.zingnex.cn/forum/thread/squat-coach-ai
- Markdown 来源: floors_fallback

---

## 【Introduction】Squat-Coach: Edge AI Enables Offline Real-Time Squat Guidance

Squat-Coach is an innovative edge AI project that combines MediaPipe pose estimation and the local Gemma model to deliver fully offline real-time squat guidance. Using a teacher-student training paradigm, it generates training data via Gemini to fine-tune the local model, providing professional fitness feedback while protecting privacy.

## Project Background & Vision: Addressing Pain Points in Professional Fitness Guidance

In the fitness field, correct posture is crucial for training effectiveness and safety, but professional coaches are expensive and cannot be available at all times. Squat-Coach aims to provide an always-accessible, fully private virtual fitness coach using edge AI technology. Its core vision is to create a coach agent with visual input, edge inference, real-time feedback, and intelligent decision-making capabilities—not just a simple overlay of AI text on pose data.

## System Architecture: Layered Design for Intelligent Guidance

Adopting a layered architecture:
- **Visual Perception Layer**: MediaPipe Pose Lite (runs in browser via WASM/WebGL, 4MB model, extracts 3D key points)
- **Biomechanical Interpretation Layer**: Calculates angles of knees/hips/torso/shins, identifies squat phases (standing/squatting/bottom/rising)
- **Intelligent Coach Layer**: Fine-tuned Gemma model as the 'brain' to output guidance suggestions
- **Agent Decision Loop**: Setup (check framing), Calibration (personalized thresholds), Real-Time Guidance, Inter-Set Summary

## Privacy First: Fully Offline Operation

The core selling point is that all processing is done locally—workout videos never leave the device, and it works even without an internet connection. Dependencies (MediaPipe WASM, JS packages, pose model) are downloaded once via the setup.sh script, with no network requests during runtime, ensuring privacy and reliability.

## Training Method: Teacher-Student Paradigm

Innovative training method:
- **Teacher Model**: Cloud-based Gemini analyzes real squat videos + rules to generate annotated datasets
- **Student Model**: Gemma is fine-tuned with the dataset to learn to generate feedback based on pose data
- **Data Quality**: Emphasizes the importance of real data; recommends collecting standard side views and error actions (shallow squats/forward lean/knee valgus, etc.) to cover scenarios.

## Technical Details: Input, Output & Configuration

Technical implementation:
- **Input Features**: 7 core features (phase/rep_count/knee_angle/hip_angle/torso_angle/shin_angle/hip_below_knee)
- **Output Specification**: JSON format (say: guidance text; priority: priority level; ui: UI instructions; cooldown_s: cooldown time)
- **Runtime Configuration**: Sampling parameters (temperature/top_p/top_k), repetition penalty, maximum length, random seed, etc.

## Application Value & Future Outlook

**Application Value**:
- Provides users with private, low-cost professional guidance—only requires a regular computer + camera
- Offers developers a reference solution for edge AI integration (large model + computer vision)

**Project Status**: Core functions (local pose detection/rep counting/angle calculation, etc.) are implemented; Gemma fine-tuning and dataset auditing are ongoing
**Future Directions**: Support more exercise types, personalized training plans, integration with fitness devices.