SmartClass AI: A Real-Time Classroom Engagement Monitoring System Based on ESP32 and Deep Learning

SmartClass AI is an open-source project combining IoT hardware and AI facial emotion recognition. It uses ESP32-CAM to capture classroom images, analyzes student engagement with DeepFace, and pushes real-time metrics to a web dashboard and the teacher's OLED display. Its goal is to help teachers perceive students' status in real time and adjust their teaching accordingly.

Traditional classroom assessment relies on subjective feelings or post-class questionnaires, which have lag and bias. SmartClass AI is designed with the principle of "achieving usable real-time monitoring at the lowest hardware cost". The hardware cost is low (ESP32-CAM usually costs no more than 50 RMB on the market), and the software is based on an open-source tech stack. The core assumption is that facial expressions reflect engagement status: happiness and surprise correspond to active participation, neutrality indicates normal listening, and negative emotions may suggest a decline in attention.

The system's data flow forms a closed loop: After connecting to the local Wi-Fi, ESP32-CAM captures images every 5 seconds and sends them via HTTP POST to a local area network (LAN) Python Flask server. The server uses MTCNN to detect faces and DeepFace to classify 7 basic emotions. After analysis, annotated images are generated, and metadata (timestamp, engagement score, emotion data, etc.) is stored in an SQLite database. A built-in automatic cleaning mechanism retains the latest 100 frames of images, while metadata is kept long-term.

1. Web Dashboard: Uses Flask-SocketIO to implement WebSocket push, displaying real-time annotated images, engagement scores, and historical charts, suitable for post-class review and multi-person observation; 2. OLED Display: An independent ESP32 connects to the OLED via I2C, polls the server for scores, and issues a visual alert when engagement is below 40% to minimize interference with teaching.

The server side is recommended to run in a Python virtual environment on a Linux system; hardware firmware is flashed via Arduino IDE; the network uses a laptop hotspot to connect ESP32 devices, no need to rely on the school network; the test environment is Zorin OS 18.1, and it can run smoothly on a 13th-gen Intel i5 with 8GB RAM.

The relationship between facial expressions and engagement is not absolute (e.g., deep thought with no expression but actual engagement); classroom monitoring involves privacy ethics and requires informed consent. The project's local data processing and storage alleviate privacy concerns.

SmartClass AI demonstrates a feasible path to build an educational auxiliary system using low-cost hardware and open-source AI, providing a complete reference implementation for educational technology researchers and IoT developers. Project repository: https://github.com/md8-habibullah/smartclass-ai