# NAO Social Robot Project: Graduation Design Practice on Multimodal Human-Robot Interaction

> This is an intelligent interaction system developed based on the SoftBank NAO humanoid robot platform, integrating computer vision, speech recognition, and the ChatGPT large language model. It realizes three core functions: face recognition, natural dialogue, and autonomous dancing, with behavior control implemented using a Finite State Machine (FSM) architecture.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-19T01:42:11.000Z
- 最近活动: 2026-05-19T01:53:07.256Z
- 热度: 154.8
- 关键词: NAO 机器人, ChatGPT, 人机交互, 多模态, 有限状态机, 计算机视觉, 语音识别, 大语言模型, 情感计算, 机器人开发
- 页面链接: https://www.zingnex.cn/en/forum/thread/nao
- Canonical: https://www.zingnex.cn/forum/thread/nao
- Markdown 来源: floors_fallback

---

## NAO Social Robot Project Introduction

This project is an intelligent interaction system developed based on the SoftBank NAO humanoid robot platform, integrating computer vision, speech recognition, and the ChatGPT large language model. It realizes three core functions: face recognition, natural dialogue, and autonomous dancing, with behavior control using a Finite State Machine (FSM) architecture. It is a graduation design practice on multimodal human-robot interaction.

## Project Background: Exploration of Socialization in Humanoid Robots

In the field of human-robot interaction, traditional rule-based or pre-scripted interaction methods are rigid and lack flexibility. With the maturity of large language models (LLMs), embedding AI dialogue capabilities into physical robots has become possible. NAO is a classic humanoid robot platform developed by SoftBank Robotics, equipped with rich sensors, flexible movement capabilities, and a mature development framework. As a graduation design, this project explores combining ChatGPT's intelligent dialogue capabilities with NAO's physical interaction capabilities to create a robot system with social attributes.

## System Architecture and Technical Methods

### System Architecture
Adopting a Finite State Machine (FSM) architecture, three core behavior states are defined:
- **Idle State**: Real-time face detection, user identity recognition, new user registration, voice command monitoring
- **Dialogue State**: Speech-to-text conversion, ChatGPT response generation, speech synthesis output, context memory
- **Dance State**: Music detection, dance choreography execution, smooth transition

### Technology Stack
- Core Platform: NAOqi SDK, Python, OpenCV
- Perception & Interaction: Google Speech API/Offline Speech Recognition, Audio Processing, Face Detection & Recognition
- Intelligent Core: OpenAI API, Finite State Machine Design Pattern

### State Machine Design
Advantages: Clear state boundaries, simple transition logic, easy to debug and extend; Trigger events include voice commands (Hey NAO/Dance NAO, etc.), face recognition events, and audio detection.

## Function Implementation and Verification

### System Workflow
1. Start and enter the idle state
2. Environment scanning (face detection, greeting/inviting registration)
3. Monitor commands:
   - Hear "Hey NAO" → Dialogue state
   - Hear "Dance NAO" → Dance state
4. Exit conditions: Saying "Goodbye" during dialogue / Saying "Stop NAO" during dance returns to idle state

### Verification & Highlights
- Video demonstration provided (link in README)
- Technical highlights: Multimodal fusion (visual/auditory/language), LLM physicalization (ChatGPT empowers natural dialogue), preliminary exploration of emotion computing (dance expresses emotions)

## Team Collaboration and Division of Labor

Development team division:
- Computer Vision Integration: Face detection, recognition, and user management modules
- Robot Behavior Programming: NAOqi framework calls and action choreography
- ChatGPT Dialogue System Design: OpenAI API integration and dialogue flow design
- Audio Processing & State Control: Music detection and state machine implementation

Team members: David Shi, Ousama Alabdullah, Humaira Saddat

## Limitations and Improvement Directions

Limitations of the system and improvement suggestions:
- Offline Capability: Relies on OpenAI API; dialogue function unavailable offline
- Multilingual Support: Chinese-English switching is not intelligent enough
- Dance Diversity: Only preset sequences; real-time choreography can be added
- Emotion Recognition: Currently only able to express emotions, cannot recognize user emotions

## Summary and Insights

### Summary
This project is an excellent graduation design work, demonstrating the combination of cutting-edge AI technology and a mature robot platform. It realizes the orderly coordination of three functions through the FSM architecture, providing a reference example for robot development and human-robot interaction learning.

### Insights
- State machines are an effective model for robot behavior management
- LLMs transform robots from "executors" to "dialoguers"
- Multimodality is the inevitable path to natural human-robot interaction