When Large Language Models Meet Curling: New Explorations of Intelligent Interaction in VR Scenarios

The research team from Technical University of Munich (TUM) has introduced Large Language Models (LLM) into the VR curling training environment, exploring the potential fusion of sports teaching and generative AI. This initiative aims to address challenges in curling popularization such as scarce professional venues, high equipment costs, and climate constraints, providing a new direction for AI-assisted sports teaching.

Curling, known as the "chess on ice", is famous for its strategic nature and team collaboration, but its popularization faces challenges like scarce professional venues, high equipment costs, and climate constraints. In recent years, VR technology has provided new possibilities for sports training, and generative AI has driven changes in interactive learning. Therefore, the TUM team has carried out innovative research on integrating LLM into VR curling training.

The project uses the Vite build tool combined with modern front-end frameworks to achieve smooth 3D rendering, supporting VR devices like Meta Quest. The core lies in integrating the natural language understanding capabilities of LLM to real-time parse users' voice/text inputs and provide technical guidance, tactical suggestions, and feedback. It also builds an immersive curling venue environment, simulating the curling process and physical properties.

The research results have formed complete academic outputs: the preprint is published on the arXiv platform (arXiv:2408.09285) for global researchers to review and cite; the official publication is in Springer with DOI 10.1007/978-3-031-91572-7_11, ensuring quality through peer review.

Interdisciplinary integration of computer science (VR, NLP) and sports science; emphasis on user experience, collecting data such as interaction efficiency, satisfaction, and learning effects through usability tests; the methodology can be transferred to fine skill learning scenarios like golf, tennis, and surgical training.

The open-source nature of the project ensures reproducibility and facilitates expansion by other researchers; it provides practical cases for learners in VR development, LLM applications, or sports technology; the integration configuration of Vite and VR, as well as the interaction ideas between LLM and 3D scenes, provide references for technical personnel.

Currently, it is an exploratory prototype (prompts are only for testing). Future directions include: integrating multi-modal interactions such as gesture recognition/eye-tracking; personalized feedback based on user progress; multi-person collaborative virtual training; optimizing the physics engine to improve simulation accuracy.

This research demonstrates the potential of combining LLM and VR to redefine skill learning. Curling is just the starting point; AI+VR is expected to bring revolutions in sports education, vocational training, and other fields. For developers, it proves that Web technologies can support complex VR+AI applications; for practitioners, it provides a technical route; for users, it foreshadows the future of AI personal coaches.