ChordMiniApp: An AI-Powered Full-Stack Music Analysis Application to Make Music Learning Smarter

ChordMiniApp is an open-source full-stack music analysis application that leverages AI to simplify music learning and creation. It integrates core features like chord recognition, beat tracking, guitar fingering diagrams, piano visualization, lyrics transcription, and context-aware LLM analysis (via Google Gemini). Users can upload local audio files or analyze YouTube videos directly. This tool benefits music learners, teachers, producers, and researchers by lowering barriers to music analysis and providing intuitive visualizations.

Learning music involves multiple pain points: guitarists struggle to identify chord progressions via trial and error; piano learners need guidance on complex harmony; singers face difficulties syncing lyrics with rhythm. ChordMiniApp addresses these issues by using AI to automate these tasks, making music analysis accessible to both beginners and professionals.

ChordMiniApp uses a three-layer architecture:

• Frontend: Next.js-based responsive interface with dark mode, real-time visualizations, and YouTube integration.
• Backend: Flask framework integrating specialized models like Beat-Transformer (beat tracking), Chord-CNN-LSTM (chord recognition), and SongFormer (song segmentation).
• AI Services: Google Gemini API for context-aware analysis (Roman numeral harmony, lyrics translation, music theory explanations). Workflow: Audio extraction → preprocessing → model-based detection → result alignment (chords with beats).

Key features include:

1. Chord Recognition & Beat Tracking: Aligns detected chords with beats and annotates song structure (intro, verse, chorus).
2. Guitar Finger Diagrams: Uses @tombatossals/chords-db to show multiple chord positions, slash chords, and syncs with the beat grid.
3. Piano Visualization: Piano roll display with real-time note highlighting and MIDI export for DAW integration.
4. Lyrics Transcription & AI Analysis: Integrates Music.ai, LRClib, and Genius API for lyrics; Gemini provides translation, theme analysis, and cultural context.

Local Development:

• System requirements: Node.js18+, Python3.9+, Git LFS.
• Steps: Clone repo (with submodules), install dependencies, configure env vars (API keys), run Flask backend and Next.js frontend. Docker Deployment: Use provided docker-compose.prod.yml and .env.docker files to launch in production. Required API Keys: Firebase (storage/auth), YouTube Data API v3, Music.ai, Google Gemini, Genius.

ChordMiniApp serves various users:

• Learners: Quick access to chord charts and fingering for practice.
• Teachers: Generate visual teaching materials for harmony or instrument lessons.
• Producers: Export MIDI for DAW use and analyze harmony logic.
• Researchers: Batch process songs for harmony or structure analysis.

Current Limitations: Large model checkpoints (Git LFS required), multiple API dependencies, language support constraints, and resource needs for real-time analysis. Future Directions: Expand to bass/ukulele support, real-time microphone input analysis, collaborative features, and native mobile apps.