AI 2048 Coach: A Neural Network System for Real-Time Guidance of iPhone 2048 via Screen Mirroring

This is an innovative AI game coach system for the iPhone version of 2048, developed by Illya97 and released on GitHub on July 12, 2026 (link: https://github.com/Illya97/2048). It captures the iPhone screen via AirPlay mirroring, recognizes the game board in real time, and uses a self-learned n-tuple neural network combined with expectimax search to provide optimal sliding suggestions within 45 milliseconds, displayed as on-screen arrows.

Key technologies involved: 2048 game screen recognition, n-tuple neural network, TD/TC reinforcement learning, expectimax search, real-time system integration.

The AI coach system assists iPhone 2048 players with a 4-step workflow:

1. Capture screen via AirPlay mirroring.
2. Recognize 16-grid game state.
3. Compute optimal move using trained n-tuple neural network + 4-step expectimax search.
4. Show suggested direction via green arrows.

The core decision component is an n-tuple neural network trained on 9 million self-play games using TD/TC reinforcement learning, accelerated by numba with a pure Python alternative available.

Real-time Screen Recognition

• Adaptive palette: Learns any game theme's color config.
• Anti-interference: Robust to mirroring noise and fast-slide blurriness.
• Full grid: Accurately identifies all 16 cells.

N-tuple Neural Network

• Architecture: 9 templates ×8 symmetries, numba-accelerated.
• Training: 9M self-play games with TD/TC learning.
• Search: 4-step expectimax (≈45ms per move).

Smart State Management

• Round prediction: Shows estimated remaining rounds and completes missing steps.
• Restart advice: Warns of weak rounds at ~300 steps if below average.
• End control: Suggests 1024+1024 merge only if expected score >47k; manual end via 🏁 button.
• Endgame protection: Hides outdated arrows when multiple high-value blocks exist.

The system uses intuitive cues:

• Green arrows: Recommended sliding direction.
• No arrows/⏳ Thinking...: System computing move.
• Optional sound prompts: Notify when suggestion is ready.

These help players quickly understand system state and follow recommendations.

Environment Requirements

• Python ≥3.10; iPhone + computer on same Wi-Fi.

Installation

1. Clone repo: git clone https://github.com/Illya97/2048.git && cd 2048
2. Install dependencies: pip install -r requirements.txt

AirPlay Setup

1. Install receiver (e.g., LetsView) on computer.
2. iPhone: Control Center → Screen Mirroring → Select computer.
3. Open 2048 game.

Calibration & Launch

• Calibrate: python -m coach --calibrate (verify 16-grid recognition).
• Start: python -m coach.

Custom Theme Support

1. Capture board screenshot as shot.png.
2. Learn colors: python -m coach --learn shot.png (input grid numbers to train config).

Pre-trained Weights

• Stored in weights/ntuple.npz (not in repo; expectimax as backup).

Continue Training

• Command: python scripts/train_ntuple.py --tc --resume --games 1000000
• Options: --parallel (all CPU cores), --warm-from path.npz (hot-start).

Benchmark

• Command: python scripts/benchmark.py --farm --agent ntuple --nt-depth4 --games100 --seed200

Evaluates move quality and speed.

Significance

• Integrates CV, RL, and real-time systems into a practical tool.
• Applications: Game AI research, real-time image processing, RL demonstration, cross-device interaction.
• For players: Learning tool for high-score strategies.

Limitations

• Single monitor required.
• Re-calibrate if mirror window is moved.
• Classic theme: Blocks >4096 share same color.