Sketch-AI: When Hand-drawn Whiteboard Meets Large Language Models

Sketch-AI combines Excalidraw's hand-drawn style with large language models (LLMs) to address the pain point that creating high-quality educational charts requires professional skills and a lot of time, thus creating an 'what you think is what you get' intelligent visual learning tool. It can automatically generate dynamic charts, flowcharts, and animations, making complex STEM concepts intuitive and easy to understand, and lowering the threshold for knowledge visualization.

In learning and teaching, complex concepts often require charts to aid understanding, but creating high-quality educational charts requires professional skills and a lot of time, which deters many educators and learners. The Sketch-AI project was born to solve this pain point, pioneering a new model of intelligent visual learning.

Sketch-AI's core features include:

1. Intelligent Chart Generation: Generate corresponding charts (e.g., blockchain principle diagrams) with natural language descriptions;
2. Multi-domain Support: Optimized for STEM education, covering physics, mathematics, engineering, computer science, etc.;
3. Dynamic Animation Demonstration: Generate animation sequences to show concept processes (e.g., sorting algorithm steps);
4. Hand-drawn Style Preservation: Retain Excalidraw's iconic hand-drawn style to reduce formality. The tech stack is based on Excalidraw, LLM, animation engine, and React+TypeScript front-end architecture.

Sketch-AI's technical architecture is divided into four layers:

1. Concept Parsing Layer: LLM extracts key concepts and relationships, converting them into structured knowledge graphs;
2. Layout Planning Layer: Plans the spatial layout of the chart, handling element positions and layers;
3. Graphic Generation Layer: Generates Excalidraw JSON format data (shapes, coordinates, styles);
4. Animation Choreography: Generates timeline information to define element transformation effects. Solutions to technical challenges: Force-directed graph for optimized layout, concept-graphic mapping knowledge base, learning from teaching videos to optimize animation timing.

Application Scenarios:

• Online education: Teachers quickly generate course illustrations to improve students' understanding efficiency;
• Technical documentation: Developers generate architecture diagrams/flowcharts to make documents more intuitive;
• Knowledge sharing: Creators make popular science illustrations to lower the threshold for content creation;
• Personal learning: Convert text to charts to deepen memory. Tool Comparison:

  Tool	Automation Level	Hand-drawn Style	Animation Support	Open Source
  Sketch-AI	High (AI-generated)	Yes	Yes	Yes
  Excalidraw	Low (manual drawing)	Yes	No	Yes
  Mermaid	Medium (code-generated)	No	Limited	Yes
  D2	Medium (code-generated)	No	No	Yes
  Canva	Medium (template-driven)	Partial	Yes	No

Conclusion: Sketch-AI uses technology to lower the threshold for knowledge visualization and redefines the way we learn and share knowledge. Open Source and Future:

• Open source ecosystem: MIT license, welcome community contributions; development priorities include more domain templates, multi-language support, collaboration features, multi-format export;
• Future outlook: Personalized learning paths, interactive exploration, cross-modal fusion (voice + text + visual); Quick Experience: Clone the repository → install dependencies → configure LLM key → start server → input description to generate charts.