LinguScan: An AI Language Learning Platform That Converts Real-World Text into Personalized Study Cards

LinguScan is an AI-based language learning platform. Its core function is to extract English text from real-world sources (such as books, menus, street signs, etc.) using OCR technology, generate context-aware translations with large language models, and convert them into personalized flashcards. It uses the SM-2 spaced repetition algorithm to help users master vocabulary efficiently, addressing the pain point of traditional vocabulary learning being disconnected from real scenarios.

Most traditional language learning apps use a "top-down" model (learn word banks first, then apply them), but research shows that learning in real contexts is more effective. LinguScan adopts a "bottom-up" model, converting actual text encountered by users into learning materials. Its advantages include:

1. Real context: Text comes from life scenarios, enhancing relevance;
2. Personalized content: Reflects individual interests and needs;
3. Active learning: Users actively participate in shooting and extraction;
4. Contextual memory: Vocabulary is associated with image scenes, making memory more solid.

LinguScan is a full-stack application with core components including: Backend: Built on the FastAPI framework, integrating EasyOCR (text extraction), Ollama/Llama3 (local context translation), DeepL as an alternative (high-quality translation), and PostgreSQL (storing user data and learning progress); Frontend: Built on React Native and Expo, providing interactive OCR boxes, a flashcard learning interface based on the SM-2 algorithm, and a "My Decks" management function; Deployment: One-click startup of the database, backend, and Ollama services via Docker Compose, simplifying installation and configuration.

The usage flow is intuitive:

1. Upload image: Users select/shoot an image containing English text;
2. OCR detection: The backend uses EasyOCR to recognize text and renders it in color boxes on the original image;
3. Context translation: Click the text box to call Llama3/DeepL to generate context-relevant translations;
4. Add to deck: Add interested sentences/vocabulary to personal decks;
5. Spaced repetition learning: Start quizzes in "My Decks", with the SM-2 algorithm dynamically adjusting review intervals.

SM-2 Spaced Repetition Algorithm: A classic memory optimization technique with core ideas:

• Spacing effect: Reviewing at appropriate times strengthens memory;
• Personalized scheduling: Adjust review time based on memory performance (Again/Hard/Good/Easy);
• Efficiency optimization: Extend intervals for easy-to-remember cards and shorten intervals for difficult ones. Advantages of Local AI: Using the local Llama3 model brings benefits such as privacy protection (data does not leave the device), offline availability, cost control, fast response speed, and customizability (adjusting model parameters).

Applicable to multiple scenarios:

• Travelers: Shoot menus, street signs, etc., for instant learning;
• Students: Extract new words from English books/papers to form personalized materials;
• Professionals: Learn professional terms in English emails/documents;
• Language enthusiasts: Collect daily English text to build a unique vocabulary library.

LinguScan represents an innovative direction of AI in the language learning field. By integrating OCR, local LLM, and spaced repetition algorithms, it converts the real world into personalized learning materials. This "learning from real scenarios" concept enhances learning relevance and efficiency, providing new ideas for language learning app design. For learners, it is a brand-new learning method; for developers, it demonstrates a practical case of integrating multiple AI technologies. Future progress in local AI models will drive innovation in more fields.

The project currently has limitations, and improvement directions include:

1. Language support: Expand to more languages (need to adapt OCR and translation models);
2. Translation quality: Optimize local models or enhance the switching mechanism with professional APIs;
3. UI optimization: Improve interface smoothness and navigation intuitiveness;
4. Learning analysis: Add detailed statistical functions to help users understand their learning progress and weak areas.