Sinhala Scorer: An Automated Sinhala Homework Grading System Based on a Local LLM Four-Agent Pipeline

Sinhala Scorer is an intelligent grading system designed specifically for Sinhala. It uses a four-agent NLP pipeline and local large language models to automatically evaluate student answers in a fully offline environment, addressing the pain point of the lack of automated grading tools in low-resource language education.

Natural language processing (NLP) technology mainly benefits mainstream languages like English, while intelligent tools available for low-resource languages such as Sinhala are scarce. In the education sector, teachers spend a lot of time grading homework, but automated grading tools often do not support local languages. The Sinhala Scorer project addresses this pain point by providing a complete localized intelligent grading solution.

The core of the system is a modular four-agent architecture:

1. Input Parsing and Preprocessing: Process Sinhala text (character normalization, word segmentation, etc.) and convert grading criteria into internal representations;
2. Content Understanding and Semantic Matching: Use local LLMs for semantic comparison to determine whether the core points of the answer cover the grading points;
3. Grading Decision and Weight Calculation: Synthesize factors such as completeness and accuracy to assign score proportions;
4. Result Generation and Feedback Output: Generate scores and detailed feedback. Reasons for choosing local LLMs: Privacy protection, offline operation to adapt to environments with poor network connectivity, and reduced API costs. Fully offline implementation: Pre-download model weights, local inference engine, quantized and compressed models, and RAG technology to introduce external knowledge; grading criteria adopt a structured design to ensure objectivity and flexibility.

System reliability is ensured through the following methods: Establishing a manually graded benchmark dataset to verify accuracy; calculating human-machine grading consistency metrics such as Cohen's Kappa to quantify performance; designing a confidence mechanism where low-confidence results prompt manual review.

Application scenarios of Sinhala Scorer include: Assisting in preliminary screening and standardized grading for large-scale exams; providing instant feedback on daily homework to accelerate the learning loop; serving as a tool for calibrating grading consistency in teacher training. This system is expected to improve the efficiency and fairness of Sinhala education evaluation.

Current system limitations: More suitable for objective questions, with limited ability to grade creative and open-ended questions. Future directions: Introduce multimodal support (e.g., handwritten answer recognition), develop adaptive learning mechanisms to optimize accuracy, and expand to other South Asian languages.

Sinhala Scorer successfully applies LLM technology to low-resource language education scenarios, balancing privacy protection and practicality. Its four-agent architecture provides a reference for the design of complex NLP tasks, and the fully offline operation mode points the way for the popularization of educational technology in areas with weak network infrastructure.