Building an English-Spanish Translation System from Scratch: End-to-End Neural Network Implementation Based on Transformer

The english-spanish-translator project introduced in this article provides a complete solution for building an English-Spanish translation system from scratch, covering the full workflow of custom Transformer model implementation, OPUS corpus training, FastAPI service deployment, and RAG-assisted review. The project processes over 4 million aligned sentence pairs, achieves a test sacreBLEU score of 31.41, and is suitable for machine learning practitioners, students, and code reviewers to learn from. It also provides a directly deployable FastAPI interface.

Since its introduction in 2017, Transformer has changed the landscape of NLP. It processes sequences in parallel based on attention mechanisms, making it more efficient. The project adopts an encoder-decoder structure: the encoder encodes English sentences into context vectors, and the decoder autoregressively generates Spanish translations. Each layer includes multi-head self-attention and a feed-forward neural network, and positional encoding compensates for the lack of sequential modeling. The project implements Transformer from scratch using PyTorch (source/Model.py), with advantages including deep understanding of key technologies, flexible architecture customization, and high educational value.

High-quality data is key to success. The project builds a complete data pipeline: using the OPUS open-source parallel corpus (including sources like web pages and government documents); preprocessing steps include corpus download, text cleaning (removing HTML, normalizing whitespace, filtering low-quality sentence pairs), subword segmentation (BPE/SentencePiece), and sequence truncation (60 tokens). Finally, about 4.39 million aligned sentence pairs are obtained, with 3.51 million in the training set and 870,000 in the test set.

Training configuration: NVIDIA RTX PRO 6000 Blackwell GPU, 30 epochs, batch size 640, maximum sequence length 60, learning rate 4.5e-4, Adam optimizer. Weights & Biases are used to monitor training, with the best validation loss of 2.5055 (at epoch 29). Evaluation uses sacreBLEU (standardized BLEU), and the test set score is 31.41, indicating that the custom model performs well.

The project provides a FastAPI RESTful service with endpoints including health check (/health), direct translation (/translate), and institutional review translation (/institutional-review). It supports Docker containerization deployment; when the container starts, it automatically downloads the pre-trained model, making the service immediately available. Example: A curl request for health check returns {"status":"ok"}, and a POST request to /translate can get translation results.

RAG is introduced to improve the translation quality of formal documents: ChromaDB is used to build a translation memory from the Europarl corpus (with standardized language and accurate terminology); the institutional-review process includes retrieving similar example sentences → combining with model output → optional polishing with GPT-4o-mini. This hybrid method combines the fluency of neural translation with the accuracy of retrieved terminology, making it suitable for fields like law and healthcare.

The project demonstrates good engineering practices: clear directory organization (e.g., .github/, agent/, rag/, source/); GitHub Actions automatically run code checks (ruff), unit tests (pytest), and dependency audits; a REPRODUCE.md document is provided to ensure result reproducibility.

This project is an excellent end-to-end machine translation case, covering the entire lifecycle from data preparation to model deployment. It is of reference value to developers who want to deeply understand the principles of neural machine translation and engineers who need to customize translation systems. Its clear code, complete documentation, and engineering practices make it an ideal resource for learning and reference.