WARDEN: Speech Recognition and Translation for Endangered Indigenous Languages with Only 6 Hours of Data

Language diversity is an important part of human cultural heritage, but thousands of languages worldwide are facing the threat of extinction. Traditional speech recognition and translation technologies rely on large amounts of labeled data, which endangered languages precisely lack. The latest research proposes the WARDEN system, which uses a two-stage architecture (speech-to-phoneme + phoneme-to-English translation), combined with cross-language transfer and dictionary-enhanced large model reasoning. With only 6 hours of labeled audio data, it achieves high-quality transcription and translation for Wardaman, an endangered indigenous language in Australia, opening up new possibilities for low-resource language processing.

Wardaman is an endangered indigenous language in northern Australia with very few speakers. The research team faced three major challenges: only 6 hours of labeled audio (far less than the thousands of hours of data for mainstream languages), no existing Wardaman-English parallel corpus, and limited expert resources. Traditional end-to-end speech recognition-translation methods rely on large amounts of data to learn direct mappings, which is completely infeasible under such extremely low-resource conditions.

WARDEN's core innovation is its phased architecture, decomposed into two subtasks:

1. Speech-to-phoneme transcription: Convert audio into phonemes (the smallest speech units), which is a simpler task with lower data requirements;
2. Phoneme-to-English translation: Eliminate the complexity of speech recognition and better utilize existing NLP technologies. Advantages of the phased approach: Reduce single-stage complexity, enable modular training, and isolate errors (transcription errors do not propagate directly).

To address the scarcity of transcription data, a cross-language transfer strategy is adopted:

• Bridge language selection: Sundanese is phonetically similar to Wardaman;
• Phoneme embedding initialization: Use phoneme embeddings from a pre-trained Sundanese model to initialize the corresponding embeddings of the Wardaman transcription model, accelerating convergence, improving generalization (handling rare phonemes), and preserving Wardaman's unique phoneme patterns. Experiments show that this strategy significantly improves transcription performance.

To address the lack of parallel corpora in the translation phase, dictionary-enhanced large model reasoning is used:

• Expert dictionary construction: Extract high-frequency Wardaman-English vocabulary and key concept mappings from expert annotations;
• LLM combined with dictionary: Add relevant dictionary entries in prompts to guide understanding, dynamically retrieve dictionary entries corresponding to input phonemes, and generate multiple candidates for filtering and ranking. Advantages: Leverage the generalization ability of LLMs, inject domain knowledge, and improve interpretability.

Evaluation results on the Wardaman dataset:

1. Outperforms open-source models: Better performance than larger open-source models like Whisper, indicating that language-specific optimization is more important than model size;
2. Outperforms proprietary APIs: Even surpasses commercial proprietary services, proving that dedicated systems can outperform general services in specific domains;
3. Ablation experiments: Verify that the phased architecture, cross-language initialization, and dictionary enhancement all significantly improve performance.

WARDEN's success has important implications:

• Lower technical barriers: Only 6 hours of data are needed to build a practical system, reducing the cost of digitizing endangered languages;
• Community participation: Communities can organize data collection and annotation on their own and participate in technical development;
• Archive processing: Convert historical recordings into searchable text;
• Cross-language transfer: Provide a knowledge-sharing path for processing other endangered languages.

WARDEN still has room for improvement:

• Data scale: Explore semi-supervised learning, data augmentation, and active learning to expand data;
• Dialect variants: Research dialect adaptation techniques to handle language diversity;
• Multilingual expansion: Identify suitable bridge languages and build dictionaries;
• Real-time applications: Optimize inference speed and latency to support conversational translation. The research team has open-sourced the data and code, and looks forward to the community advancing research on endangered language technologies.