MIRA-2: A Non-Autoregressive Medical Foundation Model Eliminating Hallucinations in Medical AI via Structural Constraints

MIRA-2 is a non-autoregressive medical foundation model targeting hallucination issues in the medical AI field. Its core lies in three architectural innovations: Mamba-2 state space model, prefix tree constrained decoding, and sequential POMDP reasoning, which fundamentally eliminate the possibility of hallucinations, achieve a 100% guarantee of ontological validity, and address the limitation of traditional methods (such as scaling models and fine-tuning) that only reduce hallucinations probabilistically.

Hallucination issues in medical AI hinder practical applications: general-purpose LLMs have a severe harm rate of 22.2% in medical Q&A. Traditional mitigation methods (scaling models, fine-tuning with medical data, retrieval augmentation) only reduce hallucinations probabilistically and cannot eliminate them completely. The deeper problem is that the medical field has a strictly structured knowledge system (ontological systems like ICD-10 and CPT), while traditional autoregressive models lack structural constraints.

Non-Autoregressive State Space Model

Uses Mamba-2 as the backbone, with O(L) computational complexity to enhance long-sequence processing capabilities. Deterministic state transitions are suitable for medical rigor, and LoRA fine-tuning (22 million parameters) efficiently adapts to medical scenarios.

Prefix Tree Constrained Decoder

Pre-constructs a medical ontology prefix tree. During decoding, valid tokens are restricted via logit masking, mathematically ensuring that generated codes strictly comply with ontologies (e.g., ICD-10) and eliminating invalid outputs.

Sequential POMDP Reasoning Framework

Models medical decision-making (triage → differential diagnosis → examination → treatment) as a POMDP, trained with a CQL network to simulate real clinical thinking processes.

Processing flow:

1. Input gating (QCCS-S) extracts medical record-related sentences
2. Mamba-2 2.8B generates hidden states
3. Phase routing assigns decision-making tasks
4. Constrained decoding generates ontology-valid codes
5. POMDP reasoning optimizes sequential decisions
6. Multi-agent verification (diagnosis/questioning/safety check)
7. Dual safety layers (constellation classifier + knowledge graph contraindication check)

The final output includes triage level, code list, reasoning process, confidence level, etc.

Metric	MIRA-2	MedGemma 4B	GPT-4	AMIE
MedQA USMLE (%)	67.5	64.4	86.7	85.5
PubMedQA (%)	74.8	68.5	75.2	74.8
NOHARM Harm Rate (%)	8.7	—	15.1	12.4
Ontological Validity (%)	100	—	71.3	74.0
ECE Calibration Error (↓)	0.04	—	0.09	0.08

MIRA-2 has a much smaller parameter count than GPT-4, yet achieves 100% ontological validity and a lower harm rate.

Training is divided into 6 phases (Modal cloud GPU orchestration):

1. Backbone LoRA fine-tuning (20,000 steps, 4×A100-80GB)
2. Code head fine-tuning (EHRSHOT/MedAlign data)
3. POMDP offline reinforcement learning (CQL)
4. Reasoning head distillation (Qwen2.5-7B teacher trajectory)
5. Safety integration
6. Comprehensive evaluation

The project is open-sourced under the MIT license, and training data includes public medical benchmarks (MedQA, PubMedQA, etc.).

MIRA-2 demonstrates the possibility of structured safety: achieving mathematical safety guarantees through architectural constraints, which can be extended to fields such as law (regulatory prefix trees), finance (product code constraints), and engineering (standard part coding).

Insight: Vertical domain AI should integrate ontological constraints from the architectural design stage; "safety by design" is the key path to highly reliable AI.