Validating Large Language Model Reasoning with Temporal Graph Constraints: A Structured Evaluation Approach

An MSci thesis project from the University of Edinburgh proposes a four-layer evaluation framework (Prediction, Validation, Scoring, Reporting) that converts the temporal reasoning outputs of large language models into temporal graphs for structured validation, supporting four temporal relationship labels: BEFORE/AFTER/SIMULTANEOUS/UNKNOWN. This method not only focuses on the consistency between predictions and standard answers but also detects internal contradictions in the reasoning process, providing a new paradigm for evaluating the temporal reasoning capabilities of LLMs.

Large language models perform well in natural language understanding tasks, but their temporal reasoning reliability is questionable. Temporal reasoning involves the order, duration, and overlap of events, which is crucial for applications such as document summarization and question answering. Existing evaluation methods only focus on the correctness of the final answer and ignore the internal consistency of the reasoning process. This project proposes converting the temporal reasoning outputs of LLMs into temporal graphs and performing structured validation through temporal logic constraints to address this issue.

The core of the project is a four-layer architecture:

1. Prediction Layer: Parse model outputs into events, relationships, and reasoning steps, supporting four relationship labels (BEFORE/AFTER/SIMULTANEOUS/UNKNOWN) and allowing the model to abstain when uncertain.
2. Validation Layer: Conduct reference-free validation of the internal validity of the temporal graph, checking transitive closure consistency, cyclic dependencies, conflicting constraints, and the satisfaction of temporal logic formulas.
3. Scoring Layer: Use a dual strategy to compare predictions with standard answers: direct edge scoring (comparing direct temporal edges) and closure-level scoring (comparing the complete temporal sequence after transitive closure). AFTER is normalized to the inverse of BEFORE, SIMULTANEOUS is compressed into a single node, and UNKNOWN is treated as abstention.
4. Reporting Layer: Generate structured outputs to ensure reproducibility, including config.json (configuration and version), predictions.jsonl (task results), report.json (aggregated metrics), and visual charts.

The project's technical highlights include:

• Temporal Graph Construction and LTL Validation: A lightweight temporal graph builder converts text into directed graphs, and the validation engine combines a typed invariant library with a basic subset of LTL to perform temporal checks.
• Multi-Dataset Support: Compatible with Canonical Synthetic (self-built synthetic dataset), TempEval-3, MAVEN-ERE, MATRES, and other standard temporal reasoning datasets.
• Ollama Integration: Supports local inference engines for batch evaluation of multiple models, configures experiments via JSON manifests, and generates comparison reports.
• Browser-Based Visualization Tool: verifier_explorer.html allows interactive checking of prediction results without requiring a server.

The project follows strict reproducibility standards:

1. Deterministic Execution: Supports setting random seeds to ensure reproducible results.
2. Version Control: Records code versions and dataset versions.
3. Complete Logs: Optionally records original model outputs for debugging.
4. Error Recovery: Breakpoint resumption function—failure of a single task does not interrupt the overall scan.

The significance and prospects of this work are:

1. Fine-Grained Diagnosis: Locate specific failure points in the reasoning chain.
2. Intrinsic Quality Assessment: Detect reasoning defects without standard answers.
3. Interpretability: Intuitively understand model reasoning paths through temporal graph visualization.
4. Benchmarking: Provide standardized evaluation tools for the development of temporal reasoning models. The four-layer framework can also be extended to other complex reasoning NLP tasks.

The current validator is a practical subset of an LTL model checker. Future directions include:

• Extending support for more complex temporal logic formulas.
• Integrating more open-source and commercial large language models.
• Developing a real-time reasoning visualization interface.
• Exploring the use of validation feedback for model fine-tuning.