LeakBench: A Forensic Tool to Catch LLM 'Exam Cheating'

LeakBench is an open-source tool for detecting benchmark data contamination in large language models (LLMs). It uses statistical testing methods to identify whether a model has "seen" test data during training, addressing the issue of declining benchmark credibility, providing "forensic" assurance for LLM evaluation, and promoting transparency and standardization in AI assessment.

LLM capability evaluation relies on benchmark systems such as GLUE, SuperGLUE, HumanEval, and MMLU. However, data contamination erodes the credibility of these evaluations: training data may include test sets (direct leakage), similar texts (indirect leakage), or task instructions (task description leakage), just like students getting exam questions in advance—their scores fail to reflect their true abilities.

LeakBench detects contamination using four statistical testing methods:

1. Perplexity Analysis: Compare the perplexity distribution of the test set with that of a clean reference set; low perplexity suggests contamination.
2. Prefix Completion Test: Truncate the prefix of a test sample and let the model continue writing; the extent to which it matches the actual suffix reflects the model's familiarity with the data.
3. Membership Inference Attack: Analyze the output confidence distribution—training samples are more "confident".
4. Multi-Model Consistency Check: Compare the performance of independent models; unusual advantages may stem from contamination.

LeakBench has the following application scenarios:

1. Model Release Self-Inspection: Developers check if their models are accidentally contaminated to maintain evaluation fairness.
2. Third-Party Model Auditing: Downstream users verify the authenticity of model benchmark scores.
3. Benchmark Optimization: Maintainers identify leaked samples to improve dataset construction.
4. Academic Research Validation: Researchers prove that performance improvements come from methodological innovation rather than contamination.

When using LeakBench, the following points should be noted:

1. Statistical Threshold Issue: Detection results are probabilistic; trade-offs between false positive and false negative risks are needed.
2. Adversarial Evasion: Malicious actors may evade detection through weight reduction or forgetting learning.
3. New Contamination Forms: Methods need to be continuously updated to address hidden contamination.
4. Black-Box Model Limitation: Closed-source models do not allow access to internal states, limiting detection capabilities.

The open-source nature of LeakBench promotes transparency and standardization in LLM evaluation, providing a technical foundation for building a credible model capability assessment system. In today's era of rapid AI development, reliable evaluation methods are as important as excellent models, and LeakBench is a key step in this direction.