Hybrid Smart Contract Vulnerability Detection: A DeFi Security Framework Combining Static Analysis, LLM, and RAG

This article introduces an innovative Ethereum DeFi smart contract vulnerability detection framework that combines the traditional static analysis tool Slither, the large language model GPT-4.1-mini, and Retrieval-Augmented Generation (RAG) technology. It continuously improves detection capabilities through the DavidAgent multi-agent adversarial iteration mechanism, aiming to address the security challenges posed by complex attack vectors in the DeFi space.

In 2024, there were 410 security incidents in the DeFi sector, causing losses exceeding $2 billion. Traditional static analysis tools like Slither and Mythril can detect some vulnerabilities, but they struggle with complex DeFi-specific attack vectors such as flash loan attacks, price oracle manipulation, and reentrancy attacks. These new types of attacks require deeper semantic understanding, which is exactly where LLMs excel.

The framework adopts a three-stage hybrid detection process:

1. Rapid Static Scanning: Uses Slither for initial scanning, taking an average of 2.2 seconds per contract to provide basic context;
2. RAG Knowledge Retrieval: Based on the ChromaDB vector database (containing over 100 DeFi attack incidents from sources like DeFiHackLabs and Rekt News), performs semantic matching via text-embedding-3-small to retrieve relevant vulnerability patterns and cases;
3. LLM Semantic Analysis: Inputs Slither results and RAG-retrieved knowledge into GPT-4.1-mini to make vulnerability judgments by synthesizing program logic, attack paths, and historical cases.

To continuously improve detection capabilities, the DavidAgent system is designed, consisting of 5 agents:

• Teacher Agent: Scans SWC vulnerability classifications and generates challenge contracts for undercovered types;
• Student Agent: Executes the LLM+RAG process to identify vulnerabilities in challenges;
• Red Team Agent: Generates variant attack samples (variable renaming, code rearrangement, etc.) for cases where vulnerabilities were not detected;
• Foundry Validator: Uses Foundry to compile variants and verify vulnerability exploitability;
• Knowledge Updater: Adds valid attack patterns to the RAG knowledge base. After 3 iterations, the EVMbench detection rate increased from 7.69% to 28.21% (a 266% improvement).

Evaluated using the SmartBugs dataset (143 vulnerable contracts, 100 safe contracts, covering 9 types of vulnerabilities), the results are as follows:

Detection Method	Recall	Precision	F1 Score	False Positive Rate	Average Time
Slither Static Analysis	94.41%	61.64%	74.59%	84.00%	2.20s
Mythril Symbolic Execution	75.00%	71.43%	73.17%	30.00%	36.24s
LLM Baseline Detection	100.00%	60.08%	75.07%	95.00%	2.81s
LLM+RAG Enhancement	99.30%	71.36%	83.04%	57.00%	2.76s
Hybrid Framework	99.30%	73.20%	84.27%	52.00%	5.76s
The hybrid framework achieved the highest F1 score (84.27%), and RAG technology reduced the false positive rate from 95% to 57% (a 40% decrease). McNemar's test showed: LLM+RAG vs LLM baseline p<0.001, hybrid framework vs Slither p=0.046, both reaching statistical significance.

In testing with the EVMbench dataset released by OpenAI and Paradigm in February 2026 (46 Code4rena projects, 120 high-severity vulnerabilities), the framework performed well on known vulnerability patterns (access control flaws, fee distribution errors, etc.), but still faced challenges with new complex vulnerabilities requiring in-depth understanding of business logic. Additionally, the phenomenon of "tool context drift" was observed: for example, in the secondswap project, the hybrid framework failed to detect the core releaseRate calculation vulnerability because Slither's static information distracted the LLM, revealing that the hybrid framework needs to balance the impact of additional information.

The framework has practical deployment value:

• Cost-Effectiveness: The API cost for GPT-4.1-mini to detect 243 contracts is only $2-$5;
• Reproducibility: Provides one-click reproduction scripts and prediction CSVs, with verifiable results;
• Scalability: Modular design supports integration of new tools or models;
• Continuous Learning: DavidAgent ensures the system evolves with new attack types. Conclusion: This hybrid framework combines traditional and AI technologies to provide a powerful tool for DeFi security, capable of identifying risks before deployment and reducing financial losses. With multi-agent iterations, detection capabilities are expected to further improve, safeguarding the blockchain ecosystem.