Appatch: An Adaptive Prompt-Based Automatic Vulnerability Repair System for Large Language Models

Appatch is an adaptive prompt-based automatic vulnerability repair system for large language models (LLMs). Its core uses vulnerability semantic reasoning and adaptive prompt techniques to guide LLMs to deeply understand the behavior of vulnerable code and generate high-quality patches. The system supports multiple mainstream LLMs (such as Claude 3.5 Sonnet, GPT-4, etc.), provides open-source code implementation and complete evaluation datasets. Experiments have verified its advantages in vulnerability repair quality and efficiency, providing an important benchmark for AI-driven software security repair.

Software vulnerability repair is one of the challenges in software engineering. Traditional Automatic Program Repair (APR) techniques have limited effectiveness when dealing with complex real-world vulnerabilities. The code understanding and generation capabilities of LLMs bring new possibilities for vulnerability repair, but the key difficulty lies in enabling LLMs to truly understand the behavior of vulnerable code—not only identifying patterns, but also grasping semantic-level content such as the root cause of the vulnerability, trigger conditions, and impact scope. Appatch is a system built around this core problem.

The two key innovations of Appatch:

1. Vulnerability Semantic Reasoning: Guides LLMs to analyze the root cause, trigger conditions, impact scope of vulnerabilities, and the semantic constraints that repairs must satisfy, ensuring that patches are semantically reasonable rather than just syntactically correct;
2. Adaptive Prompting: Dynamically adjusts prompt strategies based on vulnerability types (e.g., emphasizing pointer lifecycle for memory safety vulnerabilities, focusing on boundary checks for input validation vulnerabilities, etc.), supporting the handling of diverse vulnerability types.

Details of Appatch's implementation and evaluation:

• Multi-Model Support: Compatible with mainstream and open-source LLMs such as Claude 3.5 Sonnet, GPT-4, Gemini 1.5 Pro, Llama 3.1, etc.;
• Ablation Experiments: Designed experiments like noslice, rand_exemplars, fixed_exemplars to verify the contribution of each component;
• Comparison Baselines: Compared with traditional APR tools VulRepair and Getafix;
• Dataset Coverage: Evaluated using authoritative datasets such as PatchDB+CVEFixes, zeroday_repair, and extractfix_dataset.

Appatch explores integration with CodeQL to verify the feasibility of real-world workflows:

• codeql_appatch: A fully automated end-to-end experiment;
• codeql_human_appatch: A semi-automated experiment close to real-world scenarios; Both demonstrate the complete pipeline from vulnerability detection to repair generation.

Practical significance of Appatch: Improves repair quality (reduces the risk of new vulnerabilities), lowers repair costs (automation reduces manual work), supports multiple models, and has interpretable reasoning processes. Future directions include: supporting more programming language frameworks, deep integration with CI/CD, automatic verification of repair solutions, optimization of production environment deployment, etc.