# VCAO: A Game Theory-Based Verifier-Centered Agent Orchestration System for Strategic Discovery of OS Vulnerabilities

> This paper proposes the VCAO framework, which models OS vulnerability discovery as a Bayesian Stackelberg search game. It dynamically allocates analysis budgets via a large reasoning model orchestrator, combining verifiers like static analysis, fuzz testing, and memory detectors to achieve efficient automated vulnerability mining.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-04-09T14:27:10.000Z
- 最近活动: 2026-04-10T02:28:03.212Z
- 热度: 139.0
- 关键词: VCAO, 漏洞发现, 操作系统安全, 博弈论, 斯塔克尔伯格博弈, 智能体编排, Linux内核, 自动化安全分析
- 页面链接: https://www.zingnex.cn/en/forum/thread/vcao
- Canonical: https://www.zingnex.cn/forum/thread/vcao
- Markdown 来源: floors_fallback

---

## [Introduction] VCAO: A Game Theory-Based Agent Orchestration System to Improve OS Vulnerability Discovery Efficiency

This paper proposes the VCAO (Verifier-Centered Agentic Orchestration) framework, which models OS vulnerability discovery as a Bayesian Stackelberg search game. It dynamically allocates analysis budgets via a large reasoning model orchestrator, combining verifiers like static analysis, fuzz testing, and memory detectors to achieve efficient automated vulnerability mining. The system performs excellently in Linux kernel vulnerability discovery, significantly outperforming existing methods.

## Background: Challenges and New Paradigms in OS Vulnerability Discovery

The OS kernel is the cornerstone of computing systems, but its large code size (e.g., Linux kernel has over 30 million lines) makes vulnerability discovery difficult. Traditional methods (static analysis, fuzz testing, symbolic execution) each have limitations: static analysis has high false positives, fuzz testing has limited coverage, and symbolic execution is constrained by path explosion. The rise of large language models brings new possibilities for vulnerability discovery, but it requires intelligent orchestration of heterogeneous tools to work together—thus VCAO was born.

## Methodology: Game Theory Modeling and Six-Layer Architecture Design

VCAO models vulnerability discovery as a repeated Bayesian Stackelberg search game: the defender (system) allocates budgets, the attacker looks for missed vulnerabilities, and Bayesian beliefs are updated over multiple rounds to adjust strategies. The system uses a six-layer architecture: Surface Mapping Layer (code structure analysis), Intra-Kernel Attack Graph Construction Layer (data flow/control flow dependencies), Game-Theoretic File/Function Ranking Layer (core decision-making, MILP for optimal allocation), Parallel Executor Agent Layer (manages verifier tool instances), Cascaded Verification Layer (cross-validation to reduce false positives), and Security Governance Layer (system monitoring).

## Evidence: Experimental Evaluation and Performance Comparison Results

Experiments were conducted on 5 Linux kernel subsystems: 1. Historical CVE Reproduction: Rediscovered 847 known CVEs; 2. Real-Time Discovery: Found 0-day vulnerabilities in the latest kernel snapshot. Performance comparison: Per unit budget, it found 2.7x more valid vulnerabilities than coverage-guided fuzz testing, 1.9x more than static analysis, and 1.4x more than non-game-theoretic multi-agent pipelines; false positive rate was reduced by 68%.

## Key Insights: Core Factors for VCAO's Efficiency

Key factors for VCAO's success: 1. Dynamic Adaptability: Adjust strategies based on real-time results; 2. Heterogeneous Tool Collaboration: Optimize complementary tool combinations; 3. Strategic Prioritization: Focus on high-risk components via attack graphs and Bayesian beliefs, ensuring resources are used where they matter most.

## Open Source Contributions and Future Outlook

The research team open-sourced the simulation framework, synthetic attack graph generator, and evaluation toolchain to facilitate reproduction, extension, and standardized evaluation. Limitations: Currently focused on Linux kernel, attack graphs rely on static analysis, and game parameters need calibration. Future directions: Extend to user-space applications, introduce more verifier tools, and explore collaboration between multiple VCAO instances.

## Conclusion: Significance of VCAO for the Vulnerability Discovery Field

VCAO combines game theory, Bayesian reasoning, and large language models to build an intelligent adaptive vulnerability discovery system. In today's era of severe cybersecurity threats, such automated tools can help security teams quickly locate risks and fix vulnerabilities proactively. AI will play an increasingly important role in the software security field.