RUSH：用专家反馈和高推理模型构建策略图系统

RUSH (Reinforcement learning Using SME feedback and High-reasoning models) is a strategy graph system designed to solve the problem of scaling expert (SME) decision quality to non-expert annotators for high-quality AI data annotation. Its core solution is to make strategies explicit, versioned, graph-structured, and validated with gold standards.

In AI data annotation, a key challenge is enabling non-experts to reach SME-level quality. Traditional solutions have limitations: detailed guides can't cover all edge cases; multi-round audits are costly and slow; model consensus may lead to correlated failures. RUSH addresses this by rethinking strategy representation.

RUSH's core principles include: 1) Strategy as product (nodes with definition, standards, examples, source anchors); 2) Graph-structured strategy (Obsidian-style Markdown for clarity); 3) Auditable strategy diffs (patches for SME approval); 4)分层指标 with clear denominators. Its project structure includes policy-graph, schemas, validation scripts, and a web demo.

RUSH's first pilot is GenAI image classification. Process: collect data → build strategy graph → capture SME-LLM分歧 → propose patches → SME approval. Binary labels: gen_ai (AI-generated) vs not_gen_ai (real, edited, CGI, or insufficient evidence). Subcategories: e.g., impossible hands, garbled text, plastic skin. Boundaries: photo editing, CGI/game renders, low-quality uncertain cases.

Explicit strategy beats implicit knowledge: auditable (trace decisions to nodes), iterable (patch-managed changes), teachable (fast onboarding), measurable (coverage as metric). Graph structure beats linear docs: visual coverage, modularity, linkability, progressive完善.

Limitations: seed metrics are simulated; current web flow doesn't call LLMs; needs local image清单. Applicable scenarios: high-quality annotation with limited budget, frequent strategy changes, multi-annotator collaboration, strict compliance requirements.

RUSH is a 'strategy-first' method to ensure data annotation quality. Future milestones: M1 (graph parsing/cold start), M2 (gold standard registry), M3 (annotation queue), M4 (metrics dashboard), M5 (strategy diff workflow), M6 (LLM integration)