# RUSH: Building a Strategy Graph System Using Expert Feedback and High-Reasoning Models

> RUSH is a strategy graph system that extends the decision quality of Subject Matter Experts (SMEs) to non-expert annotators through explicit, versioned, graph-structured strategies, enabling high-quality AI data annotation.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-21T04:43:19.000Z
- 最近活动: 2026-05-21T04:51:49.430Z
- 热度: 150.9
- 关键词: 数据标注, SME, 策略图, Obsidian, LLM, 生成式AI, 图像分类, 质量控制
- 页面链接: https://www.zingnex.cn/en/forum/thread/rush
- Canonical: https://www.zingnex.cn/forum/thread/rush
- Markdown 来源: floors_fallback

---

## RUSH: Strategy Graph System for Scaling Expert-Level Data Annotation

RUSH (Reinforcement learning Using SME feedback and High-reasoning models) is a strategy graph system designed to solve the problem of scaling expert (SME) decision quality to non-expert annotators for high-quality AI data annotation. Its core solution is to make strategies explicit, versioned, graph-structured, and validated with gold standards.

## Background: The Scaling Dilemma of Annotation Quality

In AI data annotation, a key challenge is enabling non-experts to reach SME-level quality. Traditional solutions have limitations: detailed guides can't cover all edge cases; multi-round audits are costly and slow; model consensus may lead to correlated failures. RUSH addresses this by rethinking strategy representation.

## Core Approach: Strategy as Explicit Graph Product

RUSH's core principles include: 1) Strategy as product (nodes with definition, standards, examples, source anchors); 2) Graph-structured strategy (Obsidian-style Markdown for clarity); 3) Auditable strategy diffs (patches for SME approval); 4) hierarchical metrics with clear denominators. Its project structure includes policy-graph, schemas, validation scripts, and a web demo.

## Pilot: GenAI Image Classification Strategy

RUSH's first pilot is GenAI image classification. Process: collect data → build strategy graph → capture SME-LLM disagreements → propose patches → SME approval. Binary labels: `gen_ai` (AI-generated) vs `not_gen_ai` (real, edited, CGI, or insufficient evidence). Subcategories: e.g., impossible hands, garbled text, plastic skin. Boundaries: photo editing, CGI/game renders, low-quality uncertain cases.

## Key Insights: Advantages of Explicit Graph Strategy

Explicit strategy beats implicit knowledge: auditable (trace decisions to nodes), iterable (patch-managed changes), teachable (fast onboarding), measurable (coverage as metric). Graph structure beats linear docs: visual coverage, modularity, linkability, progressive improvement.

## Limitations & Applicable Scenarios

Limitations: seed metrics are simulated; current web flow doesn't call LLMs; needs local image list. Applicable scenarios: high-quality annotation with limited budget, frequent strategy changes, multi-annotator collaboration, strict compliance requirements.

## Conclusion & Future Milestones

RUSH is a 'strategy-first' method to ensure data annotation quality. Future milestones: M1 (graph parsing/cold start), M2 (gold standard registry), M3 (annotation queue), M4 (metrics dashboard), M5 (strategy diff workflow), M6 (LLM integration)