Oz: A Wizard-of-Oz Framework for UI-First Intelligent Coding Workflows

The Oz project is a Python package based on the Wizard-of-Oz method, designed to support building intelligent coding workflows in a UI-first way, providing flexible experimental and prototyping tools for AI-assisted programming. It helps developers and researchers quickly verify the feasibility of AI-human collaboration interfaces, focusing on the exploration of user experience and collaboration models.

With the popularity of AI programming assistants like GitHub Copilot and Cursor, developers face many challenges in collaborating with AI: How to balance automation and human control? How to trade off between efficiency and code quality? How to flexibly switch collaboration modes?

The Wizard-of-Oz method is a classic human-computer interaction technique: on the surface, users interact with an intelligent system, but in reality, it is controlled by a human operator. This allows rapid verification of interaction design feasibility without building a complete AI system.

The design of Oz revolves around four core principles:

1. UI-First: Design workflows starting from the user interface, not AI capabilities;
2. Rapid Prototyping: Minimize the time and code required to set up an experimental environment;
3. Flexible and Controllable: Support human operator intervention to precisely control the boundaries of AI behavior;
4. Observability: Provide rich logs and state tracking for easy analysis and iteration.

The Oz architecture consists of four main components:

• Frontend Interface Layer: A responsive UI based on modern web technologies, supporting code editing, conversational interaction, and state visualization;
• Agent Orchestration Engine: Manages the lifecycle and collaboration of multiple AI agents, supporting dynamic switching between manual and automatic modes;
• Operator Console: A backend interface for human operators to view real-time status and intervene in control;
• Event Bus: A unified asynchronous messaging system connecting the frontend, agents, and operators.

Oz is suitable for three types of scenarios:

1. AI-Assisted Programming Research: Quickly set up experimental environments to test different AI interaction modes (e.g., fully automatic generation vs. human-AI iteration);
2. Product Design Validation: Use human simulation of AI responses to collect user feedback and guide design before developing a complete AI backend;
3. Educational Training: Train operators to familiarize themselves with AI collaboration processes, preparing them to operate real AI systems.

Key technical features of Oz:

1. Progressive Automation: Supports transition from fully manual to fully automatic, gradually transferring tasks to real AI;
2. Multi-Agent Collaboration: Natively supports multi-agent scenarios (e.g., code generation + review), allowing operators to intervene at any stage;
3. Plug-and-Play Backend: Easily switch backends (human operator → LLM API → local model) with just configuration.

Oz provides important infrastructure for the research and development of intelligent coding tools, lowering the threshold for experimenting with new interaction modes and allowing researchers to focus on user experience rather than engineering implementation.

Its 'human-centered' design methodology emphasizes the importance of human developers effectively utilizing AI capabilities. Open-source release is expected to promote collaboration in the field and drive the evolution of AI-human collaboration models toward more mature and human-friendly directions.