Cube Pets Office: An AI Agent OS with Visualized Workflows

This article introduces Cube Pets Office—an AI Agent operating system that combines a 3D virtual office environment with real task execution capabilities. It aims to solve the black-box problem of traditional AI assistants. Through three core designs: visible workflows, real execution, and a 3D office shell, users can intuitively observe and participate in the complete lifecycle from natural language instructions to task completion, promoting AI interaction towards transparency and controllability.

Most current AI assistants work as a black box for users: after inputting instructions, they wait for results with no knowledge of the intermediate processes. This opacity reduces user trust and makes debugging and optimization difficult. The Cube Pets Office project attempts to change this situation by building an AI Agent OS with 'visible workflows, real execution, and a 3D office shell', allowing users to intuitively participate in the full task lifecycle.

Three Core Design Concepts

• Visible Workflows: Task decomposition steps are visualized, allowing users to clearly see the agent's process of understanding instructions, formulating plans, executing actions, and handling exceptions.
• Real Execution: Interacts with real external APIs, databases, file systems, etc., rather than just text generation.
• 3D Office Shell: Abstract task flows are embodied in a virtual 3D office environment, enabling users to intuitively observe the agent's work process.

Four Stages of the Task Lifecycle

1. Plan: Parses natural language instructions, formulates task decomposition and strategies; visualization helps users understand the parsing logic and planning methods.
2. Run: Executes according to the plan; in the 3D environment, this is represented by virtual character interactions, data flow animations, etc.
3. Review: Provides a review interface after task completion; users can point out problems, and the agent adjusts accordingly.
4. Replay: Supports full process replay, facilitating debugging, learning, and optimization.

From the project description, we can infer the technical choices:

• Multimodal Interaction Integration: The 3D environment needs to handle spatial visualization, animation rendering, and voice/gesture input, reflecting the evolution of AI interaction towards multimodality.
• Complexity of State Management: Recording the state, context, and branch paths at each decision point ensures the reproducibility of task visualization and replay, which is crucial for production-level systems.
• User Participation Design: The Review stage embodies a human-machine collaboration model where users can intervene to correct; hybrid intelligence is a practical implementation path.

Potential application scenarios include:

• Automated Office: Intelligent secretaries handle email classification, schedule arrangement, etc., while users maintain control over the process.
• Development and Operations: Execute code deployment, server monitoring, etc.; visualization allows non-technical personnel to understand the process.
• Education and Training: 3D visualization serves as a teaching tool, enabling students to intuitively observe the logic of AI solving problems.
• Customer Service: Handle complex requests; customer service can monitor in real-time and intervene to ensure quality.

Compared with traditional frameworks such as AutoGPT, LangChain Agent modules, and RPA tools, Cube Pets Office's uniqueness lies in its emphasis on 'visualization' and 'interactivity'. Traditional frameworks focus on function implementation and ignore user experience; this project treats user experience and function equally—it not only completes tasks but also allows users to understand and trust the process. This aligns with the research direction of 'explainable AI'. As AI undertakes important tasks, transparency and auditability have become key requirements. Conclusion: The project demonstrates new possibilities for AI Agent interaction—making intelligence perceivable, understandable, and participatory through visualization. The concept of transparent AI is the key to promoting AI popularization.

Challenges and recommendations for the project:

• Balancing Performance and Experience: Visualization and state recording may bring performance overhead; careful engineering optimization is needed to maintain a smooth experience.
• Trade-off Between Versatility and Specialization: General systems struggle to be optimal in specific domains, while specialized systems lack flexibility—finding the right level of abstraction is necessary.
• Security and Permission Management: Real execution capabilities carry potential risks; reasonable permission boundaries, exception handling, and manual confirmation mechanisms need to be designed.