AgentJIT: A JIT Compiler That Compiles Repetitive AI Workflows into Zero-Token Skills

An in-depth analysis of how AgentJIT automatically compiles time-consuming random workflows into deterministic zero-token skills by observing the execution traces of AI agents, achieving an efficiency leap from "ten thousand deliberations" to "hundreds of invocations."

Modern AI coding assistants (such as Claude Code, Codex, Cursor, etc.) have profoundly changed the way developers work. They can understand natural language instructions, execute complex tool call sequences, and complete various tasks from code review to deployment and operation. However, this capability comes at a cost: each task execution requires multiple rounds of LLM inference, consuming a large number of tokens and generating significant delays and costs.

Consider a typical DevOps scenario: troubleshooting service failures. An agent may need to perform the following steps:

1. Run kubectl get pods to check pod status
2. Execute kubectl logs to get logs of a specific pod
3. Use grep to search for error keywords in the logs
4. Infer possible causes based on error information
5. Modify configuration files or restart services
6. Verify the repair results

If such failures occur frequently (e.g., caused by a known bug), the agent will repeat the same reasoning process every time. While this has become "muscle memory" for humans, for AI, each time is a new deliberation, consuming about 10,000 tokens and more than 30 seconds.

AgentJIT's core insight is: these repetitive workflows can be learned, compiled, and optimized, eventually transformed into near-instant deterministic execution.

AgentJIT (Agent Just-In-Time Compiler) is a background JIT compiler designed specifically for autonomous programming agents. It achieves a qualitative change in efficiency through the following mechanisms:

AgentJIT silently observes the agent's execution traces through the hook mechanism of the agent framework (such as Claude Code's PostToolUse, SessionStart, SessionEnd hooks). It does not interfere with normal operations, but faithfully records tool call sequences, parameter patterns, and execution results.

This observation is continuous. As the agent performs more tasks, AgentJIT accumulates more execution data and identifies repeatedly occurring patterns.

When a workflow is identified as a "hotspot" (i.e., repeated execution reaches a certain threshold), AgentJIT triggers the compilation process. The compiler analyzes the execution traces, extracts parameterizable parts, and generates a deterministic skill script.

Compiled skills have the following characteristics:

• Parameterized: Variable parts in the script are extracted as parameters, enabling it to adapt to similar but slightly different scenarios
• Deterministic: The execution path is predefined, no longer requiring LLM inference
• Zero-Token: The execution process does not involve any LLM calls, only executing local commands

Compiled skills are stored in the local file system (default location ~/.aj/skills/). When the agent encounters a matching scenario again, AgentJIT intercepts the request and directly executes the corresponding skill script.

Execution results are monitored. If the skill execution fails, the system will silently fall back to the original LLM workflow, and the user will not perceive any interruption. This fault-tolerant design ensures the reliability of the system.

AgentJIT's architecture consists of four core components, forming a complete feedback loop:

This is the entry point for integration with the agent framework. Currently, it supports Claude Code's native hooks, and future plans include expanding to Codex, Gemini CLI, GitHub Copilot, Cursor, etc.

Hook types include:

• PostToolUse: Triggered after each tool call, recording call details
• SessionStart/SessionEnd: Mark session boundaries and aggregate session-level statistics