CodePromptZip: An Intelligent Prompt Compression Technique for Code Tasks, Achieving 41% Token Reduction with Accuracy Balance

This article introduces the open-source CodePromptZip technique, an intelligent prompt compression solution designed specifically for code Retrieval-Augmented Generation (RAG). Using type-aware token priority ranking and the CopyCodeT5 neural network compressor, it achieves a 41% token reduction with only a 12% accuracy loss on the Java Bug2Fix task, providing a practical solution for optimizing the inference cost of code LLMs.

With the application of LLMs in tasks such as code generation and repair, the RAG architecture improves performance but introduces the problem of prompt length bloat, resulting in high API costs and long inference delays. Traditional text compression methods (random deletion, suffix truncation, etc.) have limited effectiveness in code scenarios. Because code has a strict grammatical structure, blind compression breaks logical integrity, so a dedicated intelligent compression solution is required.

Semantic Classification of Code Tokens

Divide code tokens into 5 categories (priority from high to low: identifiers → method calls → structural keywords → symbols → method signatures), based on the difference in importance of different elements to the task (e.g., identifier redundancy in bug fixes).

Greedy Compression Algorithm

Steps: Parse tokens → classify → sort by type, word frequency, position → greedily remove high-priority tokens → reconstruct syntactically complete code.

CopyCodeT5 Neural Network Compressor

Based on CodeT5-Base, introduce a copy mechanism (generate or copy input tokens) to avoid spelling errors and preserve structure; trained with 45,000 pairs of samples covering 9 compression ratios.

Core Metrics

On the Java Bug2Fix task, the best result is at τ=0.5: 41% actual compression rate, CodeBLEU 80.36 (only a 12% loss), recommended as the default value.

Performance Curve Phenomenon

Performance does not decrease monotonically: mild compression (τ<0.4) leads to chaos → moderate (τ=0.5) pattern matching rebounds → severe (τ>0.6) performance decreases.

Baseline Comparison

Outperforms random deletion, suffix truncation, space removal, and simple TF-IDF, achieving over 40% compression rate with controllable loss.

• Cost Optimization: Reduce token usage to lower API costs (e.g., GPT-4 input billing), with significant long-term benefits for high-frequency calls.
• Latency Reduction: Shorten prompts to improve inference speed, enhancing the experience of real-time code completion and online review.
• Context Expansion: Include more code examples within fixed window limits to improve RAG recall quality.

Current Limitations

1. Only supports the Bug2Fix task (assertion generation, code suggestions not implemented); 2. Only supports Java; 3. Evaluation relies on CodeLlama-13B-Instruct.

Future Directions

Expand tasks, try larger models (CodeT5-Large), systematically compare with other compression methods, support multiple languages, integrate into real RAG systems to track cost savings.

CodePromptZip combines type-aware ranking with neural network compression to achieve a balance between 41% token reduction and 12% accuracy loss, providing an efficient cost optimization strategy for code RAG scenarios. The open-source implementation includes a complete training and evaluation process, offering a starting point for researchers and engineers to explore.