Pangkas: Open-Source LLM Context Optimization Middleware for Developers

Pangkas is an open-source LLM context optimization middleware and API gateway for developers, designed to address cost issues in LLM usage. Acting as a bridge between IDEs and AI providers (Claude, OpenAI, etc.), it helps developers reduce token consumption and costs of API calls through techniques like semantic comment pruning, intelligent whitespace compression, and historical conversation management—without changing existing workflows.

With the popularization of LLMs in development workflows, AI-assisted programming tools (such as OpenCode, Cursor) bring convenience but also lead to excessive token consumption due to carrying large amounts of context in each conversation, resulting in significant API costs. Pangkas (meaning "trim" in Indonesian) is designed to address this pain point as an intermediate layer to optimize LLM usage costs.

Pangkas's core optimization mechanisms include:

1. Semantic Comment Pruning: Through pruner.js, it intelligently identifies and removes unnecessary comments while retaining key logic comments;
2. Intelligent Whitespace Compression: compressor.js judges whether whitespace can be safely removed based on context, maintaining code structure integrity;
3. Historical Memory & Summarization: history-manager.js automatically generates historical summaries when conversations conversations are too long, reducing token consumption and helping models grasp the core context.

Pangkas adopts a modular architecture with clear responsibilities for each component:

• Entry point (index.js/ts): InitialInitializes and coordinates modules;
• pruner.js: Core of comment pruning;
• compressor.js: Whitespace compression logic;
• history-manager.js: Historical management and summarization;
• Other modules: Configuration loading, log statistics, reserved routing, semantic caching, etc. The modular design facilitates expansion and maintenance.

Pangkas's implementation has semantically-aware capabilities:

• When pruning comments, it distinguishes between docstrings, inline comments, and other types, processing them based on semantic importance;
• When generating historical summaries, it identifies key decision points, code modifications, and conclusions in conversations to ensure information integrity; It is not a simple text processing tool but an intelligent system that understands code semantics and conversation context.

For different scenarios, Pangkas provides flexible configuration recommendations:

• Short conversations: Default conservative mode, optimizing carefully without losing useful information;
• Large code reviews: Set compression level to 0.5 for aggressive optimization to reduce token consumption;
• Long conversations: Default configuration, automatically triggers summarization after 30 messages;
• Complex debugging: Disable user message pruning to retain complete debugging information.

Pangkas uses the MIT license, encouraging community contributions and secondary development. Its clear module boundaries and TypeScript type definitions facilitate expansion. The reserved router (multi-model routing) and semantic caching module provide directions for future improvements in cost optimization capabilities.

Pangkas provides a pragmatic cost optimization solution for high-frequency AI-assisted programming developers:

• Transparent intermediate layer that does not change existing workflows;
• Small token optimizations can lead to significant cost savings;
• Its design philosophy (concise but meaningful) provides a reference paradigm for efficiency optimization of AI tools.