PrivacyProxy: Use Top-Tier AI Models for Free While Protecting Privacy Locally

PrivacyProxy is an open-source OpenAI-compatible gateway designed to address the balance between privacy and model capabilities faced by AI developers. By anonymizing prompts and tool outputs locally, it enables users to safely use free but data-logging top-tier large language models (such as Nemotron, GPT-OSS, etc.) on OpenRouter, allowing sensitive data to stay on local devices while enjoying powerful reasoning capabilities.

The OpenRouter platform offers many free top-tier LLM APIs, but its terms of service clearly state that user inputs will be logged for product improvement, which poses risks to users handling confidential or private data (such as enterprise developers, data analysts, etc.). Traditional solutions either give up free models for paid ones or accept the risk of data leakage. PrivacyProxy proposes a third path: desensitize data locally before sending requests, then restore the results.

PrivacyProxy adopts a "Privacy Sandwich" architecture, consisting of three core components:

1. Local Gateway Layer: An OpenAI-compatible API proxy. AI Agents can connect by simply modifying the base_url without changing code;
2. Anonymization and Restoration Pipeline: Uses deterministic rules (regular expressions, entropy detection, place name dictionaries) to identify sensitive information, replace it with placeholders, and Vault stores the reversible mapping relationships;
3. Local Lightweight Model: Falcon-H1 or Qwen small models, serving as the second line of defense for sensitive information detection.

Threat Model: Targets the service provider's legitimate logging and training processes, ensuring that confidential data does not enter requests sent to OpenRouter. Core Design Principles:

• Determinism First: Basic anonymization uses rules implemented in pure Rust; statistical methods are only auxiliary;
• Fail-Safe: Requests that cannot be proven safe will be blocked;
• Reversible and Session-Consistent: Transformations are reversible and mappings remain unchanged within a session;
• Transparent Transformation: Only converts data formats, does not execute Agent tools;
• Tool Output Priority: Detection covers tool outputs (not just prompts).

The workflow is divided into three stages:

1. Anonymization: Regular expressions identify structured information such as emails/IPs/API keys; entropy detection identifies high-randomness strings; place name dictionaries identify geographic locations—all replaced with placeholders;
2. Cloud Inference: Desensitized requests are sent to OpenRouter's free model endpoints;
3. Restoration: Based on Vault mappings, placeholders in responses are replaced back to original data and returned to the AI Agent.

Applicable Scenarios:

• Enterprise Developers: Use AI-assisted programming (including proprietary code) in CI/CD;
• Data Analysts: Process customer data (compliance requirements);
• Individual Developers: Experiment with top-tier models (unwilling to expose project details). Value Proposition: Privacy stays local, reasoning capabilities are rented from the cloud—no extra privacy costs, no sacrifice in model quality.

The project is currently in the early development stage. The architecture documentation is complete, and the Rust workspace scaffolding is under construction. Maintainers emphasize strict privacy contracts—PRs that weaken security will not be merged. Contribution Suggestions:

• Improve the desensitization rule library;
• Optimize Vault encrypted storage;
• Enhance the accuracy of local detection models. (It is recommended to read the ARCHITECTURE.md document before participating.)

PrivacyProxy represents a new approach to AI privacy protection: instead of paying for privacy, it uses local technical means to achieve data protection. The "Privacy Sandwich" architecture not only retains the capabilities of free top-tier models but also ensures sensitive data remains within the user's control, providing a practical solution for data sovereignty in the AI era—worth the attention of developers and enterprises.