SharedRequest: A Privacy-Preserving Inference Scheme for Large Language Models Based on Model-Agnostic Discrimination Mechanism

SharedRequest proposes an innovative privacy-preserving inference framework that identifies and filters sensitive information from user queries before they enter large language models (LLMs) using a model-agnostic discrimination method, balancing data privacy and model utility. Addressing the pain points of existing privacy-preserving technologies (high overhead of homomorphic encryption, quality degradation from differential privacy), this scheme adopts a layered processing architecture, fully decoupled from the underlying LLM, and can flexibly adapt to various language models, providing an efficient and secure solution for AI applications in sensitive scenarios.

With the widespread application of large language models (LLMs), the risk of user privacy leakage has become prominent. Sensitive information in user queries (names, addresses, medical records, etc.) may be received, stored, or used for training in plaintext, violating privacy regulations (such as GDPR, Personal Information Protection Law), and restricting enterprises' willingness to adopt AI in sensitive scenarios. Existing solutions fall into two categories: technical routes based on homomorphic encryption/secure multi-party computation have huge computational overhead and are difficult for real-time inference; differential privacy methods add noise to protect privacy but significantly reduce the quality of model outputs. The industry needs an intermediate solution that balances privacy protection and performance experience.

The core innovation of SharedRequest is the 'model-agnostic discrimination mechanism', which detects and filters sensitive information through a specially trained discrimination module before queries reach the LLM. It is decoupled from the underlying LLM and can adapt to various models such as GPT, Claude, and Llama. The technical route adopts layered processing: 1. Sensitive information identification (a discriminator fine-tuned based on RoBERTa accurately identifies privacy entities); 2. Information desensitization (replacing sensitive content with neutral placeholders or semantic generalization); 3. Secure inference (sending desensitized queries to the LLM for processing and mapping results back to the original context).

The system architecture includes three core components: 1. Discrimination model training module (contrastive learning strategy, supports GPU-accelerated training, users can fine-tune with custom annotation sets); 2. Online inference filtering module (token-level analysis, multi-GPU deployment supports high concurrency); 3. Model-agnostic interface layer (standardized interfaces adapt to backends such as OpenAI API, local Ollama, and vLLM). The deployment process is simple: requires Windows 10+/16GB memory/8GB NVIDIA GPU/Python 3.10+ environment, install dependencies via pip, execute training scripts to build the model, call online filtering commands to start the pipeline, and adjust the identification threshold to balance privacy and usability.

The model-agnostic feature of SharedRequest applies to multiple scenarios: in the medical field, it protects patient privacy while assisting case analysis; in the financial field, it filters customer account information to allow intelligent customer service to handle inquiries safely; in the enterprise office scenario, it allows employees to confidently ask questions about trade secrets. For AI service providers, integrating this scheme can reduce compliance risks, provide an additional security layer for locally deployed private LLMs, and meet the data sovereignty requirements of government and enterprises.

Current limitations: The accuracy of the discrimination model depends on the coverage of training data, which may lead to missed detections or false positives for new sensitive information patterns; desensitization may change semantics and cause result deviations (especially in complex query scenarios). Future directions: Introduce advanced NER technology to improve discrimination accuracy; explore context-aware dynamic desensitization strategies; study hybrid architectures of privacy computing and discrimination filtering; integrate federated learning and TEE (Trusted Execution Environment) technologies to build an end-to-end privacy protection chain.