PII Data Desensitization: A Dual Protection Scheme Combining Encoder Models and Large Language Models

Core point: This article explores a PII data desensitization scheme integrating fine-tuning of BERT/RoBERTa encoders and prompt engineering of Large Language Models (LLMs). Through dual-model collaboration (encoder for precise positioning + LLM for semantic verification), it achieves efficient identification and automatic masking of sensitive information like names and emails, addressing the limitations of traditional rule/regex methods in complex scenarios and providing a feasible path for privacy protection in AI applications.

In the digital age, PII protection is a core issue in data security. Training data and interaction content in LLM applications contain a large amount of sensitive information, making balancing AI capabilities and privacy protection a key challenge. Traditional PII desensitization relies on rule matching or regular expressions, which have obvious limitations in recognition accuracy and generalization ability when facing complex text formats and multilingual environments. This project proposes a dual-model collaborative architecture, combining the advantages of encoder's precise classification and LLM's semantic understanding to build a robust PII detection and masking pipeline.

Encoder Model Layer

By fine-tuning BERT/RoBERTa for the domain, using the NER task paradigm (BIO annotation system: e.g., B-PER/I-PER for names, B-EMAIL/I-EMAIL for emails) to achieve token-level sequence labeling, accurately capturing entity boundaries. It has the advantages of fast inference speed and low computational overhead, serving as the first filtering line of defense.

Large Language Model Layer

Using prompt engineering, the LLM undertakes semantic verification and complex scenario processing: understanding context to infer implicit PII (e.g., indirectly disclosed email information), handling coreference resolution in multi-turn dialogues, and making up for the encoder's deficiencies at the semantic level.

The complete desensitization pipeline consists of four stages:

1. Preprocessing and Tokenization: Standardize text (unify encoding, remove abnormal characters), split into token sequences using a matching tokenizer;
2. Encoder Inference: The fine-tuned model outputs label probability distribution, obtains the annotation sequence via Viterbi decoding, and initially identifies suspected PII;
3. LLM Enhancement: Input candidate PII and context into LLM for verification, supplementing missing information detected;
4. Masking Strategy Execution: Select placeholder replacement ([NAME]/[EMAIL]), partial masking (li***@example.com), or hashing according to business needs to generate safe text.

Multilingual Support

Challenge: PII expressions vary by language and culture (e.g., Chinese names of 2-4 characters vs. Western full names). Solution: Adopt mBERT/XLM-RoBERTa multilingual pre-trained models and fine-tune them on multilingual PII corpora.

Boundary Ambiguity

Challenge: Some texts are in the gray area between PII and non-PII (e.g., common English names). Solution: Introduce LLM semantic judgment, combining context analysis to reduce false positive rates.

Adversarial Samples

Challenge: Malicious users bypass detection through special formats (spaces, homophones, mixed case). Solution: Complementary dual-model architecture—encoder captures explicit patterns, LLM understands semantic deformations.

This scheme has significant application value across multiple domains:

• Enterprise Data Compliance: Meet regulations like GDPR/CCPA, automatically remove sensitive information before data analysis and model training;
• Customer Service Dialogue Processing: Protect customer privacy while retaining the business value of dialogues for quality analysis;
• Medical Text Analysis: Desensitize patient identity information in electronic medical records/doctor-patient dialogues to support medical research and clinical decision-making;
• Educational Data Mining: Protect the privacy of minors when analyzing student interaction data.

Deployment recommendations:

1. Training Data Quality: Build an annotated dataset covering multiple PII types, different expression forms, and balanced positive/negative samples; enhance data through back-translation and synonym replacement;
2. Inference Efficiency Optimization: Reduce overhead via encoder model quantization, knowledge distillation, and ONNX conversion; call LLM on demand (only trigger when encoder results are uncertain);
3. Continuous Monitoring and Iteration: Establish a feedback loop, regularly evaluate performance on actual data, and adjust models and strategies in time to address new risks.

PII desensitization is a cornerstone technology for privacy protection in the AI era. This project's dual-model collaborative scheme combines the encoder's efficiency and precision with the LLM's deep understanding, providing a feasible path for secure AI applications. With the development of privacy computing technology, we look forward to more innovative schemes emerging to balance data value and privacy protection.