ADB: A Measurement Framework for Safety Alignment Drift in Model Quantization Compression

This article introduces the Alignment Drift Benchmark (ADB) framework, which is the first to quantify the impact of model compression techniques on the safety alignment capabilities of large language models (LLMs). The core viewpoint is: while model compression improves efficiency, it may compromise safety alignment. The ADB framework reveals this drift phenomenon through a dual-track evaluation system, providing a quantitative basis for deployment decisions in production environments, and emphasizing that efficiency optimization should not come at the cost of safety.

Large model deployment costs are high (e.g., a 70-billion-parameter FP16 model requires 140GB of VRAM), and quantization compression (INT8, INT4, etc.) is key to implementation. However, the industry is increasingly concerned: does compression weaken the model's ability to identify/reject harmful requests? The ADB framework addresses this issue by systematically quantifying the differential impact of compression on safety alignment, filling the gap in industry evaluations.

Dual-Track Evaluation System:

• General Capability Track: common sense reasoning, reading comprehension, code generation, math reasoning, etc.;
• Safety Alignment Track: harmful request rejection, jailbreak defense, bias fairness, authenticity assessment, etc. Quantization Configurations: test FP16, INT8, INT4, GPTQ, AWQ, and other schemes. Drift Metrics: absolute drift, relative drift, drift ratio, critical threshold.

1. Universality: INT8 quantization leads to a 5-15% drop in safety performance, INT4 up to 20-40%, and GPTQ/AWQ still have a 10-25% drift despite improvements;
2. Asymmetry: general capability only drops by 2-8% (INT4), while safety alignment drops by 20-40%, with a drift ratio of 2:1 to 5:1;
3. Model Size Impact: small models have larger relative drift; large models have high absolute scores but still decline; medium-sized models are robust in some configurations; 4.** Attack Surface Changes**: defense against some jailbreak techniques decreases, certain harmful requests are allowed, and rejection reasons are vague.

Risk Stratification:

• Low Risk (Internal Tools): INT4/GPTQ + anomaly monitoring;
• Medium Risk (Public Chat): INT8/AWQ + input/output filtering;
• High Risk (Sensitive Fields): FP16/INT8 + red team testing + ensemble. Checklist: post-quantization verification, red team testing, monitoring mechanism, rollback plan. Optimization Directions: mixed precision, safety layer enhancement, dynamic quantization, continuous fine-tuning.

ADB promotes the industry to include safety alignment in compression evaluation standards (traditionally only focusing on perplexity/downstream accuracy); reveals the trade-off between efficiency and safety; open-source code and datasets facilitate fair comparison, helping to establish best practices for safe deployment.

Current Limitations: incomplete coverage of evaluation sets, lack of multilingual scenarios, limited dynamic attack evaluation. Future Directions: alignment-aware quantization algorithms, real-time drift monitoring, multimodal expansion, standardized safety evaluation benchmarks.