SafeWeights: Identifying and Intervening on Safety-Critical Parameters of Large Language Models Without Retraining

The SafeWeights project proposes an innovative method to effectively mitigate the risk of jailbreak attacks without retraining by identifying safety-critical parameters in large language models (LLMs), providing a new technical path for AI safety alignment. Its core idea is to focus on specific subsets of parameters inside the model that affect safety behaviors, enabling precise intervention while balancing security and the model's general performance.

As LLM capabilities improve, jailbreak attacks have become a major security threat—attackers use carefully designed prompts to induce models to generate harmful content. Traditional protection methods include training-time alignment (e.g., RLHF), inference-time filtering, and prompt engineering, but they have limitations such as high cost, easy bypassing, and ongoing attack-defense cycles.

SafeWeights adopts a parameter-level intervention approach, consisting of three steps: 1. Identification of safety-critical parameters: Based on gradient analysis, compare parameter gradient changes between safe and unsafe scenarios to select the most impactful parameters; 2. Parameter intervention strategies: Targeted adjustment, constraint optimization (to maintain general performance), and hierarchical processing; 3. No retraining required: Directly edit parameter values, completing enhancement in minutes and reducing deployment costs.

Parameter importance evaluation uses an improved Fisher Information Matrix (importance score = E[(∂L/∂θ)^2]), combined with contrastive learning to calculate the safety-critical score (|importance in safe scenarios - importance in unsafe scenarios|). The intervention algorithm follows the principles of minimal intervention, performance preservation, and reversibility, using a projection method (θ_new = θ_original + α*direction). The project provides open-source tools: parameter analysis scripts, intervention modules, evaluation frameworks, and example notebooks (supporting models like Llama/Qwen).

Defense effect: On AdvBench, HarmBench, and custom attack datasets, it reduces the jailbreak success rate by 60-80%, with an effect comparable to RLHF but at only 1/1000th the cost. General performance: A decrease of <2% on benchmarks like MMLU/GSM8K, with no significant drop in open-task quality. Comparison with other methods: SafeWeights achieves the best balance in terms of computational cost (extremely low), defense effect (strong), impact on general performance (low), and deployment flexibility (high).

Applicable scenarios include: 1. Rapid security patches: Deploy quickly in response to new jailbreak attacks without retraining; 2. Security enhancement for open-source models: Provide low-cost security solutions for open-source models lacking alignment; 3. Customized security strategies: Adjust parameters according to scenarios without affecting core capabilities; 4. Security research tool: Help understand model security mechanisms and discover vulnerabilities.

Limitations: Facing adaptive attack risks, possible omissions in parameter identification, cross-model generalization differences, and the need to combine with other methods in extreme scenarios. Future directions: Automated parameter optimization, expansion to multi-dimensional security (privacy/fairness), real-time adaptation to new attacks, and deepening theoretical understanding (relationship between parameters and safety behaviors).