PruningLab: Defending Large Language Models Against Jailbreak Attacks Using Neural Network Pruning Techniques

PruningLab is a research framework developed by lmajcen196 (source: GitHub, link: https://github.com/lmajcen196/PruningLab, published on 2026-06-12). It explores activation pruning and magnitude pruning as defense mechanisms against LLM jailbreak attacks, aiming to enhance model security while preserving normal functionality. This framework serves as an experimental platform to evaluate these pruning methods' effectiveness in mitigating jailbreak risks.

With LLMs' rapid advancement, jailbreak attacks have become a severe security challenge—attackers use crafted prompts to bypass safety mechanisms and generate harmful content. Traditional defenses like prompt filtering or output moderation have limitations: high false positives affect user experience, and they struggle with evolving attacks. PruningLab offers a structural solution by using neural network pruning to weaken jailbreak effectiveness from within the model.

PruningLab uses two key strategies:

Activation Pruning

Targeted at jailbreak-related neurons: 1) Collect neuron activation patterns with calibration data; 2) Compare activation differences between rejected and successful jailbreak prompts;3) Score neurons' relevance to attack success;4) Remove most relevant neurons. Pros: Precise, preserves normal model capabilities.

Magnitude Pruning

General method based on weight values:1) Extract all model weights;2) Calculate absolute values;3) Set threshold;4) Zero out weights below threshold. Pros: No prior attack knowledge needed, applicable to supported models.

Supported Models

Model	Parameter Count	Activation Pruning	Magnitude Pruning
Llama-3-8B-Instruct	8B	✅	✅
Gemma-2-9B-Instruct	9B	✅	✅
Mistral-7B-Instruct-v0.2	7B	✅	✅

Supported Attack Types

15 types including: DAN series (DAN, DAN6, etc.), role-playing (STAN, Mongo Tom), encoding confusion (ASCII Art, Base64), language deformation (Ubbi Dubbi), logic manipulation (Chain of Questions).

This wide coverage helps evaluate pruning defenses' generalization.

Core Evaluation Metrics

• Attack Success Rate (ASR): Lower is better.
• Accuracy: Model performance on benign tasks.
• Safety Classification: Distribution of safe/unsafe outputs.
• Baseline Comparison: Behavior difference before/after pruning.

Experiment Flow

1. Select target model from supported list.
2. Choose jailbreak attack type.
3. Configure pruning method (activation/magnitude) and ratio.
4. Run multiple experiments for statistical significance.
5. Compare baseline and pruned model performance.
6. View aggregated results and visualizations.

The framework provides a complete experimental platform via web interface.

Practical Significance

• Theoretical: Reveals jailbreak-neuron links; proves structural defense feasibility; quantifies utility-safety tradeoff.
• Practical: Hardens models before deployment; standardizes security testing; enables continuous protection against new attacks.

Limitations

• Activation pruning relies on precomputed activation scores (limits new model applicability).
• High pruning ratio may harm benign task performance.
• Attackers may adapt to pruned models.

Future Directions

• Develop finer neuron importance evaluation.
• Explore dynamic pruning (real-time adjustment based on input).
• Combine multiple defenses for deep protection.
• Extend to larger models (70B+ parameters).