Carbon-Taxed Transformers: A Green Model Compression Framework Inspired by Carbon Tax Concepts

This paper proposes Carbon-Taxed Transformers (CTT), a systematic multi-architecture compression process inspired by the carbon tax principle in economics, aiming to address the sustainability crisis of AI in the software engineering field. The framework achieves up to 49x memory reduction and 81% carbon emission reduction in software engineering tasks while maintaining 98% clone detection accuracy, providing a new solution for balancing model efficiency, environmental benefits, and precision.

Large Language Models (LLMs) are rapidly growing in application in the software engineering field, covering tasks such as code clone detection, summary generation, and automated synthesis. However, they face issues like large size, slow deployment, high memory consumption, and significant carbon footprint, which threaten the scalability, accessibility, and long-term environmental sustainability of AI-driven software engineering. Thus, efficiency and environmental costs need to be treated as first-class design constraints.

The CTT framework is inspired by the carbon tax principle in economics and achieves systematic compression through the following mappings: 1. Calculate carbon tax: Tax inefficient designs at the architecture level; 2. Compression reward: Reward deployment-ready compression schemes; 3. Systematic ordering: Organize multi-stage processes by compression efficiency, focusing on resource consumption efficiency across the entire compression chain.

To verify the universality of CTT, the evaluation covers three core software engineering tasks: code clone detection, code summary generation, and code generation; it also includes three mainstream architectures: encoder-only, encoder-decoder, and decoder-only, ensuring the robustness and universality of the conclusions.

CTT performs excellently across multiple dimensions: up to 49x memory efficiency reduction; significant improvement in inference speed (8-10x for clone detection, 3x for summary generation, 4-7x for code generation); 81% reduction in carbon emissions; and good precision retention (about 98% accuracy for clone detection, about 89% performance for summary generation, 91% text metrics and 68% pass@1 for code generation).

Ablation studies show that: the order of compression stages has a significant impact on the effect; CTT optimizes the ordering strategy to maximize synergistic effects; each compression component needs to be used in combination—single technologies cannot achieve the overall effect, highlighting the value of the systematic approach.

CTT provides a feasible path for responsible AI in the software engineering field, and its concept of incorporating environmental costs into design decisions can be extended to broader AI systems. Industry can directly apply this compression blueprint, and researchers can explore interdisciplinary integration of economics, environmental science, and computer science. In the future, we need to emphasize the three-dimensional trade-off between performance, efficiency, and sustainability, and establish the design ethics that excellent AI systems are both intelligent and green.