Introduction: Core Overview of E2E-LLM-Watermark Framework

The E2E-LLM-Watermark proposed by the research team from Hong Kong University of Science and Technology is an end-to-end logits watermark framework. By jointly optimizing the encoder and decoder, it improves watermark robustness while maintaining text quality. This work has been accepted by the top international machine learning conference ICML 2025.

Original Authors and Sources

• Authors: Kahim Wong, Jicheng Zhou, Jiantao Zhou, Yain-Whar Si
• Source: GitHub (Project link: https://github.com/KahimWong/E2E-LLM-Watermark)
• Paper link: https://arxiv.org/abs/2505.02344
• OpenReview: https://openreview.net/forum?id=9sNiCqi2RD

This framework aims to address the trust crisis of LLM-generated content: distinguishing between human and AI text, enabling provenance tracking without sacrificing generation quality.

With the leap in capabilities of LLMs like GPT, Claude, and Llama, AI-generated content has permeated academic, news, and social domains, but it poses three major risks:

1. Academic integrity crisis
2. Spread of misinformation
3. Ambiguity in copyright and ownership

Traditional content provenance methods are not applicable to LLM outputs—text can be easily rewritten, translated, or summarized. There is a need for an implicit, robust, and quality-preserving marking mechanism.

Most existing LLM watermark schemes use post-processing strategies (modifying token embeddings to embed watermarks after generation), facing a dilemma:

• Enhancing watermark strength requires modifying more tokens → text quality degradation
• Maintaining quality leads to weak watermark signals → easily erased by attacks like rewriting or translation

Modern paraphrasing tools can deeply restructure text (while preserving semantics), making traditional watermarks ineffective.

Three key innovations of E2E-LLM-Watermark:

1. End-to-end joint optimization: Integrate the watermark encoder and decoder into the same training framework, allowing direct gradient flow for collaborative evolution (different from the pipeline of encoding first then detecting).
2. Logits-level perturbation: Perform context-aware small adjustments to the logits of candidate tokens at each generation step (based on the watermark key, constrained by Top-K).
3. Online prompting strategy: Bypass the non-differentiability of sampling operations in text generation, enabling effective training via gradient estimation.

The research team evaluated on multiple datasets: C4, HumanEval (code generation), WMT16 German-English translation. Attack scenarios covered: no-attack baseline, context replacement, Dipper paraphrasing attack. Quality metrics include: perplexity, log diversity, BLEU score, pass@1 code pass rate.

The results verify that the framework improves attack resistance while maintaining quality.

The project is built on Python3.9 and PyTorch2.1:

• Before training: Configure experiment paths and hyperparameters in train/config.py; set Hugging Face token in train/main.py.
• Key configurations: Optimization parameters, watermark generation settings, experiment output paths; perturbation strength during inference, Top-K candidate size, context window size.

• Academic status: Accepted by ICML2025, representing the latest progress in the LLM watermark field. Public checkpoints facilitate community reproduction.
• Extensibility: The end-to-end design can be extended to multi-modal generation domains such as images, audio, and video.
• Practical value: Flexible configuration space supports high-security scenarios (strong watermark) and quality-sensitive scenarios (low perturbation), providing a technical foundation for AI regulation.