Uni-CTR: A Unified Framework for Multi-Domain CTR Prediction Based on Large Language Models

A multi-domain CTR prediction framework accepted by ACM TOIS 2024, which uses large language models to learn cross-domain semantic representations, addresses the poor generalization and seesaw effect of traditional models, and supports zero-shot prediction for new domains.

• Original Authors/Maintainers: Applied-Machine-Learning-Lab
• Source Platform: GitHub
• Original Title: Uni-CTR: A Unified Framework for Multi-Domain CTR Prediction via Large Language Models
• Original Link: https://github.com/Applied-Machine-Learning-Lab/Uni-CTR
• Paper Link: https://arxiv.org/abs/2312.10743
• Source Publication Time: December 2023
• Paper Acceptance: Accepted by ACM Transactions on Information Systems (TOIS, CCF-A) in August 2024

---

Click-Through Rate (CTR) prediction is a core task in online recommendation systems, used to estimate the probability of users clicking on ads or products. With the diversification of e-commerce platform businesses—covering multiple vertical domains such as online shopping, ride-hailing, food delivery, and professional services—recommendation systems need to have cross-domain prediction capabilities, namely Multi-Domain CTR Prediction (MDCTR).

However, traditional multi-domain CTR prediction faces two core challenges:

Challenge 1: Lack of Semantic Representation for Domains

Traditional MDCTR models usually encode domains as discrete identifiers (e.g., domain_id=1,2,3), completely ignoring the rich semantic relationships between domains. For example, "books" and "electronics" have significant differences in product attributes and user behavior patterns, but traditional models cannot capture these semantic differences and commonalities. This makes it difficult for models to generalize to new domains not seen during training.

Challenge 2: Seesaw Phenomenon

In multi-domain joint training, models are often biased by certain dominant domains with large data volumes, leading to a significant drop in performance in other domains. This seesaw effect of trade-offs severely restricts the practical application effect of multi-domain models.

---

Uni-CTR proposes an innovative three-layer architecture that fully leverages the semantic understanding capabilities of Large Language Models (LLMs) to address the above problems:

Uni-CTR uses a pre-trained large language model (e.g., Llama-2-7B) as the backbone network, inputting the feature texts of users and products into the LLM, and extracting semantic representations from its different hidden layers. The strong language understanding ability of LLMs can automatically capture the commonalities (such as general user interest patterns) and differences (such as domain-specific terms and preferences) between different domains. Unlike traditional discrete domain encoding, the semantic representations generated by LLMs are naturally generalizable—even when facing completely new domains, as long as their text descriptions have semantic associations with the training domains, the model can make reasonable predictions.

To learn the unique features of each domain, Uni-CTR is equipped with independent domain-specific networks for each known domain. These networks receive representations from different layers of the LLM Backbone and extract domain-specific signals through a specialized learning process. The key design is the Masked Loss Strategy: during training, each sample only updates the domain-specific network corresponding to its domain, while keeping other domain networks unchanged. This decoupled design brings great flexibility—when a domain needs to be added or removed, only the corresponding domain network needs to be added or deleted, and there is no need to retrain the LLM Backbone or other domain networks.

In addition to domain-specific networks, Uni-CTR also includes a general network that learns shared representations of all known domains. When encountering completely new unseen domains, the model can fall back to this general network for prediction, achieving true Zero-Shot reasoning capability.

---

Considering the huge number of parameters of LLMs, Uni-CTR uses LoRA (Low-Rank Adaptation) technology for parameter-efficient fine-tuning. In experiments, the authors used a LoRA adapter with rank r=4, which only requires training a small number of parameters to adapt the LLM to the CTR prediction task, greatly reducing computational costs.