Knowledge Distillation Empowers Sequential Recommendation: Injecting User Semantic Understanding into Recommendation Systems via Pre-trained LLMs

Sequential recommendation systems excel at modeling temporal behaviors but have limitations in capturing semantic information. This article proposes an innovative knowledge distillation method that distills text-based user profiles generated by pre-trained LLMs into sequential recommendation models, achieving a balance between recommendation quality and system efficiency without requiring LLM online inference.

Sequential recommendation systems (e.g., SASRec, BERT4Rec) have achieved success by modeling the temporal sequence of user behaviors, but they over-rely on interaction matrices, simplifying users into sets of behavior sequences and ignoring deep semantic intentions (such as marathon training or weight loss goals behind buying running shoes), leading to limitations in semantic understanding.

Although LLMs can capture rich user semantics, direct integration faces three major challenges: high inference cost (difficult to meet millisecond-level response requirements), throughput limitations (unable to handle high-concurrency requests), and stability risks (affecting core business metrics). Existing solutions like LLM-as-Ranker perform well offline but are difficult to deploy online.

The core idea is to use LLMs offline to generate user semantic representations and distill them into lightweight sequential models. The steps include: 1. Text-based user profile generation: Collect multi-dimensional user information (interaction sequences, product descriptions, etc.), organize them into natural language input to LLMs to generate semantic summaries; 2. Distillation architecture design: Align hidden layer representations between the teacher (LLM) and student (sequential model), perform multi-task training (next-item prediction + semantic alignment loss), and distill knowledge progressively. This solution supports direct application of existing sequential models without modifying their architecture.

Validation on public datasets shows: 1. Improved recommendation performance: HR@10 and NDCG@10 metrics are significantly better than baselines; 2. Maintained inference efficiency: Online services rely on lightweight models, with latency and throughput comparable to traditional systems; 3. Cross-domain generalization: Better performance in cold-start scenarios, reducing reliance on domain-labeled data.

Practical references: 1. Mature system teams: Gradually upgrade via offline distillation without sacrificing online performance; 2. New system teams: Decouple LLM inference from recommendation services to achieve capability transfer. Future directions: Optimize distillation objective functions, integrate multi-modal profiles, design hybrid architectures, etc.

The combination of LLMs and sequential recommendation should be an efficient transfer of capabilities. Knowledge distillation decouples capabilities between inference and training phases, injecting semantic understanding while ensuring efficiency, providing a worthy exploration plan for teams pursuing a balance between quality and efficiency.