Large Language Model Course Practice: From Text Preprocessing to Word Embeddings and Few-Shot Learning

This open-source repository records a complete set of large language model (LLM) course assignments, compiled and shared by learner sepanta007. Starting from basic text preprocessing, the course content gradually deepens into word embedding modeling and few-shot learning methods based on modern pre-trained models, forming a progressive learning path from traditional NLP to contemporary large model technologies, covering IMDB dataset preprocessing, Word2Vec training, and few-shot sentiment classification practice.

This course assignment system is presented as an open-source repository with a progressive learning design: starting from basic text preprocessing, it gradually transitions to word embedding modeling, and finally advances to few-shot learning methods in the era of large models, helping learners clearly grasp the evolution of NLP from traditional technologies to modern large models.

The first assignment (HW1) focuses on text preprocessing of IMDB movie review data, including core steps such as text cleaning, tokenization, stopword filtering, and stemming. It requires handling issues like HTML tags, special symbols, and case unification in raw text, understanding the impact of different tokenization strategies on subsequent analysis, and establishing the NLP awareness of "data quality first".

The second assignment (HW2) implements Word2Vec model training, covering two classic architectures: CBOW (predicting the central word from context) and Skip-gram (predicting context from the central word). The repository provides 64-dimensional and 128-dimensional word embedding training checkpoints, with training parameters including window size 5, negative sampling ratio 5, and batch size 512, reflecting the trade-off between vector dimension, computational cost, and overfitting risk.

The advanced part of the course introduces few-shot learning, with transfer learning at its core: pre-trained models acquire language and world knowledge through massive text, and can quickly adapt to new tasks with only a few example prompts or lightweight fine-tuning. In the assignment, few-shot sentiment classification is explored through imdb_few_shots.ipynb, which is suitable for scenarios with high annotation costs or scarce data.

The course assignments use Jupyter Notebooks as the experimental carrier, facilitating step-by-step code execution and result observation. Key technical points include data loading and batch processing, neural network forward/backward propagation, loss function and optimizer configuration, model evaluation metrics (accuracy, recall, F1 score), and checkpoint saving and loading mechanisms. The repository provides trained model checkpoints as practical evidence.

The value of this repository lies in its end-to-end completeness and progressive teaching design, helping learners master the evolution of NLP technologies. A solid foundation (such as Word2Vec mechanisms and text preprocessing details) is a prerequisite for mastering large models, and few-shot learning practice is an essential skill for applying large models to business scenarios.

Learning large language models requires attention to theoretical foundations, data processing workflows, and internal model mechanisms. It is recommended to further study the pre-training-fine-tuning paradigm of Transformer architectures such as BERT and GPT after mastering Word2Vec; and to deeply research cutting-edge technologies like prompt engineering and Retrieval-Augmented Generation (RAG) on the basis of few-shot learning. A solid foundation is the starting point for technical advancement.