NLP Course Project: Exploring the Impact of Prompt Variations on LLM Output Style and Emotional Consistency

This project is a natural language processing course research. Its core goal is to analyze how prompt variations affect the consistency of writing style and stability of emotional expression in model outputs by comparing two large language models with different architectures: Flan-T5 and GPT. The research results will provide references for prompt engineering practices and reliable applications of AI systems.

With the widespread application of LLMs in text generation tasks, prompt engineering has become a key factor affecting output quality. However, subtle changes in prompts may lead to model-generated content with drastically different styles, which poses challenges for applications requiring stable styles (e.g., brand consistency). This project aims to systematically explore the impact of prompt variations on model outputs, focusing on two dimensions: style consistency and emotional stability.

Dataset: A dataset containing 1000 stories (1k_stories_100_genre.csv) is used, covering 100 literary genres, providing diverse materials for style testing. Experimental Models: Two types of models are compared—Flan-T5 (encoder-decoder architecture, instruction-tuned) and GPT (autoregressive decoder architecture)—to reveal differences in prompt sensitivity due to different designs. Project Components: Includes Jupyter Notebooks for the two models (flant5_model.ipynb, gpt_model.ipynb) and an auxiliary script fix_notebooks.py.

Core Questions:

1. How do prompt variations affect output content?
2. How consistent are the models in writing style?
3. Is emotional expression predictable? Methodology: The experimental process is implemented using Jupyter Notebooks, including data loading and preprocessing, prompt template design, batch generation, style and emotion analysis; the auxiliary script fix_notebooks.py handles engineering issues.

Based on the research design, expected findings include:

1. Differences in prompt sensitivity caused by model architecture variations (e.g., Flan-T5 is more sensitive to semantic structure, while GPT relies more on pattern matching);
2. Boundary conditions for model style consistency;
3. Systematic biases in emotional expression (e.g., some models tend to lean toward specific emotional polarities).

The value of this research for LLM application developers and prompt engineers:

1. Best practices for prompt design: avoid wording that leads to sudden style changes;
2. Model selection reference: choose appropriate models based on the consistency requirements of the scenario;
3. Quality assessment framework: migrate consistency assessment methods to application testing systems.

Limitations: Possible unbalanced sample distribution, non-state-of-the-art model versions, and limited evaluation dimensions (not covering factual accuracy, etc.). Future Directions: Multilingual expansion, exploration of long-text consistency, research on user intent alignment, and development of real-time consistency monitoring tools.