# Automatic Academic Paper Title Generation: Fine-tuned PEGASUS-large Model Outperforms GPT-3.5-turbo

> Researchers used open-source pre-trained models to implement automatic academic paper title generation. Experiments show that the fine-tuned PEGASUS-large model outperforms GPT-3.5-turbo (zero-shot generation) and the fine-tuned LLaMA-3-8B model across multiple evaluation metrics.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-06-03T16:49:57.000Z
- 最近活动: 2026-06-04T05:56:06.529Z
- 热度: 137.9
- 关键词: 学术论文, 标题生成, 自然语言处理, PEGASUS, GPT-3.5, 文本生成, 机器学习, 自动摘要
- 页面链接: https://www.zingnex.cn/en/forum/thread/pegasus-largegpt-3-5-turbo
- Canonical: https://www.zingnex.cn/forum/thread/pegasus-largegpt-3-5-turbo
- Markdown 来源: floors_fallback

---

## Introduction: Fine-tuned PEGASUS-large Model Outperforms GPT-3.5-turbo in Academic Paper Title Generation

This study focuses on the task of automatic academic paper title generation. By fine-tuning the open-source pre-trained model PEGASUS-large, experiments show that it outperforms GPT-3.5-turbo (zero-shot generation) and the fine-tuned LLaMA-3-8B model in multiple evaluation metrics such as ROUGE and METEOR. The research verifies the feasibility and reliability of AI-generated titles, providing a new solution for researchers to efficiently create high-quality paper titles.

## Research Background: Importance of Paper Titles and Necessity of Automatic Generation

The title of an academic paper is the "face" of the paper, undertaking key functions such as conveying core ideas, attracting readers' attention, facilitating retrieval and discovery, and establishing an academic impression. However, writing an appropriate title poses many challenges: balancing multiple goals such as accuracy and attractiveness, professionalism and readability within a limited number of words, which is especially difficult for non-native English speakers. Therefore, automatic title generation technology has emerged, aiming to assist researchers in efficiently creating high-quality titles.

## Research Methods: Dataset Construction and Model Comparison

**Dataset Construction**: Three datasets were used:
1. CSPubSum dataset: Abstract-title paired data in the field of computer science;
2. LREC-COLING-2024 dataset: Title-abstract pairs from computational linguistics conference papers;
3. SpringerSSAT dataset (newly constructed): Social science journal data to enhance domain diversity and cross-domain generalization capabilities.

**Model Comparison**:
- Open-source models: PEGASUS-large (dedicated to abstract tasks), LLaMA-3-8B (Meta open-source model), both fine-tuned;
- Closed-source model: GPT-3.5-turbo, using zero-shot setting (generation without examples).

## Evaluation Metrics: Multi-dimensional Quality Evaluation System

Five automatic evaluation metrics were used to measure title quality from multiple dimensions:
1. ROUGE: Based on n-gram overlap, including ROUGE-1/2/L;
2. METEOR: Introduces stemming and synonym matching to capture semantic similarity;
3. MoverScore: Calculates text movement distance based on word embeddings;
4. BERTScore: Uses the BERT model to compute semantic similarity;
5. SciBERTScore: Uses SciBERT (pre-trained on scientific literature) to evaluate academic semantic quality.

## Experimental Results: PEGASUS-large Performs Best, and ChatGPT Has Creative Advantages

**Key Findings**: The fine-tuned PEGASUS-large outperforms the fine-tuned LLaMA-3-8B and GPT-3.5-turbo (zero-shot) in most metrics, reflecting the value of task-specific architecture, the importance of fine-tuning, and the competitiveness of open-source models.
**ChatGPT Advantages**: It performs well in creative title generation; although it does not excel in automatic metrics, its titles are more attractive. It is recommended to use them in combination: PEGASUS generates basic titles to ensure accuracy, and ChatGPT polishes them to enhance creativity.

## Quality Analysis and Practical Application Value

**Quality Analysis**: AI-generated titles are generally appropriate and reliable, characterized by accurate content, standardized language, and complete structure; however, there are issues such as insufficient innovation, limitations in domain specificity, and neglect of cultural sensitivity.
**Application Value**:
- For researchers: Improve efficiency, gain inspiration, multi-language support, version comparison;
- For publishing institutions: Assist in review, standardization, SEO optimization;
- For academic platforms: Automatic indexing, recommendation systems, data cleaning.

## Technical Insights and Future Directions

**Model Selection Recommendations**:
- For accuracy: Choose fine-tuned dedicated models like PEGASUS;
- For creativity: Use large models like ChatGPT;
- Balanced solution: Dedicated model generates basic title + large model polishes it.
**Future Directions**: Multimodal title generation, interactive generation, style transfer, cross-language generation, citation prediction, etc.