# How Software Engineering Students Use Large Language Models to Write Research Papers: An Experience Report > This paper reports an educational experience of integrating reflective LLM use into an empirical methods assignment in software engineering education. It analyzes 146 students' disclosure statements on LLM usage, revealing how students integrated LLMs into activities such as brainstorming, methodology clarification, organizing research findings, and writing polishing, while also reporting concerns about inaccuracies and the need to verify generated content. - 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm) - 发布时间: 2026-06-03T17:19:43.000Z - 最近活动: 2026-06-04T06:22:30.659Z - 热度: 135.9 - 关键词: LLM教育, 软件工程, 学术研究, AI素养, 实证方法, 教育技术, 批判性思维 - 页面链接: https://www.zingnex.cn/en/forum/thread/llm-arxiv-2606-05114v1 - Canonical: https://www.zingnex.cn/forum/thread/llm-arxiv-2606-05114v1 - Markdown 来源: floors_fallback --- ## [Introduction] Core Summary of the Experience Report on Software Engineering Students Using LLMs to Write Research Papers This paper is an experience report on software engineering students using large language models (LLMs) to write research papers, based on an analysis of 146 students' LLM usage disclosure statements. Key findings include: students mainly use LLMs in four scenarios—brainstorming, methodology clarification, organizing research findings, and writing polishing; at the same time, there are concerns about the accuracy of LLM content, the need to verify generated content, and the risk of over-reliance. The study aims to explore effective ways to integrate LLMs into education and provide empirical references for software engineering education. ## Research Background and Motivation Large language models (LLMs) are rapidly integrating into software engineering education, especially in empirical software engineering and evidence synthesis activities. With the popularity of tools like ChatGPT and Claude, educators face a key question: how to effectively integrate LLMs into teaching practices while maintaining academic integrity? This study comes from an empirical methods assignment in a third-year software architecture course, which requires students to write short papers using rapid reviews or grey literature reviews and disclose their LLM usage to cultivate critical thinking skills about AI tools. ## Research Methodology Design The research team collected 146 students' disclosure statements and adopted a mixed analysis method: 1. LLM-assisted classification: using LLMs to initially classify the statements; 2. Manual verification and refinement: researchers manually verified and refined the results of automatic classification. This meta-analysis method reflects an academic mirror of using LLMs to study LLM usage. ## Main Scenarios of Students' LLM Usage ### 1. Brainstorming and Topic Exploration Many students use LLMs to generate research topics, explore problem perspectives, understand the current state of the field, and quickly obtain background information and research directions. ### 2. Methodology Clarification Rapid reviews and grey literature reviews are unfamiliar to undergraduate students. Students use LLMs to understand methodological concepts, steps, and best practices, and master the standardized process of academic research. ### 3. Organizing Research Findings When organizing literature, students use LLMs to help organize ideas, build argument logic, and identify relationships between literatures, reflecting LLMs' knowledge integration capabilities. ### 4. Writing Polishing and Language Improvement (Assumed) non-native English speakers use LLMs to improve the quality of academic writing, correct grammar, and optimize expressions, reducing the impact of language barriers. ## Concerns and Issues of LLM Usage Reported by Students ### 1. Accuracy Concerns Many students realize that LLMs may produce 'hallucinations' (seemingly reasonable but incorrect information) and learn to fact-check generated content. ### 2. Need for Verifying Generated Content Students report the need to verify the accuracy of LLM outputs, especially for research methods and literature citations. Verification awareness is an important part of academic literacy. ### 3. Risk of Over-reliance Some students reflect on the risk that over-reliance on LLMs may weaken their own critical thinking and research abilities. ## Educational Implications and Curriculum Design Recommendations ### 1. Importance of Transparent Disclosure Requiring students to disclose LLM usage cultivates awareness of academic integrity and also provides teachers with valuable data to understand students' use of AI tools. ### 2. Cultivation of Critical Thinking Students showed surprisingly strong critical thinking when asked to reflect on LLM usage, indicating that AI tools do not necessarily weaken critical thinking— the key lies in the design of teaching activities. ### 3. Opportunities for Methodology Education LLMs can assist in teaching research methods and help students understand complex academic concepts, but teachers need to guide students to recognize their limitations. ### 4. Future Curriculum Design Recommendations - Include LLM usage as an explicit component of curriculum design - Provide guidance on effective and responsible use of LLMs - Emphasize the importance of verification and fact-checking - Encourage students to reflect on the impact of AI tools on the learning process ## Conclusion and Future Research Prospects This experience report provides valuable insights for integrating AI-assisted technologies into empirical software engineering education. As LLM technology develops, educators need to continuously explore how to leverage the advantages of AI while cultivating students' critical thinking and independent research abilities. Future research can further explore LLM usage patterns in different disciplines and the long-term impact of LLM usage on students' academic ability development.