LLM4IAS: Cutting-Edge Exploration of Using Large Language Models to Control Industrial Automation Systems

The LLM4IAS project explores the application of large language models (LLMs) to control industrial automation systems. Through supervised fine-tuning (SFT) technology, general-purpose LLMs are enabled to master professional equipment control skills, achieving significant performance improvements in both standardized operating procedure (SOP) tasks and abnormal event response tasks. Developed by the team of Yuchen Xia from RWTH Aachen University in Germany, this project continues the previous work related to GPT4IndustrialAutomation, adopting more refined system design and comprehensive testing and fine-tuning, thus providing new possibilities for Industry 4.0 and intelligent manufacturing.

The traditional industrial automation field has long relied on preset program logic and strict standard operating procedures (SOPs) to control production equipment. However, this approach is rigid when facing complex and changing environments, especially lacking flexible response capabilities in handling unexpected abnormal events. In recent years, the natural language understanding and generation capabilities of LLMs have sparked a thought: Can LLMs directly understand and control industrial automation systems? Based on this, Yuchen Xia's team developed the LLM4IAS project, continuing the previous work on GPT4IndustrialAutomation, with more refined system design and comprehensive testing and fine-tuning.

The core concept of the LLM4IAS system is to let LLMs act as the "intelligent brain" of industrial automation systems. It receives operation instructions through a natural language interface, understands the intent, and then generates control commands. Its advantages include: lowering the threshold for human-machine interaction (non-technical personnel can directly communicate with equipment), and LLMs' strong context understanding ability (making decisions by integrating system status, historical operations, and external environment). Technically, a complete prompt engineering framework is designed, and prompt_example.txt is provided for testing.

To adapt general-purpose LLMs to industrial control tasks, the team adopted supervised fine-tuning (SFT) technology and collected a training dataset with 200,000 tokens. The experimental models include Llama-3-70B-Instruct, Llama-3-8B-Instruct, Qwen2-7B-Instruct, Mistral series, etc. Qwen2-72B-Instruct uses LoRA fine-tuning (Rank 32, alpha 32). Additionally, OpenAI API is used to fine-tune GPT-4o for comparison. The training configuration is unified: 1 epoch, learning rate of 1e-5, and batch size of 16.

Automation control tasks are divided into two categories: 1. Standardized process tasks (SOP): Operations according to preset guidelines (e.g., starting equipment in a fixed order); 2. Abnormal event response tasks: Handling unexpected situations without clear guidance (e.g., equipment failure, raw material shortage), which requires reasoning and adaptation. Evaluation is based on 100 test points, using two indicators: generated command accuracy (%) and reasoning rationality score (average of 1-5 points).

Key findings from experimental results: 1. Pre-trained model comparison: GPT-4o performs best (81% accuracy, 4.7 reasoning score), followed by Llama-3-70B (75%, 4.3), and Llama-3-8B is relatively poor (37%); 2. Task performance differences: GPT-4o achieves 100% in SOP tasks but only 41% in abnormal response tasks; Qwen2-7B has 63% in SOP but 69% in abnormal response, which exceeds GPT-4o; 3. Significant improvement from fine-tuning: Llama-3-70B increases from 75% to 95%, Llama-3-8B from 37% to 96% (small model surpasses large model after adaptation); 4. LoRA has poor effect on Qwen2-72B (70% →66%), presumably due to resource constraints.

LLM4IAS provides new possibilities for Industry 4.0 and intelligent manufacturing: 1. More intelligent human-machine collaboration (natural language interaction reduces training costs); 2. Adaptive production scheduling (independently adjusts plans to deal with emergencies); 3. Knowledge inheritance (coding engineers' experience becomes intelligent assets). The technical maturity meets the NASA TRL standard, and it has the potential to move from the laboratory to practical applications.