Combining Large Language Models and Knowledge Graphs to Enhance the Interpretability of Machine Learning in Manufacturing

This article proposes an innovative framework combining knowledge graphs (KG) and large language models (LLM) to address the interpretability issue of machine learning models in manufacturing. The framework associates and stores domain knowledge with ML results, designs a selective retrieval mechanism, and uses LLMs to generate user-friendly explanations. Evaluations in manufacturing environments show that this method performs excellently in accuracy, consistency, clarity, and practicality.

Introduction: The Black Box Dilemma of AI in Manufacturing

Manufacturing is undergoing an AI-driven transformation, with ML models widely used in quality control, predictive maintenance, and other processes. However, modern ML models are mostly "black boxes" that cannot explain the basis for their decisions. This makes it difficult for engineers to trust model recommendations and fails to meet industry regulations requiring decisions to be auditable and interpretable.

Knowledge Graph + Large Language Model: A Powerful Combination

Core Idea

The framework is based on two key insights: 1) Manufacturing domain knowledge is suitable for representation using knowledge graphs, which support explicit relationships and logical reasoning; 2) LLMs excel at converting structured information into fluent natural language. By combining the two, KG provides accurate domain knowledge, and LLM generates user-friendly explanations.

System Architecture: Three-Layer Design

Data Layer: Stores domain knowledge, ML results, and their associations;

Retrieval Layer: Selectively retrieves relevant KG triples based on ML result features and prunes them;

Generation Layer: Inputs retrieved knowledge into LLM and generates professional explanations via prompt templates.

Evaluation: XAI Question Bank and Manufacturing Field Tests

Evaluation Design

Based on 33 XAI questions (including standard and complex custom questions), metrics such as accuracy, consistency (quantitative), clarity, and practicality (qualitative) were used.

Key Findings

1. Accuracy improvement: After combining KG, accuracy increased from 75% to over 90%, reducing LLM hallucinations;
2. Consistency guarantee: KG structure ensures logical consistency of explanations;
3. Clarity and practicality: Explanations are fluent and easy to understand, providing actionable insights;
4. Complex problem handling: KG supports chain reasoning, with strong ability to handle multi-step problems.

Practical Applications: Decision Support in Manufacturing

Predictive Maintenance: When the model predicts a failure, it explains the contribution of abnormal temperature/vibration and the correlation with historical failures, and provides maintenance suggestions;

Quality Control: When marking non-conforming products, it links process parameters with quality standards and suggests adjustment measures;

Production Optimization: When suggesting plan adjustments, it integrates logic such as equipment status and order priority.

Limitations and Future Directions

Current Limitations

• High KG construction cost, requiring significant manual input;
• Knowledge update delay, which may lag behind actual changes;
• Limited ability for complex creative reasoning.

Future Directions

• Automatic knowledge extraction: Automatically update KG from documents/logs;
• Dynamic knowledge fusion: Integrate real-time sensor data;
• Multimodal explanation: Combine visualization to enhance understanding;
• Personalized explanation: Generate content in different styles based on user roles.

Conclusion: Towards Trustworthy AI in Manufacturing

This study demonstrates the potential of the KG+LLM combination in manufacturing XAI, overcoming the factual accuracy issues of pure LLM and the expression limitations of pure KG to achieve accurate and easy-to-understand explanations. This framework provides a technical path for human-machine collaborative manufacturing intelligence: ML discovers data patterns, KG carries domain knowledge, and LLM acts as a "translator". As manufacturing digital transformation deepens, explainable AI will become an essential capability, and this study provides a feasible solution and practical verification.