Multimodal Retail Decision Intelligence: Practical Integration of Graph Neural Networks and Large Language Models

This project explores the integration of Graph Neural Networks (GNN), Large Language Models (LLM), and multimodal embedding technologies to build an intelligent decision-making system that can understand retail multimodal data. It addresses the problem that traditional models struggle to handle associations between heterogeneous data, supporting retail decision tasks such as intelligent recommendation and demand forecasting.

The modern retail environment is complex, and consumer decisions are influenced by multiple factors such as price, images, and reviews. Traditional models only handle single data types and struggle to capture deep associations. Retail data has multimodal characteristics: transaction records (structured), product descriptions (text), product images (visual), and user relationships (graph structure). Unified modeling of heterogeneous data is a key challenge.

The core technology integration includes three points:

1. GNN: Models entity relationship networks such as users/products, learning node embeddings and topological structures;
2. LLM: Provides text semantic understanding, capturing emotional tendencies and implicit needs;
3. Multimodal embedding: Maps data from different modalities to a unified semantic space. Model layered design: Data preparation → Embedding learning → Graph construction → GNN modeling → LLM reasoning enhancement → Interpretability analysis → System evaluation.

Application Scenarios:

• Intelligent recommendation: Improve accuracy by combining multi-dimensional information;
• Demand forecasting/inventory optimization: Capture product relationships using graph structures;
• Customer analysis: Extract fine-grained preferences from reviews;
• Interpretable insights: Provide decision evidence and logic. Technical Highlights: Notebook-driven reproducibility, validation using public datasets, MIT open-source license supporting commercial deployment.

Limitations: Currently a master's project, focusing on technical verification rather than production deployment; real-time performance and model lightweighting need optimization. Future Directions:

• Optimize real-time inference services;
• Explore model compression and edge deployment;
• Introduce Causal AI to enhance decision robustness;
• Expand multilingual support to adapt to global scenarios.

This project demonstrates the application potential of multimodal AI in the retail field. It is a decision support framework rather than a single recommendation system. For practitioners, it provides a learning template for building a multimodal system from scratch and designing reproducible processes.