Agent Workflow Based on LangChain: FashionMNIST Image Classification Practice

This project shows how to build an agent AI workflow using LangChain and Ollama, achieving intelligent image classification on the FashionMNIST dataset through four dedicated tools: image loading, category prediction, confidence check, and result explanation. It embodies the core advantages of agent AI: dynamic decision-making, interpretability, and tool combination.

Traditional machine learning models are encapsulated as API services, with a deterministic process but limitations such as inability to adjust behavior, explain decisions, or seek help. Agent AI endows the system with tool-using capabilities and decision-making autonomy, allowing dynamic combination of operations and evaluation of intermediate results. This project is a practice of the agent AI concept, building a workflow with a perception-decision-action loop through LangChain and Ollama.

The FashionMNIST dataset contains 10 categories of clothing images. This project focuses on demonstrating a flexible system combining large language models with dedicated tools. Technology selection: LangChain provides agent orchestration capabilities (tool definition, chain calls, etc.), Ollama supports local running of open-source models (e.g., Llama 3.2, Phi3) to ensure privacy, and the tool architecture uses a modular design for easy expansion.

1. Image Loading Tool: Preprocesses images into a format that the model can handle and verifies input quality—it is the "perception" link of the agent. 2. Category Prediction Tool: Outputs category labels and probabilities based on a pre-trained CNN, providing only candidate results. 3. Confidence Check Tool: Evaluates the reliability of predictions, introduces "metacognitive" capabilities, and can take remedial measures when confidence is insufficient. 4. Result Explanation Tool: Generates natural language explanations, explains decision-making basis, and improves system transparency.

Typical workflow: Receive request → Call image loading tool → Obtain prediction results → Call confidence check → Decide subsequent actions based on confidence → Call explanation tool to generate results. The workflow is dynamically conditional; the agent adjusts steps based on intermediate results instead of following a fixed sequence.

• Model Management: Ollama is responsible for model download and inference, supporting multi-model selection (lightweight Phi3 is suitable for edge devices, Llama3.2 for servers). - Tool Definition: Python functions + LangChain decorators, with clear input and output modes. - Prompt Engineering: System prompts define the agent's role and tool usage logic. - Error Handling: Multi-layer mechanisms including tool exception capture, agent retries, and user-friendly feedback.

Application Scenarios: E-commerce product recognition, industrial quality inspection, medical image assistance, document processing. Expansion Directions: Add perception tools such as OCR/object detection, introduce memory mechanisms, integrate external knowledge bases, and implement multi-agent collaboration.

Although this project is small in scale, it demonstrates the core ideas of agent AI (tool combination + dynamic decision-making). The combination of LangChain and Ollama enables local deployment of the agent architecture, providing an introductory reference for the transformation from traditional ML to the agent paradigm.