LLM Art Valuation Research: Do Cutting-Edge Visual Models Truly Understand Art or Just Memorize Prices?

Introduction

This study compares four multimodal models—GPT-5.4, Claude, Gemini, and Qwen—by testing their art valuation performance under three conditions: pure image, metadata, and complete information. Key finding: Current models rely heavily on metadata knowledge rather than visual art understanding, revealing the true boundaries of AI's art cognition.

Artificial intelligence has made significant breakthroughs in the image domain, but when it comes to artworks—an area dependent on subjective aesthetics, cultural context, and market cognition—does AI truly understand art or just retrieve price tags? This question is critical to technical evaluation and the boundaries of AI cognition. Art valuation requires integrating multiple factors such as style, technique, and history, and AI's performance will expose its limitations in abstract concept comprehension and aesthetic judgment.

The study selected 20 paintings from different genres, periods, and price ranges as samples, and set up three control groups:

• Pure image condition: Only the painting image is provided to test visual feature extraction ability
• Metadata condition: Only background information such as artist and era is provided to test knowledge reasoning ability
• Complete information condition: Both image and metadata are provided to simulate real scenarios This design separates the contributions of visual understanding and knowledge memory to valuation accuracy.

The study selected four cutting-edge models:

• GPT-5.4: OpenAI's flagship model with excellent visual understanding performance
• Claude: Anthropic's series, known for reasoning ability and safety
• Gemini: Google's native multimodal model
• Qwen: Alibaba's open-source model with good performance in Chinese and English multimodal tasks Cross-vendor comparison helps identify the impact of architecture and training strategies on art valuation.

Experimental results show:

• Pure image condition: All models' valuation accuracy decreased significantly, reflecting the shortcoming of inferring value from visual features
• Metadata condition: Performance improved greatly, suggesting models rely on memorized market prices of known artists
• Cross-model differences: Some models have more sensitive visual encoders, while others rely more on textual knowledge, reflecting ability biases.

A highlight of the study is fully open data; the repository includes:

• Complete evaluation logs: Input and output records of each model call
• Reasoning traces: Thinking processes of chain-of-thought models
• Valuation dataset: Detailed information and reference prices of the 20 test works
• Comparative analysis scripts: Code to reproduce the conclusions Transparency facilitates other researchers to verify, expand experiments, or conduct in-depth analysis of specific categories of artworks.

Implications of the study for AI art applications:

• Current limitations: Do not over-rely on AI for independent valuation; it is prone to training data biases and unreliable for emerging artists or non-mainstream styles
• Human-AI collaboration: AI should be used as an auxiliary tool to help experts retrieve information, identify similar works, and organize market data
• Future improvements: Need fine-tuning for the art domain, integration of art criticism knowledge, and reinforcement learning with human feedback.

This study uses empirical data to show that current cutting-edge visual models rely on metadata rather than visual understanding in art valuation. This is not a denial of AI's capabilities, but a clear recognition of the current state of technology—art, as a complex and subjective human creative activity, is still a domain that AI needs to approach with humility.