Data Imputation Using Large Language Models: A New Paradigm of Prompt Engineering

This article explores how the LLMsImputation project applies large language models (LLMs) to the task of missing data imputation, using prompt engineering techniques to achieve a data repair solution without traditional machine learning training. The project redefines data imputation as a natural language generation problem, leveraging the general knowledge and context understanding capabilities of LLMs to provide a new paradigm for improving data quality.

In the field of data science, data quality is key to model performance, but real-world datasets often have missing values (due to technical failures, users not filling in, etc.). Traditional imputation methods such as mean imputation, regression imputation, and K-nearest neighbor imputation require training on specific datasets and have limited understanding of complex data patterns and contextual relationships.

LLMsImputation was developed by Arthur Mangussi. Its core innovation is redefining the data imputation task as a natural language generation problem. Unlike traditional methods that require specially trained models, it uses the general knowledge capabilities of LLMs and prompt templates to enable the model to understand context and generate reasonable filled values. This method can handle multiple data types such as text, numerical values, and categories, and capture semantic relationships between columns (e.g., the relationship between name, occupation, and income in customer information).

The core of LLMsImputation is prompt engineering design. The prompt template includes: 1. Context description (data source and meaning); 2. Data examples (reference for complete rows); 3. Records to be imputed (missing fields marked); 4. Task instructions (infer and fill). This design leverages the in-context learning capabilities of LLMs to understand data patterns without fine-tuning, and in some scenarios, its performance can surpass traditional algorithms.

LLMsImputation is suitable for: 1. Small sample datasets (no need for large amounts of training data); 2. Multimodal mixed data (unified processing of structured and free text); 3. Scenarios with rich domain knowledge (e.g., medical, finance, using domain descriptions in prompts). Advantages include strong interpretability (the imputation process is presented in natural language, making it easy to audit).

The current method faces challenges: 1. High computational cost (calling LLM APIs is more expensive than traditional algorithms); 2. Privacy and security (sending sensitive data to external APIs has leakage risks, requiring local deployment or differential privacy); 3. Insufficient stability (generated results are random). Improvement directions: Explore consistency constraints, deterministic generation strategies, and establish a comprehensive benchmark testing system.

LLMsImputation represents a trend in data engineering: using the general capabilities of LLMs to solve traditional data quality problems. It inspires a rethinking of the relationship between data and knowledge—massive knowledge internalized by LLMs can be used to repair incomplete data. As LLM capabilities improve and costs decrease, prompt engineering-based data processing methods are expected to be more widely applied, providing practitioners with tools to address complex data quality issues.