Large Language Models Combined with Genetic Programming: Automatically Discovering Interpretable Kalman Filter Variants

A research team from ETH Zurich proposed a new method combining large language models (LLMs) and Cartesian Genetic Programming, which can automatically discover and optimize Kalman filtering algorithms from raw data. This method outperforms traditional Kalman filters in adversarial environments, and the generated symbolic expressions are interpretable, providing a new paradigm for the automatic improvement of classic algorithms.

Since 1960, the Kalman filter has been a cornerstone in the field of state estimation, but it relies on precise system modeling and is limited when facing challenges such as model mismatch, nonlinear systems, and non-Gaussian noise. In recent years, studies like FunSearch have shown that combining LLMs with evolutionary algorithms can automatically discover complex mathematical structures and algorithms, inspiring the team to explore applying this paradigm to Kalman filters. The core question is: Can AI automatically rediscover and improve the Kalman filter?

Two complementary evolutionary search strategies are adopted:

1. Cartesian Genetic Programming (CGP)：A graph-based variant of genetic programming that uses a fixed-size genotype grid to express state update equations, generating compact and interpretable symbolic expressions.
2. LLM-assisted FunSearch-style Mutation：Integrates the DeepSeek-R1-Distill-Qwen-14B model to generate semantically meaningful code variants based on the best programs in the population, combining LLM code understanding and evolutionary exploration capabilities. Experimental setup: 4×NVIDIA H100 GPUs, 64-core CPU node, each experiment takes about 3 days, maximum model generation length is 3024 tokens, and 4 independent evolutionary islands run in parallel.

Two core experiments are designed:

1. Ideal Condition Evaluation：Linear dynamic system + Gaussian noise scenario, verifying whether the classic Kalman update rules can be rediscovered from input-output trajectories, using Mean Squared Error (MSE) to compare with the theoretical optimal solution.
2. Adversarial and Non-Ideal Condition Evaluation：Testing robustness, including scenarios that are difficult for traditional Kalman filters to handle, such as observation delays, nonlinear systems, and non-Gaussian noise (heavy-tailed/skewed).

1. Near-Optimal Algorithm Recovery：Both pure CGP or LLM-assisted search can rediscover Kalman-like update rules from data, and their performance is close to the theoretical optimal under ideal conditions.
2. Surpassing Classic Kalman Filters：Under non-ideal conditions, the symbolically evolved programs systematically outperform traditional Kalman filters and can automatically adapt to model mismatches.
3. Interpretability Advantage：The output is symbolic, human-readable mathematical expressions, allowing engineers to understand and verify the logic, which is superior to black-box neural networks.

• Contribution to Control Theory: Data-driven methods can automatically explore the algorithm space, discover improvement schemes that human experts may overlook, and break through the traditional design model that relies on mathematical skills and domain knowledge.
• Implications for Engineering Practice: Suitable for scenarios where system dynamics are unknown/time-varying, noise is complex and difficult to parameterize, performance and efficiency need to be balanced, and interpretability is required.
• Methodological Inspiration: LLMs (such as the 14B-parameter DeepSeek model) can act as "intelligent mutation operators" in evolutionary computing, opening up new directions for neuro-symbolic AI.

Limitations: High computational cost (each experiment takes about 3 days of GPU time), problem specificity (customized for Kalman filtering), and lack of convergence theory guarantees. Future Directions: Explore more efficient LLM calling strategies, apply to other classic algorithms (such as PID controllers, particle filters), combine neural networks with symbolic programs, and develop a convergence theory framework for LLM-assisted evolution.