SymBOL: A Bayesian Optimization-Enhanced Large Model Symbolic Learner

SymBOL (Symbolic Learner) is a general symbolic learning framework that innovatively combines large language models (LLM) with Bayesian optimization (BO) to enable efficient scientific discovery. Its core idea is to use BO to guide LLM in searching for symbolic expressions, leveraging LLM's semantic understanding and code generation capabilities alongside BO's search efficiency to address the challenge of automatic symbolic law discovery from observational data.

Scientific discovery often requires finding concise mathematical expressions, but traditional symbolic regression methods like genetic programming face low search efficiency and difficulty handling high-dimensional data. Neural networks are powerful but lack interpretability. LLMs have strong semantic understanding and code generation abilities but lack a systematic search mechanism. These gaps motivate the fusion of LLM and BO in SymBOL.

SymBOL's architecture integrates two key components:

1. Bayesian Optimization Framework: Uses Gaussian process as surrogate model (modeling performance distribution with mean and uncertainty) and acquisition functions (EI, UCB, info gain) to guide search.
2. LLM-Enhanced Candidate Generation: LLM acts as an intelligent agent to generate candidate expressions via prompt-based methods (incorporating existing performance data, mathematical operations, nonlinear relationships). The iterative loop: Initialize → Evaluate → Update surrogate → LLM generate → Select via acquisition → Repeat until convergence.

• Expression Representation: Uses tree structure (e.g., x1*x2 + sin(x3) as a tree) and prefix notation (e.g., (+ (* x1 x2) (sin x3)) for easy LLM handling.
• LLM Prompt Design: Uses in-context learning (providing examples like free fall or Ohm's law) and chain-of-thought (guiding step-by-step reasoning).
• BO Adaptation: Handles discrete expression space with suitable kernels and distance metrics; supports multi-objective optimization (fitting accuracy, complexity, interpretability).

SymBOL applies to multiple scientific domains:

• Physics: Rediscovers Newton's second law (F=ma) and ideal gas law (PV=nRT) using observational data.
• Chemistry: Finds reaction rate equations (e.g., r=k*[A]^m*[B]^n) from concentration and rate data.
• Biology: Models population growth, enzyme kinetics (Michaelis-Menten equation), and neural network activity patterns.

vs Genetic Programming: SymBOL uses BO-guided LLM generation (faster convergence, better use of prior knowledge) vs GP's random mutation (slow, local optima). vs Pure LLM: SymBOL has systematic search (BO avoids repetition, uses full history) vs pure LLM's low systematicity. vs Neuro-symbolic Methods: SymBOL uses explicit BO search (interpretable iterations, flexible domain knowledge integration) vs end-to-end learning.

• LLM Hallucination: Use syntax checks, code models (e.g., Codex), and few-shot examples.
• Evaluation Cost: Use surrogate models for prediction, parallel evaluation, and early stopping.
• Expression Equivalence: Normalize representations (sort operands), symbol simplification, hash deduplication.
• High-dimensional Data: Feature selection, hierarchical search (single variable first), LLM-based variable correlation judgment.

Future Directions: Multimodal extension (visual data integration), active learning (optimal experiment design), causal discovery (distinguish correlation vs causation), domain adaptation (physics/chemistry/biology-specific constraints). Conclusion: SymBOL represents an important direction in AI for Science, combining LLM's semantic abilities with BO's search efficiency. It retains symbolic methods' interpretability while leveraging LLM's prior knowledge, promising to assist scientists in discovering new laws and models.