Evaluation of Large Language Models' Small Molecule Drug Design Capabilities: From Benchmark Testing to Reinforcement Learning Post-Training

This paper constructs ChemRL, a drug design task benchmark based on chemical principles, and formalizes it as a reinforcement learning (RL) environment to evaluate the small molecule drug design capabilities of cutting-edge large language models (LLMs). The study finds: Cutting-edge models are becoming increasingly proficient in chemical tasks, but there is still room for improvement in low-data scenarios; crucially, RL-based post-training can significantly improve performance, enabling smaller models to reach the level of cutting-edge models.

Traditional new drug R&D takes 10-15 years, costs billions of dollars, has an extremely low success rate, and faces a "productivity crisis". AI, especially LLMs, with their cross-modal reasoning capabilities, are expected to accelerate drug design, but their practical utility in professional fields is still unclear, and the core obstacle is the lack of benchmarks that reflect real-world scenarios.

The study proposes the ChemRL benchmark suite, which formalizes drug design tasks as an RL environment. It covers three core tasks: 1) Molecular property prediction (e.g., ADMET, target affinity); 2) Molecular representation conversion (e.g., mutual conversion between SMILES and molecular graphs); 3) Molecular design (e.g., multi-objective optimization, scaffold hopping). The RL environment includes a state space (current partial solution/context), action space (predicting properties, modifying molecules, etc.), and reward function (continuous feedback, chemical rationality penalties, etc.), supporting iterative optimization and post-training.

Experimental evaluation of mainstream models finds: 1) Cutting-edge models (e.g., GPT-4, Claude3) perform well in SMILES parsing, basic property prediction, and simple molecule generation; 2) Performance drops significantly in low-data scenarios (e.g., new targets), multi-objective optimization, and under chemical rationality constraints; 3) RL post-training (e.g., PPO algorithm) enables smaller models to reach the level of cutting-edge models on the ChemRL benchmark.

The advantages of RL post-training are: 1) Shifting from passive learning to active exploration, where the model accumulates design intuition through trial and error; 2) Fine-grained continuous rewards provide richer learning signals; 3) Internalizing task structures (legal operations, target constraints, etc.) through interaction to improve generalization ability.

For the pharmaceutical industry: 1) Prioritize small models that have undergone specialized post-training over general-purpose large models; 2) Adopt iterative human-machine collaboration (model proposes candidates → expert evaluation → feedback training); 3) Attach importance to organizing high-quality domain data. For AI research: 1) Design benchmarks that support training (e.g., ChemRL); 2) Explore efficient post-training strategies instead of only pursuing pre-training scale; 3) Strengthen the integration of domain knowledge and AI (e.g., encoding chemical rules into reward functions).

Current limitations: ChemRL still has simplified assumptions (e.g., accurate property prediction), high RL training costs, and unproven generalization to real-world scenarios. Future directions: Integrate real experimental feedback, explore multi-agent collaboration, quantify uncertainty, and enhance model interpretability.