Bolek: A Compact Multimodal Language Model for Molecular Reasoning

Bolek is a compact multimodal language model with 4 billion parameters. Its core innovation lies in injecting Morgan fingerprint embeddings into the text decoder to enable natural language reasoning based on molecular structures. The model has demonstrated performance surpassing large models in drug discovery tasks, combining interpretability with deployment efficiency and providing a new solution for AI-assisted drug discovery.

Molecular property models are crucial for high-risk drug discovery decisions but have significant pain points: traditional predictors only return scores without reasoning basis, while language models can generate explanations but have weak connections to the actual molecular structure. This situation creates development opportunities for molecular reasoning models that combine performance and interpretability.

Core Design

Bolek anchors natural language reasoning to molecular structures by injecting Morgan fingerprint embeddings into an instruction-tuned text decoder.

Training Strategy

The model fine-tuning tasks include:

1. Molecular alignment tasks (molecular description, RDKit descriptor prediction, substructure detection)
2. Downstream reasoning tasks (based on 15 TDC binary classification tasks, using synthetic chain-of-thought anchored to molecular features)

Performance Comparison

• Compared to Qwen3-4B-Instruct: All superior in yes/no mode; 13 out of 15 tasks are better in chain-of-thought mode
• Average AUC increased from 0.55 to 0.76
• Compared to TxGemma-9B-Chat: With less than half the number of parameters, it is better in 13 out of 15 tasks

Interpretability Advantages

• Descriptor reference frequency is 10-100 times that of the baseline
• Values are highly consistent with RDKit key descriptors (Spearman correlation coefficient: 0.87-0.91)

Generalization Capability

• Cross-task: 5 out of 15 unseen TDC classification tasks are on par with TxGemma
• Cross-domain: Despite not being exposed to regression tasks, it has non-trivial rank correlation in 3 reserved regression endpoints

Technical Insights

1. Targeted modal injection: Effectively integrates structured information such as molecular fingerprints
2. Verifiable feature binding: Ensures the reasoning process is traceable and verifiable
3. Efficient use of compact models: Small models surpass large models through sophisticated design

Conclusions

Bolek verifies that through targeted modal injection and reasoning supervision with verifiable molecular feature binding, a compact and auditable molecular reasoning model can be built, opening up new possibilities for AI-assisted drug discovery.

In the field of drug discovery, Bolek provides a high-performance and auditable molecular reasoning solution: its compact architecture is easy to deploy, and its solid interpretability meets the transparency requirements for high-risk decisions, making it promising to become an important tool for AI-assisted drug research and development.