# Bolek: A Compact Multimodal Language Model for Molecular Reasoning

> Bolek is a 4-billion-parameter multimodal language model. By injecting Morgan fingerprint embeddings into the text decoder, it enables natural language reasoning based on molecular structures and has demonstrated performance surpassing large models in drug discovery tasks.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-04T15:46:39.000Z
- 最近活动: 2026-05-05T02:39:32.768Z
- 热度: 127.1
- 关键词: 分子推理, 多模态模型, 药物发现, Morgan指纹, 可解释AI, TDC基准
- 页面链接: https://www.zingnex.cn/en/forum/thread/bolek
- Canonical: https://www.zingnex.cn/forum/thread/bolek
- Markdown 来源: floors_fallback

---

## 【Introduction】Bolek: A Compact Multimodal Language Model for Molecular Reasoning

Bolek is a compact multimodal language model with 4 billion parameters. Its core innovation lies in injecting Morgan fingerprint embeddings into the text decoder to enable natural language reasoning based on molecular structures. The model has demonstrated performance surpassing large models in drug discovery tasks, combining interpretability with deployment efficiency and providing a new solution for AI-assisted drug discovery.

## Challenges and Opportunities in Molecular Reasoning

Molecular property models are crucial for high-risk drug discovery decisions but have significant pain points: traditional predictors only return scores without reasoning basis, while language models can generate explanations but have weak connections to the actual molecular structure. This situation creates development opportunities for molecular reasoning models that combine performance and interpretability.

## Bolek Model Architecture and Training Strategy

### Core Design
Bolek anchors natural language reasoning to molecular structures by injecting **Morgan fingerprint embeddings** into an instruction-tuned text decoder.
### Training Strategy
The model fine-tuning tasks include:
1. Molecular alignment tasks (molecular description, RDKit descriptor prediction, substructure detection)
2. Downstream reasoning tasks (based on 15 TDC binary classification tasks, using synthetic chain-of-thought anchored to molecular features)

## Performance and Generalization Capability Verification

### Performance Comparison
- Compared to Qwen3-4B-Instruct: All superior in yes/no mode; 13 out of 15 tasks are better in chain-of-thought mode
- Average AUC increased from 0.55 to 0.76
- Compared to TxGemma-9B-Chat: With less than half the number of parameters, it is better in 13 out of 15 tasks
### Interpretability Advantages
- Descriptor reference frequency is 10-100 times that of the baseline
- Values are highly consistent with RDKit key descriptors (Spearman correlation coefficient: 0.87-0.91)
### Generalization Capability
- Cross-task: 5 out of 15 unseen TDC classification tasks are on par with TxGemma
- Cross-domain: Despite not being exposed to regression tasks, it has non-trivial rank correlation in 3 reserved regression endpoints

## Technical Insights and Core Conclusions

### Technical Insights
1. Targeted modal injection: Effectively integrates structured information such as molecular fingerprints
2. Verifiable feature binding: Ensures the reasoning process is traceable and verifiable
3. Efficient use of compact models: Small models surpass large models through sophisticated design
### Conclusions
Bolek verifies that through targeted modal injection and reasoning supervision with verifiable molecular feature binding, a compact and auditable molecular reasoning model can be built, opening up new possibilities for AI-assisted drug discovery.

## Application Prospects of Bolek

In the field of drug discovery, Bolek provides a high-performance and auditable molecular reasoning solution: its compact architecture is easy to deploy, and its solid interpretability meets the transparency requirements for high-risk decisions, making it promising to become an important tool for AI-assisted drug research and development.