ClinSeekAgent: A Multimodal Evidence Active Acquisition Agent Framework for Clinical Reasoning

This article introduces ClinSeekAgent, a clinical agent framework that shifts from passive evidence consumption to active evidence acquisition. It can automatically query medical knowledge bases, navigate electronic health records (EHRs), and call medical imaging tools to achieve dynamic multimodal evidence collection and clinical decision synthesis. It addresses the pain point of existing clinical AI relying on manually organized evidence and promotes real-scenario implementation.

Large language models and agent systems have great potential in the field of medical decision support, but existing studies assume that the required medical evidence has been manually organized and directly provided to the model. However, in real clinical settings, doctors face raw, scattered, and heterogeneous multimodal data sources (unstructured EHR texts, medical imaging DICOM files, laboratory results, medical literature databases). This gap between ideal and reality makes current clinical AI difficult to implement: doctors need to manually collect and organize evidence, and AI can only passively process pre-prepared inputs, lacking active exploration and dynamic adaptation capabilities.

Core Paradigm Shift

ClinSeekAgent实现从被动证据消费到主动证据获取的转变，包含三个创新：

1. Autonomous Evidence Collection: No pre-organization of evidence is required. After receiving clinical queries and data source access permissions, it independently decides which data sources to query, constructs queries, and integrates results. It can operate three core data sources: medical knowledge bases, EHR systems, and medical imaging tools;
2. Iterative Hypothesis Refinement: Simulates the doctor's diagnostic process: after each round of evidence collection, it evaluates the support for hypotheses, identifies information gaps, plans the next round of strategies, and gradually narrows the diagnostic scope;
3. Multimodal Evidence Fusion Reasoning: Unifies the representation of text, numerical, and image multimodal information through a dedicated module for joint reasoning.

Hierarchical Agent Architecture

由三个组件协同：

• Planning Layer: Decomposes clinical queries into subtasks, analyzes intentions, identifies hypotheses, and formulates evidence acquisition plans;
• Execution Layer: Interacts with data sources/tools and encapsulates professional interfaces (SQL constructor, literature retrieval API, imaging SDK, etc.);
• Reasoning Layer: Integrates evidence, evaluates weights and logical consistency, and outputs structured recommendations.

Tool Usage and API Orchestration

Maintains an extensible tool library, and the agent independently selects tool combinations. For example, in a suspected pneumonia case: call EHR to query symptom history → use laboratory data interface to obtain blood routine results → use imaging analysis tool to interpret X-rays → use literature retrieval tool to check guidelines.

1. Auxiliary Diagnostic Decision-Making: In complex case consultations, it quickly integrates patient history, test results, and imaging data, generates a structured differential diagnosis list, and provides evidence support score to reduce missed diagnoses and misdiagnoses;
2. Treatment Plan Recommendation: Based on the patient's individual characteristics (age, comorbidities, allergy history) and the latest guidelines, it recommends personalized plans, explains the basis, and automatically checks for drug interactions and contraindications;
3. Medical Education and Training: As an interactive learning tool, it demonstrates the complete clinical reasoning process, helping learners understand evidence collection and thinking construction from initial symptoms to diagnosis.

1. Data Privacy and Security: Adopts a federated learning architecture to process sensitive data locally and only send desensitized feature vectors to the cloud; implements fine-grained access control to ensure queries only access authorized data;
2. Evidence Credibility Evaluation: Introduces an evidence quality evaluation module that assigns credibility weights based on source authority, timeliness, sample size, etc., to avoid interference from low-quality evidence;
3. Interpretability Requirements: Generates detailed reasoning logs, records the decision basis and reasoning chain for each step of evidence acquisition, allowing doctors to review the logic and intervene.

There is still room for exploration in ClinSeekAgent:

• Multi-agent Collaboration: How different professional agents collaborate in multidisciplinary consultations for complex cases;
• Real-time Learning and Adaptation: Continuously learn from doctor feedback to optimize evidence acquisition strategies;
• Edge Computing Deployment: Deploy to edge devices to support decision-making in offline environments.

ClinSeekAgent provides new ideas for complex clinical decision-making through its active evidence acquisition capability. Technically, it realizes multimodal fusion and autonomous reasoning, and more importantly, it embodies the design concept that AI systems should be centered on real work processes. With the progress of LLM and agent technologies, we look forward to such systems entering the clinical frontline and becoming trusted intelligent assistants for doctors.