AI-Powered Intelligent Query System for Clinical Trial Data: Natural Language to SQL Conversion and Visualization

This project is AI-in-Clinical-Trials-Explorer, maintained by paighowal on GitHub (released on May 26, 2026). Its core function is to build a natural language query interface based on large language models, enabling non-technical users (such as medical researchers, clinicians, etc.) to query clinical trial databases without SQL knowledge. The system automatically generates SQL and returns visualization results, addressing the pain point that non-technical personnel find it difficult to access complex databases.

Clinical trial data is a core asset for medical research, but it is usually stored in complex databases that require professional SQL knowledge to query. Non-technical users such as medical researchers, clinicians, and policymakers lack SQL skills, making it difficult to access these valuable data, leading to data being underutilized. This project aims to build a natural language query interface by combining large language models and database technology, allowing non-technical users to easily explore clinical trial data.

Overall Architecture

User query → SQL generation (LLM + database schema context) → SQL execution (SQLite) → Data insights (parallel: LLM analysis + chart selection) → Visualization generation → Result display

Core Functions

1. Multi-LLM Support: Compatible with OpenAI GPT, Google Gemini, Ollama (local), Groq (high-speed inference), users can choose as needed.
2. Intelligent SQL Generation: Accurate SQL generated via schema awareness (obtain column names, data types, etc.), semantic mapping (user intent to field correspondence), and example guidance.
3. Automatic Visualization: Select appropriate charts like bar charts, line charts, pie charts based on data characteristics.
4. Data Insight Generation: Call LLM to analyze results and provide data meaning and potential value.

Data Layer

Uses SQLite for storage, supports CSV import. Key fields include nct_id (trial identifier), enrollment (number of participants), phase (trial phase), sponsor_name (sponsor institution), etc.

Concurrency Processing

Multi-threaded parallel execution of data insight generation and chart selection decisions to reduce waiting time.

Error Handling

Includes mechanisms like LLM initialization failure fallback, SQL execution error recovery, chart generation failure fallback to tables, invalid column name handling, empty result set management, etc.

Typical Query Examples

• Top 10 trials sorted by number of participants
• Distribution of studies by phase
• Institutions sponsoring the most trials
• Diabetes-related trials
• Number of recruiting vs completed trials

Target Users

Medical researchers (quick trial screening), clinicians (find trials for specific diseases), policymakers (analyze trial distribution), pharmaceutical companies (monitor competitor progress).

Advantages

• Lower Threshold: Non-technical users can query data using natural language.
• Intelligent Visualization: Automatically select optimal charts to improve understanding efficiency.
• Context Awareness: Generate data insights to provide additional value.
• Multi-model Support: Flexibly choose LLM to balance speed and accuracy.

Limitations

• SQLite is suitable for GB-level data; ultra-large-scale data requires migration to professional databases.
• Visualization is suitable for data below 10,000 rows.
• Dependent on LLM capabilities and data quality.

Usage Recommendations

• Ensure data quality to improve query accuracy.
• Add query validation and rate limits in production environments.
• Monitor API costs of cloud LLMs.

Future Directions

Plan to support multi-table joins, query optimization, caching mechanisms, result export (CSV/JSON), advanced filtering, custom metrics, etc.

Conclusion

This system is a typical application of AI empowering data analysis. By converting SQL queries into conversations via a natural language interface, it allows domain experts to directly access data. This Text-to-SQL model can be extended to fields like medical records and scientific research databases, and will become a standard tool for knowledge workers in the future, improving the efficiency of data-driven decision-making.