TB-Vision Pro: An Intelligent Tuberculosis Diagnosis System Based on Multimodal Deep Learning

TB-Vision Pro is an open-source medical diagnosis platform with a consensus architecture integrating Vision Transformer, ResNet-50, and MobileNetV2. Combined with MedSAM segmentation technology and clinical data analysis, it enables early screening of tuberculosis, lesion visualization, and rehabilitation tracking, providing an innovative solution to the time-consuming and subjective misdiagnosis issues in traditional diagnostic processes.

As a major global public health challenge, tuberculosis causes millions of infections each year. Early detection and accurate diagnosis are key to controlling the epidemic. Traditional diagnosis relies on doctors' experience and tedious image analysis, which is time-consuming and prone to misdiagnosis due to subjective factors.

TB-Vision Pro adopts a consensus architecture with parallel analysis of three models:

• Vision Transformer (ViT)：Captures the global structure and long-range dependencies of lung images, excels at identifying diffuse lesions;
• ResNet-50：Extracts multi-scale texture features via residual networks, accurately identifies typical pathological manifestations such as nodules and cavities;
• MobileNetV2：Achieves fast inference with depthwise separable convolution, suitable for resource-constrained environments. The final results are integrated via a weighted fusion algorithm to reduce the bias of a single model and improve robustness.

The system integrates MedSAM segmentation technology, automatically annotates affected lung areas and generates color-coded severity maps, enabling visual interpretation of diagnostic results and enhancing doctors' trust. It also supports image upload via Ctrl+V paste, simplifying the operation process and improving work efficiency.

The platform has a built-in longitudinal analysis function that can automatically identify follow-up patients and generate rehabilitation progress charts, showing the trend of TB risk probability changes, which helps evaluate treatment effects and adjust plans. This design realizes the transition from a diagnostic tool to a comprehensive management platform.

Technology Stack: Frontend built with React18+Vite+TailwindCSS, backend using FastAPI asynchronous framework, AI layer integrating TensorFlow/Keras ecosystem; Data Insight: Real-time dashboard provides screening speed monitoring, patient demographic analysis, and recent activity streams, offering data support for medical institution management and public health decision-making.

TB-Vision Pro represents the evolution direction of AI medical diagnosis from a single function to a comprehensive platform, providing end-to-end prevention and control support. The project states that it is only for research and clinical demonstration; actual diagnosis requires the participation of professional medical personnel. In the future, through the maturity of MedSAM technology, multi-center validation, and data accumulation, it will help tuberculosis prevention and control in resource-poor areas and promote the achievement of global elimination goals.