DERMA-SCAN.AI: A Multi-Model Fusion Medical-Grade Skin Lesion Classification System

Hello everyone! Today I'd like to introduce the open-source project DERMA-SCAN.AI on GitHub — a medical-grade skin lesion classification system built with FastAPI and React. This system uses nine-model voting integration, OpenCV preprocessing (Dull-Razor algorithm), and a visual large model (VLLM) fallback mechanism to achieve skin lesion classification and severity assessment, aiming to help medical workers improve diagnostic efficiency and consistency. The project is maintained by HARISHPG21, with the original link at https://github.com/HARISHPG21/DERMA-SCAN.AI, updated on June 16, 2026.

Project Background and Clinical Significance

Skin cancer is one of the most common types of cancer globally, and early detection and accurate diagnosis are crucial for improving the cure rate. Traditional diagnosis relies on the professional experience of dermatologists; AI systems can provide assistance in the screening phase, identify suspicious lesions, and improve efficiency and consistency.

As a medical AI platform for skin lesion analysis, DERMA-SCAN.AI's design considerations include high accuracy, interpretability, robustness, and deployability. It integrates technical solutions of ensemble learning, traditional image processing, and visual large models, reflecting the combination of engineering practice and clinical needs.

Technical Architecture Analysis

Backend: Built with the FastAPI framework, it has the characteristics of a modern high-performance web framework, supporting asynchronous processing of image uploads and inference requests, automatically generating OpenAPI documents for easy integration, and a modular design that facilitates maintenance and expansion. Frontend: Developed with React+Vite, it provides a smooth interactive experience. Vite's fast hot reload and optimized build improve development and deployment efficiency, and the interface design considers medical scenario needs (clear image display, intuitive result presentation, etc.).

Core Technical Innovations (1): Multi-Model Integration and Preprocessing

1. Nine-Model Voting Integration: By combining the prediction results of nine machine learning models, it reduces the bias and variance of a single model and improves stability; it can also quantify uncertainty (high credibility if models agree, mark for manual review if there are disagreements).
2. OpenCV Dull-Razor Preprocessing: Designed for dermoscopy images, it removes hair interference through morphological operations and interpolation techniques, preserves lesion details, and improves the input quality of subsequent classification models.

Core Technical Innovations (2): Visual Large Model Fallback Mechanism

When the prediction confidence of the traditional ensemble model is low, the system calls a visual large language model (VLLM) for secondary analysis. Pre-trained on massive data, VLLM has rich visual knowledge and reasoning capabilities, and can synthesize contextual information such as the overall characteristics of the lesion and surrounding skin conditions, balancing the system's response speed and the ability to analyze difficult cases.

Application Scenarios and Usage Flow

Application Scenarios:

• Primary screening: Assess lesion risk in primary/telemedicine settings and identify suspicious cases that need referral;
• Teaching and training: Medical students compare AI results with their own observations to accelerate professional competence development;
• Research support: Provide standardized image analysis to assist data collection and statistics.

Usage Flow: Upload skin lesion image → System automatically preprocesses and classifies → Returns results and confidence → Doctor makes final diagnosis based on the suggestions.

Technical Challenges and Ethical Considerations

Technical Challenges:

• Data quality and diversity: Mitigated through preprocessing pipelines and diverse training data;
• Model generalization ability: Ensemble learning improves adaptability to images from different populations/devices;
• Balance between efficiency and accuracy: Optimize inference process + VLLM fallback mechanism to control response time.

Limitations and Ethics:

• Regulatory compliance: Must comply with local medical device regulations for clinical use;
• Diagnostic accuracy: May still misjudge (rare lesions or those outside the training distribution);
• Responsibility attribution: Need a clear legal framework to define liability for AI-assisted errors; Important Note: The system is an auxiliary tool; the final diagnosis must be made by professional medical personnel.

Summary and Outlook

DERMA-SCAN.AI is an open-source medical AI project with an advanced technical architecture. Its combined solution of multi-model integration, preprocessing, and fallback mechanism demonstrates the technical possibilities of skin lesion classification systems. FastAPI+React ensures performance and experience, and the modular design facilitates expansion.

This project not only implements functions but also provides a reference example for medical AI development. With the progress of AI technology and the accumulation of medical data, AI diagnosis of skin lesions will become more accurate and reliable. DERMA-SCAN.AI lays the foundation for intelligent medical auxiliary tools and is worthy of in-depth research by developers and researchers.