OllamaOnDemand: A ChatGPT-like Large Model Interaction Interface Designed for High-Performance Computing Clusters

OllamaOnDemand is an open-source Gradio web interface developed by the Louisiana State University (LSU) HPC team, specifically designed for High-Performance Computing (HPC) clusters. It addresses the complex configuration issues researchers face when running Large Language Models (LLMs) on supercomputing clusters, provides an intuitive ChatGPT-like interaction experience, and natively supports Open OnDemand integration—allowing users to utilize local large models without diving into the underlying infrastructure.

Background: Pain Points of Running Large Models on Supercomputing Clusters

With the widespread application of LLMs in scientific research, teams want to deploy models on HPC clusters but face many challenges: complex container configurations, tedious environment dependency management, and compatibility issues with scheduling systems like Slurm. Additionally, researchers accustomed to ChatGPT's web interface have a learning curve with command-line operations and complex configuration files. Balancing HPC computing power with a simple user experience has become an urgent problem to solve.

Core Features and Technical Implementation

Native Open OnDemand Support

Open OnDemand is an NSF-supported HPC portal platform. OllamaOnDemand natively supports subpath operation and can be directly deployed as an interactive application without additional reverse proxy configuration.

Multimodal Capability Support

It includes the multimodal.py module, supporting text dialogue and multimodal inputs like images—suitable for scenarios such as scientific research charts and experimental image analysis.

Session Management and Remote Model Support

chatsessions.py enables conversation persistence, and remotemodels.py supports connecting to remote model services with flexible endpoint configuration.

User Configuration System

It provides user-level configuration options such as model parameter adjustment and interface theme customization via usersettings.json and usersettings.py.

Deployment and Application Scenarios

Typical Deployment Process

1. Install the Ollama service on compute nodes
2. Configure the Python environment and install dependencies
3. Register as an Open OnDemand interactive application
4. Users launch personal instances via the cluster portal

Application Scenarios

• Sensitive Data Processing: Running on local clusters ensures data does not leave the environment
• Customized Models: Using fine-tuned domain models for inference
• Batch Experiments: Integrating with Slurm scheduling for automated evaluation
• Teaching and Training: Providing a user-friendly LLM entry point for HPC beginners

Highlights of Technical Architecture

The project uses a modular design, with key components including:

• main.py: Core application logic (≈78KB)
• arg.py: Command-line parameter parsing
• grblocks.css: Interface style customization
• head.html: HTML header template
• container/: Containerized deployment configuration

It uses the MIT license, lowering the barrier for academic institutions to use it.

Limitations and Outlook

Currently, the project's README is concise, and the documentation needs improvement; users unfamiliar with HPC environments still need onboarding guidance; the star count (2 stars) indicates it is in the early stage, with limited community contributions and deployment cases.

However, based on the LSU HPC team's professional background and the real pain points it addresses, the project has good development potential and is expected to become a reference solution for LLM deployment in HPC environments.

Conclusion

OllamaOnDemand focuses on solving practical problems—retaining HPC computing power while lowering the barrier to using LLMs, without pursuing technical novelty. For cluster centers running Open OnDemand, it is a tool worth paying attention to and trying.