Experiment Tracker: A Self-Hosted Machine Learning Experiment Tracking Platform for Research Scenarios

Experiment Tracker is an open-source self-hosted machine learning experiment tracking tool designed specifically for research-intensive workflows. Its core features include metric comparison, scalar curve analysis, step-aware artifact management, and experiment lineage tracking. Built with FastAPI+Next.js+PostgreSQL+ClickHouse+MinIO architecture, it has a clear positioning—focusing on "experiment understanding" to help researchers answer key experiment questions, rather than being a full-featured MLOps suite.

TensorBoard excels at local visualization, but Experiment Tracker complements it with project-level research context:

• Metric-prioritized model selection tables, supporting project-wide grid comparisons
• Scalar curves for multi-experiment comparison (smoothing, hovering, zooming, etc.)
• Step-aware named artifact management
• Editable experiment lineage tracking (search, highlighting, layout persistence) Its positioning is a clear and focused research workspace, not a tool for training orchestration or deployment.

1. Metric Comparison and Model Selection

Provides dense tables with support for filtering, sorting, exporting, highlighting extreme values, and viewing metadata via side panels

2. Scalar Curve Analysis

Powered by ClickHouse, with features like synchronized axes, smoothing, view saving, etc. ###3. Step-Aware Artifact Review Grouped by type/name, with association to training step context ###4. Experiment Lineage Tracking Research tree view, parent-child relationships, online difference comparison ###5. File Comparison Side-by-side difference highlighting, inline change display

These features follow research intuition: first compare metrics, then dive into dynamics.

Architecture Component Division

Component	Purpose
PostgreSQL	Relational state (users, projects, permissions, etc.)
ClickHouse	High-volume scalar time series and artifact metadata
MinIO/S3	Large file storage
FastAPI	Backend orchestration layer
Next.js	Frontend interface
Python SDK/CLI	Training logging and command-line interaction

Tech Stack Highlights

Python3.10+, FastAPI, Next.js, PostgreSQL, ClickHouse, MinIO/S3, Docker (self-hosting support)

The architecture separates data by workload to match the form of experimental data.

Suitable Scenarios

• Comparing multiple experiment metrics to select the best model
• Analyzing training/validation curves to understand learning dynamics
• Tracking the relationship between artifacts and training steps
• Understanding the evolutionary relationship of experiments
• Self-hosting needs, data sovereignty

Unsuitable Scenarios

• Training orchestration, infrastructure management
• Model registry/production deployment
• Hyperparameter auto-search/GPU queueing
• Full-featured AI platform requirements

For such MLOps needs, W&B or ClearML can be chosen.

Experiment Tracker adopts a "counter-trend" design: against the backdrop of complex full-featured MLOps platforms, it focuses on the core research need—experiment understanding. By integrating key features into a lightweight self-hosted platform, it provides a practical choice for teams that need data sovereignty and want to upgrade their experiment tracking capabilities.

Recommendation: Teams that are building an internal ML platform, upgrading from TensorBoard, or do not want to use heavyweight commercial solutions should consider evaluating this tool.