Experiment tracking with MLflow

Generated by Claude through GigAI 0.1.11.10 on 2026-10-11.

Concept: MLflow · also covers Experimentation to production, Model training · Step 11 of 35 in MLOps / ML platform engineer · version 1

Experiment tracking records what was run, with which inputs, and what came out, so any model can be traced back to its code, data and hyperparameters. MLflow Tracking is the most common open-source implementation: a client API that logs runs, and a server that stores metadata and artifacts. For a platform engineer the job is mostly running that server reliably and making logging the default path for every training job.

How it is done in practice

A platform team runs the tracking server as a small replicated deployment on Kubernetes in front of a managed PostgreSQL database and an object storage bucket, behind the company's single sign-on. Training job templates inject the tracking address and default tags as environment variables, and a shared helper library starts the run, turns on autologging and attaches the git commit and image digest. Experiments are named per team or project, old runs are pruned by a retention job, and database and bucket are backed up because losing tracking history means losing the lineage of every registered model. Teams that use Weights and Biases instead follow the same pattern with a hosted backend.

Technologies

Diagram

flowchart LR
  A[Training job with MLflow client] -->|REST: params, metrics, tags| B[MLflow tracking server]
  A -->|artifact upload| B
  B --> C[(Backend store: SQL database)]
  B --> D[(Artifact store: object storage)]
  E[Engineer in MLflow UI] --> B
  B --> F[Model registry]

flowchart LR
  A[Training job with MLflow client] -->|REST: params, metrics, tags| B[MLflow tracking server]
  A -->|artifact upload| B
  B --> C[(Backend store: SQL database)]
  B --> D[(Artifact store: object storage)]
  E[Engineer in MLflow UI] --> B
  B --> F[Model registry]

Code

import os
import mlflow

# MLFLOW_TRACKING_URI is injected by the platform
mlflow.set_experiment("fraud-classifier")

with mlflow.start_run(run_name="baseline") as run:
    mlflow.set_tag("git_commit", os.environ.get("GIT_SHA", "unknown"))
    mlflow.log_params({"lr": 3e-4, "batch_size": 256})
    for epoch in range(3):
        loss = 1.0 / (epoch + 1)  # stand-in for a real training step
        mlflow.log_metric("train_loss", loss, step=epoch)
    with open("report.txt", "w") as f:
        f.write("evaluation summary")
    mlflow.log_artifact("report.txt")
    print("run id:", run.info.run_id)

Subtopics

The tracking data model: experiments, runs, params, metrics, artifacts

What gets recorded

MLflow organises work into experiments, each holding many runs. A run is one execution of training code and carries four kinds of data. Parameters are key-value inputs such as learning rate or batch size, written once. Metrics are numeric values that can be logged repeatedly with a step and timestamp, which gives loss and accuracy curves. Tags are free-form metadata such as git commit, owner or dataset version. Artifacts are files of any size: model weights, plots, config files, evaluation reports. Every run has a unique run ID, a start and end time, and a status, and that ID is the handle everything downstream uses.

Why the split matters

Parameters, metrics and tags are small and queryable, so they live in a database. Artifacts are large and opaque, so they live in object or file storage and the run only stores their location. This split drives the server architecture and the cost profile: metadata queries must be fast for the UI and search API, while artifact storage must be cheap and durable. Teams usually agree on a small set of mandatory tags so runs can be filtered consistently across projects.

Tracking server architecture: backend store and artifact store

Two stores behind one server

A production MLflow deployment has three parts. The tracking server is a stateless HTTP service that exposes the REST API and the UI. The backend store is a SQL database, typically PostgreSQL or MySQL, holding experiments, runs, params, metrics and tags. The artifact store is object storage such as S3, GCS or Azure Blob. Clients can upload artifacts directly to the bucket with their own credentials, or the server can proxy artifact access so that clients only need to reach the server and never hold bucket credentials. Proxied access simplifies identity management but puts large uploads through the server, so it needs enough replicas and sensible timeouts.

Instrumenting training code: explicit logging and autologging

Two ways to log

Explicit logging wraps training in a run context and calls log_param, log_metric and log_artifact where needed. It is precise but relies on every author remembering to do it. Autologging is switched on with a single call and hooks into supported frameworks such as PyTorch Lightning, scikit-learn, XGBoost and Keras to capture parameters, per-epoch metrics and the trained model without code changes. Platforms usually combine both: autologging for the baseline, plus a thin wrapper that adds mandatory tags like git commit, container image and dataset version. The client finds the server through the MLFLOW_TRACKING_URI environment variable, which the platform injects into every training pod so user code contains no addresses.

Comparing runs and making results reproducible

From a list of runs to a decision

The value of tracking appears when runs are compared. The MLflow UI and the search API filter runs by parameter, metric or tag and plot metric curves side by side, which answers questions like which learning rate converged fastest. Reproducibility needs more than hyperparameters: the run should also record the source revision, the environment or image, the dataset version and the random seed. With those in place, a run ID becomes a complete recipe. The same ID then links forward into the model registry, so a model version in production can be traced back to the exact run, metrics and artifacts that produced it.

Sources

• docMLflow Tracking (mlflow.org)
• tutorialMLflow Getting Started (mlflow.org)
• repoMLflow repository (github.com)

Related lessons

• Model registry, packaging and versioning
• Model evaluation: regression suites and release gates
• Distributed training across multiple GPU nodes
• Storage for ML: lakehouse, warehouse and operational data stores
• MLOps and the ML lifecycle: from experiment to production system