Practice: Kubernetes for ML workloads: containers, deployments and jobs

Generated by Claude through GigAI 0.1.11.10 on 2026-10-11.

Main tool: Kubernetes · Kubernetes is the one tool where the lesson's image, Deployment, Job and resource primitives can all be practised together on a free local cluster. · Alternatives: HashiCorp Nomad is a simpler single-binary scheduler with a much smaller ML ecosystem, Docker Compose or Docker Swarm run containers without a reconciling control plane that suits batch and GPU workloads, managed cloud ML platforms hide the cluster but cannot run on a laptop, and Slurm schedules batch training well but has no native model for long-running services.

Every step links into the tool's own documentation; links were checked on 2026-10-11 · version 1

Setup (one time)

1. Install a container engine, the kubectl command line tool and a local cluster tool such as minikube or kind, then start a single-node cluster.
Every later step needs a running cluster and a configured kubectl. A local cluster costs nothing and needs no GPU.
Explained in: List All Container Images Running in a Cluster | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: kubectl reports one node in Ready state and the client and server versions print without error.
2. Check which container runtime your local node uses by reading the wide node listing.
Images you build with Docker are run by containerd or another runtime inside the node. Knowing which one explains later image loading and pull behaviour.
Explained in: Find Out What Container Runtime is Used on a Node | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: The node listing shows a container runtime column with a runtime name and version.

Steps

1. Run the first Kubernetes Basics modules on your local cluster: create the cluster, deploy the sample app and explore it.
This is the shortest guided path from nothing to a running workload. It gives you the kubectl verbs used in every later step.
Explained in: Learn Kubernetes Basics | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: The sample app runs as a Pod and you can see its logs and description with kubectl.
2. Create a Deployment for a small CPU model server with three replicas, then delete one Pod and watch the cluster replace it.
This shows the cluster model and desired state in action: you declare a replica count and the controller keeps it true. Inference services depend on this behaviour.
Explained in: Deployments | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: The Deployment shows three of three replicas ready again within seconds of the Pod being deleted.
3. Run a rolling update of the model server Deployment to a new image tag, check the rollout history, then roll back to the previous revision.
Shipping a new model version and reverting a bad one is the daily Deployment workflow for model servers. Rollout history is the record you compare revisions against.
Explained in: Deployments | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: Rollout history lists at least two revisions and the running Pods use the earlier image again.
4. Write CPU and memory requests and limits into the model server container spec and apply the change.
Requests drive scheduling and limits cap usage. Without them, ML workloads on shared nodes starve each other.
Explained in: Resource Management for Pods and Containers | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: Describing the Pod shows the requests and limits you set, and describing the node shows them counted in allocated resources.
5. Run a training script as a run-to-completion Job with a deliberately low memory limit, watch it get killed, then raise the limit and run it again to completion.
Jobs are the shape for training and batch inference, and running out of memory is their most common failure. You need to recognise the termination reason quickly.
Explained in: Resource Management for Pods and Containers | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: The first Job Pod shows a terminated state with an out of memory reason and the second Job reports one successful completion.
6. Read how custom resources and custom controllers extend the built-in API, which is how training operators and batch queueing systems add their own job types.
The built-in Job covers single-container batch work. Distributed training and queued batch inference are modelled as custom resources reconciled by controllers, so you need this model before installing any of them.
Explained in: Custom Resources | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: You can explain in two sentences how a training job custom resource differs from a built-in Job and what its controller does.
7. Create a Pod that requests a GPU-style extended resource. A real GPU needs a GPU node and a vendor device plugin, so on a laptop advertise a fake node-level extended resource on your node and request that instead.
GPUs are exposed as extended resources that must be requested as whole integers in limits. A fake resource reproduces the same scheduling behaviour without hardware.
Explained in: Resource Management for Pods and Containers | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: A Pod requesting the available amount runs, and a Pod requesting more than the node advertises stays Pending with a FailedScheduling event.
8. Install kube-state-metrics in the local cluster and query its metrics to compare desired and available replicas for your Deployment and the status of your Jobs.
Object state metrics persist what the cluster believes about your workloads. They let you compare declared state against actual state over time rather than by eye.
Explained in: Metrics for Kubernetes Object States | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: A metrics query returns replica and Job status values that match what kubectl shows for the same objects.
9. Package your model server code into a container image with an explicit version tag, load it into the local cluster and set the image pull policy so the node uses the loaded image.
The image is the unit you ship for ML code. Pull policy decides whether a node uses a cached image or contacts a registry, a frequent source of stale or missing model code.
Explained in: Images | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: The Deployment runs your own image and the Pod events show the image was already present on the node, not pulled.
10. Start a local container registry with authentication, push your image to it, create a registry credential Secret and reference it from the Pod template.
Teams keep ML images in private registries. Registering the image and wiring pull credentials is the promote step every workload depends on.
Explained in: Images | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: A fresh Pod pulls the image from the local registry successfully, and removing the Secret reference makes a new Pod fail with an image pull error.
11. Reproduce the deployment from a clean cluster using manifests that pin the image by digest instead of a mutable tag, then list all images running in the cluster.
A digest fixes exactly which bytes run, so a training or serving run can be repeated later or in CI. Listing running images verifies nothing drifted.
Explained in: Images | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: After recreating the cluster and applying the manifests, the listed running image carries the same digest you pinned.
12. Configure a multi-node local cluster as a production-like setup: verify that dynamic resource allocation is enabled and create a DeviceClass. Real devices need GPU nodes and a vendor driver, so on a laptop stop at the DeviceClass or use an example driver.
Production ML clusters separate node pools and hand out accelerators through device classes and drivers. Verifying the API and creating the class is the part you can rehearse locally.
Explained in: Set Up DRA in a Cluster | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: The resource API group is listed by the cluster and your DeviceClass appears when you list device classes.

Back to Kubernetes for ML workloads: containers, deployments and jobs