This is part 2 of a series on building a homelab Kubernetes cluster. Part 1: One Box, Real DNS covers the first node. Part 3: Real DNS Names, Real Certificates covers moving off a bad domain and getting real Let's Encrypt certs for internal-only services.

The first node is mf1: 16 threads, no GPU, always on, running the control plane and everything else by itself for now. The second node is a gaming PC. Ryzen 9 5950X, 128GB RAM, an RTX 3080 Ti with 12GB VRAM sitting mostly idle. It's a much bigger machine than mf1 in every dimension that matters for AI workloads, and it dual-boots Windows, which means it isn't always there.

This is the writeup of adding it to the cluster anyway, on terms that account for that, and the model research detour that almost went sideways before I actually checked the source.

The tradeoff, stated plainly

Standalone Ollama on the gaming box would have been the lower-effort path: install it, run it, point things at its IP by hand. Cluster membership costs real setup, NVIDIA's container toolkit, a containerd runtime class, a device plugin DaemonSet, none of which standalone needs.

What it buys back: the DNS pipeline from the last post applies here for free. Anything I deploy on this node gets a *.k8s.myhomenetwork.local name the same way Grafana or podinfo would, no separate reverse proxy or manual DNS entry to babysit for one box. Scheduling and resource isolation come from the same mechanism everything else in the cluster already uses. And it's one coherent system instead of two, which matters more as more pieces get added later.

The part that actually needed solving before any of that was worth it: a node that gets powered off for gaming can't be allowed to behave like a normal cluster member when it vanishes.

Making disappearance safe

Two mechanisms handle this together, not one.

Resource requests do the heavy lifting. A pod that asks for nvidia.com/gpu: 1 can only be scheduled on a node advertising that resource. mf1 never will, since it has no GPU device plugin running. So GPU workloads structurally can't land on mf1 even by accident, not because of a rule someone has to remember, but because the scheduler has nowhere else to put them.

A taint stops the reverse. nvidia.com/gpu=present:NoSchedule on the GPU node keeps ordinary pods from parking there and competing for CPU and RAM while it's up for other reasons.

Deployments, not StatefulSets, for anything that runs here. When the node disappears mid-session, a Deployment's replacement pod just sits Pending until the node comes back, clean and self-healing. StatefulSets are more conservative about a vanished node and can get stuck needing a manual force-delete.

Drain before shutdown, uncordon after boot. Without this, powering the node off leaves pods sitting in Terminating for the default five-minute eviction window before anything gets cleaned up. kubectl cordon and kubectl drain --ignore-daemonsets right before shutting down make that instant instead. It's a manual two-command step for now, not automated with a shutdown hook yet. Worth doing if it turns out to be annoying to remember by hand.

The GPU stack

Node prep matched mf1's kubeadm setup (swap off, kernel modules, sysctls, containerd with CRI enabled), plus the GPU-specific pieces: NVIDIA Container Toolkit, then nvidia-ctk runtime configure --runtime=containerd --set-as-default to point containerd's default runtime at nvidia. kubeadm join from there was routine.

The NVIDIA device plugin DaemonSet is what makes nvidia.com/gpu visible to the scheduler at all. Its default manifest already tolerates a nvidia.com/gpu:NoSchedule taint, so no changes needed there, but it has no node selector by default, meaning it tries to run everywhere, including mf1, where it just crash-loops looking for a GPU that isn't there. A nodeSelector for a nvidia.com/gpu.present: "true" label fixed that, scoping it to the one node that actually has a card.

Proof it worked end to end: a pod requesting nvidia.com/gpu: 1, with a matching toleration, scheduled onto the GPU node and ran nvidia-smi inside the container. It saw the RTX 3080 Ti correctly, driver version and all.

Choosing a serving engine, and nearly trusting the wrong search result

Ollama versus vLLM came down to a single practical fact about this specific node: vLLM reserves a chunk of GPU memory up front and expects to hold it for as long as it's running, no automatic idle release. Ollama unloads a model after a configurable idle period, freeing the card for other things. On a node that's supposed to also be available for gaming, that difference decides it. vLLM's real advantages, continuous batching, prefix caching for repeated large contexts, matter most at higher concurrency than one household generates. Ollama it was.

Picking the model is where it got interesting. A blog post claimed a "DeepSeek-Coder V3 (Distilled)" as the strongest option for a 12GB card, a 16B dense model with V3.2's reasoning traces baked in. It sounded exactly right and I nearly built the whole deployment around that name.

Before pulling anything, I checked Ollama's actual library instead. That model doesn't have a downloadable tag anywhere. It's likely SEO-aggregator content describing something that was never packaged for local use, or describing weights that only exist on Hugging Face in a form nothing local can load yet. The same check on the other candidate, Qwen3-Coder, turned up a real problem too: its smallest tag is 30B at a 19GB download, which doesn't fit in 12GB VRAM and would spill into system RAM, working, but slower than a model sized for the card.

deepseek-coder-v2:16b was the one that actually held up: a real, pullable tag, a 16B MoE model with only 2.4B active parameters, an 8.9GB download that fits the 3080 Ti with room to spare. The lesson generalizes past this one model: a roundup blog confidently naming a model and its specs is not the same as that model existing in a form you can actually run. Check the registry, not the summary of the registry.

DeepSeek V4 Flash, while I was looking

Since the model search was already turning up 2026-era releases past what I'd normally know offhand, I looked into DeepSeek V4 Flash too, since it kept coming up as "the approachable one" next to V4 Pro. Worth writing down what "approachable" actually means at this scale: 284B total parameters, 13B active, a genuinely capable model. Even the most aggressive quantization anyone's gotten working needs around 81GB, and the realistic homelab entry point people are citing is 96GB of VRAM or 128GB of unified memory. Neither node here is in range, and it wouldn't matter if they were, since stable Ollama and llama.cpp don't support the V4 architecture yet, only experimental forks do. Good to know where the ceiling actually is before assuming anything called "Flash" is automatically the small one.

The deployment

The manifest ended up simple, once the model choice was settled: a Deployment with a nodeSelector for the GPU label, a toleration for the taint, resources.limits: nvidia.com/gpu: 1, and a hostPath volume so pulled models survive a pod restart instead of re-downloading every time. OLLAMA_KEEP_ALIVE set to 30 minutes, so an idle model releases VRAM back to the card instead of holding it hostage indefinitely.

The Service is a plain LoadBalancer with the same external-dns hostname annotation every other service in this cluster uses. No new DNS plumbing needed at all, the pipeline from the last post just picked it up: ollama.k8s.myhomenetwork.local resolved and worked on the first try.

Pulled the model into the running pod, then tested it for real over the LAN name, not the pod IP:

curl http://ollama.k8s.myhomenetwork.local/api/generate -d '{
  "model": "deepseek-coder-v2:16b",
  "prompt": "Write a one-line Python function that returns the factorial of n.",
  "stream": false
}'

Got a correct, working one-liner back.

Using it

Ollama speaks its own API and an OpenAI-compatible one at /v1/chat/completions, which matters more than it sounds like it should: anything already built against OpenAI's API works here by changing one base URL. Continue.dev and Aider both point at http://ollama.k8s.myhomenetwork.local/v1 and just work, no API key needed since there's nothing to authenticate against on a home LAN.

For a quick interactive session, kubectl exec -it into the pod runs the normal Ollama chat loop, though it took a second try to get right: kubectl exec -it needs a real TTY, and ssh won't allocate one for a remote command unless told to with -t. Without it, the outer SSH session runs the command in a pipe, and ollama run's TTY check fails immediately. The working version:

ssh -t mf1 "kubectl -n ai exec -it deploy/ollama -- ollama run deepseek-coder-v2:16b"

What's next

The GPU node is intermittent by design, and everything above was built around that instead of around pretending it isn't. More models can go on the same box later, bounded by however much fits in 12GB at once, since Ollama can hold several loaded simultaneously if the VRAM budget allows it. If this cluster ever needs more than one GPU-backed service running at the same time, the device plugin supports time-slicing the same physical card across multiple pods, cooperative sharing with no memory isolation between them, not set up yet, not needed yet either.

Next: Part 3: Real DNS Names, Real Certificates covers what happened when it turned out the domain both of these posts pointed at had a bad reputation, and the actual fix.