Homelab Kubernetes, Part 4: Self-Sufficiency, Not Self-Signed


A glowing padlock-and-shield connected to a NAS storage tower, with a firework bursting overhead in the same soft glow-dot style as the rest of the series

This is part 4 of a series on building a homelab Kubernetes cluster. Part 1: One Box, Real DNS covers the first node. Part 2: A GPU Node That Comes and Goes covers the second. Part 3: Real DNS Names, Real Certificates covers the domain and TLS work this post builds on.

Fitting enough that this landed on the 4th. Part 3 already got the whole cluster off self-signed certificates for good, every internal hostname under one real, browser-trusted wildcard. Independence, this time, is a different and narrower kind: not depending on a public registry to store images this homelab builds itself, hosted on a machine that was never part of the cluster and was never going to be. The standard set in Part 3 still applies here too, though, no self-signed fallback, no insecure-registry workaround, just because the hardware sits outside the cluster's boundary. This is the writeup of getting a Synology NAS a real certificate for exactly that, without handing it anything close to standing access, and the two bugs that stood between "imported the cert" and "the cert is actually what's being served."

The boundary decided before any commands ran

Every other piece of automation in this series (cert-manager, external-dns, the whole DNS chain) runs inside the cluster and gets to act on its own, because the cluster is the thing being built and it's reasonable for it to manage itself. The NAS is different on purpose. It holds file shares, personal data, and now a registry, and the working rule became: no automated tool, container or otherwise, gets standing DSM admin credentials. Not a cron job, not a webhook, not a background daemon quietly renewing certificates on a schedule.

That ruled out the obvious path pretty quickly. acme.sh's Synology deploy hook is a real, maintained, well-documented way to solve exactly this, issue a cert and push it into DSM automatically. It also needs a DSM account with certificate-management rights sitting in a container's environment variables, indefinitely, for as long as that container runs. Looked into whether DSM has some narrower delegated role for just certificate management, short of full admin, the way it does for a handful of other operations. It doesn't, as far as I could confirm, certificate management is gated the same way Container Manager access is: administrators group or nothing.

What actually got built instead

The cluster's wildcard certificate already covers any hostname under *.k8s.abhijeetapsunde.com. The NAS's registry just needed a hostname under that same domain, registry.k8s.abhijeetapsunde.com, and it would already be covered without issuing anything new at all. No second ACME account, no Cloudflare token handed to a second system, no separate renewal clock that could drift out of sync with the cluster's.

That leaves one piece: getting the cert out of the cluster and onto the NAS, without automating the last step. A small script pulls the current cert and key out of the Kubernetes Secret cert-manager manages, writes them to disk, and stops there. A person uploads them to DSM by hand. Renewal, whenever cert-manager rotates it (every 60 days or so, 90-day Let's Encrypt certs renewed with a 30-day buffer), means running the script again and re-uploading, a couple of minutes, six times a year. That's the actual cost of the boundary, and it's a small one, especially against the alternative of a container holding admin credentials indefinitely.

Bug one: a cert that looks fine in a browser and fails everywhere else

First upload, first test, and curl refused the connection: unable to get local issuer certificate. Chrome, checked separately, showed the cert as completely fine, real Let's Encrypt issuer, correct hostname, green padlock. Two clients disagreeing about the same connection like that is a specific, recognizable signature: an incomplete certificate chain. Browsers routinely fetch a missing intermediate certificate on their own if a server doesn't send one, quietly, via the AIA extension embedded in the leaf cert. curl doesn't. Neither does containerd, which was the client that actually mattered here.

The cause was upstream of any of that. DSM's certificate import form has three separate fields, Certificate, Private Key, and Intermediate Certificate, and the extraction script had been writing one combined file with all three PEM blocks concatenated together. Uploaded into a single field, only the first block, the leaf certificate, actually took. The intermediate silently never made it in. Splitting the script's output into three separate files, one per field, fixed it outright: curl came back clean, SSL certificate verify ok, no flags needed.

Bug two: imported doesn't mean assigned

Even with a complete chain, the very first attempt to reach the registry over HTTPS still showed a synology-issued self-signed certificate, not the Let's Encrypt one that had just been uploaded. Importing a certificate into DSM and having a specific service actually use it are two separate steps. There's a per-service certificate mapping (Control Panel → Security → Certificate → Settings), and everything on the NAS, including the reverse proxy handling this exact hostname, was still pointed at System Default, still set to the built-in self-signed cert. Reassigning that one dropdown was the whole fix.

What was actually running underneath

The registry itself turned out to already exist, a Container Manager project from some earlier, forgotten session, sitting there doing nothing. Wiring DSM's reverse proxy (a newer wizard-driven "Portal" flow in current DSM, same underlying idea as the classic Reverse Proxy page) to that project's published port surfaced one more error before everything lined up: a 502, DSM's own generic failure page, which turned out to mean the hostname routing itself was fine but the port the portal forwarded to didn't match the port the container actually published on. Lining those up was the last piece.

TLS terminates at DSM's reverse proxy, not inside the registry container itself, deliberately. The container just serves plain HTTP on an internal port and never sees a certificate file. That matters for the same reason the CoreDNS Corefile didn't reload itself back in Part 3: a container terminating its own TLS would need a restart on every renewal to pick up new cert files. DSM's reverse proxy just starts using whatever's assigned the moment it's reassigned, no container restart, one less moving part to get wrong twice.

Proof, not assumption

https://registry.k8s.abhijeetapsunde.com/v2/ returns {}, the Docker Registry API's specific healthy-empty response, with a clean certificate chain behind it. From mf1: pulled hello-world, tagged it for the new registry, pushed it, removed the local copies, pulled it back down. Every step worked with zero insecure-registry configuration anywhere, no --insecure-registry flag, no manually trusted CA cert dropped onto a node, because the certificate is genuinely, properly trusted the same way any public site's certificate is.

What four parts actually added up to

One node became two. A made-up domain that turned out to have a real, bad reputation became one actually owned. The cluster stopped depending on self-signed certificates anywhere inside it, and now there's somewhere to keep custom-built images that isn't a public repository either, on hardware the cluster doesn't control and never will, without lowering the same standard just because the boundary moved. Whatever gets built on top of this next inherits all of it for free.