I’ve had a few attempts before at running homelab services but manual administration and lack of reproducibility always meant that I ended up not really trusting the services and then using them less. I decided to tackle a homelab again recently, taking a more disciplined approach — reproducible builds, no code shipped without going through CI and code review.

This post covers how I use Flux CD and k3s to manage my home lab with GitOps. For the HTTPS/certificate side of things, see Automated Wildcard HTTPS Behind NAT with Let’s Encrypt.

The full configuration is public — you can browse every manifest referenced here in stjohnb/homelab.

The end result

Every service I run is declared in a single apps/kustomization.yaml in my fleet-infra repo:

yaml
resources:
  - home-assistant
  - plex
  - immich
  - grafana
  - prometheus

Adding a new service is adding a directory with manifests and a line in this file. Open a PR, wait for green CI and a clean review, then merge to main, and Flux deploys it within a minute.

Architecture overview

graph TD
    repo["GitHub Repo<br>(fleet-infra)"] -->|"git pull (1 min)"| flux

    subgraph cluster["k3s Cluster (192.168.0.251)"]
        flux["Flux CD"] --> infra["Infrastructure<br>(Traefik, cert-manager)"]
        flux --> config["Config<br>(certificates, issuers)"]
        flux --> apps["Apps<br>(apps/)"]
        apps --> pods["immich · plex · HA · ..."]
    end

One repo, layered reconciliation

Everything lives in a single repository, fleet-infra, split into two directories:

  • clusters/my-cluster/ — Platform infrastructure: Flux itself, Traefik, cert-manager, external-secrets, certificates, ClusterIssuers, RBAC. Changes rarely.
  • apps/ — Application services: deployments, services, ingresses for each app. Changes often.

Flux reconciles the directories as layered Kustomizations within the one repo — the platform layer applies first, the app layer last. Because the app layer dependsOn the platform layer, Flux never tries to create an Ingress before Traefik exists. The order is explicit, not accidental.

The dependency chain

Flux uses Kustomization resources to define what to deploy and in what order. Four Kustomizations form a dependency graph:

clusters/my-cluster/infrastructure-kustomization.yaml — Deploys platform components:

yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: infrastructure
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./clusters/my-cluster/infrastructure
  prune: true
  wait: true

clusters/my-cluster/config-kustomization.yaml — Deploys configuration that depends on infrastructure:

yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: config
  namespace: flux-system
spec:
  dependsOn:
    - name: infrastructure
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./clusters/my-cluster/config
  prune: true

clusters/my-cluster/migrations-kustomization.yaml — Runs one-off setup jobs (secret generation and the like) that also depend on infrastructure, alongside config:

yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: migrations
  namespace: flux-system
spec:
  dependsOn:
    - name: infrastructure
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./clusters/my-cluster/migrations
  prune: true

clusters/my-cluster/apps-kustomization.yaml — Deploys the application services once both config and migrations are ready:

yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  dependsOn:
    - name: config
    - name: migrations
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps
  prune: true

The ordering matters: infrastructure installs Traefik and cert-manager first, then config and migrations run in parallel — config creates certificates and issuers while migrations seed secrets — and only once both finish does Flux deploy your apps. Without dependsOn, Flux might try to create an Ingress before Traefik exists.

All four layers read from the same flux-system GitRepository source that flux bootstrap created — there’s no second repository. The apps Kustomization just points at path: ./apps in that one source. A root clusters/my-cluster/kustomization.yaml explicitly lists these four Kustomizations rather than letting Flux auto-discover the directory; that safeguard stops Flux from picking up the subdirectories on its own and applying them all at once, which would bypass the dependsOn ordering entirely.

k3s setup

k3s is installed without the built-in Traefik — Traefik is managed via Flux and git instead:

bash
curl -sfL https://get.k3s.io | sh -s - --disable traefik

Bootstrapping Flux

bash
flux bootstrap github \
  --owner=stjohnb \
  --repository=fleet-infra \
  --branch=main \
  --path=clusters/my-cluster \
  --personal

This creates a deploy key, installs Flux controllers, and sets up the GitOps sync for fleet-infra. Flux watches the whole tree from the flux-system source it just created.

Deploying services: two patterns

In-cluster services

Most services run as pods in the cluster. Here’s immich as an example — you need three manifests plus a kustomization:

immich/deployment.yaml — Pod specification (image, ports, volumes, etc.)

immich/service.yaml — ClusterIP service:

yaml
apiVersion: v1
kind: Service
metadata:
  name: immich
spec:
  selector:
    app: immich
  ports:
    - port: 2283
      targetPort: 2283

immich/ingress.yaml — Makes it accessible via HTTPS:

yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: immich
spec:
  ingressClassName: traefik-traefik
  tls:
    - hosts:
        - immich.home.bstjohn.net
      secretName: wildcard-home-tls
  rules:
    - host: immich.home.bstjohn.net
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: immich
                port:
                  number: 2283

immich/kustomization.yaml:

yaml
resources:
  - deployment.yaml
  - service.yaml
  - ingress.yaml

The ingress references a wildcard TLS certificate — see the HTTPS post for how that’s set up.

External service proxy

Some services run on other machines but still benefit from the cluster’s ingress and certificates. Home Assistant runs on a separate server at 192.168.0.73:8123. To proxy through the cluster, use a Service without selectors and explicit Endpoints:

home-assistant/service.yaml:

yaml
apiVersion: v1
kind: Service
metadata:
  name: home-assistant
spec:
  ports:
    - port: 8123
      targetPort: 8123

home-assistant/endpoints.yaml:

yaml
apiVersion: v1
kind: Endpoints
metadata:
  name: home-assistant
subsets:
  - addresses:
      - ip: 192.168.0.73
    ports:
      - port: 8123

home-assistant/ingress.yaml:

yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: home-assistant
spec:
  ingressClassName: traefik-traefik
  tls:
    - hosts:
        - home-assistant.home.bstjohn.net
      secretName: wildcard-home-tls
  rules:
    - host: home-assistant.home.bstjohn.net
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: home-assistant
                port:
                  number: 8123

Now https://home-assistant.home.bstjohn.net terminates SSL at Traefik and proxies to the external server. The service running on the other machine doesn’t need to know anything about certificates.

Adding a new service

The workflow is simple:

bash
cd fleet-infra/apps

# Create service directory with manifests
mkdir my-service
# ... create deployment.yaml, service.yaml, ingress.yaml, kustomization.yaml

# Add to the apps kustomization
echo "  - my-service" >> kustomization.yaml

# Deploy
git add . && git commit -m "Add my-service" && git push

# Flux deploys automatically within ~1 minute

Disaster recovery

This is the real selling point of GitOps for a home lab. If the cluster dies, rebuilding is straightforward:

  1. Install k3s with --disable traefik
  2. Bootstrap Flux pointing at fleet-infra
  3. Recreate the handful of secrets that aren’t stored in Git (API keys, deploy keys)

Flux reads the repo and rebuilds everything. The cluster converges to the declared state. All your services, their configuration, the infrastructure — it’s all in Git. No danger of forgetting what’s deployed or how it’s configured.

For the HTTPS and certificate management side of this setup, see Automated Wildcard HTTPS Behind NAT with Let’s Encrypt.