Gitfed
bastien-mrq/gitfed / deploy / k8s / README.md
README.md Code Preview

Deploying gitfed alongside ess-helm (k3s + Traefik + cert-manager)

This assumes the same single-node k3s setup ess-helm's own install guide sets up: k3s with its bundled Traefik ingress controller, and cert-manager already installed with a ClusterIssuer for Let's Encrypt. gitfed reuses both rather than standing up anything new.

Why this shape

  • One pod, two containers, one PVC. gitfed-server owns the bbolt store (single-writer, exclusive file lock) and a Unix admin socket at /data/admin.sock. gitfed-web talks to it over that socket instead of opening the database itself — see internal/opsconnect. That only works if they share a filesystem with the server, hence one pod.
  • replicas: 1, strategy: Recreate, forever. bbolt, the admin socket, and the SSH hostPort are all single-instance by construction — this isn't a temporary limitation, don't try to scale it out.
  • gitfed-web is a real multi-user app now, not an admin-only tool: public repo browsing needs no login; self-service (your own keys, repos, collaborators) and the admin section both sit behind a real username/password login (session cookies, bcrypt-hashed passwords). That's why, unlike an earlier iteration of this deployment, it has both a Service and an Ingress — it's meant to be reachable from the internet.
  • Password login is web-only. It never grants git access — git clone/push always goes over SSH with a key/certificate, completely independent of this login system.
  • Port 22 is the VPS's own sshd — gitfed's git+ssh listens on 2222 instead, exposed via hostPort since this is a single-node cluster (no separate load balancer needed).

1. Build and import the image

The image tag you build must match the tag already written in deployment.yaml, not :latest — check it first:

grep 'image:' deploy/k8s/deployment.yaml   # e.g. "image: gitfed:1.2.4"
cat VERSION                                # should match

deployment.yaml is checked into git with whatever version was live on the original deploy — deploy/update.sh moves that pin forward on every release (see below), it's never :latest. Build and tag with VERSION, not a hardcoded tag, so this can't drift:

Build on the VPS itself, not your workstation — docker build targets whatever architecture it runs on, and a workstation that isn't x86_64 (an Apple Silicon Mac, say) would produce an image the VPS can't run. This is exactly what deploy/update.sh already does under the hood (rsync the source over, then docker build via SSH on the VPS), so doing it by hand the same way for the first deploy keeps one mental model instead of two:

# from a checkout on the VPS (rsync/clone it there first)
docker build -f deploy/docker/Dockerfile -t gitfed:$(cat VERSION) .
docker save gitfed:$(cat VERSION) | sudo k3s ctr images import -

imagePullPolicy: Never in deployment.yaml means k3s will never try to pull this from a registry — it only ever uses what you've imported under that exact tag. Get the tag wrong (e.g. build :latest while deployment.yaml still says :1.2.4) and the pod sits in ErrImageNeverPull forever, since there's nowhere else k3s will look — fix it by importing an image under the tag deployment.yaml actually asks for, not by editing the tag in deployment.yaml unless you mean to.

Whenever you rebuild, redo this import and then kubectl rollout restart deployment/gitfed -n gitfed — importing a new image under the same tag does not restart pods that are already running.

For every update after the first, use deploy/update.sh instead of the steps above — it does the same build/import/rollout, but tags the image with the actual version (gitfed:0.3.1, not just :latest) so kubectl rollout history means something, bumps VERSION, and commits. Add the new version's ## X.Y.Z section to CHANGELOG.md first, then:

deploy/update.sh          # patch bump
deploy/update.sh minor    # or minor/major/an explicit X.Y.Z

2. DNS

Add one more A/AAAA record to the same zone ess-helm's subdomains live in:

git.example.com  ->  <VPS public IP>

Same IP as your matrix./chat./etc. records — Traefik will route by hostname for HTTP(S), and the SSH hostPort listens directly on the node's network interface regardless of Traefik.

3. Check the placeholders

configmap.yaml and ingress.yaml are filled in already for git.neuromancer.ovh / letsencrypt-prod — double check those still match before applying if anything changed since.

4. Apply

kubectl apply -f deploy/k8s/namespace.yaml
kubectl apply -f deploy/k8s/configmap.yaml
kubectl apply -f deploy/k8s/pvc.yaml
kubectl apply -f deploy/k8s/deployment.yaml
kubectl apply -f deploy/k8s/service.yaml
kubectl apply -f deploy/k8s/ingress.yaml

Watch it come up:

kubectl -n gitfed get pods -w

Both containers must reach Running/Ready — web waits in a loop for server's admin socket before starting, so a brief 0/2 is normal on first boot.

5. Verify

curl https://git.neuromancer.ovh/.well-known/gitfed.json
curl https://git.neuromancer.ovh/          # gitfed-web homepage

6. Bootstrap the first (admin) account

There's no account yet, and there's no self-registration by design. Create the first one directly against the live admin socket:

kubectl -n gitfed exec -it deployment/gitfed -c server -- \
  gitfed-tui -config /etc/gitfed/gitfed.json

Go to Users → a (add user), fill in username, SSH public key, a password, and answer y to the admin prompt. Then log in at https://git.neuromancer.ovh/login with that username/password — you'll land on the dashboard with an Admin link in the nav.

From here on, that account (or any other admin account) can create further users from Admin → Users in the web UI itself — gitfed-tui is only needed for this one-time bootstrap, or later if the web UI is ever unreachable.

Then clone for real, over SSH (not the web login):

git clone ssh://git@git.neuromancer.ovh:2222/<user>/<repo>

Backups

Everything that matters lives on the gitfed-data PVC: gitfed.db (users, password hashes, sessions, repos/ACLs, trust store, audit log), ca/ (the instance's signing key — losing this invalidates every certificate and breaks every federated trust relationship pointing at this domain, there's no recovery short of everyone re-establishing trust), host_key, and repos/ (the actual bare git repos). Back up the whole PVC, not just the git data.

From your workstation:

deploy/backup.sh              # snapshot now, prune to the 14 most recent
deploy/backup.sh --keep 30    # keep more (or --keep 0 to keep everything)

This streams a tarball straight off the running container over SSH into ./backups/ (gitignored) — it doesn't touch the VPS's own disk, so a lost VPS doesn't take the backups with it. It's a live copy taken while gitfed-server keeps running, not a transactional snapshot; see the script's header comment for why that's an acceptable trade here.

There's nothing scheduling this for you — it's a plain script, so wire it into whatever you already use for recurring jobs (a cron entry or a launchd/systemd user timer running deploy/backup.sh from a checkout of this repo works fine).

Restoring from a backup

  1. Stop gitfed so nothing is writing to /data while you overwrite it:

    kubectl -n gitfed scale deployment/gitfed --replicas=0

  2. Start the throwaway restore pod (same PVC, no gitfed code running):

    kubectl apply -f deploy/k8s/restore-pod.yaml
kubectl -n gitfed wait --for=condition=Ready pod/gitfed-restore

  3. Copy the backup in and unpack it, replacing whatever's currently on the PVC:

    kubectl -n gitfed cp backups/gitfed-<timestamp>.tar.gz gitfed-restore:/tmp/backup.tar.gz
kubectl -n gitfed exec gitfed-restore -- sh -c \
  'find /data -mindepth 1 -delete && tar xzf /tmp/backup.tar.gz -C /data'

  4. Clean up and start gitfed again:

    kubectl -n gitfed delete pod gitfed-restore
kubectl -n gitfed scale deployment/gitfed --replicas=1

Further hardening (not included here, worth doing later)

  • deploy/k8s/networkpolicy.yaml exists but isn't part of the "Apply" step above, and k3s' bundled flannel CNI doesn't enforce NetworkPolicy at all. Applying it as-is is documentation of intent, not an active control, until the cluster runs a policy-capable CNI (Cilium, Calico, ...) — worth doing if this cluster ever hosts anything you don't fully trust alongside gitfed.
  • Rotating cert_ttl_hours down and setting up gitfed-renew-cert on a timer for any users who script access, instead of the default 48h/manual gitfed-cert request.

Login already has rate limiting (per-account and per-IP, see cmd/gitfed-web/ratelimit.go) — it used to be listed here as missing; it isn't anymore.