Gitfed
bastien-mrq/gitfed / deploy / k8s / README.md
README.md Code Preview
# Deploying gitfed alongside ess-helm (k3s + Traefik + cert-manager)

This assumes the same single-node k3s setup ess-helm's own install guide
sets up: k3s with its bundled Traefik ingress controller, and cert-manager
already installed with a `ClusterIssuer` for Let's Encrypt. gitfed reuses
both rather than standing up anything new.

## Why this shape

- **One pod, two containers, one PVC.** `gitfed-server` owns the bbolt
  store (single-writer, exclusive file lock) and a Unix admin socket at
  `/data/admin.sock`. `gitfed-web` talks to it over that socket instead of
  opening the database itself — see `internal/opsconnect`. That only works
  if they share a filesystem with the server, hence one pod.
- **`replicas: 1`, `strategy: Recreate`, forever.** bbolt, the admin socket,
  and the SSH `hostPort` are all single-instance by construction — this
  isn't a temporary limitation, don't try to scale it out.
- **`gitfed-web` is a real multi-user app now**, not an admin-only tool:
  public repo browsing needs no login; self-service (your own keys, repos,
  collaborators) and the admin section both sit behind a real
  username/password login (session cookies, bcrypt-hashed passwords).
  That's why, unlike an earlier iteration of this deployment, it has both a
  Service and an Ingress — it's meant to be reachable from the internet.
- **Password login is web-only.** It never grants git access — `git
  clone`/`push` always goes over SSH with a key/certificate, completely
  independent of this login system.
- **Port 22 is the VPS's own sshd** — gitfed's git+ssh listens on 2222
  instead, exposed via `hostPort` since this is a single-node cluster (no
  separate load balancer needed).

## 1. Build and import the image

**The image tag you build must match the tag already written in
`deployment.yaml`**, not `:latest` — check it first:

```sh
grep 'image:' deploy/k8s/deployment.yaml   # e.g. "image: gitfed:1.2.4"
cat VERSION                                # should match
```

`deployment.yaml` is checked into git with whatever version was live on the
original deploy — `deploy/update.sh` moves that pin forward on every
release (see below), it's never `:latest`. Build and tag with `VERSION`,
not a hardcoded tag, so this can't drift:

**Build on the VPS itself, not your workstation** — `docker build` targets
whatever architecture it runs on, and a workstation that isn't x86_64 (an
Apple Silicon Mac, say) would produce an image the VPS can't run. This is
exactly what `deploy/update.sh` already does under the hood (`rsync` the
source over, then `docker build` via SSH on the VPS), so doing it by hand
the same way for the first deploy keeps one mental model instead of two:

```sh
# from a checkout on the VPS (rsync/clone it there first)
docker build -f deploy/docker/Dockerfile -t gitfed:$(cat VERSION) .
docker save gitfed:$(cat VERSION) | sudo k3s ctr images import -
```

`imagePullPolicy: Never` in `deployment.yaml` means k3s will never try to
pull this from a registry — it only ever uses what you've imported **under
that exact tag**. Get the tag wrong (e.g. build `:latest` while
`deployment.yaml` still says `:1.2.4`) and the pod sits in
`ErrImageNeverPull` forever, since there's nowhere else k3s will look —
fix it by importing an image under the tag `deployment.yaml` actually asks
for, not by editing the tag in `deployment.yaml` unless you mean to.

**Whenever you rebuild, redo this import and then**
`kubectl rollout restart deployment/gitfed -n gitfed` — importing a new
image under the same tag does not restart pods that are already running.

**For every update after the first**, use `deploy/update.sh` instead of the
steps above — it does the same build/import/rollout, but tags the image
with the actual version (`gitfed:0.3.1`, not just `:latest`) so
`kubectl rollout history` means something, bumps `VERSION`, and commits.
Add the new version's `## X.Y.Z` section to `CHANGELOG.md` first, then:

```sh
deploy/update.sh          # patch bump
deploy/update.sh minor    # or minor/major/an explicit X.Y.Z
```

## 2. DNS

Add one more A/AAAA record to the same zone ess-helm's subdomains live in:

```
git.example.com  ->  <VPS public IP>
```

Same IP as your `matrix.`/`chat.`/etc. records — Traefik will route by
hostname for HTTP(S), and the SSH `hostPort` listens directly on the node's
network interface regardless of Traefik.

## 3. Check the placeholders

`configmap.yaml` and `ingress.yaml` are filled in already for
`git.neuromancer.ovh` / `letsencrypt-prod` — double check those still match
before applying if anything changed since.

## 4. Apply

```sh
kubectl apply -f deploy/k8s/namespace.yaml
kubectl apply -f deploy/k8s/configmap.yaml
kubectl apply -f deploy/k8s/pvc.yaml
kubectl apply -f deploy/k8s/deployment.yaml
kubectl apply -f deploy/k8s/service.yaml
kubectl apply -f deploy/k8s/ingress.yaml
```

Watch it come up:

```sh
kubectl -n gitfed get pods -w
```

Both containers must reach `Running`/`Ready` — `web` waits in a loop for
`server`'s admin socket before starting, so a brief `0/2` is normal on
first boot.

## 5. Verify

```sh
curl https://git.neuromancer.ovh/.well-known/gitfed.json
curl https://git.neuromancer.ovh/          # gitfed-web homepage
```

## 6. Bootstrap the first (admin) account

There's no account yet, and there's no self-registration by design. Create
the first one directly against the live admin socket:

```sh
kubectl -n gitfed exec -it deployment/gitfed -c server -- \
  gitfed-tui -config /etc/gitfed/gitfed.json
```

Go to **Users → a** (add user), fill in username, SSH public key, a
password, and answer `y` to the admin prompt. Then log in at
`https://git.neuromancer.ovh/login` with that username/password — you'll
land on the dashboard with an **Admin** link in the nav.

From here on, that account (or any other admin account) can create further
users from **Admin → Users** in the web UI itself — `gitfed-tui` is only
needed for this one-time bootstrap, or later if the web UI is ever
unreachable.

Then clone for real, over SSH (not the web login):

```sh
git clone ssh://git@git.neuromancer.ovh:2222/<user>/<repo>
```

## Backups

Everything that matters lives on the `gitfed-data` PVC:
`gitfed.db` (users, password hashes, sessions, repos/ACLs, trust store,
audit log), `ca/` (the instance's signing key — **losing this invalidates
every certificate and breaks every federated trust relationship pointing
at this domain**, there's no recovery short of everyone re-establishing
trust), `host_key`, and `repos/` (the actual bare git repos). Back up the
whole PVC, not just the git data.

From your workstation:

```sh
deploy/backup.sh              # snapshot now, prune to the 14 most recent
deploy/backup.sh --keep 30    # keep more (or --keep 0 to keep everything)
```

This streams a tarball straight off the running container over SSH into
`./backups/` (gitignored) — it doesn't touch the VPS's own disk, so a lost
VPS doesn't take the backups with it. It's a live copy taken while
`gitfed-server` keeps running, not a transactional snapshot; see the
script's header comment for why that's an acceptable trade here.

There's nothing scheduling this for you — it's a plain script, so wire it
into whatever you already use for recurring jobs (a cron entry or a
`launchd`/systemd user timer running `deploy/backup.sh` from a checkout of
this repo works fine).

### Restoring from a backup

1. Stop gitfed so nothing is writing to `/data` while you overwrite it:

   ```sh
   kubectl -n gitfed scale deployment/gitfed --replicas=0
   ```

2. Start the throwaway restore pod (same PVC, no gitfed code running):

   ```sh
   kubectl apply -f deploy/k8s/restore-pod.yaml
   kubectl -n gitfed wait --for=condition=Ready pod/gitfed-restore
   ```

3. Copy the backup in and unpack it, replacing whatever's currently on the
   PVC:

   ```sh
   kubectl -n gitfed cp backups/gitfed-<timestamp>.tar.gz gitfed-restore:/tmp/backup.tar.gz
   kubectl -n gitfed exec gitfed-restore -- sh -c \
     'find /data -mindepth 1 -delete && tar xzf /tmp/backup.tar.gz -C /data'
   ```

4. Clean up and start gitfed again:

   ```sh
   kubectl -n gitfed delete pod gitfed-restore
   kubectl -n gitfed scale deployment/gitfed --replicas=1
   ```

## Further hardening (not included here, worth doing later)

- **`deploy/k8s/networkpolicy.yaml` exists but isn't part of the "Apply"
  step above, and k3s' bundled flannel CNI doesn't enforce `NetworkPolicy`
  at all.** Applying it as-is is documentation of intent, not an active
  control, until the cluster runs a policy-capable CNI (Cilium, Calico,
  ...) — worth doing if this cluster ever hosts anything you don't fully
  trust alongside gitfed.
- Rotating `cert_ttl_hours` down and setting up `gitfed-renew-cert` on a
  timer for any users who script access, instead of the default 48h/manual
  `gitfed-cert` request.

Login already has rate limiting (per-account and per-IP, see
`cmd/gitfed-web/ratelimit.go`) — it used to be listed here as missing;
it isn't anymore.