Gitfed
bastien-mrq/gitfed/ Commits/ 5e69f28

Add deploy/backup.sh and a documented restore procedure

Streams a tar of /data straight off the running server container over SSH into ./backups/ (gitignored), with pruning. The only previous backup guidance was "copy the whole PVC by hand" — given that losing the CA key breaks every federated trust relationship irrecoverably, V1 needs an actual tested procedure, not just a warning in the docs. Restore uses a throwaway pod (deploy/k8s/restore-pod.yaml) mounting the same PVC while gitfed is scaled to zero, documented step by step in deploy/k8s/README.md.

bastien-mrq 2026-07-28 21:57 commit 5e69f2856cad8b228be83d073bc8ccb1aaca38a8 parent 5ae01b4489db590eda034c7168f67222d3b7d630
5 files changed +136 −0
M .gitignore +1 −0
M CHANGELOG.md +4 −0
A deploy/backup.sh +57 −0
M deploy/k8s/README.md +49 −0
A deploy/k8s/restore-pod.yaml +25 −0
.gitignore
diff --git a/.gitignore b/.gitignore index 12fa2eb..49996a9 100644 --- a/.gitignore +++ b/.gitignore @@ -2,4 +2,5 @@ /bin/ /data/ /demo/ +/backups/ *.db
CHANGELOG.md
diff --git a/CHANGELOG.md b/CHANGELOG.md index 90dd729..7755561 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,9 @@ # Changelog +## 0.9.8 + +- Added `deploy/backup.sh`: snapshots the whole `gitfed-data` PVC (bbolt store, CA key, host key, bare repos) to a timestamped tarball downloaded off the VPS, with automatic pruning. Previously the only documented backup procedure was "copy the whole PVC by hand." Restore procedure and a throwaway `deploy/k8s/restore-pod.yaml` documented in `deploy/k8s/README.md`. + ## 0.9.7 - Capitalized "Gitfed" as a proper noun throughout the nav and UI copy (was lowercase "gitfed" everywhere). Also fixed the page `<title>` showing the brand name twice on the landing and security pages.
deploy/backup.sh
diff --git a/deploy/backup.sh b/deploy/backup.sh new file mode 100755 index 0000000..159bf0a --- /dev/null +++ b/deploy/backup.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +# Snapshots everything gitfed can't function without — the bbolt store, +# the instance's CA key, the SSH host key, and every bare repo, all under +# /data on the gitfed-data PVC (see deploy/k8s/README.md "Backups") — to a +# timestamped tarball downloaded into ./backups/ on this machine. Keeping +# the only copy on the VPS defeats the point: this is meant to survive +# losing the VPS entirely. +# +# Streams the tar straight off the running container (kubectl exec | ssh), +# so it needs no elevated access beyond what deploy/update.sh already uses, +# and doesn't depend on the underlying storage class's on-disk layout. +# +# This is a live copy, not a transactional snapshot — gitfed-server keeps +# running and writing while it's taken. bbolt is crash-consistent by +# design (the same guarantee as surviving a power loss mid-write), so this +# is safe, but a push landing in the exact instant of the tar could be +# split across two backups. Acceptable for a personal/small-team instance; +# not a substitute for a maintenance-window backup if that ever matters. +# +# Usage: +# deploy/backup.sh # snapshot now, prune to the 14 most recent +# deploy/backup.sh --keep N # snapshot now, prune to the N most recent +# deploy/backup.sh --keep 0 # snapshot now, keep everything + +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")/.." + +VPS_HOST="${GITFED_VPS_HOST:-ubuntu@51.77.215.92}" +NAMESPACE="gitfed" +KEEP=14 + +if [ "${1:-}" = "--keep" ]; then + KEEP="${2:?--keep needs a number}" +elif [ -n "${1:-}" ]; then + echo "usage: $0 [--keep N]" >&2 + exit 1 +fi + +mkdir -p backups +timestamp="$(date -u +%Y%m%dT%H%M%SZ)" +out="backups/gitfed-$timestamp.tar.gz" +tmp="$out.part" + +echo "==> archiving /data on $VPS_HOST (namespace $NAMESPACE)" +ssh "$VPS_HOST" "kubectl -n $NAMESPACE exec deployment/gitfed -c server -- tar czf - -C /data ." > "$tmp" +mv "$tmp" "$out" + +size="$(du -h "$out" | cut -f1)" +echo "==> done: $out ($size)" + +if [ "$KEEP" -gt 0 ]; then + pruned="$(ls -1t backups/gitfed-*.tar.gz 2>/dev/null | tail -n +"$((KEEP + 1))")" + if [ -n "$pruned" ]; then + echo "==> pruning to the $KEEP most recent backups" + echo "$pruned" | xargs -I{} rm -v -- {} + fi +fi
deploy/k8s/README.md
diff --git a/deploy/k8s/README.md b/deploy/k8s/README.md index 86fd9ab..2eea4c4 100644 --- a/deploy/k8s/README.md +++ b/deploy/k8s/README.md @@ -144,6 +144,55 @@ at this domain**, there's no recovery short of everyone re-establishing trust), `host_key`, and `repos/` (the actual bare git repos). Back up the whole PVC, not just the git data. +From your workstation: + +```sh +deploy/backup.sh # snapshot now, prune to the 14 most recent +deploy/backup.sh --keep 30 # keep more (or --keep 0 to keep everything) +``` + +This streams a tarball straight off the running container over SSH into +`./backups/` (gitignored) — it doesn't touch the VPS's own disk, so a lost +VPS doesn't take the backups with it. It's a live copy taken while +`gitfed-server` keeps running, not a transactional snapshot; see the +script's header comment for why that's an acceptable trade here. + +There's nothing scheduling this for you — it's a plain script, so wire it +into whatever you already use for recurring jobs (a cron entry or a +`launchd`/systemd user timer running `deploy/backup.sh` from a checkout of +this repo works fine). + +### Restoring from a backup + +1. Stop gitfed so nothing is writing to `/data` while you overwrite it: + + ```sh + kubectl -n gitfed scale deployment/gitfed --replicas=0 + ``` + +2. Start the throwaway restore pod (same PVC, no gitfed code running): + + ```sh + kubectl apply -f deploy/k8s/restore-pod.yaml + kubectl -n gitfed wait --for=condition=Ready pod/gitfed-restore + ``` + +3. Copy the backup in and unpack it, replacing whatever's currently on the + PVC: + + ```sh + kubectl -n gitfed cp backups/gitfed-<timestamp>.tar.gz gitfed-restore:/tmp/backup.tar.gz + kubectl -n gitfed exec gitfed-restore -- sh -c \ + 'find /data -mindepth 1 -delete && tar xzf /tmp/backup.tar.gz -C /data' + ``` + +4. Clean up and start gitfed again: + + ```sh + kubectl -n gitfed delete pod gitfed-restore + kubectl -n gitfed scale deployment/gitfed --replicas=1 + ``` + ## Further hardening (not included here, worth doing later) - A `NetworkPolicy` restricting which namespaces/pods can reach
deploy/k8s/restore-pod.yaml
diff --git a/deploy/k8s/restore-pod.yaml b/deploy/k8s/restore-pod.yaml new file mode 100644 index 0000000..8ec4d43 --- /dev/null +++ b/deploy/k8s/restore-pod.yaml @@ -0,0 +1,25 @@ +# Temporary pod for restoring a deploy/backup.sh tarball onto the +# gitfed-data PVC — see deploy/k8s/README.md "Restoring from a backup". +# Not part of the normal deployment: apply only while restoring, then +# delete it. Deliberately not gitfed's own image — this only ever needs +# `tar`, and running something other than gitfed-server/gitfed-web against +# a live PVC while the real deployment is scaled down keeps the two from +# ever fighting over the bbolt file's exclusive lock. +apiVersion: v1 +kind: Pod +metadata: + name: gitfed-restore + namespace: gitfed +spec: + restartPolicy: Never + containers: + - name: restore + image: debian:bookworm-slim + command: ["sleep", "infinity"] + volumeMounts: + - name: data + mountPath: /data + volumes: + - name: data + persistentVolumeClaim: + claimName: gitfed-data