Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.poolside.ai/llms.txt

Use this file to discover all available pages before exploring further.

The Poolside deployment bundle includes db-snapshot.sh for database backup and restore operations. Use this script before upgrading your deployment or to recover from data loss or accidental deletion.

​
Prerequisites

Before running db-snapshot.sh, ensure you have:
  • kubectl configured with access to your Poolside namespace
  • skopeo and jq installed on your system (required for image discovery)
  • A running core-api deployment in your namespace
  • Access to the poolside-database-client container image in your registry. The Poolside deployment bundle includes this image, so your installation process should have already uploaded it to your registry.

​
Location

The script’s location depends on your deployment type. cd into the relevant directory before running the examples below: 
cd <bundle-root>/scripts

​
How it works

The script creates an ephemeral “jump pod” using the poolside-database-client image, which includes psql and pg_dump utilities. This pod:
  1. Reads database connection configuration from the running core-api deployment
  2. Mounts the PostgreSQL password from the existing secret (without exposing it)
  3. Executes the backup or restore operation
  4. Deletes itself after completion

​
Image discovery

The script determines which image to use in the following order. Use --image or --registry to override the default discovery behavior:
  1. If you provide --image=<full-reference>, the script uses that exact image.
  2. If you provide --registry=<host>, the script finds the latest poolside-database-client tag in that registry.
  3. Otherwise, the script reads the registry from FORGE_API_IMAGE_REPOSITORY in the core-api deployment, then finds the latest tag there.
If the registry is unreachable, the script falls back to tag v0.0.3. In air-gapped environments where this tag does not exist, use --image=<registry>/poolside-database-client:<tag> to specify the image directly.

​
Create a backup

The script saves snapshots to ~/.poolside/snapshots/ by default, with file names of the form db-snapshot-<version>-<timestamp>.tar. 
./db-snapshot.sh create

# Specify a custom registry (use when running in an air-gapped environment)
./db-snapshot.sh --registry=<registry-host> create

# Specify a full image reference (use when the image tag differs from the registry's latest)
./db-snapshot.sh --image=<image> create

# Specify a custom backup directory (use for centralized backup storage)
./db-snapshot.sh --dir=<backup-directory> create

​
What gets backed up

The script uses pg_dump with --format=tar to create a logical backup of the database. The backup includes:
  • All table data and schema definitions
  • The schema_migrations table (used to determine schema version compatibility during restore)
  • All application data, but no ownership or privilege information

​
Backup location

By default, backups are saved to ~/.poolside/snapshots/. You should copy these files to a secure, persistent storage location after creation. Each backup includes the schema version in the filename, so you can identify the right snapshot quickly.

​
Restore from a backup

The restore process includes safety checks to prevent data loss. The script compares the live database’s schema version against the snapshot’s schema version and chooses one of three paths:
  • Use data-only restore when: Schema versions match. The schema stays in place and only table data is replaced.
  • Use drop-and-restore when: Live database is newer than the snapshot. The script prompts to confirm, then scales core-api to 0 replicas, drops database objects, runs pg_restore, and scales core-api back.
  • Abort and upgrade first when: Live database is older than the snapshot. This prevents data loss.

​
Interactive workflow

When you run ./db-snapshot.sh restore without specifying a file, the script:
  1. Lists all available .tar files in the backup directory
  2. Prompts you to select a snapshot by number
  3. Compares schema versions between the live database and the selected backup
  4. If versions differ, prompts for confirmation before proceeding
  5. If dropping is required, warns that core-api will be temporarily unavailable
  6. Prompts for final confirmation before starting the restore
./db-snapshot.sh restore

# Restore a specific snapshot file (use when you know which backup to restore)
./db-snapshot.sh --file=<snapshot-file> restore

# Force drop-and-restore (use when restoring an older backup onto a newer database)
./db-snapshot.sh --file=<snapshot-file> --drop restore
Restore operations require downtime. The core-api deployment is scaled to 0 replicas during a drop-and-restore operation.

​
Log files

Log files are saved to ~/.poolside/logs/ with filenames like db-snapshot-restore-<timestamp>.log. Check these logs when a backup or restore operation fails.

​
Additional commands

# List existing snapshots
./db-snapshot.sh list

# Print the schema version stored in a snapshot
./db-snapshot.sh --file=<snapshot-file> extract-version

​
List snapshots

The list command displays all .tar files in your backup directory with their sizes and modification dates. This helps you identify which snapshot to restore when you have multiple backups.

​
Extract schema version

Use extract-version to check a snapshot’s schema version without starting a restore. This is useful when you need to verify backup compatibility before planning a restore operation.

​
Advanced usage

​
Debug with an interactive jump pod

For troubleshooting or manual database operations, you can create a jump pod with jump-create. This leaves a pod running until you delete it, so you can exec into it for interactive psql sessions. 
./db-snapshot.sh jump-create

# The script outputs the connection details. To open an interactive psql session:
kubectl exec -it db-snapshot-jump -n <namespace> -- psql -h <postgres-host> -d <database-name> -U <postgres-user>
When finished, delete the pod: 
./db-snapshot.sh jump-delete
Jump pods are ephemeral and do not persist. Do not use jump-create for long-running operations. Always run jump-delete when finished to clean up resources.

​
Command reference

FlagDescription
--dir=<path>Backup directory path. Default: ~/.poolside/snapshots/
--file=<filename>Specific snapshot filename to use
--dropForce drop-and-restore regardless of schema version
-n, --namespace=<namespace>Kubernetes namespace. Default: poolside
--registry=<host>Container registry for the jump pod image
--image=<image>Full image reference for the jump pod (overrides --registry)
--force-insecureDisable TLS verification. Use only for HTTP-only registries in air-gapped environments

​
Troubleshooting

​
Image pull failures

If the script fails with an image pull error, the poolside-database-client image may be missing from your registry. To resolve: 
./db-snapshot.sh --registry=<registry-host> --image=<registry-host>/poolside-database-client:<tag> create
To find available tags in your registry: 
skopeo list-tags docker://<registry-host>/poolside-database-client

​
Optional: Additional backup types

Depending on your organization’s backup and compliance requirements, you may also want to back up other Poolside components beyond the database. These backups are not required for standard upgrades but may be part of your disaster recovery or compliance procedures.

​
S3-compatible storage

Poolside stores usage data, telemetry, and model checkpoints in S3-compatible storage. Use this backup when your organization requires archiving usage data or model checkpoints. You can back up this storage using your preferred backup or replication method, such as an S3-compatible client, storage replication, or rsync against a mounted storage path. Configure S3-compatible tools to use the SeaweedFS endpoint in your deployment.

​
Kubernetes objects

Use this backup when you need to preserve deployment configuration for disaster recovery. Kubernetes object backups are typically part of cluster-level backup procedures. You can back up these objects by exporting YAML manifests directly, or by using Velero or another Kubernetes backup solution. Export Deployments, ConfigMaps, Secrets, and Ingresses from the Poolside namespace to preserve your deployment configuration.
Kubernetes object backups are not compatible across schema versions. Do not restore these objects after upgrading Poolside. Use this backup when you need configuration recovery before any version changes.
  • Upgrade on-premises: Database backup is a prerequisite step before upgrading a Poolside deployment.