Install on-premises

Overview

This guide describes how to deploy the Poolside platform on a GPU workstation or server host. The installation supports both online and air-gapped environments. Installation time typically takes less than 30 minutes when the installation bundle is cached on the host. The Poolside platform includes:

Agent and chat model: Laguna family
Code completion model: Point
Poolside Chat: Browser-based chat interface
Poolside Console: Browser-based interface for managing models, configuring agents and integrations, controlling access, and monitoring usage
Built-in authentication: Keycloak identity provider
S3-compatible object storage: SeaweedFS for models and telemetry
Database: PostgreSQL for Poolside deployment
cert-manager: Self-signed certificate provider

Prerequisites

Before you begin, ensure that the host meets the following prerequisites. The installation process installs and configures GPU drivers and Poolside platform services.

A supported operating system: Ubuntu 22.04 LTS, Ubuntu 24.04 LTS, or Red Hat Enterprise Linux (RHEL) 9.6
sudo access on the host
Network access for online installations, or the full bundle and artifacts available locally for air-gapped installations
The nouveau driver is not loaded on the host. For instructions, see Disable Nouveau in the NVIDIA documentation.

Prerequisites for RHEL 9.6

RHEL can upgrade the host to a newer minor release when new updates become available through dnf update or yum update. Before you install packages, lock the release to RHEL 9.6 to prevent automatic minor version upgrades.

# List available versions
subscription-manager release --list

# Lock the release to version 9.6
subscription-manager release --set=9.6

yum clean all

Install iptables-nft using yum (version 1.8.10-11.el9)
Install container-selinux using yum
Install curl using yum
Install jq using yum (version 1.6 or later)
Install yq (version v4.49.2 or later) from the yq releases page
- Download yq_linux_amd64.tar.gz and install it to /usr/local/bin/yq
Install unzip using yum (version 6.00 or later)
Install skopeo using yum (package skopeo-1.18.1-2.el9_6.x86_64 or later)

Install kubectl by adding the Kubernetes repository (ensure that the kubectl version is the same as or newer than the RKE2 Kubernetes version):

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.33/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.33/rpm/repodata/repomd.xml.key
EOF

Then run:

sudo yum install -y kubectl

Install terraform (version 1.8.5) from the Terraform 1.8.5 releases page
- Download and install the binary to /usr/local/bin/terraform
- unzip is required to extract the Terraform binary

In RHEL 9.x, /usr/local/bin is not included in the secure_path setting in /etc/sudoers by default. As a result, sudo terraform can return a command not found error.Run Terraform with the absolute path: /usr/local/bin/terraform.

Confirm that the nouveau graphics driver is not loaded. For instructions, see Disable the nouveau driver in the NVIDIA documentation. Run the following command to check whether the nouveau driver is loaded. If the command returns output, follow the next steps to turn off the driver and reboot.

lsmod | grep nouveau

If the nouveau driver is loaded:

# Check for nouveau in the GRUB configuration.
grep GRUB_CMDLINE_LINUX /etc/default/grub

# If this command does not show that nouveau is blacklisted, ensure that the
# GRUB_CMDLINE_LINUX line in /etc/default/grub contains "modprobe.blacklist=nouveau",
# for example, at the end of the line.

cat <<EOF | sudo tee /etc/modprobe.d/blacklist-nouveau.conf
blacklist nouveau
options nouveau modeset=0
EOF

# Regenerate the grub config file and add a boot menu entry for EFI firmware configuration.
sudo dracut --force
grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

# Reboot the system.
sudo systemctl reboot

# After reboot, confirm that the nouveau driver is not loaded.
lsmod | grep nouveau

Prerequisites for Ubuntu

These steps apply to both Ubuntu 22.04 LTS and Ubuntu 24.04 LTS.

Install kubectl using sudo snap install kubectl --classic
Install curl using sudo apt install -y curl
Install jq using sudo apt install -y jq
Install yq (version v4.49.2 or later) from the yq releases page
- Download yq_linux_amd64.tar.gz and install it to /usr/local/bin/yq
Install terraform (version 1.8.5) from the Terraform 1.8.5 releases page
- Download and install the binary to /usr/local/bin/terraform
- unzip is required to extract the Terraform binary
Install skopeo (version v1.18 or later) from the skopeo-binary releases page
- Download skopeo-linux-amd64 and install it to /usr/local/bin/skopeo
- Ensure the containers trust policy at /etc/containers/policy.json allows skopeo to access the RKE2 registry with the minimum required permissions to load container images into the registry during installation
```
{
  "default": [
    {
      "type": "insecureAcceptAnything"
    }
  ]
}
```

lsmod | grep nouveau

If the nouveau driver is loaded:

cat <<EOF | sudo tee /etc/modprobe.d/blacklist-nouveau.conf
blacklist nouveau
options nouveau modeset=0
EOF

# Regenerate the kernel initramfs.
sudo update-initramfs -u

# Reboot your system:
sudo reboot

# After reboot, ensure nouveau is NOT loaded.
lsmod | grep nouveau

Kernel parameters

Some operating systems require kernel parameter (sysctl) adjustments to support the Poolside platform’s runtime needs. Ubuntu 22.04 and 24.04 The platform’s file watchers exceed the Ubuntu default for inotify instances. Set the following parameter to 65535 or higher:

fs.inotify.max_user_instances = 65535

To apply, add the parameter under /etc/sysctl.d/ and reload:

echo "fs.inotify.max_user_instances = 65535" | sudo tee /etc/sysctl.d/99-poolside.conf
sudo sysctl --system

Install steps

The Poolside installation bundle includes all required Terraform providers for a linux/amd64 host to support fully offline (air-gapped) deployments.

Step 0 (optional): Set up an air-gapped installation

This configuration is required for air-gapped installations. It uses a bundled Terraform CLI configuration file instead of a standard .terraformrc file so Terraform behaves consistently when commands run as both root and a non-root user. In internet-accessible environments, you can skip this step.

To use the local Terraform provider cache included in the bundle, configure Terraform to load providers from the bundled terraform.d directory.

Locate poolside-terraform.tfrc in the root of the unpacked installation bundle.
Replace the $POOLSIDE_INSTALL_DIR placeholder with the fully qualified path to the bundle’s root directory.
For Terraform commands in the installation and post-installation steps, prefix the command with the Terraform CLI configuration file path:
```
TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc terraform <command>
```

Setting this variable ensures that both root and non-root users reference the same cached Terraform providers.

You can configure Terraform using alternative methods, such as a .terraformrc file, as described in the official HashiCorp documentation. Because the installation process runs as both root and a local user, you must ensure that both accounts are configured to reference the cached providers correctly.

Step 1: Install and configure RKE2 on the host

The 01-infra-rke2 directory contains the Terraform module that installs RKE2 on the host system. Using sudo, run the following commands from the 01-infra-rke2 directory.

You must run the RKE2 installation using sudo from the same user account that will run the Poolside platform after deployment.
Terraform uses the original user and group IDs from the sudo environment to set ownership and permissions required by later installation stages.

Air-gapped environment:

sudo TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc /usr/local/bin/terraform init
sudo TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc /usr/local/bin/terraform apply

Online environment:

sudo /usr/local/bin/terraform init
sudo /usr/local/bin/terraform apply

Step 2: Generate RKE2 credentials and access files

The 02-rke2-credentials directory contains the Terraform module that connects to the RKE2 cluster and extracts the configuration files required for later installation stages. These files allow Terraform to authenticate to the RKE2 cluster and complete the Poolside platform deployment. Using sudo, run the following commands from the 02-rke2-credentials directory.

You must run this step using sudo from the same user account that will run the Poolside platform after deployment.
Terraform uses the original user and group IDs from the sudo environment to set the permissions required for RKE2 cluster access in later stages.

Air-gapped environment:

sudo TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc /usr/local/bin/terraform init
sudo TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc /usr/local/bin/terraform apply

Online environment:

sudo /usr/local/bin/terraform init
sudo /usr/local/bin/terraform apply

If RKE2 certificates or credentials change, re-run this step to regenerate the configuration files and restore cluster access.

Step 3: Install supporting infrastructure services

The 03-infra-services directory contains the Terraform module that uses the previously generated RKE2 credentials to access the cluster and deploy the supporting infrastructure required by the Poolside platform. This step installs:

A local container registry
A PostgreSQL database
An S3-compatible object store
Keycloak for user authentication
cert-manager, which generates a local self-signed certificate authority (CA) on the cluster and uses it to issue TLS certificates for the platform’s ingress and supporting services

Using sudo, run the following commands from the 03-infra-services directory.

Use custom TLS certificates

The default self-signed CA is suitable for evaluation and development. For production deployments, you are responsible for providing your own CA and server certificate. Configure the bring-your-own (BYO) certificate variables in 03-infra-services/terraform.tfvars before you run terraform apply. The custom_certificates schema accepts a separate certificate and key for each of the platform’s TLS-terminating services (poolside, services.storage, services.storage_s3, and services.identity), so you can use multiple per-service certificates if your Public Key Infrastructure (PKI) requires it. A single certificate that covers every service is the recommended pattern. Across all certificates you provide, the Subject Alternative Names (SANs) should cover the following ingress hostnames:

<poolside-ingress-host>.<domain-name>
<keycloak-ingress-host>.<domain-name>
seaweedfs.<domain-name>
seaweedfs-s3.<domain-name>

Where:

<poolside-ingress-host> is the value of poolside_ingress_host in 03-infra-services/terraform.tfvars (default poolside).
<keycloak-ingress-host> is the value of keycloak_ingress_host in 03-infra-services/terraform.tfvars (default keycloak).
<domain-name> is the value of domain_name in 03-infra-services/terraform.tfvars (default poolside.local).

A SAN list that includes all of these names covers every service with one certificate. Alternatively, a wildcard certificate with *.<domain-name> as a SAN covers every service.

PostgreSQL is not exposed through an ingress and is not reachable from outside the cluster, so its certificate is not configurable through custom_certificates. cert-manager continues to issue it for in-cluster mTLS connections.

Place your CA certificate, server certificate, and private key in a directory accessible to Terraform. The example below uses <path-to-bundle>/poolside-install/byo-certs/. The poolside-install/ subdirectory holds the installation’s persistent state and is preserved across cluster resets, so it is the recommended location for BYO certificate files.
```
<path-to-bundle>/poolside-install/byo-certs/
├── ca.crt       # CA certificate (root, or root and intermediate chain)
├── server.crt   # Server certificate signed by the CA
└── server.key   # Server private key
```
You must reference these files using fully qualified (absolute) paths in the next step. Relative paths are not supported.

In 03-infra-services/terraform.tfvars, set the BYO variables:

custom_ca_trust_chain = {
  root_ca_path = "<path-to-bundle>/poolside-install/byo-certs/ca.crt"
}

custom_certificates = {
  poolside = {
    cert_path = "<path-to-bundle>/poolside-install/byo-certs/server.crt"
    key_path  = "<path-to-bundle>/poolside-install/byo-certs/server.key"
  }
  services = {
    storage = {
      cert_path = "<path-to-bundle>/poolside-install/byo-certs/server.crt"
      key_path  = "<path-to-bundle>/poolside-install/byo-certs/server.key"
    }
    storage_s3 = {
      cert_path = "<path-to-bundle>/poolside-install/byo-certs/server.crt"
      key_path  = "<path-to-bundle>/poolside-install/byo-certs/server.key"
    }
    identity = {
      cert_path = "<path-to-bundle>/poolside-install/byo-certs/server.crt"
      key_path  = "<path-to-bundle>/poolside-install/byo-certs/server.key"
    }
  }
}

custom_ca_trust_chain.root_ca_path must point to the CA that signed server.crt. The platform mounts the CA bundle into core-api and other workloads so they can verify in-cluster TLS connections, such as the OpenID Connect (OIDC) discovery call from core-api to the identity provider (which is Keycloak by default). When you run terraform apply, the module creates Kubernetes secrets with a -byo suffix from these files. Step 4 reads the certificate paths from step 3’s exported configuration and uses them for the platform’s ingress secrets, so you don’t need to configure certificates in 04-poolside-deployment.

Apply the changes

sudo TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc /usr/local/bin/terraform init
sudo TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc /usr/local/bin/terraform apply

This step can take some time to complete. The process loads container images into the local RKE2 registry to support air-gapped operation and improve Poolside platform startup performance.

Step 4: Deploy the Poolside platform

The 04-poolside-deployment directory contains the Terraform module that deploys the Poolside platform into the RKE2 cluster using the infrastructure services installed in the previous steps. This deployment includes optional configuration and must align with any modifications you made in earlier phases. Review and update the terraform.tfvars file as needed before running Terraform. Run the following commands from the 04-poolside-deployment directory.

TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc terraform init
TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc terraform apply

Step 5: Upload Poolside models

This step uploads Poolside models to the S3-compatible storage used by the deployment. The 05-poolside-model-upload directory contains a Terraform module that creates a Kubernetes job to sync model files from a local host directory into the poolside-models S3 bucket.

Copy the Poolside model files into the local host directory: /opt/poolside/poolside-model-uploads If you customized host volume paths during the RKE2 installation step (01-infra-rke2), use the corresponding directory instead.

Run the following commands from the 05-poolside-model-upload directory to trigger the model upload job.

TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc terraform init
TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc terraform apply

To upload additional or updated models later, repeat these steps. Uploads are additive and do not remove existing models from the deployment.

Step 6: Deploy Poolside models

The final phase 06-poolside-inference deploys the Poolside models from S3-compatible storage into the RKE2 cluster. The directory contains a Terraform module, which wraps the inference Helm chart used to deploy models across all modalities.

Verify the model paths for all uploaded Poolside models in S3-compatible storage. The default bucket path is s3://poolside-models/, followed by the model folder. These paths will be used for model_s3_uri in terraform.tfvars.
```
ls /opt/poolside/poolside-model-uploads
```
Example output:
```
laguna_xs_fp8_fp8kv_04_2026  malibu_v2_agent_int4_04_2026 point_v2_04_2026
```
From this example, the model_s3_uri values would be:
- s3://poolside-models/laguna_xs_fp8_fp8kv_04_2026
- s3://poolside-models/malibu_v2_agent_int4_04_2026
- s3://poolside-models/point_v2_04_2026

In the 06-poolside-inference directory, create or update the terraform.tfvars file.

deployment_name should match the value used for each previous phase.
model_s3_uris should include the S3 URIs for each model listed in the previous step.
models_config should include the configuration for each model, including the number of replicas and GPUs for your deployment’s hardware configuration.

deployment_name = "poolside-server"

model_s3_uris = {
  agent_small = "s3://poolside-models/laguna_xs_fp8_fp8kv_04_2026"
  agent = "s3://poolside-models/malibu_v2_agent_int4_04_2026"
  completion = "s3://poolside-models/malibu_v2_completion_int4_04_2026"
}

models_config = {
  agent_small = {
    replicas = 1
    gpus = 1
  }
  agent = {
    replicas = 1
    gpus = 2
  }
  completion = {
    replicas = 1
    gpus = 1
  }
}

Run the following commands to deploy the models.

TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc terraform init
TF_CLI_CONFIG_FILE=<path-to-bundle>/poolside-terraform.tfrc terraform apply

After the models are deployed, run the following command to retrieve the name and url for the deployed models.
```
terraform output model_urls
```
To deploy additional models or change the GPU or replica counts, modify terraform.tfvars and run terraform apply again.

Next steps: Post-installation configuration

Step 1: Configure local DNS

Add these hostname mappings on the deployment host. This ensures that local service callbacks resolve to the local system instead of external DNS.

# Add to /etc/hosts
echo "127.0.0.1 poolside.poolside.local" | sudo tee -a /etc/hosts
echo "127.0.0.1 keycloak.poolside.local" | sudo tee -a /etc/hosts
echo "127.0.0.1 seaweedfs.poolside.local" | sudo tee -a /etc/hosts
echo "127.0.0.1 seaweedfs-s3.poolside.local" | sudo tee -a /etc/hosts

Step 2: Set up Poolside

The following steps describe how to complete initial setup using the internal Keycloak service as the identity provider. If you are integrating with an external identity provider, see OIDC authentication.

Open a web browser and navigate to: https://poolside.poolside.local If you configured a different hostname, use that value instead. Ensure DNS resolution is working for the selected hostname.
Retrieve identity configuration values from Terraform output.
- Change to the 03-infra-services directory.
- Run the following command:
  terraform output identity_config
- Note the values for client_api_credentials and oidc_provider_url.
Create the organization.
- When prompted in the Poolside Console, create your organization. For Poolside realm settings, enter the following values:
  - Provider URL: oidc_provider_url from the Terraform output
  - Client ID: client_api_credentials.id from the Terraform output
  - Client Secret: client_api_credentials.secret from the Terraform output
Register the first user.
- Click Login, then Register.
- Create your administrator user account. This user is automatically assigned the tenant-admin role.
Link the deployed models in the Poolside Console.
- In the Poolside Console, navigate to Agents > Models.
- Click New Model.
- Enter the Model Name and Base URL retrieved from Step 6: Deploy Poolside models, and optionally a Description or Name override.
- Click Connect to Model.

Verification

Your installation is successful when:

All pods are running: kubectl get pods -A
The web interface is accessible at https://poolside.poolside.local
Models show a Healthy status in the Poolside Console
Chat functionality works in the Poolside Chat

Troubleshooting

Common issues

Model pods stuck in ContainerCreating

Verify GPU availability on the host using nvidia-smi
Scale the affected deployment to zero replicas, then scale it back up:
- Identify the deployment: kubectl get deploy -n poolside-models
- Scale down and back up as needed

Certificate warnings in the browser

Trust the cluster CA certificate in the poolside-install directory of your installation bundle, or from the OS trust store under /usr/local/... after installation
Certificate installation steps vary by operating system

Models not loading

Verify that services are running:
```
kubectl get pods -n poolside-services
```
Confirm that models were successfully uploaded during the model upload step

Useful commands

# Check overall cluster status
kubectl get pods -A

# Monitor key namespaces
kubectl get pods -n poolside-models    # Model workloads
kubectl get pods -n poolside           # API and platform services
kubectl get pods -n poolside-services  # Database supporting services

# Inspect pod and deployment details
kubectl describe pod <pod-name> -n <namespace>
kubectl describe deploy <deployment-name> -n <namespace>

# View logs
kubectl logs <pod-name> -n <namespace>

# View logs for model download or initialization containers
kubectl logs <pod-name> -c model-downloader -n poolside-models

# View recent events in a namespace
kubectl get events -n <namespace>

Overview

Cloud deployment

On-premises deployment

Configuration

Metrics and telemetry

Legacy

Install on-premises

Overview

Prerequisites

Prerequisites for RHEL 9.6

Prerequisites for Ubuntu

Kernel parameters

Install steps

Step 0 (optional): Set up an air-gapped installation

Step 1: Install and configure RKE2 on the host

Step 2: Generate RKE2 credentials and access files

Step 3: Install supporting infrastructure services

Use custom TLS certificates

Apply the changes

Step 4: Deploy the Poolside platform

Step 5: Upload Poolside models

Step 6: Deploy Poolside models

Next steps: Post-installation configuration

Step 1: Configure local DNS

Step 2: Set up Poolside

Verification

Troubleshooting

Common issues

Useful commands

Overview

Cloud deployment

On-premises deployment

Configuration

Metrics and telemetry

Legacy

Documentation Index

​Overview

​Prerequisites

​Prerequisites for RHEL 9.6

​Prerequisites for Ubuntu

​Kernel parameters

​Install steps

​Step 0 (optional): Set up an air-gapped installation

​Step 1: Install and configure RKE2 on the host

​Step 2: Generate RKE2 credentials and access files

​Step 3: Install supporting infrastructure services

​Use custom TLS certificates

​Apply the changes

​Step 4: Deploy the Poolside platform

​Step 5: Upload Poolside models

​Step 6: Deploy Poolside models

​Next steps: Post-installation configuration

​Step 1: Configure local DNS

​Step 2: Set up Poolside

​Verification

​Troubleshooting

​Common issues

​Useful commands

Overview

Prerequisites

Prerequisites for RHEL 9.6

Prerequisites for Ubuntu

Kernel parameters

Install steps

Step 0 (optional): Set up an air-gapped installation

Step 1: Install and configure RKE2 on the host

Step 2: Generate RKE2 credentials and access files

Step 3: Install supporting infrastructure services

Use custom TLS certificates

Apply the changes

Step 4: Deploy the Poolside platform

Step 5: Upload Poolside models

Step 6: Deploy Poolside models

Next steps: Post-installation configuration

Step 1: Configure local DNS

Step 2: Set up Poolside

Verification

Troubleshooting

Common issues

Useful commands