Skip to main content

​
AWS deployment options

Poolside supports two different AWS deployment options. Work with your Poolside representative to determine the most appropriate deployment option for your organization. See AWS Cost Modeling for a breakdown of services and associated costs for each deployment option.

Amazon EC2

  • Complete control over infrastructure
  • Self-hosted GPU instances via Amazon EC2
  • Full infrastructure cost responsibility
  • Ideal for organizations requiring maximum control

Amazon Bedrock

  • Serverless model inference
  • No GPU infrastructure management
  • Pay-per-request pricing model
  • Reduced operational overhead

​
Deployment management

Poolside AWS deployments are self-managed so you have full control over the installation process. Poolside provides the necessary scripts and documentation so you can install and manage the platform independently. Poolside still provides support for product installation and configuration. This typically requires about one hour of effort from your organization.

​
Architecture

The following diagram illustrates the Poolside high-level architecture after a successful deployment on AWS. This is fundamental to help illustrate the available options: Architecture The diagram illustrates 4 main areas:
  • Installation hub: Poolside online service (control plane) used during installation and maintenance
  • Application plane: Represents all application-related services that support Poolside
  • Model plane: The inference space that can be deployed in your VPC via Amazon EC2 or via Amazon Bedrock
  • Data plane: The model registry service

​
Additional considerations

For AWS deployments, be aware of the following additional considerations:

​
Regional availability

Poolside deployments can be configured in any AWS region that supports the required services. Key considerations for regional deployment include:

Amazon EC2

  • Requires regions with p5.48xlarge/p5e.48xlarge GPU instance availability
  • All standard AWS services (EKS, RDS, S3) must be available
  • See AWS Services by Region for up-to-date service availability

Amazon Bedrock

  • Deployment region must have Amazon Bedrock service available
  • Standard platform services available in most regions
  • See Amazon Bedrock endpoints and quotas for supported regions

​
Cross-region support in Amazon Bedrock

Poolside supports cross-region inference through Amazon Bedrock, which enables the use of an Amazon Bedrock endpoint in a different region from the VPC where the Poolside Application Plane and Data Plane are deployed. See Data Privacy in Amazon Bedrock for more information.

​
Costs

For a detailed breakdown of AWS infrastructure costs, including both fixed operating costs and variable usage costs, see AWS Cost Modeling. This guide covers infrastructure requirements, deployment options, and cost estimation guidance for both Amazon EC2 and Amazon Bedrock deployments. Please note that all services and components are required.

​
Security

In any AWS deployment:
  • Deploying Poolside does not require root access. Follow AWS best practices for securing the root user.
  • AWS resources used by Poolside, such as Amazon S3 and Amazon RDS, contain sensitive data and must be protected using appropriate access controls.
  • All supported AWS deployment modalities encrypt data at rest by default. Poolside uses customer-managed KMS keys when provided (C2E and other AWS modalities), and otherwise falls back to AWS-managed KMS keys for each service.
  • Secrets and credentials required by Poolside are managed by the deployment and stored in AWS Systems Manager Parameter Store. AWS Secrets Manager is not supported, and external credential rotation can disrupt service operation. For example, database passwords are generated during installation and set directly on the database; rotating them outside the deployment can break connectivity.

​
Patches and upgrades

Patches and upgrades are to be coordinated directly with Poolside. Please contact your Poolside representative to discuss your specific patch and upgrade plan (and schedule) based on your deployment.
The platform’s Kubernetes-based architecture and underlying AWS infrastructure are managed through Terraform. Most Kubernetes resources are provisioned directly, with select third-party components included in the same Terraform-managed workflow. Infrastructure provisioning and patching are also handled through Terraform.

​
Backup and recovery

Poolside follows AWS’s shared responsibility model for backup and recovery. While AWS provides infrastructure redundancy and availability measures, customers are responsible for implementing and testing their own backup and disaster recovery procedures based on their requirements.

​
Default configuration

  • Amazon S3: Multi-Availability Zone durability by default
  • Amazon RDS: Automated weekly backups with 7-day retention; multi-AZ replication is not enabled by default
  • EKS Cluster: No additional backup measures beyond AWS’s control plane management
  • Amazon ECR: Container images are stored in private ECR registries within the customer’s AWS account, supporting isolated deployments that do not require continuous internet access (aside from AWS-managed EKS add-ons)

​
Critical data considerations

When planning backup and recovery, pay particular attention to the following data sources:
  • Model artifacts stored in Amazon S3
  • Database state in Amazon RDS, including conversation history and user preferences
  • Platform and deployment configuration
  • Container images stored in Amazon ECR
Based on your recovery requirements, the following measures are recommended:
  1. Amazon S3:
    • Enable S3 bucket backups
    • Consider cross-region replication for disaster recovery
    • Implement appropriate retention policies
  2. Amazon RDS:
    • Automated backups are enabled by default with 7-day retention; increase retention as needed
    • Consider multi-AZ deployment for high availability
    • Test recovery procedures regularly
  3. Amazon ECR:
    • Consider enabling replication rules or periodic exports to preserve critical images
    • Document image provenance and retention policies for recovery
  4. EKS cluster:
    • The EKS control plane is managed by AWS
    • Consider backing up critical Kubernetes objects (secrets, configmaps)
    • Document platform configuration for disaster recovery
    • Maintain Infrastructure as Code (IaC) in version control
Backup and recovery requirements should be assessed based on your organization’s:
  • Recovery Point Objective (RPO)
  • Recovery Time Objective (RTO)
  • Compliance requirements
  • Data retention policies
Contact your Poolside representative for guidance on backup strategies that align with your specific needs.