Technical components
- EKS cluster with two node groups: one for CPU-based activities and another for GPU-based activities. If using Amazon Bedrock for serverless inference, GPU nodes are not required in your EKS cluster.
- Minimum of two sub-networks: one for connectivity of the services that compose Poolside and another one for management of the cluster.
- Plugin installed in the IDE of the developer: interact with the Poolside API to handle all model requests.
- Web interface: that allows for RBAC activities and to manage all model-related activities.
- Poolside API: is the main business logic that handles AuthZ/AuthN. This API orchestrates the connection with the inference engine and all other activities related to managing a tenant.
- OpenID connectivity: to allow connectivity between the Poolside API and the customer IdP.
- Single Postgres RDS: database that supports all relational metadata for the system to operate.
- Three S3 buckets:
- poolside.logs: a bucket to handle all the logs, that can used with any monitoring or observability tools (like Splunk or Grafana).
- poolside.artifacts: a bucket to handle all model versions and other assets necessary for Poolside to operate.
- poolside.code: a bucket to handle all code related to RAG activities.
To promote proper audits all API requests are tracked by a middleware that follows OpenTelemetry standards.
Poolside terminology
Core-API: Serves as the endpoint orchestrating communications between Poolside models and Poolside Assistant. Inference engine: Layer responsible for processing input data sent to the Poolside model, model computation, and model outputs. Poolside Agent CLI (pool): A command-line interface for running and managing Poolside agents that run in headless environments, such as CI/CD pipelines.
Poolside Assistant: An IDE extension for VS Code and Visual Studio that facilitates user inputs and model outputs, enabling agent-based workflows and code assistance directly in the development environment.
Poolside Console: The web-based administrative interface for Poolside. Previously known as the Admin Dashboard.
Poolside Splash CLI: The legacy command-line interface (CLI) for administering and managing Poolside.
Poolside Web Assistant: The browser-based Poolside interface that provides chat capabilities outside the IDE.
RAG engine: Retrieval augmented generation layer that orchestrates the selection and injection of additional context to the Poolside model.