Skip to main content

​
Introduction

Poolside’s partnership with AWS allows customers to benefit from the capacity of running Poolside’s models in Amazon Bedrock. This allows any customer to manage inference in a simplified manner, while benefiting from the fact that they can scale inference to 0.

​
Architecture

The following diagram illustrates the Poolside infrastructure-level architecture after a successful deployment: Architecture To keep customer data secure, Poolside deploys in the customer infrastructure. Poolside automatically sets up Amazon Bedrock through an AWS PrivateLink with the correct Security Groups, keeping the model plane and registry fully secure and isolated.

​
List Existing Models

First, let’s verify what models are currently available in the same region where API is running: 
splash models list --engine bedrock
In case of success, splash should then return an output similar to: 
MODEL_ID     AWS-ARN    VERSION
malibu       <arn>      1.1
point        <arn>      1.0
Note: Splash will return No model is available in region us-west2 when a region doesn’t support Poolside’s models.

​
Testing the API

After successfully loading the model we can also test the API to ensure that everything is working properly. First, you need to get the bearer token for your logged in user. We can do this by checking the Splash config file for the token with a ps prefix. Run: 
splash config edit
Note down the bearer token and then use either cURL or an API testing tool like Postman to hit the core-api. The cURL command can be structured as follows: 
curl --location 'https://<api.domain.com>/v0/prompt' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ps-rest_of_your_token' \
  --data '{
  "prompt": "briefly explain cURL",
  "intent": "chat",
  "context": {
  "elements": []
  }
}'
Note: Remember to insert your own token as the value for the auth header. Detailed documentation on the Poolside API can be found by navigating to http://<api.domain.com>/docs