# AGENTS.md instructions
Source: https://docs.poolside.ai/agent-instructions
Set up instructions for Poolside across personal, project, and directory scopes.
Use `AGENTS.md` files to give Poolside instructions to follow during a session. Add instructions for coding standards, project structure, common commands, response preferences, and constraints.
To learn more about `AGENTS.md` across tools, see [https://agents.md](https://agents.md/).
## How Poolside uses instructions
Poolside uses instructions from personal, project, and directory-level `AGENTS.md` files. When multiple files apply to the same path, Poolside follows this priority order:
| Scope | File | Use it for |
| ------------------------- | ------------------------------------------ | -------------------------------------------------------------------------------- |
| 1. Directory instructions | Nested `AGENTS.md` files in subdirectories | Rules for one part of the codebase, such as frontend, backend, or generated code |
| 2. Project instructions | `AGENTS.md` at the repository root | Project architecture, setup commands, coding standards, and common workflows |
| 3. Personal instructions | `~/.config/poolside/AGENTS.md` by default | Preferences and workflows that should follow you across projects |
At the start of a session, Poolside reads your personal instructions and the `AGENTS.md` files that apply to the working directory. If the working directory is inside a Git repository, Poolside reads `AGENTS.md` files from the Git repository root through the working directory. If the working directory is not inside a Git repository, Poolside treats the working directory as the root for instruction loading.
As Poolside works in different directories, it also reads any nested `AGENTS.md` files that apply to those paths.
In the following example, three `AGENTS.md` files apply when Poolside works in `app/ui/`: `app/ui/AGENTS.md`, `app/AGENTS.md`, and the root `AGENTS.md`. If you start `pool` from `app/`, Poolside reads the root `AGENTS.md` and `app/AGENTS.md` at the start of the session. Because `app/ui/AGENTS.md` is the most specific file, it takes priority for work in `app/ui/`.
```text theme={null}
repo/
├── AGENTS.md # instructions for the whole repository
├── app/
│ ├── AGENTS.md # instructions for all app code
│ └── ui/
│ └── AGENTS.md # instructions for UI code
└── api/
└── AGENTS.md # instructions for API code
```
Because model behavior is not deterministic, responses might not always follow the instruction priority perfectly.
## Set up project instructions
Start with project instructions at the repository root. Use directory instructions only when one part of the codebase needs different rules. Use personal instructions for preferences that should follow you across projects.
1. Create an `AGENTS.md` file at the root of your repository.
2. Add a few clear rules about project structure, coding conventions, and common workflows. See [Example project AGENTS.md](#example-project-agents-md).
3. Refine your instructions if Poolside misses important context or behaves inconsistently.
### Example project AGENTS.md
```markdown theme={null}
## Project overview
This repository contains the main application code. The `src/` directory contains product code, `tests/` contains automated tests, and `docs/` contains user-facing documentation.
## Development
- Check `README.md` for setup steps before installing dependencies.
- Run the relevant test command before finishing a code change.
- Use the existing formatter and linter configuration.
## Conventions
- Keep changes focused on the requested task.
- Add or update tests when behavior changes.
- Follow existing naming and file organization patterns.
```
## Add directory-specific instructions
For rules that apply only to one part of a repository, add a nested `AGENTS.md` file in the relevant directory.
Use directory instructions for areas with different setup steps, commands, conventions, or constraints. For example, you might add separate instructions for frontend code, backend code, generated files, or documentation.
### Example directory AGENTS.md
```markdown theme={null}
## Directory overview
This directory contains frontend code for the application UI.
## Development
- Run frontend tests before finishing changes in this directory.
- Use the existing component patterns.
- Do not edit generated files manually.
## Conventions
- Keep components small and focused.
- Follow existing naming and file organization patterns.
```
## Create personal instructions
Use personal instructions for preferences that are not tied to one repository.
1. Create an `AGENTS.md` file in your Poolside config directory.
By default, Poolside looks for personal instructions at `~/.config/poolside/AGENTS.md` on macOS and Linux. To use a different config directory, set the `XDG_CONFIG_HOME` environment variable.
2. Add personal preferences that should follow you across projects. See [Example personal AGENTS.md](#example-personal-agents-md).
3. Keep project-specific rules in project `AGENTS.md` files instead.
Personal instructions use the same Markdown format as project `AGENTS.md` files. This Markdown file is separate from the `.poolside/` directory that stores settings such as [permissions](/permissions).
### Example personal AGENTS.md
```markdown theme={null}
## Response preferences
- Put the code before the explanation.
- Keep explanations concise unless I ask for more detail.
- When you suggest shell commands, explain any destructive steps.
## Coding preferences
- Prefer immutable patterns over mutation where reasonable.
- Add docstrings to new public functions.
```
## Tips for effective instructions
* Keep instructions short and specific.
* Use Markdown headings to separate project structure, commands, conventions, and workflow rules.
* Include commands Poolside should run before or after common changes.
* Link to existing project documentation instead of duplicating long reference material.
* Keep instructions current. Stale instructions can cause incorrect behavior.
## Understand instruction behavior
* `AGENTS.md` files are guidance, not strict enforcement. For rules that must always apply, such as blocking specific tools, use [permissions](/permissions) instead.
* Poolside skips `AGENTS.md` files in ignored directories, including `.git/`, `node_modules/`, common cache or vendor directories, and paths ignored by repository ignore rules.
* Instructions you type during a session can override `AGENTS.md` instructions.
* Empty `AGENTS.md` files are not sent to the model.
* The model might not follow instructions consistently.
## Related resources
* [Poolside Agent CLI](/cli/pool)
* [Permissions](/permissions)
# OpenAI-compatible API examples
Source: https://docs.poolside.ai/api/openai-api-examples
Make OpenAI-compatible API requests to Poolside models.
This page builds on [Make your first API call](/api/overview#make-your-first-api-call) with examples for listing models, interpreting responses, controlling generation and thinking, streaming responses, and using tools. For request parameters and response schemas, see [List models](/inference-api/models/list-models) and [Create chat completion](/inference-api/chat/create-chat-completion).
The examples use Poolside-hosted inference. For other access methods, see [Access methods](/api/overview#access-methods).
You may need to adapt an example for another access method because base URLs, model IDs, and supported parameters can differ. For OpenRouter, use the sections that name OpenRouter explicitly, and check the `supported_parameters` field in [OpenRouter's model metadata](https://openrouter.ai/api/v1/models) before adapting any other example.
## Prerequisites
Before you get started, you need:
1. Any authentication required by your access method: a [Poolside Platform](https://platform.poolside.ai/) or OpenRouter API key, or the authentication configured for your self-managed model endpoint or gateway. To configure authentication, see [Authentication](/api/overview#authentication).
2. `curl` or another tool that can make API requests.
If API key authentication is off for your model endpoint or gateway, omit the `Authorization` header.
## List available models
Most API requests require a model `id`. For the full request and response reference, see [List models](/inference-api/models/list-models).
```bash title="List models" theme={null}
curl --request GET \
--url https://inference.poolside.ai/v1/models \
--header 'Authorization: Bearer '
```
OpenRouter serves models from many providers, so filter its catalog for Poolside model IDs. This request does not require an API key:
```bash title="List Poolside models on OpenRouter" theme={null}
curl --silent --request GET \
--url https://openrouter.ai/api/v1/models |
jq '[.data[] | select(.id | startswith("poolside/")) | {id, name}]'
```
```json title="Response example" theme={null}
[
{
"id": "poolside/laguna-s-2.1",
"name": "Poolside: Laguna S 2.1"
},
{
"id": "poolside/laguna-s-2.1:free",
"name": "Poolside: Laguna S 2.1 (free)"
}
]
```
OpenRouter may offer free and paid Poolside models with different context lengths. To see current availability in the browser, see [Poolside models on OpenRouter](https://openrouter.ai/poolside).
## Read a chat response
A chat completion returns the model's message and the tokens the request used. The response below is abbreviated for readability:
```json title="Response example" theme={null}
{
"id": "chatcmpl-abc123",
"model": "poolside/laguna-s-2.1",
"object": "chat.completion",
"choices": [
{
"index": 0,
"message": {
"reasoning_content": "The user is asking what cURL is. I should explain that it is a command-line tool, name the protocols it supports, and give the common uses...",
"content": "cURL is a powerful command-line tool used for transferring data to or from a server, supporting various protocols such as HTTP, HTTPS, FTP, and more.",
"role": "assistant"
},
"finish_reason": "stop"
}
],
"usage": {
"completion_tokens": 88,
"prompt_tokens": 447,
"total_tokens": 535
}
}
```
Poolside-hosted inference enables thinking by default. Responses from models that think can include `reasoning_content` in addition to `content`. The `usage` object reports the tokens used by the request and response. Poolside-hosted inference responses do not include `logprobs` or `stop_reason`.
To give the model information it does not have, put that context in the message `content` as ordinary text. For guidance on what to include, see the [Prompting guide](/resources/prompting-best-practices).
## Control generation
Poolside-hosted inference supports these generation parameters:
| Parameter | Default | Documented range |
| ------------- | ------: | ------------------------- |
| `max_tokens` | `32768` | From `1` through `262144` |
| `temperature` | `1.0` | From `0` through `2` |
| `top_k` | `20` | Not documented |
| `min_p` | `0` | Not documented |
Include only the values you want to override:
```bash title="Set generation parameters" theme={null}
curl --request POST \
--url https://inference.poolside.ai/v1/chat/completions \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ' \
--data '{
"model": "poolside/laguna-s-2.1",
"messages": [
{
"role": "user",
"content": "Explain cURL"
}
],
"max_tokens": 4096,
"temperature": 0.2,
"top_k": 20,
"min_p": 0
}'
```
## Turn off thinking
Poolside-hosted inference enables thinking by default. To turn it off for a request, set `chat_template_kwargs.enable_thinking` to `false`:
```bash title="Turn off thinking" theme={null}
curl --request POST \
--url https://inference.poolside.ai/v1/chat/completions \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ' \
--data '{
"model": "poolside/laguna-s-2.1",
"messages": [
{
"role": "user",
"content": "What are channels in Go?"
}
],
"chat_template_kwargs": {
"enable_thinking": false
}
}'
```
The response returns the answer in `content` and sets `reasoning_content` to `null`.
## Preserve reasoning in agentic workflows
For agentic workflows with Poolside models, preserve `reasoning_content` from assistant responses when you include those responses in follow-up requests. Dropping previous reasoning content can prevent the model from reasoning in later steps. For examples, see the [Control reasoning](https://huggingface.co/poolside/Laguna-S-2.1#controlling-reasoning) sections on the Laguna model pages.
## Stream responses
To receive the response as a series of chunks returned as server-sent events, set `stream` to `true`. This is useful for real-time applications. Poolside-hosted inference includes a running token usage total on every chunk and the completed total on the final chunk. Setting `stream_options.include_usage` to `false` does not suppress these totals:
```bash title="Stream chat prompt" theme={null}
curl --request POST \
--url https://inference.poolside.ai/v1/chat/completions \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ' \
--data '{
"messages": [
{
"content": "Explain cURL",
"role": "user"
}
],
"model": "poolside/laguna-s-2.1",
"stream": true
}'
```
Generation chunks carry a `delta` object. When a model produces reasoning, the stream returns `reasoning_content` before `content`. The stream ends with a chunk that has an empty `choices` array and the completed `usage` object. Chunks below are abbreviated for readability; each also includes `created` and `model`:
```text title="Response example" theme={null}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","reasoning_content":"The user"}}],"usage":{"prompt_tokens":447,"completion_tokens":1,"total_tokens":448}}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" wants"}}],"usage":{"prompt_tokens":447,"completion_tokens":2,"total_tokens":449}}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"cURL"}}],"usage":{"prompt_tokens":447,"completion_tokens":45,"total_tokens":492}}
...
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":447,"completion_tokens":88,"total_tokens":535}}
data: [DONE]
```
## Extend models with tools
You can extend a model's capabilities by providing tools that it can call during a conversation. This lets the model retrieve real-time data, run calculations, or interact with external systems.
Poolside-hosted inference allows parallel tool calls by default. Set `parallel_tool_calls` to `false` when you want the model to call at most one tool in a response.
Tool calling takes two requests. First, include a `tools` array and define the functions the model can call:
```bash title="Define a tool" theme={null}
curl --request POST \
--url https://inference.poolside.ai/v1/chat/completions \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ' \
--data '{
"model": "poolside/laguna-s-2.1",
"messages": [
{
"role": "user",
"content": "what is the weather forecast for San Francisco"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_forecast",
"description": "Get weather forecast for a city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name"
}
},
"required": [
"city"
],
"additionalProperties": false
}
}
}
]
}'
```
When the model needs a tool, it returns a `tool_calls` array and a `finish_reason` of `tool_calls`. The `arguments` value is a JSON string:
```json title="Response example" theme={null}
{
"id": "chatcmpl-abc123",
"model": "poolside/laguna-s-2.1",
"created": 1753997542,
"object": "chat.completion",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"tool_calls": [
{
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_forecast",
"arguments": "{\"city\": \"San Francisco\"}"
}
}
]
},
"finish_reason": "tool_calls"
}
],
"usage": {
"completion_tokens": 95,
"prompt_tokens": 338,
"total_tokens": 433
}
}
```
Second, run the tool yourself and send the result back. Include the original user message, the assistant message containing the `tool_calls`, and a `tool` message whose `tool_call_id` matches.
Copy the assistant message from the previous response rather than rebuilding it, so that `reasoning_content` travels with it when the model returned it. The assistant message below is abbreviated for readability:
```bash title="Send a tool result" theme={null}
curl --request POST \
--url https://inference.poolside.ai/v1/chat/completions \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ' \
--data '{
"model": "poolside/laguna-s-2.1",
"messages": [
{
"role": "user",
"content": "what is the weather forecast for San Francisco"
},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_forecast",
"arguments": "{\"city\": \"San Francisco\"}"
}
}
]
},
{
"role": "tool",
"tool_call_id": "call_abc123",
"content": "{\"temperature\": 25}"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_forecast",
"description": "Get weather forecast for a city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name"
}
},
"required": [
"city"
],
"additionalProperties": false
}
}
}
]
}'
```
The model then answers using the tool result.
## Control reasoning through OpenRouter
OpenRouter uses its own `reasoning` object rather than `chat_template_kwargs`. To control reasoning effort for OpenRouter-compatible models, include a `reasoning` object:
```json theme={null}
{
"model": "",
"messages": [
{
"content": "Explain cURL",
"role": "user"
}
],
"reasoning": {
"effort": "max"
}
}
```
OpenRouter's generic effort values are `max`, `xhigh`, `high`, `medium`, `low`, `minimal`, and `none`, but provider and model support varies. Replace `` and `max` with values that your selected model supports. To check which parameters a model accepts, see the `supported_parameters` field in [OpenRouter's model metadata](https://openrouter.ai/api/v1/models).
The `reasoning` field is OpenRouter-style: a top-level object, which differs from the OpenAI Chat Completions `reasoning_effort` parameter. Use it only with OpenRouter or another provider that accepts this field. A direct connection to the OpenAI Chat Completions API at `https://api.openai.com/v1/chat/completions` rejects the `reasoning` field.
Effort-level support varies by provider and model. Some providers ignore effort settings or apply thinking automatically. Check your provider's documentation for how it handles the `reasoning` field.
# Poolside API
Source: https://docs.poolside.ai/api/overview
Choose an access method, authenticate, and send your first request to the Poolside OpenAI-compatible API.
Call Poolside models through an OpenAI-compatible API. This page covers access methods, available endpoints, authentication, and your first request with cURL or an OpenAI-compatible SDK.
## Access methods
The API base URL depends on how you access Poolside.
| Access method | Base URL |
| ------------------------------- | ------------------------------------------------------------------------------ |
| Poolside-hosted inference | `https://inference.poolside.ai/v1` |
| Self-managed Poolside inference | `https:///v1`, or the base URL exposed by an inference gateway |
| OpenRouter | `https://openrouter.ai/api/v1` |
## Available endpoints
Poolside-hosted inference supports these endpoints:
| Method | Endpoint | What it's for |
| ------ | -------------------------------------------------------------------- | --------------------------------------- |
| `GET` | [`/v1/models`](/inference-api/models/list-models) | List available models |
| `POST` | [`/v1/chat/completions`](/inference-api/chat/create-chat-completion) | Generate a response from a conversation |
## Authentication
### Get an API key
Get the API key for your access method:
* **Poolside-hosted inference**: Sign in to [Poolside Platform](https://platform.poolside.ai/), open the **API Keys** tab, and click **New key**.
* **OpenRouter**: Go to [OpenRouter API keys](https://openrouter.ai/keys), sign in, and create an API key.
* **Self-managed Poolside inference**: Use the API key configured for your model endpoint or inference gateway. If API key authentication is off, you do not need a key.
### Send the API key
Poolside-hosted inference and OpenRouter require an API key. Send it as a Bearer token in the `Authorization` header:
```text theme={null}
Authorization: Bearer
```
Authentication for self-managed Poolside inference depends on how your endpoint or gateway is configured. If API key authentication is off, omit the `Authorization` header.
API keys are secrets. Store them securely and never commit them to source control.
## Make your first API call
This quickstart uses Poolside-hosted inference and the [`poolside/laguna-s-2.1`](/api/openai-api-examples#list-available-models) model ID.
**Prerequisites**
* An API key from [Poolside Platform](https://platform.poolside.ai/)
* `curl`, Python, or Node.js
**Steps**
Export your API key as an environment variable. The cURL command below reads it from `$POOLSIDE_API_KEY`.
```bash theme={null}
export POOLSIDE_API_KEY=""
```
Send a `POST` request to the Chat Completions endpoint:
```bash theme={null}
curl https://inference.poolside.ai/v1/chat/completions \
-H "Authorization: Bearer $POOLSIDE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "poolside/laguna-s-2.1",
"messages": [{ "role": "user", "content": "Explain cURL" }]
}'
```
Poolside returns a JSON response containing the message:
```json title="Response example" theme={null}
{
"id": "chatcmpl-abc123",
"model": "poolside/laguna-s-2.1",
"created": 1751993576,
"object": "chat.completion",
"choices": [
{
"index": 0,
"message": {
"reasoning_content": "The user is asking what cURL is. I should explain that it is a command-line tool, name the protocols it supports, and give the common uses...",
"content": "cURL is a powerful command-line tool used for transferring data to or from a server, supporting various protocols such as HTTP, HTTPS, FTP, and more. It's widely used for testing APIs, downloading files, and automating HTTP requests. cURL allows you to specify headers, methods (GET, POST, PUT, DELETE, etc.), and data payloads, making it versatile for a range of web-related tasks.\n",
"role": "assistant"
},
"finish_reason": "stop"
}
],
"usage": {
"completion_tokens": 88,
"prompt_tokens": 447,
"total_tokens": 535
}
}
```
Poolside-hosted inference enables thinking by default, so the response can include `reasoning_content` in addition to `content`. See [OpenAI-compatible API examples](/api/openai-api-examples#turn-off-thinking) to turn it off.
Export your API key as an environment variable.
```bash theme={null}
export POOLSIDE_API_KEY=""
```
```bash theme={null}
pip install openai
```
Create a file called `quickstart.py`. The code below reads your key from `POOLSIDE_API_KEY`:
```python title="quickstart.py" theme={null}
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["POOLSIDE_API_KEY"],
base_url="https://inference.poolside.ai/v1",
)
completion = client.chat.completions.create(
model="poolside/laguna-s-2.1",
messages=[{"role": "user", "content": "Explain cURL"}],
)
print(completion.choices[0].message.content)
```
```bash theme={null}
python quickstart.py
```
Export your API key as an environment variable.
```bash theme={null}
export POOLSIDE_API_KEY=""
```
`tsx` runs the TypeScript file directly.
```bash theme={null}
npm install openai tsx
```
Create a file called `quickstart.ts`. The code below reads your key from `POOLSIDE_API_KEY`:
```typescript title="quickstart.ts" theme={null}
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.POOLSIDE_API_KEY,
baseURL: "https://inference.poolside.ai/v1",
});
const completion = await client.chat.completions.create({
model: "poolside/laguna-s-2.1",
messages: [{ role: "user", content: "Explain cURL" }],
});
console.log(completion.choices[0].message.content);
```
```bash theme={null}
npx tsx quickstart.ts
```
To use OpenRouter instead, use `https://openrouter.ai/api/v1` as the base URL or SDK `base_url`/`baseURL`, pass your OpenRouter API key, and replace the model ID with an OpenRouter model ID.
## Use the API for CLI automation
For interactive Poolside Agent CLI access, see [Log in to Poolside](/get-started/log-in).
For CLI automation against an OpenAI-compatible endpoint, configure `POOLSIDE_STANDALONE_BASE_URL`, `POOLSIDE_STANDALONE_MODEL`, and `POOLSIDE_API_KEY`. If the endpoint does not validate API keys, use any non-empty value for `POOLSIDE_API_KEY`, such as `local-test`. See [Automate tasks](/cli/automated-mode#basic-usage).
## Next steps
* [OpenAI-compatible API examples](/api/openai-api-examples) for generation controls, thinking control, streaming, and tool calling
* [Supported models](/get-started/supported-models) for model guidance, context windows, and modes
* [Editors](/tools#editors) to use Poolside from editors and editor extensions
* [Desktop apps](/tools#desktop-apps) to use Poolside from desktop apps
# Automate tasks
Source: https://docs.poolside.ai/cli/automated-mode
Use `pool exec` to run one-shot tasks in scripts, CI, and automated workflows.
Run `pool exec` to send a single prompt and exit when the task is complete. Use this for scripts, CI pipelines, and one-off tasks where you don't need a back-and-forth.
For interactive work, see [Work from the terminal](/cli/interactive-mode) instead.
This documentation describes Poolside Agent CLI v1.0.16. Check your version with `pool --version`. To update, exit any active session and run `pool update` from your terminal. See [Poolside Agent CLI releases on GitHub](https://github.com/poolsideai/pool/releases) for release history.
## Basic usage
In a non-interactive environment without credentials saved by `pool login`, choose how you access Poolside and set the variables shown.
```bash theme={null}
POOLSIDE_API_KEY= \
POOLSIDE_STANDALONE_BASE_URL=https://inference.poolside.ai \
pool exec -p "What does this codebase do?"
```
To choose a model instead of using the default, also set `POOLSIDE_STANDALONE_MODEL` to its model ID.
```bash theme={null}
POOLSIDE_API_KEY= \
POOLSIDE_STANDALONE_BASE_URL= \
POOLSIDE_STANDALONE_MODEL= \
pool exec -p "What does this codebase do?"
```
For self-managed Poolside inference, use `https:///v1` for a direct model endpoint. For an inference gateway, use the base URL it exposes; the documented Bifrost and LiteLLM configurations use `https:///v1`. For another provider, use its OpenAI-compatible base URL, including any required path such as `/api/v1`.
If the endpoint does not validate API keys, use any non-empty value for `POOLSIDE_API_KEY`, such as `local-test`.
This example selects a model explicitly because availability differs by endpoint. You can omit `POOLSIDE_STANDALONE_MODEL` when the default model is available from your endpoint.
The remaining examples assume that you signed in with `pool login` or set the required environment variables.
Pass a prompt inline:
```bash theme={null}
pool exec -p "What does this codebase do?"
```
Read the prompt from a file:
```bash theme={null}
pool exec -f prompt.txt
```
Pipe the prompt from standard input:
```bash theme={null}
pool exec < prompt.txt
```
Use `-` to read from standard input when combining with other flags:
```bash theme={null}
echo "Summarize this file" | pool exec -p -
```
## Pass MCP server input variables
If an MCP server needs input from environment variables, start `pool exec` with those variables set:
```bash theme={null}
KEY=VALUE pool exec -p "test"
```
## Run in a specific directory
By default, `pool exec` uses your current directory. Use `-d` to point it somewhere else:
```bash theme={null}
pool exec -p "Review this project" -d
```
## Continue from a previous run
Resume from the last run:
```bash theme={null}
pool exec --continue -p "Now add tests for what you just wrote"
```
Resume a specific run by ID:
```bash theme={null}
pool exec --continue= -p "Now add tests for what you just wrote"
```
To find recent run IDs, use `pool history logs`.
## Output format
By default, `pool exec` prints Markdown. Use `-o json` for newline-delimited JSON:
```bash theme={null}
pool exec -p "List the exported functions in this file" -o json
```
JSON mode prints event-style records. Common types:
* `reasoning`: Raw model reasoning
* `thought`: Agent message text
* `toolCall`: Tool invocations and arguments
* `toolCallResult`: Tool results
* `oauth_url`: Browser authentication required
Use JSON mode when another tool needs to consume the output.
## Run without approval prompts
Use `--unsafe-auto-allow` when you want `pool exec` to run in automated mode without approval prompts:
```bash theme={null}
pool exec -p "Review this repository for security issues" --unsafe-auto-allow
```
Use this only in trusted non-interactive environments. Explicit deny rules still apply.
For persistent approval rules, path rules, and `settings.yaml` locations, see [Permissions](/permissions).
## Override sandbox usage
Use `--sandbox required` to require a configured sandbox for the run:
```bash theme={null}
pool exec -p "Review this repository" --sandbox required
```
Use `--sandbox disabled` only when you want to run without a configured sandbox.
## Related resources
* [Work from the terminal](/cli/interactive-mode)
* [CLI reference](/cli/cli-reference)
# CLI reference
Source: https://docs.poolside.ai/cli/cli-reference
Reference for current `pool` commands, flags, slash commands, and exit codes.
Reference for current `pool` commands, user-facing flags, slash commands, and exit codes. For an overview and usage guidance, start with [Poolside Agent CLI](/cli/pool).
This documentation describes Poolside Agent CLI v1.0.16. Check your version with `pool --version`. To update, exit any active session and run `pool update` from your terminal. See [Poolside Agent CLI releases on GitHub](https://github.com/poolsideai/pool/releases) for release history.
## pool
`pool` starts an interactive session by default. Arguments after `--` are forwarded to the configured agent server.
By default, `pool` uses the credentials saved by `pool login`. To authenticate one invocation with an API key, set `POOLSIDE_API_KEY` before the command. Set `POOLSIDE_STANDALONE_BASE_URL` to override an OpenAI-compatible endpoint.
```bash theme={null}
POOLSIDE_API_KEY= pool
```
If your MCP servers expect environment variables, start `pool` with those variables set.
```bash theme={null}
KEY=VALUE pool
```
| Flag | Short flag | Description |
| --------------------------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--directory` | `-C` | Working directory for the interactive session |
| `--worktree []` | `-w` | Run the session in a Git worktree for the named branch, creating the worktree and branch if needed. Use the flag without a branch name to generate a worktree name. |
| `--resume` | `-r` | Resume a previous session by ID, or use `-r` alone to open the session picker. In the picker, press `Tab` to switch between current-directory sessions and sessions from all directories. |
| `--model` | `-m` | Override the saved model preference for the interactive session |
| `--mode ` | | Override the saved approval mode for the interactive session. The Poolside agent server supports `default`, `accept-edits`, `auto`, and `always-allow`; `auto` requires an [Auto mode classifier](/permissions#auto-mode). This flag does not select Build or Plan. Other ACP servers can provide different values. |
| `--agent-server []` | `-s` | Agent server entry, command, or remote ACP URL to use. Use `pool -s` to open the agent server picker. |
| `--sandbox ` | | Override sandbox usage for the Poolside agent server. Supported values: `required` and `disabled`. |
| `--prompt-queue ` | `-q` | Queue a prompt to send after the interactive session and agent connection finish initializing. Repeat to queue multiple prompts. |
| `--help` | `-h` | Show help for `pool` and exit |
| `--version` | `-v` | Show the current `pool` version and exit |
Use `pool --help` to show help for a subcommand.
For keyboard shortcuts, see [Work from the terminal](/cli/interactive-mode#keyboard-shortcuts).
With the Poolside agent server, press `Enter` while a turn is running to steer a prompt into that turn. Press `Ctrl+Enter` to queue it for the next turn when your terminal supports key disambiguation. Shell input and slash commands do not steer.
## pool exec
`pool exec` runs a single prompt non-interactively and then exits. Provide the prompt with `--prompt`, `--prompt-file`, or standard input.
Files passed after `--` are added as context for the run.
To authenticate `pool exec` in a non-interactive environment, set `POOLSIDE_API_KEY` before the command. For complete setup examples, see [Automate tasks](/cli/automated-mode#basic-usage).
```bash theme={null}
POOLSIDE_API_KEY= pool exec -p "test"
```
If your MCP servers expect environment variables, start `pool exec` with those variables set.
```bash theme={null}
KEY=VALUE pool exec -p "test"
```
| Flag | Short flag | Description |
| --------------------- | ---------- | --------------------------------------------------------------------------------------------------- |
| `--prompt` | `-p` | Prompt text. Use `-` to read the prompt from standard input. |
| `--prompt-file` | `-f` | File containing the prompt |
| `--directory` | `-d` | Working directory to operate in. Defaults to the current directory. |
| `--output` | `-o` | Output format: `markdown` or `json`. JSON output is newline-delimited JSON. |
| `--unsafe-auto-allow` | | Automatically approve tool actions without confirmation |
| `--verbose` | | Print verbose tool result output |
| `--sandbox ` | | Override sandbox usage. Supported values: `required` and `disabled`. |
| `--continue` | | Continue a previous conversation. Provide a run ID, or use the flag alone to continue the last run. |
### Exit codes
| Code | Meaning |
| ----- | -------------------------------------------------------------- |
| `0` | Task completed successfully |
| `4` | The agent ran but reported that it could not complete the task |
| Other | Unexpected CLI or request error |
## pool acp
`pool acp` starts Poolside's Agent Client Protocol (ACP) server over standard input and standard output.
If your MCP servers expect environment variables, start `pool acp` with those variables set.
| Flag | Short flag | Description |
| ----------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `--sandbox ` | | Override sandbox usage. Supported values: `required` and `disabled`. |
| `--settings ` | | Apply extra settings on top of workspace and global settings. Provide a path to a YAML settings file or inline YAML content. |
| `--version` | `-v` | Show the current `pool acp` version and exit |
```bash theme={null}
KEY=VALUE pool acp
```
Configure thought level through ACP session config options when your editor exposes them.
Available thought levels depend on your provider and model.
* Poolside-hosted inference offers `max` and `none`. `max` turns thinking on and is the default. `none` turns thinking off.
* With OpenRouter, available choices come from the selected model's metadata. If `default` appears, it clears your override and lets the provider choose.
* Self-managed endpoints and other OpenAI-compatible providers do not expose thought-level choices through `pool`. Use the reasoning controls your model server supports.
### pool acp serve
`pool acp serve` is experimental. Its flags and behavior can change in any release.
`pool acp serve` serves the Poolside ACP agent over a Streamable HTTP network transport on the configured listen address instead of standard input and output. One agent instance starts for each inbound connection.
`pool acp serve` inherits the agent-configuration flags from `pool acp`, such as `--sandbox` and `--settings`.
```bash theme={null}
pool acp serve --host 127.0.0.1 --port 3284
```
When you connect with `pool --agent-server`, use the `/acp` endpoint over Streamable HTTP. With the default host and port, the remote ACP URL is `http://localhost:3284/acp`.
| Flag | Description |
| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| `--host ` | Host interface to bind. Defaults to `127.0.0.1`. Use `0.0.0.0` to accept connections from any interface. |
| `--port ` | TCP port to bind. Defaults to `3284`. |
| `--access-log ` | HTTP access log destination. Supported values are `stderr`, `stdout`, `-`, `off`, or a file path. Defaults to `stderr`. |
### pool acp setup
Use `pool acp setup` to configure Zed or JetBrains to use Poolside.
| Flag | Description |
| ------------------- | ---------------------------------------------------------------------- |
| `--editor ` | Editor to configure. Required. Supported values: `zed` and `jetbrains` |
### pool acp logs
`pool acp logs` reads ACP debug logs from the Poolside log directory for the current working directory.
| Flag | Short flag | Description |
| ----------- | ---------- | ----------------------------------- |
| `--follow` | `-f` | Follow log output |
| `--pretty` | `-p` | Pretty-print log output |
| `--session` | | Show logs for a specific session ID |
## Setup and authentication
### pool login
`pool login` runs the interactive login flow. For self-managed Poolside inference or an inference gateway, choose **Connect an OpenAI-compatible provider**. Enter `https:///v1` for a direct model endpoint. For a gateway, enter the base URL it exposes; the documented Bifrost and LiteLLM configurations use `https:///v1`.
| Flag | Description |
| ----------------- | ------------------------------------------------ |
| `--api-key ` | Configure standalone mode with the given API key |
### pool logout
`pool logout` removes locally stored credentials for the selected API URL. For OpenAI-compatible connections, revoke or rotate the API key through the system that issued it.
| Flag | Description |
| ----------------- | -------------- |
| `--api-url ` | API URL to use |
## Configuration and updates
### pool config
Prints the log, trajectory, and config directories, plus the credentials path.
### pool config settings
Opens `settings.yaml` in `VISUAL`, `EDITOR`, or `vi`, validates it after you exit the editor, and then saves it.
For approval rules, path rules, and sandbox configuration, see [Permissions](/permissions).
### pool update
`pool update [version]` updates the CLI to the latest version, or to a specific version when you provide one.
| Flag | Description |
| --------- | ---------------------------------------------------------- |
| `--force` | Reinstall even if the CLI is already on the target version |
## Slash commands
Slash command availability depends on where you use `pool`.
### Interactive slash commands
Use these commands in an interactive `pool` session.
| Command | What it does |
| --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `/model` | Open the model selector for the current session |
| `/mode []` | Open the approval mode selector, or set an approval mode. The Poolside agent server supports `default`, `accept-edits`, `auto`, and `always-allow`; `auto` requires an [Auto mode classifier](/permissions#auto-mode). Other ACP servers can provide different values. |
| `/agent-mode [build\|plan]` | Open the agent mode selector, or switch between Build and Plan. Use `/collaboration-mode` as an alias. Available only when the connected agent server provides agent modes. |
| `/thought-level []` | Open the thought-level selector, or set a thought level. Use `/effort` as an alias. Available only when the connected agent server provides thought-level choices. |
| `/new` or `/clear` | Clear conversation history and start a new session. The new session keeps the current model and approval mode. Select the agent mode again if needed. |
| `/copy` | Copy the last agent response to the clipboard |
| `/delete` | Select and delete saved sessions. Unavailable while a turn is in progress. |
| `/rewind` | Roll back to a previous turn |
| `/resume` | Open the session picker and switch to a previous session |
| `/rename` | Rename the current session |
| `/move` | Move the session to another Git worktree, or create one. If the current worktree has uncommitted changes, choose whether to move those files too. |
| `/set-option ` | Set a session configuration option by ID on the connected ACP agent |
| `/system` | Show the system prompt for the current session when available |
| `/feedback` | Open a feedback draft and optionally attach logs |
| `/logs` | Archive debug logs for the current session |
| `/logout` | Log out through the connected agent server and exit the session. Available only when the server supports logout. |
| `/quit` or `/exit` | Exit the session |
| `/debug:dump` | Write the raw agent-server message log to a local JSON file |
When you use the Poolside agent server, these additional commands are available in interactive mode:
| Command | What it does |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `/plan` | Switch to plan mode |
| `/compact []` | Compact conversation context. Add guidance after the command to tell the agent what to preserve, such as `/compact preserve tool call errors`. |
| `/share` | Get a link to the trajectory viewer for the current session |
| `/mcp` | Show MCP servers, connection status, and tools for the current session |
| `/sandbox` | Show local sandbox configuration |
| `/sandbox-apply-to-host` | Review pending sandbox filesystem changes and apply them to the host workspace when available |
| `/skills` | Refresh and list available skills, including any skill load errors |
| `/usage` | Show token usage, context window state, and session cost when available. With [subagents](/subagents#review-subagent-usage), include parent, per-subagent, and total usage. |
In interactive `pool`, skills use the `$` skill picker instead of the slash-command menu. Type `$` at the start of the prompt or after a space to open the skill picker and add a skill reference to your prompt.
### ACP slash commands
When your editor passes slash commands to `pool acp`, these commands are available:
| Command | What it does |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `/plan` | Switch to plan mode when plan mode is available |
| `/clear` | Clear conversation history and free up context |
| `/compact []` | Compact conversation context. Add guidance after the command to tell the agent what to preserve, such as `/compact preserve tool call errors`. |
| `/share` | Get the trajectory viewer URL for the current session when trajectory sharing is available |
| `/rename` | Rename the current session |
| `/mcp` | Show MCP servers, connection status, and tools for the current session |
| `/sandbox` | Show local sandbox configuration |
| `/sandbox-apply-to-host` | Review pending sandbox filesystem changes and apply them to the host workspace when available |
| `/usage` | Show token usage, context window state, and session cost for the current session when available. With [subagents](/subagents#review-subagent-usage), include parent, per-subagent, and total usage. |
| `/skills` | Refresh and list available skills, including any skill load errors |
Compatible ACP clients can expose available skills from `pool acp` as slash commands. The available skill commands depend on the skills configured for the agent and workspace.
## History
### pool history logs
Use `pool history logs []` to list recent log files or show one matching file.
| Flag | Short flag | Description |
| ---------- | ---------- | ---------------------------------------------------------------------- |
| `--all` | `-a` | Show all log files instead of the most recent 20 |
| `--latest` | `-l` | Show the most recent log file and write its filename to standard error |
| `--pretty` | `-p` | Pretty-print log contents with colors and formatting |
| `--follow` | `-f` | Follow log output like `tail -f` |
### pool history trajectories
Use `pool history trajectories []` to list recent trajectory files or show one matching file.
Use `--atif` when you need to convert a local trajectory to Agent Trajectory Interchange Format (ATIF) JSON for external tooling.
| Flag | Short flag | Description |
| ---------- | ---------- | ----------------------------------------------------------------------------------------------------- |
| `--all` | `-a` | Show all trajectory files instead of the most recent 20 |
| `--latest` | `-l` | Show the most recent trajectory file |
| `--atif` | | Render a single trajectory in ATIF JSON format. Use with `--latest` or part of a trajectory filename. |
| `--pretty` | `-p` | Pretty-print ATIF JSON. Only applies with `--atif`. |
### pool history sessions
`pool history sessions` lists recent sessions.
| Flag | Short flag | Description |
| ------- | ---------- | ----------------------------------------------- |
| `--all` | `-a` | Show all sessions instead of the most recent 20 |
```bash theme={null}
pool history sessions
pool history logs --latest --pretty
pool history trajectories --latest
```
## MCP servers and secrets
### pool mcp list
Lists configured MCP servers. Sensitive header values and environment values are masked in the output.
### pool mcp get
Use `pool mcp get ` to inspect one MCP server configuration. Sensitive header values and environment values are masked in the output.
### pool mcp remove
Use `pool mcp remove ` to remove an MCP server from `settings.yaml`.
### pool mcp add
Use `pool mcp add [command] [args...]` to add an MCP server.
Use `--transport` to add a remote server. Without `--transport`, `pool` expects a command-based server and requires a command after `--`.
| Flag | Short flag | Description |
| -------------------- | ---------- | ---------------------------------------------------------------- |
| `--transport ` | `-t` | Transport type for remote servers: `http` or `sse` |
| `--env ` | `-e` | Environment variable to store with the server. Repeat as needed. |
| `--header ` | `-H` | HTTP header for HTTP or SSE transport. Repeat as needed. |
Examples:
```bash theme={null}
# Command-based server over stdio
pool mcp add filesystem -- node filesystem-server.js
# Remote HTTP server
pool mcp add --transport http notion https://mcp.notion.com/mcp
# Remote SSE server
pool mcp add --transport sse linear https://mcp.linear.app/sse
# Pass environment variables or HTTP headers
pool mcp add --env API_KEY= myserver -- npx -y myserver-mcp
pool mcp add --transport http --header "Authorization: Bearer $TOKEN" svc https://example.com/mcp
# Inspect and remove
pool mcp list
pool mcp get
pool mcp remove
```
`pool` stores MCP server configuration under `mcp_servers` in `settings.yaml`. To share servers with a project, add them to `.poolside/settings.yaml`. To keep them personal across projects, add them to `~/.config/poolside/settings.yaml`.
### pool secrets list
Lists stored secrets and their source.
### pool secrets add
Use `pool secrets add ` to store a secret in the system keychain.
| Flag | Short flag | Description |
| ---------------------- | ---------- | -------------------------- |
| `--description ` | `-d` | Description for the secret |
### pool secrets edit
Use `pool secrets edit ` to update a stored secret.
| Flag | Short flag | Description |
| ---------------------- | ---------- | ---------------------- |
| `--name ` | | Rename the secret |
| `--description ` | `-d` | Update the description |
### pool secrets get
Use `pool secrets get ` to inspect a stored secret.
| Flag | Description |
| -------------- | ---------------------------- |
| `--show-value` | Show the stored secret value |
### pool secrets delete
Use `pool secrets delete ` to remove a secret from the keychain.
## Related resources
* [Poolside Agent CLI](/cli/pool)
* [Work from the terminal](/cli/interactive-mode)
* [Automate tasks](/cli/automated-mode)
* [Hooks](/hooks)
* [Subagents](/subagents)
# Install Poolside Agent CLI
Source: https://docs.poolside.ai/cli/install
Install `pool` and sign in.
Install the `pool` command to use Poolside from the terminal or in CI.
This documentation describes Poolside Agent CLI v1.0.16. Check your version with `pool --version`. To update, exit any active session and run `pool update` from your terminal. See [Poolside Agent CLI releases on GitHub](https://github.com/poolsideai/pool/releases) for release history.
## Prerequisites
* On macOS or Linux, `curl` or `wget`, and `tar`
* BETA On Windows, PowerShell and `tar`
## Install and authenticate
Run either command:
```bash theme={null}
curl -fsSL https://downloads.poolside.ai/pool/install.sh | sh
```
```bash theme={null}
wget -qO- https://downloads.poolside.ai/pool/install.sh | sh
```
The installer asks you to accept the Poolside End User License Agreement. In headless environments, run `export POOL_INSTALL_ACCEPT_EULA=1` before you run the installer.
The script installs `pool` to `~/.local/bin` by default unless `POOL_INSTALL_DIR` or `XDG_BIN_HOME` is set. If that directory is already on your PATH, the CLI is ready to use immediately. Otherwise, the installer either prompts to add it to your shell config or prints instructions to add it manually. You can rerun with `POOL_INSTALL_UPDATE_PATH=1` to update your shell config automatically.
By default, the installer places `pool.exe` in `%LOCALAPPDATA%\Programs\pool\bin` unless `POOL_INSTALL_DIR` is set. It also adds the install directory to your user PATH. To skip the automatic PATH update, set `POOL_INSTALL_UPDATE_PATH=0` before running the installer.
In PowerShell, run:
```powershell theme={null}
irm https://downloads.poolside.ai/pool/install.ps1 | iex
```
The installer asks you to accept the Poolside End User License Agreement. In headless environments, set `$env:POOL_INSTALL_ACCEPT_EULA = "1"` in the same PowerShell session before you run the installer.
Before you continue:
* If the installer indicates that the current session needs a PATH update, restart PowerShell or refresh PATH:
```powershell theme={null}
$env:Path = [System.Environment]::GetEnvironmentVariable('Path', 'User') + ';' + [System.Environment]::GetEnvironmentVariable('Path', 'Machine')
```
* If the installer says `pool` is not on your PATH, follow the printed instructions to add the install directory to PATH.
After PATH is refreshed, continue to authentication. If your shell does not resolve `.exe` commands, run the CLI as `pool.exe` instead of `pool`.
Choose the sign-in method that matches your setup. For help choosing a login option, see [Log in to Poolside](/get-started/log-in).
Use this for the fastest way to get free developer access to models hosted by Poolside.
```bash theme={null}
pool login
```
Choose **Use Poolside Platform for free (recommended)**. This opens `platform.poolside.ai` in your browser. Copy the API key that Poolside creates for you, then paste it into the terminal.
Use this if your organization has a Poolside deployment.
```bash theme={null}
pool login --api-url
```
Replace `` with the API URL from your Poolside administrator. Do not use the deployment's `/openai/v1` API base URL. Choose browser login or provide an API token. To sign in again later, run `pool login`.
Use this if you already use OpenRouter or need paid access to Poolside models.
```bash theme={null}
pool login
```
Choose **Use your OpenRouter account**. Enter an OpenRouter API key from [OpenRouter API keys](https://openrouter.ai/keys).
Use this for self-managed Poolside inference, an inference gateway such as Bifrost or LiteLLM, another provider, or your own model server.
```bash theme={null}
pool login
```
Choose **Connect an OpenAI-compatible provider**. For self-managed Poolside inference, enter `https:///v1` for a direct model endpoint. For an inference gateway, enter the base URL it exposes; the documented Bifrost and LiteLLM configurations use `https:///v1`. Enter the API key configured for that endpoint. If your endpoint does not validate API keys, enter any non-empty value, such as `local-test`.
If you run a local model server or use a provider that does not list models from its API, set the model name before starting `pool`:
```bash theme={null}
POOLSIDE_STANDALONE_MODEL= pool
```
```bash theme={null}
pool --version
```
Then choose the workflow you want to use:
* [Work from the terminal](/cli/interactive-mode)
* [Automate tasks](/cli/automated-mode)
* [Use Poolside in another ACP-compatible editor](/tools/other-acp)
For `pool` CLI bugs and feature requests, open an issue on [GitHub](https://github.com/poolsideai/pool).
## Configure and inspect paths
Run `pool config` to see where the CLI stores configuration, credentials, logs, and trajectories.
Run `pool config settings` to edit `settings.yaml` in your editor. After the editor exits, the CLI validates the file before saving it.
By default, Poolside stores configuration files in `~/.config/poolside`. This includes `settings.yaml`, which stores the saved API URL and other CLI settings, and `credentials.json`, where the CLI stores your saved auth token. If your environment uses a custom configuration directory, `pool` uses that location instead.
For Poolside-hosted inference, self-managed Poolside inference, OpenRouter, or another OpenAI-compatible endpoint, set `POOLSIDE_STANDALONE_BASE_URL` to override the saved base URL.
## CI and automation
Use environment variables instead of stored credentials in non-interactive environments.
| Variable | What it sets |
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------ |
| `POOLSIDE_API_KEY` | API key or token for any Poolside connection |
| `POOLSIDE_STANDALONE_BASE_URL` | OpenAI-compatible API base URL, including any required path such as `/v1` or `/api/v1`; selects standalone mode |
| `POOLSIDE_STANDALONE_CONTEXT_LENGTH` | Optional context length in tokens that `pool` uses to calculate automatic compaction thresholds in standalone mode |
| `POOLSIDE_STANDALONE_MODEL` | Optional model ID that overrides the default for standalone mode |
`pool` checks these before reading from configuration files.
When you set `POOLSIDE_STANDALONE_BASE_URL` for Poolside-hosted inference, either `https://inference.poolside.ai` or `https://inference.poolside.ai/v1` works. These docs use the shorter form.
### Configure standalone context length
In standalone mode, `pool` uses the selected model's context length from the provider's `/v1/models` response to calculate automatic compaction thresholds. Set `POOLSIDE_STANDALONE_CONTEXT_LENGTH` when the provider reports a value that differs from the context window configured on your model server:
```bash theme={null}
POOLSIDE_API_KEY= \
POOLSIDE_STANDALONE_BASE_URL= \
POOLSIDE_STANDALONE_MODEL= \
POOLSIDE_STANDALONE_CONTEXT_LENGTH= \
pool
```
Use a positive integer no greater than your model server's context window. The override does not change that window and applies only when the provider's `/v1/models` response includes the selected model.
To run the CLI or call Poolside models in a GitHub Actions workflow, see [Run Poolside in GitHub Actions](/tools/github-actions).
For authentication, `pool` checks credentials in this order:
* Poolside-hosted inference, self-managed Poolside inference, OpenRouter, or another OpenAI-compatible endpoint: `POOLSIDE_API_KEY`, then `credentials.json`
## Uninstall Poolside Agent CLI
Use these steps to remove the `pool` command and, if needed, saved local configuration and state.
If your environment uses custom configuration, log, or trajectory directories, run `pool config` before you start and note the paths.
1. To remove stored authentication credentials, log out:
```bash theme={null}
pool logout
```
`pool logout` removes locally stored credentials. For Poolside Platform, OpenRouter, model endpoint, or gateway connections, revoke or rotate the API key through the system that issued it.
2. Remove the installed binary. By default, it is at `~/.local/bin/pool` on macOS or Linux, or `%LOCALAPPDATA%\Programs\pool\bin\pool.exe` on Windows, unless `POOL_INSTALL_DIR` was set during installation. On macOS or Linux, `XDG_BIN_HOME` can also change the default location.
To locate your copy, run `command -v pool` on macOS or Linux, or `Get-Command pool` on Windows. If the command reports a path, remove that file. If it reports an alias or function, remove the alias or function from your shell configuration or PowerShell profile. On macOS or Linux, if `ls -l ` shows the file is a symlink, remove its target too.
3. Remove saved local configuration (optional). Remove the config directory from the `pool config` output you noted earlier. If you did not note a custom config directory, remove the default config directory: `~/.config/poolside` on macOS or Linux, or `%USERPROFILE%\.config\poolside` on Windows.
This directory holds `settings.yaml` and `credentials.json`. To use the CLI again later, you may need to run `pool login` and recreate local settings.
4. If the installer added a PATH entry only for `pool`, remove that entry from your shell configuration or user PATH. Do not remove a shared directory such as `~/.local/bin` from PATH if you use it for other tools.
5. Remove saved local state (optional). This includes logs, trajectories, session history, and other saved runtime state. It does not delete files from your projects. Remove the default directories for your platform:
* macOS: `~/Library/Application Support/poolside`
* Linux: `~/.local/state/poolside` and `~/.local/share/poolside`
* Windows: `%LOCALAPPDATA%\poolside`
On a custom setup, remove the `poolside` directories under the custom state or data locations you configured. The log and trajectory paths from `pool config` can help you identify the state directory.
If `pool` still runs after you remove the binary, you likely have another installation or a leftover shell alias. Run `type pool` on macOS or Linux, or `Get-Command pool` on Windows, to see what your shell is running.
# Work from the terminal
Source: https://docs.poolside.ai/cli/interactive-mode
Use `pool` to work with a Poolside agent over multiple turns in the terminal.
Run `pool` to open an interactive session. The agent can read your code, run commands, and write files. By default, `pool` asks you to approve tool actions that your settings do not already allow. Your active approval mode and settings control which actions require confirmation.
For one-shot tasks and scripts, see [Automate tasks](/cli/automated-mode) instead.
This documentation describes Poolside Agent CLI v1.0.16. Check your version with `pool --version`. To update, exit any active session and run `pool update` from your terminal. See [Poolside Agent CLI releases on GitHub](https://github.com/poolsideai/pool/releases) for release history.
## Start a session
Open a session in your current directory:
```bash theme={null}
pool
```
Open a session in a specific directory:
```bash theme={null}
pool -C
```
Run a session in a Git worktree for a branch, creating the worktree and branch if needed:
```bash theme={null}
pool --worktree
```
Use `--worktree` without a branch name to let `pool` generate a worktree name:
```bash theme={null}
pool --worktree
```
When your terminal supports it, `pool` notifies the terminal of the worktree directory. New tabs, panes, or splits can open in that directory, so you can run `git diff` or other commands while the session continues running. The shell that started `pool` remains in its original working directory.
To add a prefix to generated worktree names, set `pool.worktree_prefix` in `~/.config/poolside/settings.yaml`.
Require a configured sandbox for the Poolside agent server:
```bash theme={null}
pool --sandbox required
```
Use `--sandbox disabled` only when you want to run without a configured sandbox.
Open the session picker to resume a previous session from the current directory:
```bash theme={null}
pool -r
```
Open the agent server picker to choose a configured ACP-compatible agent server for the session:
```bash theme={null}
pool -s
```
Resume a specific session by ID (a session ID is provided when you exit a session):
```bash theme={null}
pool --resume
```
By default, `pool` uses the credentials saved by `pool login`. To authenticate one invocation with an API key, set `POOLSIDE_API_KEY` before the command:
```bash theme={null}
POOLSIDE_API_KEY= pool
```
To override the saved connection for one invocation, set `POOLSIDE_STANDALONE_BASE_URL` for an OpenAI-compatible endpoint.
If an MCP server needs input from environment variables, start `pool` with those variables set:
```bash theme={null}
KEY=VALUE pool
```
## Write prompts
Type your prompt in the input area at the bottom and press `Enter` to send.
When the connected agent server supports steering, enter another prompt while the agent is working and press `Enter`. The agent receives it at the next step and adjusts its current work. The Poolside agent server supports steering. If another server does not, the prompt waits for the next turn. Steering accepts prompts only. Shell input that starts with `!` and slash commands also wait until the current turn finishes.
To queue a prompt for the next turn instead, press `Ctrl+Enter`. Queueing is available only when your terminal supports key disambiguation. If `Ctrl+Enter` inserts a new line, your terminal cannot distinguish the queue shortcut. Press `?` with an empty input field to see the shortcuts available in your terminal, or wait for the current turn to finish before you send the next prompt.
To find the newline shortcut for your terminal, press `?` with an empty input field. Common shortcuts are:
* macOS: `Shift+Enter` when your terminal supports key disambiguation, or `Alt+Enter` as a fallback. In Apple's built-in Terminal app, if `Alt+Enter` does not insert a new line, go to **Terminal** > **Settings** > **Profiles** > select your profile > **Keyboard**, then enable **Use Option as Meta key**.
* Linux: `Shift+Enter` when your terminal supports key disambiguation, or `Alt+Enter` as a fallback.
* Windows: `Shift+Enter` when your terminal supports key disambiguation. Otherwise, use `Ctrl+Enter`, or `Alt+Enter` when connected over SSH.
Use the up and down arrow keys to browse prompt history for the current directory.
Type `!` to enter shell mode and run a command directly from the prompt input box.
You can select text in the conversation output using your terminal's selection controls. When your terminal supports it, double-click and drag to select whole words. Copied text omits line breaks that were added only to wrap text on screen.
You can also use the mouse to choose items in supported dialogs, menus, and selectors.
## Add context
Type `@` in the prompt to mention a file or directory. `pool` opens a picker so you can choose what to include.
When the connected model supports image input, the agent can view PNG, JPEG, GIF, and WebP image files in your project that are 5 MB or smaller.
To add an image from your clipboard, paste it into the prompt input box with `Ctrl+V`.
## Use a skill
Type `$` at the start of the prompt or after a space to open the skill picker. Continue typing to filter the list, then press `Tab` or `Enter` to insert the selected skill reference into your prompt.
When you send the prompt, the selected skill applies to that turn. For more information, see [Skills](/skills).
## Answer agent questions
If the agent needs clarification while it is working, it can open a question dialog instead of guessing.
To trigger this flow, tell the agent to ask when needed, for example `Ask me a question if anything is ambiguous.`
You can select one of the provided options or choose **Type your own answer** to enter a custom response. Press `Esc` to decline.
## Notifications
`pool` can send notifications when a turn ends, when an approval prompt needs your response, or when the agent asks a question. Notifications appear only when the `pool` session is not focused.
When you run `pool` in cmux, `pool` can also show notifications and status icons.
## Status line
The status line shows session details such as the current approval mode, agent mode, Git branch, working directory, context usage, and model when that information is available.
Click the current approval mode, agent mode, or model to open the corresponding selector. Click the current directory to open it in your configured editor.
Hover over context usage to see cache read and write totals, cache read percentage, and session cost when the connected agent server provides them. When the session uses subagents, the tooltip includes their cost in the session total. The model selector also shows descriptions when the connected agent server provides them.
When the connected agent server offers thought-level choices, the status line shows the current choice next to the model. Click it, or use `/thought-level` or `/effort`, to select a reasoning effort level. Available choices depend on the connected agent server and model.
## Approve tool actions
When the agent wants to run a command or write a file, it asks for approval:
* **Allow once**: Approve only that action
* **Always allow: ...**: Save an approval rule for similar actions for the rest of the session
* **Accept edits for this session**: For file write approvals, switch to Accept edits mode for the rest of the session
* **Reject**: Decline and let the agent work around it if possible
If the agent requests multiple permissions concurrently, `pool` queues them and shows one approval prompt at a time.
To approve all actions automatically, start the session with `--mode always-allow`, or switch to the **Allow all** approval mode with `/mode`.
For persistent approval rules, path rules, and `settings.yaml` locations, see [Permissions](/permissions).
## Approval modes
Approval modes control which tool actions require your confirmation. When you use the Poolside agent server, these approval modes are available:
| Approval mode | ID | What it does |
| ------------- | -------------- | ------------------------------------------------------------------------------------------------- |
| Always ask | `default` | Prompts for tool actions that are not already allowed |
| Accept edits | `accept-edits` | Auto-approves workspace file reads and writes, then prompts for everything else |
| Auto | `auto` | Uses a configured classifier to run low- and medium-risk actions and prompt for high-risk actions |
| Allow all | `always-allow` | Approves tool actions automatically |
Press `Shift+Tab` to cycle through approval modes, or use `/mode` to open the approval mode selector.
When you select an approval mode, `pool` asks whether to save it as the default.
**Auto** appears when your API connection is configured and a classifier model is set, either through `pool.auto_mode_classifier` in `~/.config/poolside/settings.yaml` or through `POOL_AUTO_MODE_CLASSIFIER_MODEL` for that `pool` invocation. For classification behavior and failure handling, see [Auto mode](/permissions#auto-mode).
Build and Plan are separate agent modes. Changing the agent mode does not change your approval mode.
## Use plan mode
Use plan mode when you want to review an implementation approach before the agent changes your code. It is useful for complex refactors, broad changes that span multiple files, or work where early decisions are hard to undo.
In plan mode, the agent can read and explore your codebase, ask clarifying questions, and write an implementation plan for you to review. Plan mode does not modify source files.
Starting in v1.0.15, the Poolside agent server uses `--mode` for approval modes only. If you run `pool --mode plan`, `pool` starts with the default approval mode and displays a warning. There is no separate startup flag for Plan mode. Start `pool`, then use `/plan` or `/agent-mode plan`.
To enter plan mode, type `/plan`, type `/agent-mode plan`, or click the current agent mode in the status line and select **Plan**.
To leave plan mode, type `/agent-mode build`, or click the current agent mode and select **Build**. Your approval mode stays unchanged when you switch between Build and Plan.
When the plan is ready, approve the prompt to switch to build mode, or decline it to keep reviewing or discussing the plan. Poolside always asks you to review this switch, including when your approval mode is **Allow all**. Declining keeps the agent in plan mode and does not cancel the task.
`pool` saves each plan as a Markdown file. Outside a sandbox, `pool` stores the plan in your local Poolside state directory. In a sandbox, it stores the plan at `.poolside/plans/` in your current workspace. The saved path is clickable in the terminal so you can open the plan in your editor.
## Keyboard shortcuts
CLI shortcuts use the key names shown by your terminal. On macOS, use `Ctrl` for CLI shortcuts such as `Ctrl+M`, `Ctrl+C`, and `Ctrl+V`. `Command` shortcuts are handled by your terminal application, not by `pool`.
| Key | What it does |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Enter` | Send the prompt. While the agent is working, steer the prompt into the running turn. |
| `Ctrl+Enter` | While the agent is working, queue the prompt for the next turn when your terminal supports key disambiguation. If it inserts a new line instead, queueing is unavailable. |
| `Shift+Enter` or `Alt+Enter` | Insert a new line. The shortcut varies by terminal and platform. |
| `Up` / `Down` | Browse prompt history when the cursor is at the start or end of the prompt |
| `/` | Show available slash commands |
| `?` | Show all keyboard shortcuts when the input field is empty |
| `Shift+Tab` | Cycle through approval modes |
| `Ctrl+M` or `Alt+M` | Open the model selector |
| `Esc` | Interrupt the agent while it is running |
| `Esc`, then `Esc` | Rewind to the previous turn when idle |
| `Ctrl+C`, then `Ctrl+C` | Exit when idle. The first press clears the input field. |
| `Ctrl+D`, then `Ctrl+D` | Exit when idle with an empty input field |
| `Ctrl+T` | Toggle tool grouping in the conversation |
| `Ctrl+G` | Open the current prompt in your configured editor |
| `Ctrl+V` | Paste clipboard text or attach a clipboard image |
| `Page Up` / `Page Down` | Scroll the conversation |
Press `?` with an empty input field to see the shortcut list for your terminal.
### Picker and menu shortcuts
| Where | Key | What it does |
| ------------------------------------------------------------- | ------------------------------------ | --------------------------------------------------------------------------- |
| Slash command menu | `Up` / `Down` | Move through commands |
| Slash command menu | `Tab` | Complete the selected command or run a command that opens immediately |
| Slash command menu | `Enter` | Complete the selected command, or send the prompt if no command is selected |
| Slash command menu | `Esc` | Close the command menu |
| File picker | `Up` / `Down` | Move through files and directories |
| File picker | `Tab` or `Enter` | Add the selected file or directory mention |
| File picker | `Esc` | Close the file picker |
| Skill picker | `Up` / `Down` | Move through skills |
| Skill picker | `Tab` or `Enter` | Add the selected skill reference |
| Skill picker | `Esc` | Close the skill picker |
| Model, approval mode, agent mode, and thought-level selectors | Select | Select an option |
| Session picker | Type text | Filter sessions |
| Session picker | `Up` / `Down` or `Ctrl+P` / `Ctrl+N` | Move through sessions |
| Session picker | `Enter` | Resume the selected session |
| Session picker | `Tab` | Switch between current-directory sessions and sessions from all directories |
| Session picker | `Esc` | Start a new session |
| Session picker | `Ctrl+C` | Cancel and exit the picker |
## Slash commands
Type `/` to open the command menu, or type a command directly.
The commands you see can include terminal UI commands and commands from the connected agent server. Non-Poolside ACP servers can expose a different set of server-provided slash commands.
These commands are built into the `pool` interface:
| Command | What it does |
| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `/model` | Open the model selector for the current session |
| `/mode []` | Open the approval mode selector, or set an approval mode. The Poolside agent server supports `default`, `accept-edits`, `auto`, and `always-allow`; `auto` requires a configured API connection and classifier model. Other ACP servers can provide different values. |
| `/agent-mode [build\|plan]` | Open the agent mode selector, or switch between Build and Plan. Use `/collaboration-mode` as an alias. Available only when the connected agent server provides agent modes. |
| `/thought-level []` | Open the thought-level selector, or set a thought level. Use `/effort` as an alias. Available only when the connected agent server provides thought-level choices. |
| `/new` or `/clear` | Clear conversation history and start a new session. The new session keeps the current model and approval mode. Select the agent mode again if needed. |
| `/copy` | Copy the last agent response to the clipboard |
| `/delete` | Select and delete saved sessions. You cannot delete sessions while a turn is in progress. |
| `/rewind` | Roll back to a previous turn |
| `/resume` | Open the session picker and switch to a previous session |
| `/rename` | Rename the current session |
| `/move` | Move the session to another Git worktree, or create one. If the current worktree has uncommitted changes, choose whether to move those files too. |
| `/set-option ` | Set a session configuration option by ID on the connected ACP agent |
| `/system` | Show the system prompt for the current session when available |
| `/feedback` | Open a feedback draft and optionally attach logs |
| `/logs` | Archive debug logs for the current session |
| `/logout` | Log out through the connected agent server and exit the session. Available only when the server supports logout. |
| `/quit` or `/exit` | Exit the session |
| `/debug:dump` | Write the raw agent-server message log to a local JSON file |
When you use the Poolside agent server, these additional commands are available:
| Command | What it does |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `/plan` | Switch to plan mode |
| `/compact []` | Compact conversation context. Add guidance after the command to tell the agent what to preserve, such as `/compact preserve tool call errors`. |
| `/share` | Get a link to the trajectory viewer for the current session |
| `/mcp` | Show MCP servers, connection status, and tools for the current session |
| `/sandbox` | Show local sandbox configuration |
| `/sandbox-apply-to-host` | Review pending sandbox filesystem changes and apply them to the host workspace when available |
| `/skills` | Refresh and list available skills, including any skill load errors |
| `/usage` | Show token usage, context window state, and session cost when available. With [subagents](/subagents#review-subagent-usage), show parent, per-subagent, and total usage. |
You can also ask `pool` about its own commands, capabilities, and current behavior. The Poolside agent server uses a built-in introspection skill to answer questions about itself.
## Change the agent
Press `Ctrl+M` or `Alt+M`, or use `/model` to change the agent for the current session or set a new default. The selector includes agent descriptions when the connected agent server provides them. The command is named `/model` because [Agent Client Protocol](https://agentclientprotocol.com/) refers to agents, as defined in Poolside, as models.
After you select a model, `pool` asks whether to save it as the default.
To start a session with a specific model:
```bash theme={null}
pool --model poolside/laguna-s-2.1
```
## Get debug logs
Use `/logs` when you need to collect debug logs for troubleshooting or support.
`/logs` creates a `logs.zip` archive for the current session and prints links to copy the archive, copy the archive path, or open the containing folder. The archive can include session logs, ACP logs, trajectory data, and session metadata. Review the archive before sharing it, because logs and trajectory data can include prompt and response text from the session.
## Rewind a turn
Press `Esc`, then `Esc` while idle, or use `/rewind` to roll back the last turn. Rewind removes that exchange from the conversation history so the agent does not see it on the next prompt.
## Sessions
`pool` saves sessions automatically. When you exit, `pool` prints the `--resume` command for that session so you can continue it later.
Use `/resume` during a session to open the session picker and switch to another saved session. You cannot resume a session while a turn is in progress.
Use `/delete` to select and delete saved sessions. If you delete the current session, `pool` starts a new session. You cannot delete sessions while a turn is in progress.
Use `/rename` to rename the current session. The session picker opened by `pool -r` or `pool --resume` shows renamed sessions from the current directory by default. Press `Tab` in the picker to switch between current-directory sessions and sessions from all directories.
## Update Poolside Agent CLI
When a newer version is available, `pool` opens a dialog before it creates the session. Choose **Update now** to exit and install the update, **Remind me later** to start the session without updating, or **Skip this version** to start the session and stop prompts for that version.
You can also exit the session and run `pool update` from your terminal.
## MCP servers
Use `/mcp` to see MCP servers, connection status, and tools for the current session.
## View the trajectory
When you use the Poolside agent server, use `/share` during a session to get a link to the web-based trajectory viewer.
After a session, browse trajectory files locally:
```bash theme={null}
pool history trajectories --latest
```
## Send feedback
Use `/feedback` to open a feedback draft for the current session. You can choose whether to attach logs.
To attach an earlier session, resume it first with `pool -r`, then run `/feedback`.
## Related resources
* [Automate tasks](/cli/automated-mode)
* [Use other agent servers](/cli/other-agent-servers)
* [CLI reference](/cli/cli-reference)
# Use other agent servers
Source: https://docs.poolside.ai/cli/other-agent-servers
Use `pool` as an Agent Client Protocol (ACP) client for another compatible agent server.
Unless you configure a different default, `pool` connects to the Poolside agent server. You can also use `pool` as an Agent Client Protocol (ACP) client for another compatible agent server, such as Claude Agent, Codex, or Gemini.
## Prerequisites
* You have another ACP-compatible agent server installed and runnable on your machine, or the URL of a remote ACP server.
* You know the command or URL that starts that agent server.
## Steps
1. Start `pool` with `--agent-server` and the command or URL for the agent you want to use:
```bash theme={null}
# Claude Agent
pool --agent-server claude-agent-acp
# Codex
pool --agent-server codex-acp
# Gemini
pool --agent-server "gemini --acp"
# Remote ACP server
pool --agent-server http://localhost:3284/acp
```
2. To choose from configured agent servers interactively instead of passing a command each time, run:
```bash theme={null}
pool -s
```
3. To reuse a server without retyping its command or URL, add an `agent_servers` entry to `~/.config/poolside/settings.yaml`. Set `pool.default_agent_server` to use it automatically, or select it later with `pool -s`.
```yaml title="Agent server example for ~/.config/poolside/settings.yaml" theme={null}
pool:
default_agent_server: claude
agent_servers:
claude:
command: claude-agent-acp
remote:
url: http://localhost:3284/acp
```
For remote ACP servers, you can also configure `headers` on the `agent_servers` entry. For full configuration details, see [Agent servers](/settings-file-reference#agent-servers).
## Related resources
* [Poolside Agent CLI](/cli/pool)
* [Work from the terminal](/cli/interactive-mode)
* [Settings file reference](/settings-file-reference)
# Poolside Agent CLI
Source: https://docs.poolside.ai/cli/pool
Use the `pool` command for interactive sessions, automated tasks, ACP editors, and other agent servers.
Poolside Agent CLI is the command-line application for Poolside's coding agent. Use the `pool` command to work directly from your terminal, automate one-shot tasks, or connect Poolside to an ACP-compatible editor. You can also use `pool` as an ACP client for another compatible agent server.
The agent can read your code, answer questions about a project, propose changes, edit files, run commands, and use configured resources such as MCP servers and skills. You control what it can do with approval prompts, permissions, and sandboxes.
This documentation describes Poolside Agent CLI v1.0.16. Check your version with `pool --version`. To update, exit any active session and run `pool update` from your terminal. See [Poolside Agent CLI releases on GitHub](https://github.com/poolsideai/pool/releases) for release history.
## Key features
* [Plan mode](/cli/interactive-mode#use-plan-mode) to review an approach before the agent changes your code
* [Skills](/cli/interactive-mode#use-a-skill) and [slash commands](/cli/interactive-mode#slash-commands) to extend and speed up common workflows
* [Sessions](/cli/interactive-mode#sessions) and [rewind](/cli/interactive-mode#rewind-a-turn) to resume work and undo turns
* [Approval modes](/cli/interactive-mode#approval-modes) to control how much the agent can do without asking
* [Subagents](/subagents) to delegate focused work with separate context or a custom ACP server
* [Hooks](/hooks) to inspect or change activity at agent lifecycle events
* [AGENTS.md instructions](/agent-instructions) to give the agent project-specific guidance
* [MCP servers](/mcp-servers) to connect external tools and resources
## Choose a workflow
Use `pool` for interactive, multi-turn sessions in your project directory.
Use `pool exec` for one-shot prompts in scripts and CI jobs.
Connect Poolside to an ACP-compatible editor with `pool acp`.
Use `pool` as an ACP client for another compatible agent server.
## Install and authenticate
[Install Poolside Agent CLI](/cli/install) to install `pool` and sign in. To compare access methods, such as Poolside-hosted inference, self-managed Poolside inference, a Poolside deployment, or OpenRouter, see [Log in to Poolside](/get-started/log-in).
Looking for a Poolside API key? Run `pool login` and choose **Use Poolside Platform for free (recommended)** for the fastest way to get free developer access to models hosted by Poolside.
## Configure the agent
[Configure Poolside](/configure) to customize agent behavior and the resources it can use across terminal, automated, and ACP sessions, including [permissions and sandboxes](/permissions) that control what the agent can access and run.
## Related resources
* [CLI reference](/cli/cli-reference) for commands, flags, slash commands, and exit codes. To see commands and flags for your installed version, run `pool --help`.
* [Troubleshoot Poolside Agent CLI](/cli/troubleshooting) for authentication, connection, session, and ACP issues.
* [Uninstall Poolside Agent CLI](/cli/install#uninstall-poolside-agent-cli) to remove `pool` and saved local data.
For `pool` CLI bugs and feature requests, open an issue on [GitHub](https://github.com/poolsideai/pool).
# Troubleshoot Poolside Agent CLI
Source: https://docs.poolside.ai/cli/troubleshooting
Diagnose common Poolside Agent CLI authentication, configuration, and ACP issues.
Use this page when `pool` starts but a session, prompt, or editor connection fails.
Run shell troubleshooting commands in your terminal, not inside the interactive `pool` prompt or an editor chat panel. Use slash commands such as `/logs` inside an interactive `pool` session. To leave an interactive `pool` session, type `/quit` or press `Ctrl+C` twice.
This documentation describes Poolside Agent CLI v1.0.16. Check your version with `pool --version`. To update, exit any active session and run `pool update` from your terminal. See [Poolside Agent CLI releases on GitHub](https://github.com/poolsideai/pool/releases) for release history.
## Check which files Poolside uses
Run:
```bash theme={null}
pool config
```
The output shows the log directory, trajectory directory, config directory, and credentials path that `pool` is using.
By default, Poolside stores configuration files in `~/.config/poolside`. For more information about configuration and credential files, see [Install Poolside Agent CLI](/cli/install#configure-and-inspect-paths).
## Fix API key or token errors
Use this section when you see an error such as:
* `403 Forbidden: please check the api-key you provided`
* `401 Unauthorized`
* `Error during ACP method session/prompt`
These errors usually mean the prompt reached the agent, but the agent or model request could not authenticate.
1. Check whether authentication environment variables are set.
```bash theme={null}
env | grep -E '^POOLSIDE_(API_KEY|API_URL|TOKEN|STANDALONE_BASE_URL|STANDALONE_CONTEXT_LENGTH|STANDALONE_MODEL)=' | sed 's/=.*/=/'
```
This command shows whether a variable is set without printing the secret value.
2. Confirm that `pool` is using the connection variables for your access method:
* For Poolside-hosted inference, self-managed Poolside inference, an inference gateway, or another OpenAI-compatible endpoint, set `POOLSIDE_STANDALONE_BASE_URL` and `POOLSIDE_API_KEY`. If your endpoint does not list models from its API, also set `POOLSIDE_STANDALONE_MODEL`.
If you used `pool login`, check the saved configuration instead. For setup details, see [Install and authenticate](/cli/install#install-and-authenticate).
3. If any variables do not match your intended connection, test without the environment overrides.
```bash theme={null}
env -u POOLSIDE_API_KEY -u POOLSIDE_API_URL -u POOLSIDE_TOKEN -u POOLSIDE_STANDALONE_BASE_URL -u POOLSIDE_STANDALONE_CONTEXT_LENGTH -u POOLSIDE_STANDALONE_MODEL pool
```
4. Sign in again.
```bash theme={null}
pool logout
pool login
```
5. Choose the sign-in method that matches your setup. For details, see [Install and authenticate](/cli/install#install-and-authenticate).
6. Start a new session and send a short text prompt.
```bash theme={null}
pool
```
If the short prompt works, retry the original workflow. If the short prompt still returns an API key or token error, the saved credentials or selected environment are not valid for the endpoint you are using.
## Fix connection errors in automation
If `pool exec` returns `agent not found` or `user is not authorized to use --unsafe-auto-allow` when you intend to use Poolside-hosted inference, set the standalone endpoint variable:
```bash theme={null}
POOLSIDE_API_KEY= \
POOLSIDE_STANDALONE_BASE_URL=https://inference.poolside.ai \
pool exec -p "" --unsafe-auto-allow
```
To choose a model instead of using the default, also set `POOLSIDE_STANDALONE_MODEL` to its model ID.
## Check feature behavior and authentication
Some feature tests can reach authentication later during prompt submission. For example, when you paste an image with `Ctrl+V`, the image paste succeeds if the prompt input box shows an image attachment before you submit. If the prompt later returns `403 Forbidden: please check the api-key you provided`, troubleshoot authentication instead of image paste.
## Fix clipboard errors on Linux
Clipboard support depends on your display server:
* On Wayland, install the `wl-clipboard` package so `wl-copy` and `wl-paste` are available.
* On X11, install `xclip` for text, image, and file clipboard support. You can use `xsel` instead if you only need text clipboard support.
Restart `pool` after installing the required commands.
## Collect interactive session logs
Use this section when an interactive `pool` session shows an error or asks you to collect logs.
1. In the interactive `pool` prompt, type:
```text theme={null}
/logs
```
2. Review the generated `logs.zip` archive before sharing it.
The archive can include session logs, ACP logs, trajectory data, and session metadata. Logs and trajectory data can include prompt and response text from the session.
## Troubleshoot editor ACP connections
If `pool` is running through an Agent Client Protocol (ACP) editor integration, check the ACP logs:
```bash theme={null}
pool acp logs -f
```
For formatted logs, run:
```bash theme={null}
pool acp logs --pretty
```
Common ACP issues:
* **`pool` not found**: Add `pool` to your `PATH`, use the full path to the binary in your editor configuration, or run `pool.exe` on Windows if your shell does not resolve `.exe` commands.
* **Authentication errors with Poolside-hosted inference, self-managed Poolside inference, OpenRouter, or another OpenAI-compatible endpoint**: Run `pool login`, choose the matching login option, enter the required credentials, then reconnect.
* **Missing mode, history, rewind, or slash command support**: ACP capabilities depend on both your editor client and your Poolside account access. The same `pool acp` server can expose a capability that an editor does not show in its UI.
After fixing authentication or configuration, reconnect the editor integration or restart the editor session.
## Related resources
* [Install Poolside Agent CLI](/cli/install)
* [CLI reference](/cli/cli-reference)
# Configure Poolside
Source: https://docs.poolside.ai/configure
Shape agent behavior and control what Poolside agents can access.
These guides configure Poolside's own agent, whether you run it through Poolside Agent CLI, Poolside Assistant, or an ACP-compatible editor. They help you guide the agent, add capabilities, and control access.
If a third-party tool such as Cline, GitHub Copilot, or Kilo Code calls Poolside models directly, or an ACP-compatible editor runs a different agent, use that tool's own setup guide instead. See [Tool integrations](/tools) to choose a guide.
## Guide the agent
Give Poolside project and personal instructions to follow.
Add reusable, task-specific instructions and resources.
Delegate work to built-in or custom agents with focused context.
## Add capabilities
Connect agents to external tools and services.
Let agents search the web and read pages.
Configure `pool` to run another ACP-compatible agent.
## Control access
Control what agents can read, write, and run.
Run commands at agent lifecycle events to inspect or change activity. Hooks are not a security boundary.
Run agents in isolated environments with file and network controls.
Store API keys and tokens without exposing raw values.
# Install on Amazon EKS
Source: https://docs.poolside.ai/deployment/cloud/aws-eks/install
Deploy Poolside model inference on Amazon EKS with Helm, using IRSA for S3 access and an Application Load Balancer for ingress.
Follow these steps to deploy Poolside model inference on your Amazon EKS cluster and serve models through an OpenAI-compatible API. For an overview of this deployment approach and architecture, see [Amazon EKS deployment](/deployment/cloud/aws-eks/overview).
This guide deploys the Poolside inference chart. Each model becomes its own `Deployment`, `Service`, and `Ingress`, reachable at its own hostname through a shared Application Load Balancer.
## Prerequisites
Poolside distributes the Helm deployment bundle as a `.tar.gz` archive. Extract it before you start:
```bash theme={null}
tar -xzf .tar.gz
cd
```
Confirm that you are working from the root of the extracted bundle. The bundle root contains the following directories:
```text theme={null}
./scripts/
./containers/
./charts/
```
### Required AWS infrastructure
You provision the following AWS foundation before you deploy the chart. For a turnkey foundation that provisions all of it, apply the Terraform reference architecture in the [`poolsideai/reference_architectures`](https://github.com/poolsideai/reference_architectures/tree/main/aws) repository, or reproduce the same architecture in your own infrastructure-as-code. For the architecture diagram and design decisions, see [Reference architecture](/deployment/cloud/aws-eks/reference-architecture).
* **EKS cluster**, Kubernetes 1.29 or later, with an IAM OIDC provider enabled. The OIDC provider is what makes IRSA work, so the model servers can read checkpoints from S3 without static credentials.
* **GPU node group** with enough GPU memory for the models you deploy. The reference architecture uses `p5e.48xlarge`; `p5` and `p5en` instances also fit. The node group runs an EKS-optimized GPU AMI that can schedule containerized GPU workloads, so each node advertises the `nvidia.com/gpu` resource. Apply the `nvidia.com/gpu=true:NoSchedule` taint to keep non-GPU workloads off these nodes; the chart's default tolerations already tolerate it. These instances are usually not available on demand and need reserved capacity. For instance shapes, model packing, and capacity reservations, see [Reference architecture](/deployment/cloud/aws-eks/reference-architecture#gpu-node-group).
* **NVIDIA GPU Operator** to expose GPUs to the cluster. Run it in one of two modes, depending on your AMI:
* If the AMI already includes the NVIDIA driver and container toolkit, such as the AL2023 or AL2 NVIDIA accelerated AMIs or the Bottlerocket NVIDIA variant, run the GPU Operator in device-plugin-only mode with the driver and toolkit subcomponents turned off. This is what the reference architecture does.
* If the AMI ships without drivers, run the full GPU Operator so it installs the driver and container toolkit.
* **AWS Load Balancer Controller**, installed and running in the cluster. It reconciles the per-model `Ingress` objects into an Application Load Balancer. Its admission webhook must be reachable, which requires the controller to have ready pods on schedulable nodes: a controller scaled to zero or stuck `Pending` rejects every `Service` and `Ingress` the chart creates, and the install fails. Tag your subnets for load balancer discovery: `kubernetes.io/role/elb=1` on public subnets for an internet-facing load balancer, or `kubernetes.io/role/internal-elb=1` on private subnets for an internal one.
* **Amazon S3 bucket** for the model checkpoints. Server-side encryption with a KMS key is recommended. Bucket versioning is unnecessary because the checkpoints are content-addressed.
* **Amazon ECR** to host the bundled container images.
* **AWS Certificate Manager certificate** covering the hostnames you assign to the models. The load balancer terminates TLS with this certificate. Decide the per-model hostnames now: you set them as the `ingressHost` values in [Step 6](#step-6-configure-the-values-file), and the certificate must cover every one of them.
An S3 gateway VPC endpoint keeps checkpoint downloads off your NAT gateways
and reduces data transfer cost. If your worker nodes pull from Amazon ECR
through private networking, also configure the Amazon ECR interface VPC
endpoints required for private image pulls.
### Workstation tools
Install the following tools on the host you use to run the deployment:
* `helm` `3.12` or later
* `kubectl`, configured for your EKS cluster
* `skopeo`, to copy the bundled images into Amazon ECR
* `aws` CLI, to create AWS resources and upload checkpoints to S3
* `jq`, to parse JSON responses from the inference API
* `tar`, to extract the deployment bundle and model checkpoints
* `curl`, to call the inference API
* `eksctl` (optional), to associate an IAM OIDC provider if your cluster lacks one, or to create the IRSA role and service account with the alternative in [Step 4](#step-4-create-the-irsa-role)
**Disk space**
Stage the deployment from a host with tens of GB free. The extracted bundle is roughly 20 GB, because it carries the `atlas` container image, and each model checkpoint is tens of GB more. On a constrained workstation, run the extraction and the uploads from an EC2 instance in the bucket's region that has enough disk, which also speeds up the checkpoint upload, and delete each local copy once it is uploaded.
## Step 1: Create the namespace
The inference stack runs in a single namespace:
```bash theme={null}
kubectl create namespace poolside-models
```
## Step 2: Upload the container images to Amazon ECR
The bundle ships the `atlas` inference server image and the `public-docs` documentation site image as OCI archives under `./containers/`. Copy them into Amazon ECR.
Amazon ECR does not create repositories on push, so create the repositories first. Each repository name must match an image name from the bundle:
```bash theme={null}
for image_name in $(find ./containers -name "*.tar" -type f -exec basename {} .tar \; | sed 's/__.*//' | sort -u); do
aws ecr describe-repositories --repository-names "$image_name" --region >/dev/null 2>&1 \
|| aws ecr create-repository --repository-name "$image_name" --region
done
```
Authenticate `skopeo` to your ECR registry:
```bash theme={null}
aws ecr get-login-password --region \
| skopeo login --username AWS --password-stdin .dkr.ecr..amazonaws.com
```
Upload the images with the provided script. Pass your ECR registry host as the target:
```bash theme={null}
chmod +x ./scripts/upload_images.sh
./scripts/upload_images.sh .dkr.ecr..amazonaws.com
```
The script pushes each archive to `.dkr.ecr..amazonaws.com/:`. The tags are specific to the bundle you received, not fixed values, and the chart's `image.name`, `image.tag`, `docs.image.name`, and `docs.image.tag` are preset to match the archives, so you set only `image.registry` at install time. You do not need to type the tags anywhere, but you can confirm what was pushed. For example, to inspect the `atlas` image:
```bash theme={null}
aws ecr describe-images --repository-name atlas --region \
--query 'sort_by(imageDetails,&imagePushedAt)[-1].imageTags' --output text
```
You do not need an image pull secret. The GPU node group's instance role
authorizes ECR pulls through the AWS-managed
`AmazonEC2ContainerRegistryReadOnly` policy, so kubelet pulls the image
directly.
## Step 3: Upload model checkpoints to S3
The model servers download their checkpoints from S3 on pod startup, so the checkpoints must be in place before you deploy the chart. Poolside provides the checkpoint files separately from the deployment bundle. Confirm the local path and the destination prefix with your Poolside contact.
Uploading checkpoints is time consuming. Start it now and continue with the remaining steps in parallel.
Poolside ships each model checkpoint as a single `.tar` archive that contains one top-level directory holding the checkpoint files. The model server loads the unpacked files (`*.safetensors`, `*.json`, and the tokenizer files) directly from the prefix you point at, so extract each archive's contents into its own directory.
```bash theme={null}
tar -xf ./checkpoints/.tar
rm -v ./checkpoints/.tar
```
Confirm the files sit at the root of the directory, not under a subfolder:
```bash theme={null}
ls ./checkpoints/
# config.yaml generation_config.json model.safetensors tokenizer/
```
You choose the S3 layout. Give each model its own prefix, because every model entry in the values file points at one prefix. The command below preserves the local directory structure, so `./checkpoints/laguna-m` and `./checkpoints/laguna-xs` upload to `checkpoints/laguna-m` and `checkpoints/laguna-xs` under the bucket:
```bash theme={null}
aws s3 cp ./checkpoints s3:///checkpoints --recursive --region
```
Note the full `s3://` prefix for each model. You reference it in the `models..model` paths in [Step 6](#step-6-configure-the-values-file), and the paths must match what you uploaded exactly: a misspelled or missing prefix downloads nothing and the pod never starts.
Checkpoints are typically tens of GB per model. To speed up the transfer, run
the upload from an EC2 instance in the same region as the bucket, and tune
`aws configure set default.s3.max_concurrent_requests` and
`default.s3.multipart_chunksize`.
## Step 4: Create the IRSA role
The model servers read checkpoints from S3 through an IAM role assumed by the `inference` service account. This is the recommended path on EKS, and it keeps static AWS credentials out of the cluster.
Save the following permissions policy to a file named `inference-pod-policy.json`. It grants read-only access to the checkpoint bucket and decrypt access to the bucket's KMS key, and nothing else:
```json title="inference-pod-policy.json" theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3ListBucket",
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": "arn:aws:s3:::"
},
{
"Sid": "S3ReadObjects",
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:GetObjectTagging"],
"Resource": "arn:aws:s3:::/*"
},
{
"Sid": "S3GetBucketLocation",
"Effect": "Allow",
"Action": ["s3:GetBucketLocation"],
"Resource": "*"
},
{
"Sid": "KMSDecryptForS3",
"Effect": "Allow",
"Action": ["kms:Decrypt", "kms:DescribeKey"],
"Resource": ""
}
]
}
```
If the checkpoint bucket uses SSE-S3 rather than SSE-KMS, omit the `KMSDecryptForS3` statement.
Save the role's trust policy to a file named `inference-pod-trust.json`. It allows the cluster's OIDC provider to assume the role only for the `inference` service account in the `poolside-models` namespace:
```json title="inference-pod-trust.json" theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam:::oidc-provider/"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
":aud": "sts.amazonaws.com",
":sub": "system:serviceaccount:poolside-models:inference"
}
}
}
]
}
```
Replace `` with your cluster's OIDC issuer host and path, such as `oidc.eks..amazonaws.com/id/`. Retrieve it from the cluster, stripping the `https://` scheme:
```bash theme={null}
aws eks describe-cluster --name --region \
--query 'cluster.identity.oidc.issuer' --output text \
| sed 's#^https://##'
```
If the command returns no issuer, the cluster has no IAM OIDC provider yet. Associate one before you create the role, either through the reference architecture's Terraform or with `eksctl utils associate-iam-oidc-provider --cluster --approve`.
The trust policy's `sub` condition embeds the namespace:
`system:serviceaccount:poolside-models:inference`. This guide deploys into
`poolside-models`. If you deploy into a different namespace, change the
namespace in the `sub` value to match, or the model servers get `AccessDenied`
when they read from S3.
Create the policy from `inference-pod-policy.json`, create the role with the trust policy from `inference-pod-trust.json`, and attach the policy to the role:
```bash theme={null}
aws iam create-policy \
--policy-name inference-pod-policy \
--policy-document file://inference-pod-policy.json
aws iam create-role \
--role-name inference-pod-role \
--assume-role-policy-document file://inference-pod-trust.json
aws iam attach-role-policy \
--role-name inference-pod-role \
--policy-arn arn:aws:iam:::policy/inference-pod-policy
```
The role's ARN is `arn:aws:iam:::role/inference-pod-role`. In [Step 6](#step-6-configure-the-values-file), you annotate the chart's service account with this ARN, and the chart creates the annotated `inference` service account for you.
`eksctl create iamserviceaccount` creates the IAM role and the Kubernetes service account together, and builds the trust policy for you, so you do not need `inference-pod-trust.json`. You still create the permissions policy first, because `--attach-policy-arn` attaches an existing policy:
```bash theme={null}
aws iam create-policy \
--policy-name inference-pod-policy \
--policy-document file://inference-pod-policy.json
eksctl create iamserviceaccount \
--cluster \
--namespace poolside-models \
--name inference \
--attach-policy-arn arn:aws:iam:::policy/inference-pod-policy \
--approve
```
Because `eksctl` already creates the service account, set `serviceAccount.create: false` and `serviceAccount.name: inference` in your values file so the chart uses the existing account instead of creating a second one.
If you cannot use IRSA, create a Kubernetes secret with static credentials and set `s3.secretName` in your values file instead:
```bash theme={null}
kubectl create secret generic aws-credentials \
--from-literal=AWS_ACCESS_KEY_ID= \
--from-literal=AWS_SECRET_ACCESS_KEY= \
-n poolside-models
```
## Step 5: Create the API key secret (recommended for internet-facing)
To require an API key on the model servers, create a secret containing the key in `poolside-models`:
```bash theme={null}
kubectl create secret generic vllm-auth \
--from-literal=VLLM_API_KEY= \
-n poolside-models
```
Reference it through `authentication.secretName` in the next step. For an internet-facing load balancer, Poolside strongly recommends enabling an API key; for an internal load balancer it is optional.
## Step 6: Configure the values file
Create an `inference_values.yaml` file in the bundle root. Set the fields that apply to your environment. The example below deploys two Laguna model variants, exposes each through its own ALB ingress, and reads checkpoints through IRSA:
```yaml title="Example: inference_values.yaml" theme={null}
# Name the release resources "inference" so the service account matches the IRSA trust subject
fullnameOverride: inference
image:
# Registry the atlas image was uploaded to in Step 2
registry: .dkr.ecr..amazonaws.com
serviceAccount:
create: true
annotations:
# ARN of the IRSA role from Step 4
eks.amazonaws.com/role-arn: arn:aws:iam:::role/inference-pod-role
# Required on EKS. Unlike OpenShift, upstream Kubernetes does not assign a user ID automatically
podSecurityContext:
runAsNonRoot: true
runAsUser: 10003
seccompProfile:
type: RuntimeDefault
authentication:
# API key auth. Strongly recommended for an internet-facing load balancer.
# Uses the Step 5 secret; set to "" to disable (reasonable only for an internal LB).
secretName: vllm-auth
ingress:
enabled: true
className: alb
annotations:
# Use "internal" for a VPC-internal load balancer
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
alb.ingress.kubernetes.io/ssl-redirect: "443"
alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS13-1-2-2021-06
alb.ingress.kubernetes.io/group.name: poolside
# ARN of the ACM certificate covering the model hostnames below
alb.ingress.kubernetes.io/certificate-arn:
alb.ingress.kubernetes.io/healthcheck-path: /health
models:
laguna-xs:
model: s3:///checkpoints/laguna-xs
modelName: Lagunaxs
modelType: agent
gpus: 1
ingressHost:
laguna-m:
model: s3:///checkpoints/laguna-m
modelName: Lagunam
modelType: agent
gpus: 4
ingressHost:
```
The checkpoint paths in `models..model` must match the locations you uploaded in [Step 3](#step-3-upload-model-checkpoints-to-s3). The `image.name` and `image.tag` fields default to the values that match the bundled archive, so you do not set them.
The example exposes the models through an internet-facing load balancer.
Poolside strongly recommends enabling API-key authentication on any
internet-facing endpoint, so the example sets `authentication.secretName` to
the secret from [Step
5](#step-5-create-the-api-key-secret-recommended-for-internet-facing). The
chart does not enforce this: if you set `secretName: ""`, the endpoint is
reachable without a key. Only do that for an `internal` load balancer.
Each model is exposed at its own hostname through a separate `Ingress` named `inference-`. The ingresses share the `poolside` load balancer group, so the AWS Load Balancer Controller provisions one Application Load Balancer for all of them. Give every model a unique `ingressHost`, and point each hostname's DNS record at the load balancer once it is provisioned.
**Model type**
Each model's `modelType` selects the default serving arguments for that class of model. It takes one of three values:
* `agent`: Defaults for agent models.
* `agent_small`: Agent defaults with the context length and batch size capped for smaller models or GPUs with less memory.
* `completion`: Defaults for completion models.
Poolside specifies the `modelType` for each model in your bundle. Use the value provided for your checkpoint; the example above reflects the current Laguna models.
**GPU count and tensor parallelism**
Set `gpus` to the number of GPUs each model needs. The model server reads the number of GPUs allocated to its pod and shards the model across them, so you do not pass a tensor-parallel-size argument. The number of GPUs must be a value the server supports, such as 1, 2, 4, or 8, and your hardware must provide enough GPU memory for the model and workload. For planning estimates, see [Supported configurations](/deployment/supported-configurations).
## Step 7: Install the chart
Install the `inference` chart into `poolside-models`:
```bash theme={null}
helm install inference ./charts/inference \
--namespace poolside-models \
-f ./inference_values.yaml
```
## Step 8: Verify the deployment
Check that the model pods are running:
```bash theme={null}
kubectl get pods -n poolside-models
```
Each model server takes time to become ready on first start because it downloads its checkpoint from S3. Watch a model's logs to track progress, where `` is the key you set under `models` in the values file, such as `laguna-m`:
```bash theme={null}
kubectl logs -f -n poolside-models deploy/inference-
```
Confirm that an ingress was created for each model and that the load balancer has an address:
```bash theme={null}
kubectl get ingress -n poolside-models
```
Create or update a DNS record for each `ingressHost`, pointing it at the load balancer address. Then confirm routing works, where `` is the `ingressHost` you set for that model:
```bash theme={null}
curl -s https:///v1/models \
-H "Authorization: Bearer "
```
If API key authentication is off, omit the `Authorization` header.
## Step 9: Call the inference API
Each model serves the OpenAI-compatible API at its own hostname. The base URL has the form:
```text theme={null}
https:///v1
```
Requests are routed to a model by hostname, so each hostname serves exactly one model. The `model` field in the request body is the served model name (`modelName`), which the server validates against what it loaded. It does not need to be unique across models, because the hostname already selects the backend: in the example values, both Laguna variants use the `modelName` `Laguna` but answer at different hostnames.
Send a chat completion request, where `` is the model's `modelName`:
```bash theme={null}
curl https:///v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "",
"messages": [{"role": "user", "content": "Write a function that reverses a string."}]
}'
```
For example, to call the `laguna-m` model served as `Laguna`:
```bash theme={null}
curl https://laguna-m.example.com/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Laguna",
"messages": [{"role": "user", "content": "Write a function that reverses a string."}]
}'
```
If you enabled API key authentication in [Step 5](#step-5-create-the-api-key-secret-recommended-for-internet-facing), include the key as a bearer token:
```bash theme={null}
curl https:///v1/chat/completions \
-H "Authorization: Bearer " \
-H "Content-Type: application/json" \
-d '{
"model": "",
"messages": [{"role": "user", "content": "Write a function that reverses a string."}]
}'
```
## TLS
The load balancer terminates TLS with the ACM certificate you set in `alb.ingress.kubernetes.io/certificate-arn`, and the `listen-ports` and `ssl-redirect` annotations in [Step 6](#step-6-configure-the-values-file) serve every model over HTTPS on port 443 and redirect HTTP to HTTPS. To cover additional hostnames, issue or import a certificate in AWS Certificate Manager that includes them, then update the `certificate-arn` annotation. Unlike the upstream Kubernetes deployment, you do not create a TLS secret in the cluster or add a `tls` block to the values file.
## Offline documentation (optional)
The bundle also ships the Poolside documentation site, which the same `inference` chart can deploy in-cluster so operators have local access to the docs. It is off by default. To enable and expose it through the Application Load Balancer, see [Set up offline documentation](/deployment/cloud/set-up-offline-documentation).
## Troubleshooting
* If a model pod is `Pending`, confirm the cluster has enough GPUs for the `gpus` value you requested and that the NVIDIA GPU Operator is healthy. Run `kubectl describe pod -n poolside-models` and check the scheduling events.
* If pods stay in `Init` or restart in a loop, check the init container logs with `kubectl logs -n poolside-models -c model-downloader`. A stale or misspelled checkpoint path syncs nothing and the pod never starts. An `AccessDenied` error usually means the IRSA role's policy does not cover the bucket or its KMS key.
* If the load balancer never receives an address, confirm the AWS Load Balancer Controller is running and that your subnets carry the `kubernetes.io/role/elb` or `kubernetes.io/role/internal-elb` tags. Check the controller logs for the `Ingress` events.
* If requests return a 5xx from the load balancer, confirm the target group is healthy. The ALB health check uses `/health` on each model's port; a model that is still downloading its checkpoint stays unhealthy until it is ready.
* If image pulls fail, confirm the GPU node group's instance role has the `AmazonEC2ContainerRegistryReadOnly` policy and that the `atlas` repository exists in ECR.
* If `helm install` fails with `no endpoints available for service "aws-load-balancer-webhook-service"`, the AWS Load Balancer Controller has no ready pods, so its admission webhook rejects the `Service` objects the chart creates. Confirm the controller is running with `kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller`, make sure the cluster has schedulable nodes, then run `helm install` again.
* If model pods never become ready and the init container logs show no checkpoint files synced, the S3 prefix may hold the checkpoint `.tar` instead of its extracted contents. The downloader only fetches unpacked files such as `*.safetensors` and `*.json`. Extract the archive and re-upload its contents, as in [Step 3](#step-3-upload-model-checkpoints-to-s3).
## Related resources
* [Amazon EKS deployment](/deployment/cloud/aws-eks/overview)
* [Manage models on Amazon EKS](/deployment/cloud/aws-eks/manage-models)
* [Upgrade on Amazon EKS](/deployment/cloud/aws-eks/upgrade)
* [Remove from Amazon EKS](/deployment/cloud/aws-eks/remove)
For questions about hardware requirements, infrastructure configuration, or deployment issues, contact Poolside support.
# Manage models on Amazon EKS
Source: https://docs.poolside.ai/deployment/cloud/aws-eks/manage-models
Add, update, or remove inference models in an existing inference deployment on Amazon EKS.
## Overview
Use this guide to change the set of models served by a running `inference` release: adding a new model, replacing a model's checkpoint, or removing a model. You edit your `inference_values.yaml` file and run `helm upgrade`; the chart reconciles the model Deployments, Services, and Ingress objects to match.
You can make these changes on their own against the current chart version, or apply them as part of a chart upgrade to a new Poolside bundle. To upgrade the chart, see [Upgrade on Amazon EKS](/deployment/cloud/aws-eks/upgrade); make the model edits described here in the same `inference_values.yaml` file before you run `helm upgrade`.
## Prerequisites
* A working deployment completed with the [Install on Amazon EKS](/deployment/cloud/aws-eks/install) guide.
* The customized `inference_values.yaml` file you used to install.
* The new model checkpoint, provided by Poolside.
* Workstation tools:
* `helm` `3.12` or later
* `kubectl`, configured for your EKS cluster
* `aws` CLI, to upload checkpoints to S3
* `jq`, to parse JSON responses from the inference API
## Downtime
Adding a model does not affect models that are already serving. Updating a checkpoint rolls that model's Deployment, and the model server re-downloads the checkpoint from S3 on restart, so expect a delay before it becomes ready again. Plan a maintenance window for single-replica models.
## Add a model
Extract the new checkpoint archive as described in [Upload model checkpoints to S3](/deployment/cloud/aws-eks/install#step-3-upload-model-checkpoints-to-s3) so its files sit at the prefix root, then upload it to your S3 bucket. Use a distinct prefix per model:
```bash theme={null}
aws s3 cp ./checkpoints/ s3:///checkpoints/ \
--recursive \
--region
```
For checkpoint upload details such as concurrency throttling, see [Upload model checkpoints to S3](/deployment/cloud/aws-eks/install#step-3-upload-model-checkpoints-to-s3).
Add a new key under `models` in your `inference_values.yaml` file. Give the model its own `ingressHost`, covered by the ACM certificate referenced in `ingress.annotations`:
```yaml title="Example: inference_values.yaml" theme={null}
models:
# ...existing models...
:
model: s3:///checkpoints/
modelName:
modelType: agent
gpus:
ingressHost:
```
Apply the change with `helm upgrade`:
```bash theme={null}
helm upgrade inference ./charts/inference \
--namespace poolside-models \
-f ./inference_values.yaml
```
The chart creates a new `Deployment`, `Service`, and `Ingress` named `inference-` for the model. Confirm the new pod starts and the ingress is created:
```bash theme={null}
kubectl get pods -n poolside-models
kubectl get ingress inference- -n poolside-models
```
Create a DNS record for the new `ingressHost`, pointing it at the load balancer address.
## Update a model checkpoint
Upload the new checkpoint to a new, versioned prefix rather than overwriting the existing one. A new path lets `helm upgrade` detect the change and roll the Deployment automatically, and it lets you roll back by pointing at the previous path. Extract the archive first, as in [Step 3](/deployment/cloud/aws-eks/install#step-3-upload-model-checkpoints-to-s3), so the files sit at the prefix root:
```bash theme={null}
aws s3 cp ./checkpoints/- s3:///checkpoints/- \
--recursive \
--region
```
Point the model's `model` field at the new path in your `inference_values.yaml` file. Update `modelName` only if the served model name changes:
```yaml title="Example: inference_values.yaml" theme={null}
models:
laguna-m:
model: s3:///checkpoints/laguna-m-
modelName: Laguna
modelType: agent
gpus: 4
ingressHost:
```
Apply the change:
```bash theme={null}
helm upgrade inference ./charts/inference \
--namespace poolside-models \
-f ./inference_values.yaml
```
The model's Deployment rolls, and the init container downloads the new checkpoint on startup. Watch the rollout:
```bash theme={null}
kubectl rollout status deploy/inference- -n poolside-models
```
If you reuse the same S3 path instead of a versioned one, `helm upgrade` detects no change to the values and does not restart the model. Force a restart so the init container re-downloads the checkpoint:
```bash theme={null}
kubectl rollout restart deploy/inference- -n poolside-models
```
## Remove a model
Delete the model's key from `models` in your `inference_values.yaml` file, then apply the change:
```bash theme={null}
helm upgrade inference ./charts/inference \
--namespace poolside-models \
-f ./inference_values.yaml
```
The chart removes that model's `Deployment`, `Service`, and `Ingress`. Confirm the resources are gone:
```bash theme={null}
kubectl get deploy,svc,ingress -n poolside-models -l app.kubernetes.io/component=inference
```
If you no longer need the model's checkpoint, delete it from the bucket:
```bash theme={null}
aws s3 rm s3:///checkpoints/ --recursive --region
```
You can also remove the model's DNS record once the ingress is gone.
## Verification
Confirm a model serves traffic, where `` is the `ingressHost` of that model:
```bash theme={null}
curl -s https:///v1/models \
-H "Authorization: Bearer " \
| jq -r '.data[].id'
```
If API key authentication is off, omit the `Authorization` header.
## Related resources
* [Install on Amazon EKS](/deployment/cloud/aws-eks/install)
* [Upgrade on Amazon EKS](/deployment/cloud/aws-eks/upgrade)
* [Remove from Amazon EKS](/deployment/cloud/aws-eks/remove)
For questions about model checkpoints or hardware requirements, contact Poolside support.
# Amazon EKS deployment
Source: https://docs.poolside.ai/deployment/cloud/aws-eks/overview
Overview of deploying Poolside model inference on Amazon EKS by using Helm, with IRSA for object storage and an ALB for ingress.
Use this page to understand how to serve Poolside models from an Amazon EKS cluster.
You provision the EKS cluster and the supporting AWS services, including the model checkpoint S3 bucket, an Amazon ECR registry, and the GPU node group. Poolside provides the deployment bundle, which contains the `inference` Helm chart. The model checkpoints are provided separately. You deploy the chart, expose each model through its own Application Load Balancer ingress, and call the OpenAI-compatible API.
This deployment uses the Poolside inference chart from the current Poolside inference bundle. It serves the model servers directly.
## Architecture
This deployment includes:
* One `Deployment` and `Service` per model. Each model server downloads its checkpoint from Amazon S3 on startup and serves an OpenAI-compatible API.
* One `Ingress` per model, reconciled by the AWS Load Balancer Controller into a shared internal or internet-facing Application Load Balancer. Each model is reachable at its own hostname.
* A single shared service account, `inference`, annotated for IAM Roles for Service Accounts (IRSA). The model servers read checkpoints from S3 through this role, so the cluster needs no static AWS credentials.
* Optionally, the Poolside documentation site, deployed in-cluster from the bundle. See [Set up offline documentation](/deployment/cloud/set-up-offline-documentation).
You are responsible for sending requests to the inference endpoints and for any authentication or routing in front of them.
## How Amazon EKS differs from upstream Kubernetes
The deployment shape matches the [upstream Kubernetes deployment](/deployment/cloud/upstream-kubernetes/overview), with these AWS-native substitutions:
* **Ingress**: an Application Load Balancer provisioned by the AWS Load Balancer Controller, instead of an in-cluster ingress controller.
* **Object storage access**: IRSA on the `inference` service account, instead of a mounted AWS credentials secret.
* **Container registry**: Amazon ECR, with image pulls authorized by the GPU node group's instance role, instead of an image pull secret.
* **TLS**: terminated at the load balancer with an AWS Certificate Manager certificate, instead of a TLS secret in the cluster.
## Required AWS foundation
You provision the AWS infrastructure that the chart runs on. The [Install on Amazon EKS](/deployment/cloud/aws-eks/install) page lists the required services and the reason for each.
For a turnkey foundation, Poolside publishes a Terraform reference architecture in the [`poolsideai/reference_architectures`](https://github.com/poolsideai/reference_architectures/tree/main/aws) repository. You can apply it as published, fork it, or reproduce the same architecture in your own infrastructure-as-code. For the architecture diagram and the key design decisions, see [Reference architecture](/deployment/cloud/aws-eks/reference-architecture).
## Related resources
* [Install on Amazon EKS](/deployment/cloud/aws-eks/install)
* [Manage models on Amazon EKS](/deployment/cloud/aws-eks/manage-models)
* [Upgrade on Amazon EKS](/deployment/cloud/aws-eks/upgrade)
* [Remove from Amazon EKS](/deployment/cloud/aws-eks/remove)
* [Cloud deployment overview](/deployment/cloud/overview)
# Reference architecture
Source: https://docs.poolside.ai/deployment/cloud/aws-eks/reference-architecture
AWS reference architecture for a Poolside model inference deployment on Amazon EKS, including the architecture diagram, the AWS layers the inference chart depends on, and the key design decisions.
Use this page to plan a model inference deployment on Amazon EKS and to align on the key decisions before you install. It describes the AWS foundation that the Poolside inference chart runs on, the diagram for that foundation, and the opinions that distinguish it from a generic EKS install.
You provision the AWS infrastructure. Poolside provides the deployment bundle with the `inference` Helm chart, and an optional Terraform reference stack that provisions the same foundation. You can apply the Terraform as published, fork it, or reproduce the architecture by hand against your own infrastructure-as-code standards. In every case the same chart from the bundle installs onto the resulting cluster.
The reference architecture is published in the [`poolsideai/reference_architectures`](https://github.com/poolsideai/reference_architectures/tree/main/aws) repository, alongside the Terraform modules, example roots, and supporting documentation.
## Architecture
![Poolside inference reference architecture for AWS, showing the VPC, EKS cluster with a GPU node group, per-model Deployments behind an Application Load Balancer, and IRSA-based access to the S3 checkpoint bucket and Amazon ECR]()
The inference deployment relies on the following AWS layers.
### Network
A VPC with public and private subnets across multiple availability zones:
* **Public subnets**: the internet-facing Application Load Balancer, when you expose models outside the VPC.
* **Private worker subnets**: the EKS worker nodes, with outbound internet through NAT gateways.
An S3 gateway VPC endpoint routes checkpoint downloads directly to Amazon S3, bypassing the NAT gateways and reducing data transfer cost. If your worker nodes pull from Amazon ECR through private networking, the network also needs the Amazon ECR interface VPC endpoints required for private image pulls.
### EKS cluster
A managed Kubernetes cluster, version 1.29 or later, with an IAM OIDC provider enabled. The OIDC provider is what makes IRSA work, so the model servers read checkpoints from S3 without static credentials.
### GPU node group
A GPU node group with enough GPU memory for the models you deploy, running an EKS-optimized GPU AMI and the NVIDIA GPU Operator so each node advertises the `nvidia.com/gpu` resource. The reference architecture sets `p5e.48xlarge` as the minimum instance type for the supported model performance profile. `p5en.48xlarge` and `p5.48xlarge` are the other supported shapes.
A `p5e.48xlarge` node provides eight H200 GPUs. You place models on the node by GPU count rather than by instance: each model's `gpus` value reserves that many GPUs, and several models share a node until its GPUs are used up. To begin planning model placement, compare its [estimated total GPU memory](/deployment/supported-configurations#planning-estimates) with the total memory available across the assigned [GPU type](/deployment/supported-configurations#gpu-types). Confirm the GPU count for each model with your Poolside account team.
**Node volume sizing**
Size each GPU node's root volume for what the model servers stage locally, not only for the operating system and image. On startup, a model server downloads its entire checkpoint, which is tens of GB, onto the node, on top of the `atlas` image. A default-sized node volume can fill before the pod becomes ready. The reference deployment uses a 300 GB node volume.
**Scheduling under the GPU taint**
The GPU nodes carry the `nvidia.com/gpu=true:NoSchedule` taint, which has two scheduling consequences. Provision a separate non-GPU node group for the cluster controllers that are not GPU workloads, such as the AWS Load Balancer Controller and the GPU Operator's controller, so they have somewhere to run. Configure the GPU Operator's node-level components, including its Node Feature Discovery worker, to tolerate the taint, or they cannot run on the GPU nodes and the nodes never advertise the `nvidia.com/gpu` resource.
**Capacity**
The supported GPU instances are in high demand and are usually not available on demand. To guarantee an instance, reserve capacity before you create the node group, either an On-Demand Capacity Reservation or an EC2 Capacity Block for ML, then launch the node group into the reservation.
* The reference Terraform consumes a reservation for you: set the capacity reservation or capacity block input on the GPU node group, and the launch template targets it.
* If you provision the node group yourself, target the reservation explicitly in the node group's launch template. A managed node group does not consume a Capacity Reservation unless its launch template names that reservation, and a RAM-shared targeted reservation from another account is not consumed automatically.
For the mechanics of reserving capacity, see the AWS documentation on [On-Demand Capacity Reservations](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/capacity-reservations-using.html) and [Capacity Blocks for ML](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-capacity-blocks.html).
### Object storage and registry
* **Amazon S3** for the model checkpoints, with server-side encryption using a customer-managed KMS key.
* **Amazon ECR** for the `atlas` inference container image, with pulls authorized by the GPU node group's instance role.
### Ingress and TLS
The AWS Load Balancer Controller reconciles the per-model `Ingress` objects into a shared Application Load Balancer. The load balancer terminates TLS with an AWS Certificate Manager certificate that covers the model hostnames.
### Access
IRSA on the shared `inference` service account grants the model servers read-only access to the checkpoint bucket and decrypt access to its KMS key, and nothing else.
## Key opinions
The reference architecture commits to the following decisions. They distinguish it from a generic Amazon EKS install. If you reproduce the architecture by hand, follow them to stay aligned with what Poolside support and the rest of this documentation expect.
* **IRSA for object storage**: the model servers reach S3 through an IAM role assumed by the `inference` service account, not a mounted credentials secret.
* **ALB ingress**: traffic enters the cluster through the AWS Load Balancer Controller, which provisions one shared Application Load Balancer for all models.
* **Customer-managed KMS key for S3**: the checkpoint bucket uses SSE-KMS with a key you control, and the IRSA policy grants decrypt access to that key.
* **Minimum GPU instance type `p5e.48xlarge`**: required for the supported model performance profile.
## Use the reference architecture
You can use the reference architecture in three ways:
* **Apply it directly**: Clone the repository, configure the example for your environment, and run `terraform apply`.
* **Fork it**: Take the example as a starting point and adapt the inputs, modules, or wrapper to your standards.
* **Reproduce it by hand**: Use the architecture and the opinions on this page as a specification, and build the equivalent foundation in your own infrastructure-as-code.
For the full set of AWS resources, the Terraform modules, and the example roots, see the [`poolsideai/reference_architectures`](https://github.com/poolsideai/reference_architectures/tree/main/aws) repository.
## Related resources
* [Amazon EKS deployment](/deployment/cloud/aws-eks/overview)
* [Install on Amazon EKS](/deployment/cloud/aws-eks/install)
* [Manage models on Amazon EKS](/deployment/cloud/aws-eks/manage-models)
* [Supported configurations](/deployment/supported-configurations)
# Remove from Amazon EKS
Source: https://docs.poolside.ai/deployment/cloud/aws-eks/remove
Instructions for removing a model inference deployment from an Amazon EKS cluster.
## Overview
These steps remove the `inference` Helm release, its Kubernetes resources, and the AWS artifacts used by a model inference deployment on Amazon EKS.
The removal process includes the following phases:
1. **Uninstall the inference release**: Uninstall the `inference` Helm release, which also tears down the Application Load Balancer.
2. **Delete cluster resources**: Delete the `poolside-models` namespace, which removes the remaining workloads, secrets, ConfigMaps, and Ingress objects.
3. **Clean up AWS artifacts**: Remove the IRSA role, ECR repository, S3 checkpoints, ACM certificate, and DNS records.
## Prerequisites
These instructions assume that you deployed model inference using [Install on Amazon EKS](/deployment/cloud/aws-eks/install). Before you start, make sure you have:
* Cluster administrator access to the EKS cluster
* `helm` `3.12` or later
* `kubectl`, configured for your EKS cluster
* `aws` CLI, with permission to delete the AWS resources you provisioned
## Step 1: Uninstall the inference release
List the Helm releases in the `poolside-models` namespace:
```bash theme={null}
helm list -n poolside-models
```
Uninstall the release. This deletes the per-model Ingress objects, which prompts the AWS Load Balancer Controller to delete the shared Application Load Balancer:
```bash theme={null}
helm uninstall inference -n poolside-models
```
Wait for the inference pods to stop. Continue when `kubectl get pods -n poolside-models` shows no Poolside pods remaining:
```bash theme={null}
kubectl get pods -n poolside-models
```
Confirm the load balancer is gone before you continue:
```bash theme={null}
kubectl get ingress -n poolside-models
```
## Step 2: Delete cluster resources
Deleting the namespace removes the remaining workloads, secrets, ConfigMaps, and any Ingress objects that the inference release created:
```bash theme={null}
kubectl delete namespace poolside-models
```
## Step 3: Clean up AWS artifacts
Clean up the following AWS resources separately. Remove only the resources that you no longer need.
**IRSA role**
Delete the inference-pod IAM role and its permissions policy. If you created them with `eksctl create iamserviceaccount`, delete them with `eksctl`:
```bash theme={null}
eksctl delete iamserviceaccount \
--cluster \
--namespace poolside-models \
--name inference
```
If you created the role and policy directly, detach the policy and delete both with the `aws iam` commands you use for IAM cleanup.
**Amazon ECR**
Delete the `atlas` repository if no other deployment uses it:
```bash theme={null}
aws ecr delete-repository --repository-name atlas --force --region
```
**Amazon S3**
Delete the model checkpoints from the bucket you referenced in `models..model` in the `inference_values.yaml` file you used to install:
```bash theme={null}
aws s3 rm s3:///checkpoints --recursive --region
```
If the bucket only stored Poolside checkpoints, you can delete the bucket itself.
**TLS certificate and DNS records**
Remove the DNS records that pointed to the model hostnames. If the ACM certificate covered only those hostnames, delete it:
```bash theme={null}
aws acm delete-certificate --certificate-arn --region
```
**Local files**
Delete the extracted Helm bundle directory and your `inference_values.yaml` file from your workstation.
## Related resources
* [Install on Amazon EKS](/deployment/cloud/aws-eks/install)
* [Amazon EKS deployment](/deployment/cloud/aws-eks/overview)
* [Cloud deployment overview](/deployment/cloud/overview)
For questions about the removal process, contact Poolside support.
# Upgrade on Amazon EKS
Source: https://docs.poolside.ai/deployment/cloud/aws-eks/upgrade
Upgrade an existing model inference deployment on Amazon EKS to a new bundle.
This guide assumes that you deployed model inference using the instructions in [Install on Amazon EKS](/deployment/cloud/aws-eks/install).
## Overview
This guide describes how to upgrade an existing model inference deployment on Amazon EKS to a new Helm bundle. The upgrade updates the `inference` Helm release.
The upgrade process includes the following phases:
1. **Prepare the new bundle**: Extract the bundle and reuse the values file from the previous deployment. Add any new values the new chart requires.
2. **Upload new container images**: Push the new bundle's container images into Amazon ECR.
3. **Upgrade the inference release**: Run `helm upgrade` against the `inference` chart.
4. **Verify**: Confirm that the new revision is deployed and the pods are healthy.
## Prerequisites
* A working model inference deployment completed with [Install on Amazon EKS](/deployment/cloud/aws-eks/install).
* The new deployment bundle provided by Poolside.
* The customized `inference_values.yaml` file used for the initial deployment.
* Workstation tools, same versions as the initial deployment:
* `helm` `3.12` or later
* `kubectl`, configured for your EKS cluster
* `skopeo`, to copy the bundled images into Amazon ECR
* `aws` CLI
## Downtime
The upgrade rolls model pods one Deployment at a time. The chart sets `maxSurge` to 0 so a rolled model does not request additional GPUs during the rollout, which means that model goes down briefly while its new pod starts. Each model server also re-downloads its checkpoint from S3 on restart, so expect a delay before a rolled model becomes ready. Plan a maintenance window if you run single-replica models.
## Step 1: Extract the new bundle
Poolside provides the new bundle as a tarball. Extract it to a directory of your choice, then set a shell variable for the new bundle root:
```bash theme={null}
export NEW_BUNDLE=
```
## Step 2: Review the values file
Reuse the `inference_values.yaml` file from your previous deployment. Poolside notes any required values changes in the release notes. The new bundle contains the reference `values.yaml` for the `inference` chart at `charts/inference/values.yaml`. Use it as a reference while reviewing your existing file.
## Step 3: Upload the new container images
The new bundle ships updated container images in `./containers/`. Authenticate `skopeo` to your ECR registry, then push the images to the same repositories that the inference release uses:
```bash theme={null}
aws ecr get-login-password --region \
| skopeo login --username AWS --password-stdin .dkr.ecr..amazonaws.com
cd $NEW_BUNDLE
./scripts/upload_images.sh .dkr.ecr..amazonaws.com
```
The image tags are specific to the new bundle, not fixed values. After the upload completes, confirm the `atlas` tag that was pushed before you continue:
```bash theme={null}
aws ecr describe-images --repository-name atlas --region \
--query 'sort_by(imageDetails,&imagePushedAt)[-1].imageTags' --output text
```
## Step 4: Dry-run the upgrade (optional)
Preview the changes before you apply them:
```bash theme={null}
helm upgrade inference \
$NEW_BUNDLE/charts/inference \
-f \
-n poolside-models --dry-run --debug | less
```
## Step 5: Apply the upgrade
Run the upgrade and watch the pods roll. The pods should return to a `Running` state when the upgrade completes:
```bash theme={null}
helm upgrade inference \
$NEW_BUNDLE/charts/inference \
-f \
-n poolside-models
kubectl get pods -n poolside-models -w
```
## Step 6: Update models (optional)
You can add, update, or remove model checkpoints as part of this upgrade rather than as a separate operation. Make the model edits in the same `inference_values.yaml` file you reviewed in Step 2, before you run the `helm upgrade` in Step 5. The single `helm upgrade` then reconciles both the new chart and the model changes.
For the full procedure to add, update, or remove models, see [Manage models on Amazon EKS](/deployment/cloud/aws-eks/manage-models). You can also run those changes separately at any time after the upgrade.
## Verification
Confirm the release is deployed:
```bash theme={null}
helm history inference -n poolside-models
```
Verify that all pods are healthy:
```bash theme={null}
kubectl get pods -n poolside-models
```
Confirm that the inference endpoints still serve traffic, where `` is the `ingressHost` of a model under `models`:
```bash theme={null}
curl -s https:///v1/models \
-H "Authorization: Bearer "
```
If API key authentication is off, omit the `Authorization` header.
## Troubleshooting
* **Pods stuck pulling images**: Verify that the new tag is present in the `atlas` ECR repository and that the GPU node group's instance role still has the `AmazonEC2ContainerRegistryReadOnly` policy.
* **Model pods stuck in `Init`**: Each model re-downloads its checkpoint from S3 on restart. Check the init container logs and confirm the checkpoint paths in `inference_values.yaml` are still valid.
## Related resources
* [Install on Amazon EKS](/deployment/cloud/aws-eks/install)
* [Manage models on Amazon EKS](/deployment/cloud/aws-eks/manage-models)
* [Amazon EKS deployment](/deployment/cloud/aws-eks/overview)
* [Remove from Amazon EKS](/deployment/cloud/aws-eks/remove)
# Install on OpenShift
Source: https://docs.poolside.ai/deployment/cloud/openshift/install
Deploy Poolside model inference on OpenShift and serve models through an OpenAI-compatible API.
Follow these steps to deploy Poolside model inference on your GPU-backed OpenShift cluster. For an overview of this deployment approach and architecture, see [OpenShift deployment overview](/deployment/cloud/openshift/overview).
## Prerequisites
Poolside distributes the Helm deployment bundle as a `.tar.gz` archive. Extract it before you start:
```bash theme={null}
tar -xzf .tar.gz
cd
```
Confirm that you are working from the root of the extracted bundle. The bundle root contains the following directories:
```text theme={null}
./scripts/
./containers/
./charts/
./binaries/
```
**Cluster requirements**
* OpenShift 4.16 or later
* GPU nodes with enough GPUs for the models you deploy
* NVIDIA GPU Operator 26.3.0, with NVIDIA driver and NVIDIA Container Toolkit 1.19.0
* DNS records that resolve to the cluster router endpoint, or use a router-generated hostname
* An S3-compatible object storage service such as NooBaa (OpenShift Data Foundation), Amazon S3, or MinIO
* A container registry that your cluster can access
**Workstation tools**
Install the following tools on the host you use to run the deployment:
* `helm` `3.12` or later
* `oc` or `kubectl`
* `skopeo`
* `aws` CLI (to upload checkpoints to S3-compatible object storage)
* `jq` (to parse JSON responses from the inference API)
* `tar` (to extract the deployment bundle)
* `curl` (to call the inference API)
* `openssl` (optional, to generate a TLS certificate for the inference endpoint)
**Minimum resource requirements**
Ensure that your cluster has enough GPUs for the models you deploy. If you have questions about the required specs, contact Poolside support.
## Step 1: Create the namespace
The inference stack runs in a single namespace:
```bash theme={null}
oc create namespace poolside-models
```
## Step 2: Upload container images
Copy the bundled images into your registry. Log in to your target registry using `docker login` or `podman login` before running any upload commands.
Authenticate skopeo against your target registry:
```bash theme={null}
skopeo login --username --password
```
Upload the images with the provided script:
```bash theme={null}
chmod +x ./scripts/upload_images.sh
./scripts/upload_images.sh
```
If your registry requires authentication, create an image pull secret in `poolside-models`:
```bash theme={null}
oc create secret docker-registry poolside-registry-secret \
--docker-server= \
--docker-username= \
--docker-password= \
-n poolside-models
```
If you use the OpenShift internal registry, push the images into the
`poolside-models` namespace. Pods in that namespace pull same-namespace
imagestreams with the default service account, so no cross-namespace
`system:image-puller` rolebinding is required.
## Step 3: Upload model checkpoints
The inference stack downloads model weights from your S3 bucket on pod startup, so the checkpoints must be in place before you deploy the chart. Poolside provides the checkpoint files separately from the deployment bundle. Confirm the local path and the destination prefix with your Poolside contact.
Uploading checkpoints is time consuming. Start it now and continue with the remaining steps in parallel.
Poolside provides model checkpoints as `.tar` archives. The inference chart does not extract archives at pod startup. It syncs unpacked checkpoint files (`*.safetensors`, `*.json`, and the tokenizer files) from the S3 prefix you set in `models..model`, so extract each archive's contents into its own directory before uploading.
```bash theme={null}
tar -xf ./checkpoints/.tar
rm -v ./checkpoints/.tar
```
Confirm the files sit at the root of the directory, not under a subfolder:
```bash theme={null}
ls ./checkpoints/
# config.yaml generation_config.json model.safetensors tokenizer/
```
Create the bucket if it does not already exist. The example uses the NooBaa endpoint; for AWS S3, omit the `--endpoint-url` flag:
```bash theme={null}
aws s3 mb s3:// --endpoint-url https:// --region
```
Note the bucket name; you reference it in the `models..model` paths in [Step 5](#step-5-configure-the-inference-values-file).
Then upload the checkpoints to the bucket:
```bash theme={null}
aws s3 cp ./checkpoints s3:///checkpoints --recursive --region
```
For a non-AWS S3 endpoint such as NooBaa or MinIO, add `--endpoint-url`:
```bash theme={null}
aws s3 cp ./checkpoints s3:///checkpoints \
--recursive \
--endpoint-url https:// \
--region
```
Checkpoints are typically tens of GiB per model. For faster throughput, or for
backends sensitive to upload concurrency such as NooBaa, run the upload from a
host inside the cluster and tune `aws configure set
default.s3.max_concurrent_requests` and `default.s3.multipart_chunksize`.
## Step 4: Create the S3 credentials secret
The model servers read checkpoints from S3 using credentials in a Kubernetes secret. Create it in `poolside-models`:
```bash theme={null}
oc create secret generic aws-credentials \
--from-literal=AWS_ACCESS_KEY_ID= \
--from-literal=AWS_SECRET_ACCESS_KEY= \
-n poolside-models
```
**API key authentication (optional)**
To require an API key on the vLLM inference servers, create a secret containing the key in `poolside-models`:
```bash theme={null}
oc create secret generic vllm-auth \
--from-literal=VLLM_API_KEY= \
-n poolside-models
```
Creating the secret does not enable API key authentication by itself. In [Step 5](#step-5-configure-the-inference-values-file), set `authentication.secretName` to `vllm-auth`.
## Step 5: Configure the inference values file
Create an `inference_values.yaml` file in the bundle root:
```bash theme={null}
cp ./charts/inference/values.yaml ./inference_values.yaml
```
Set the fields that apply to your environment. The example below deploys two models and exposes each model through its own OpenShift Route:
```yaml title="Example: inference_values.yaml" theme={null}
image:
# -- Registry you uploaded the atlas image to (required)
registry: ""
# -- Image name and tag come pre-set in the bundle to match the shipped image
name: "atlas"
tag: ""
# -- Name of the image pull secret for private registries (omit if your registry is public)
imagePullSecret: "poolside-registry-secret"
podSecurityContext:
# -- Require non-root user. Do not set runAsUser on OpenShift; the SCC injects a UID from the namespace range.
runAsNonRoot: true
seccompProfile:
# -- Seccomp profile type
type: RuntimeDefault
s3:
# -- Name of secret containing AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
secretName: "aws-credentials"
# -- Custom CA certificate bundle for S3 (required for NooBaa with the OpenShift service CA)
caBundle: ""
authentication:
# -- Name of secret containing VLLM_API_KEY for vLLM server authentication (set to "vllm-auth" if you created the optional secret in Step 4; leave empty to disable)
secretName: ""
route:
# -- Create a Route for every model
enabled: true
tls:
# -- Terminate TLS at the OpenShift router
enabled: true
termination: edge
insecureEdgeTerminationPolicy: Redirect
models:
laguna-xs:
model: s3:///checkpoints/laguna-xs
modelName: Lagunaxs
modelType: agent
gpus: 1
# -- Route host for this model (leave empty for a router-generated hostname)
routeHost: ""
laguna-m:
model: s3:///checkpoints/laguna-m
modelName: Lagunam
modelType: agent
gpus: 4
# -- Route host for this model (leave empty for a router-generated hostname)
routeHost: ""
```
The checkpoint paths in `models..model` and the image registry must exactly match the locations you uploaded from the deployment bundle. The image `name` and `tag` come pre-set to match the shipped `atlas` image.
Set each model's `gpus` to a value that provides enough GPU memory for the model and workload on your GPU type. For planning estimates, see [Supported configurations](/deployment/supported-configurations).
Each model is exposed through a separate `Route` named `inference-`. Leave `routeHost` empty to let the OpenShift router generate a hostname per model, or set an explicit host. The Route sends the host's root path directly to that model's vLLM service, so clients reach the OpenAI-compatible API at `https:///v1`.
**NooBaa and non-AWS S3 endpoints**
If your object storage is NooBaa or another non-AWS S3 service, point the model servers at the endpoint and region:
```yaml theme={null}
extraEnv:
AWS_REGION: ""
AWS_ENDPOINT_URL_S3: "https://s3.openshift-storage.svc:443"
```
For NooBaa with the OpenShift service CA, the model servers also need the service CA to trust the S3 endpoint. The inference chart takes it as inline text in `s3.caBundle`, supplied at deployment time in [Step 6](#step-6-install-the-inference-chart).
NooBaa and other S3 backends with limited concurrency need throttled downloads. Without throttling, the init container can fail after downloading 1-2 GiB and restart in an infinite loop because the `emptyDir` volume is wiped on each restart:
```yaml theme={null}
awsCliConfig:
default.s3.max_concurrent_requests: "2"
default.s3.max_queue_size: "1000"
default.s3.multipart_chunksize: "64MB"
```
## Step 6: Install the inference chart
Install the `inference` chart into `poolside-models`. If your S3 backend uses a publicly trusted certificate, install the chart directly:
```bash theme={null}
helm install inference ./charts/inference \
--namespace poolside-models \
-f ./inference_values.yaml
```
If you use NooBaa, the model servers must trust its self-signed S3 serving certificate. NooBaa's certificate is signed by the OpenShift service CA and rotates automatically, so extract it fresh from the `noobaa-s3-serving-cert` secret rather than committing it to your values file:
```bash theme={null}
oc get secret noobaa-s3-serving-cert -n openshift-storage \
-o jsonpath='{.data.tls\.crt}' | base64 -d > service-ca.crt
```
Then install with the certificate passed inline through `--set-file`:
```bash theme={null}
helm install inference ./charts/inference \
--namespace poolside-models \
-f ./inference_values.yaml \
--set-file s3.caBundle=./service-ca.crt
```
## Step 7: Verify the deployment
Check that the model pods are running. The only pods in the namespace are the per-model servers:
```bash theme={null}
oc get pods -n poolside-models
```
Each model server takes time to become ready on first start because it downloads its checkpoint from S3. Watch a model's logs to track progress. The `` is the key you set under `models` in the values file, such as `laguna-m` or `laguna-xs` in the [Step 5](#step-5-configure-the-inference-values-file) example:
```bash theme={null}
oc logs -f -n poolside-models deploy/inference-
```
Confirm a Route was created for each model and note its host:
```bash theme={null}
oc get route -n poolside-models
```
List the served models on a model's endpoint to confirm routing works, where `` is the host of that model's Route:
```bash theme={null}
curl -s https:///v1/models
```
## Step 8: Call the inference API
Each model serves the OpenAI-compatible API directly at its own Route host. The base URL has the form:
```text theme={null}
https:///v1
```
Append the OpenAI-compatible route to the base URL, such as `/chat/completions` or `/completions`.
The commands below use three placeholders. Fill the model values from the `inference_values.yaml` you wrote in [Step 5](#step-5-configure-the-inference-values-file); OpenShift assigns each model's Route host unless you set `routeHost`:
| Placeholder | Source | Example |
| --------------------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| `` | assigned by OpenShift per model (or `models..routeHost` if you set a custom host) | `inference-laguna-m-poolside-models.apps.cluster.example.com` |
| `` | a key under `models` | `laguna-m` |
| `` | `models..modelName` | `Laguna` |
Retrieve each value from the running cluster.
Retrieve the `` values. Each model deployment is named `inference-`:
```bash theme={null}
oc get deploy -n poolside-models -l app.kubernetes.io/component=inference
```
Retrieve `` from the model's Route:
```bash theme={null}
oc get route inference- -n poolside-models -o jsonpath='{.spec.host}'
```
Retrieve `` from the `id` field of that model's models endpoint:
```bash theme={null}
curl -s https:///v1/models | jq -r '.data[].id'
```
Send a chat completion request:
```bash theme={null}
curl https:///v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "",
"messages": [{"role": "user", "content": "Write a function that reverses a string."}]
}'
```
For example, to call the `laguna-m` model served as `Laguna`:
```bash theme={null}
curl https://inference-laguna-m-poolside-models.apps.cluster.example.com/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Laguna",
"messages": [{"role": "user", "content": "Write a function that reverses a string."}]
}'
```
If you set `authentication.secretName` in Step 5, include the key as a bearer token:
```bash theme={null}
curl https:///v1/chat/completions \
-H "Authorization: Bearer " \
-H "Content-Type: application/json" \
-d '{
"model": "",
"messages": [{"role": "user", "content": "Write a function that reverses a string."}]
}'
```
## TLS
The Route example in [Step 5](#step-5-configure-the-inference-values-file) uses `edge` termination, where the OpenShift router terminates TLS with its default certificate. To serve a custom certificate, provide it inline under `route.tls`. This block applies to every model's Route, so the certificate must be valid for all model Route hosts (for example, a wildcard certificate):
```yaml theme={null}
route:
enabled: true
tls:
enabled: true
termination: edge
insecureEdgeTerminationPolicy: Redirect
certificate: ""
key: ""
caCertificate: ""
```
## Offline documentation (optional)
The bundle also ships the Poolside documentation site, which the same `inference` chart can deploy in-cluster so operators have local access to the docs. It is off by default. To enable and expose it through a Route, see [Set up offline documentation](/deployment/cloud/set-up-offline-documentation).
## Troubleshooting
* If pods stay in `Init` or restart in a loop, check the init container logs with `oc logs -n poolside-models -c `. A stale or misspelled checkpoint path syncs nothing and the pod never starts.
* If model pods never become ready and the init container logs show no checkpoint files synced, the S3 prefix may hold the checkpoint `.tar` instead of its extracted contents. The downloader only fetches unpacked files such as `*.safetensors` and `*.json`. Extract the archive and re-upload its contents, as in [Step 3](#step-3-upload-model-checkpoints).
* If checkpoint downloads fail against NooBaa, confirm the S3 CA bundle is mounted and review the `awsCliConfig` throttle settings in Step 5.
* If model servers fail to pull images, run `oc describe pod -n poolside-models` and verify the image pull secret or internal-registry pull access.
* If a model pod is `Pending`, confirm the cluster has enough GPUs for the `gpus` value you requested and that the NVIDIA GPU Operator is healthy.
## Related resources
* [OpenShift deployment overview](/deployment/cloud/openshift/overview)
* [Set up offline documentation](/deployment/cloud/set-up-offline-documentation)
* [Upgrade on OpenShift](/deployment/cloud/openshift/upgrade)
* [Remove from OpenShift](/deployment/cloud/openshift/remove)
For questions about hardware requirements, infrastructure configuration, or deployment issues, contact Poolside support.
# Manage models on OpenShift
Source: https://docs.poolside.ai/deployment/cloud/openshift/manage-models
Add, update, or remove inference models in an existing inference deployment on OpenShift.
## Overview
Use this guide to change the set of models served by a running `inference` release: adding a new model, replacing a model's checkpoint, or removing a model. You edit your `inference_values.yaml` file and run `helm upgrade`; the chart reconciles the model Deployments, Services, and Routes to match.
You can make these changes on their own against the current chart version, or apply them as part of a chart upgrade to a new Poolside bundle. To upgrade the chart, see [Upgrade on OpenShift](/deployment/cloud/openshift/upgrade); make the model edits described here in the same `inference_values.yaml` file before you run `helm upgrade`.
## Prerequisites
* A working deployment completed with the [Install on OpenShift](/deployment/cloud/openshift/install) guide.
* The customized `inference_values.yaml` file you used to install.
* The new model checkpoint, provided by Poolside.
* Workstation tools:
* `helm` `3.12` or later
* `oc` or `kubectl`
* `aws` CLI (to upload checkpoints to S3-compatible object storage)
* `jq` (to parse JSON responses from the inference API)
## Downtime
Adding a model does not affect models that are already serving. Updating a checkpoint rolls that model's Deployment, and the model server re-downloads the checkpoint from S3 on restart, so expect a delay before it becomes ready again. Plan a maintenance window for single-replica models.
## Add a model
Extract the new checkpoint archive as described in [Upload model checkpoints](/deployment/cloud/openshift/install#step-3-upload-model-checkpoints) so its files sit at the prefix root, then upload it to your S3 bucket. Use a distinct prefix per model. For NooBaa or another non-AWS endpoint, include `--endpoint-url`:
```bash theme={null}
aws s3 cp ./checkpoints/ s3:///checkpoints/ \
--recursive \
--endpoint-url https:// \
--region
```
For checkpoint upload details such as concurrency throttling and the S3 CA bundle, see [Upload model checkpoints](/deployment/cloud/openshift/install#step-3-upload-model-checkpoints).
Add a new key under `models` in your `inference_values.yaml` file. Give the model its own `routeHost`, or leave it empty for a router-generated hostname:
```yaml title="Example: inference_values.yaml" theme={null}
models:
# ...existing models...
:
model: s3:///checkpoints/
modelName:
modelType: agent
gpus: 1
# -- Route host for this model (leave empty for a router-generated hostname)
routeHost: ""
```
Apply the change with `helm upgrade`. Use the same flags you used to install. If your install command used `--set-file s3.caBundle=...` because your S3 backend uses a private CA such as NooBaa, include that flag every time you run `helm upgrade` on this page:
```bash theme={null}
helm upgrade inference ./charts/inference \
--namespace poolside-models \
-f ./inference_values.yaml
```
The chart creates a new `Deployment`, `Service`, and `Route` named `inference-` for the model. Confirm the new pod starts and the Route is created:
```bash theme={null}
oc get pods -n poolside-models
oc get route inference- -n poolside-models
```
## Update a model checkpoint
Upload the new checkpoint to a new, versioned prefix rather than overwriting the existing one. A new path lets `helm upgrade` detect the change and roll the Deployment automatically, and it lets you roll back by pointing at the previous path. Extract the archive first, as in [Upload model checkpoints](/deployment/cloud/openshift/install#step-3-upload-model-checkpoints), so its files sit at the prefix root:
```bash theme={null}
aws s3 cp ./checkpoints/- s3:///checkpoints/- \
--recursive \
--endpoint-url https:// \
--region
```
Point the model's `model` field at the new path in your `inference_values.yaml` file. Update `modelName` only if the served model name changes:
```yaml title="Example: inference_values.yaml" theme={null}
models:
laguna:
model: s3:///checkpoints/laguna-
modelName: Laguna
modelType: agent
gpus: 4
routeHost: ""
```
Apply the change:
```bash theme={null}
helm upgrade inference ./charts/inference \
--namespace poolside-models \
-f ./inference_values.yaml
```
The model's Deployment rolls, and the init container downloads the new checkpoint on startup. Watch the rollout:
```bash theme={null}
oc rollout status deploy/inference-