How to run a Poolside model locally with vLLM Metal on a Mac

Use vLLM Metal to serve Laguna XS 2.1 through an OpenAI-compatible endpoint on your Mac, then connect Poolside Agent CLI to the local model. For the supported Ollama setup on Apple silicon and NVIDIA systems, see How to run a Poolside model locally with Ollama. vLLM Metal is a community-maintained vLLM hardware plugin that uses MLX to run models on Apple silicon.

​

Prerequisites

• A Mac with Apple silicon and 36 GB of unified memory or more. Rosetta and x86_64 Python are not supported.
• Enough free disk space for the approximately 22 GB checkpoint, vLLM Metal, and local caches.
• An internet connection for the installer and initial checkpoint download.

​

Steps

1. If Poolside Agent CLI is not installed, install it:
   curl -fsSL https://downloads.poolside.ai/pool/install.sh | sh
   
   Verify the installation:
   pool --version
   
   The command prints the installed version. For installation details, see Install Poolside Agent CLI.
2. Install vLLM Metal:
   curl -fsSL https://raw.githubusercontent.com/vllm-project/vllm-metal/main/install.sh | bash
   
   The installer creates a native arm64 Python 3.12 environment at ~/.venv-vllm-metal and installs vLLM, vLLM Metal, and their dependencies.
3. Activate the environment and start the local model server:
   source ~/.venv-vllm-metal/bin/activate
   
   vllm serve \
     --model poolside/Laguna-XS-2.1-NVFP4-mlx \
     --tool-call-parser poolside_v1 \
     --reasoning-parser poolside_v1 \
     --enable-auto-tool-choice \
     --served-model-name laguna \
     --port 8888 \
     --max-model-len 65536
   
   Keep this terminal open while you use the model. The first start downloads the checkpoint and can take several minutes, depending on your connection.
4. In a different terminal, open the project where you want to work and connect pool to the local server:
   cd <project-path>
   
   POOLSIDE_API_KEY=local-test \
     POOLSIDE_STANDALONE_BASE_URL=http://localhost:8888/v1 \
     POOLSIDE_STANDALONE_MODEL=laguna \
     pool
   
   The local server does not validate the example API key, but pool requires a non-empty value.
5. Send a prompt:
   Summarize this project and suggest one small improvement.
   
   The setup works when the agent responds using the laguna model served from localhost:8888.

​

Adjust the context length

Model weights and the KV cache share unified memory. If the server runs out of memory, reduce --max-model-len when you restart it. When you reconnect, pool reads the updated context length from the server and adjusts its automatic compaction threshold. The checkpoint supports up to 256K tokens, but the available context depends on your Mac’s unified memory and other running applications.

​

Related resources

• How to run a Poolside model locally with Ollama
• Laguna XS 2.1 NVFP4 MLX checkpoint
• vLLM Metal documentation
• Poolside Agent CLI