Integrate & ship
Models & providers
Which models your workspace can reach, whose account pays for them, and how to pick sensibly between them.
Bring your own key#
Connect provider credentials under Integrations and every call runs against your account: your rates, your quota, your data agreement with that provider. The operator's shared key exists only so a brand-new workspace works before anything is connected.
Why it works this way
Supported providers — all 14#
These are the provider ids the chat route accepts. Anything else is rejected before a call is made.
| Provider id | Name | What you configure |
|---|---|---|
openai | OpenAI | API key; optional organization id |
anthropic | Anthropic | API key |
gemini | Google Gemini | API key |
vertex | Google Vertex AI | Service-account credentials + project and region |
bedrock | Amazon Bedrock | AWS credentials + region. Use when data must stay in your AWS account. |
azure_openai | Azure OpenAI | Endpoint + deployment name + key |
oci_genai | OCI Generative AI | Oracle Cloud tenancy credentials |
grok | Grok (xAI) | API key |
qwen | Qwen | API key |
groq | Groq | API key — fast inference for small models |
nvidia | NVIDIA NIM | API key |
openrouter | OpenRouter | One key, many models — the simplest way to try several |
ollama | Ollama | Base URL of your Ollama host. Local models, nothing leaves your machine. |
vllm | vLLM | Base URL of your vLLM server. Self-hosted open-weight models at scale. |
Note
ollama and vllm are the two that keep inference entirely on infrastructure you control — worth knowing about if the reason you are self-hosting is that prompts must not leave your network at all.Keys are encrypted at rest. Prefer storing them in Secrets and referencing them, so a rotation is one edit rather than a hunt through every connector.
Configuring the cloud providers#
Most of the fourteen need nothing but an API key. Four ask for real cloud credentials, and two more take an optional endpoint override; the exact fields are worth having in front of you, because most failed connections are one of these values in the wrong box. The two self-hosted providers are covered separately below — they take a base URL instead of a key.
| Provider | Fields | Notes |
|---|---|---|
bedrock | region, accessKeyId, secretAccessKey, and optionally sessionToken | The session token is only for temporary STS credentials — leave it blank for a long-lived IAM user. |
vertex | projectId, location, serviceAccountJson | Paste the whole service-account JSON file. It is parsed on save, so a truncated paste fails immediately rather than at first use. |
azure_openai | endpoint, apiKey, and optionally apiVersion | Endpoint is the resource root, e.g. https://my-resource.openai.azure.com. Defaults to API version 2024-08-01-preview. |
oci_genai | region, compartmentId, tenancyOcid, userOcid, fingerprint, privateKeyPem | Six fields, all from your OCI API-key config file. style selects the GENERIC or COHERE request shape. |
anthropic | apiKey | Listed here because it also supports a stored cloud credential, not just a workspace key. |
qwen | apiKey, optional baseUrl | Set the base URL if you are pointed at a regional or self-managed endpoint. |
On Azure, the model name IS the deployment name
<endpoint>/openai/deployments/<model>/chat/completions — so whatever you type as the model must be the name you gave the deployment in the Azure portal, not the underlying model name. If you deployed gpt-4o under the name prod-chat, the model is prod-chat. A wrong value here comes back as a 404 from Azure, which reads like the endpoint is wrong.Keeping inference on your own machines#
ollama and vllm take a base URL and nothing else. Both are OpenAI-compatible, so the model id is whatever the server itself reports.
ollama http://localhost:11434
vllm http://vllm.internal:8000/v1A local URL still has to be reachable from the app
localhost means localhost on the server. Running the app in Docker with Ollama on the host means http://host.docker.internal:11434, not http://localhost:11434 — the single most common reason a local model "isn't found" when it is running perfectly well.The private-network block does not apply here
BLOCK_PRIVATE_NETWORK_FETCH guards the places a URL can arrive from a user or a model — MCP endpoints, swarm HTTP nodes, page fetching. Provider base URLs are operator configuration and are called directly, because a model server on a private address is the entire point of these two. Treat the base URL as trusted input: whoever can set it can make the app call it.Model registry#
Integrations → Model Registry curates which models appear in pickers across the app. Left alone, every model your connected providers expose is offered — which is rarely what you want on a shared instance, where a handful of sensible defaults beats a list of two hundred.
- Enable or hide models per provider.
- Record context window and cost so pickers can show the trade-off at the point of choice.
- Set the workspace default for new agents.
The registry is about visibility. To control what a particular person is allowed to run, use model rules in Access control — those are enforced server-side on every request, not just hidden in the UI.
Choosing a model#
There is no single best model; there is a fit per job. A rough guide that holds up in practice:
| Job | What to favour |
|---|---|
| Classification, routing, extraction | The smallest capable model. Cheap and fast; the task is mechanical. |
| Retrieval-grounded Q&A | Mid-tier with a large context window — the work is reading, not reasoning. |
| Multi-step tool use / swarm orchestration | A strong model. Weak models pick the wrong tool and loop. |
| Long-form drafting | A strong model, higher temperature. |
| Code generation | A code-tuned model where your provider offers one. |
| Vision (screenshots, scans) | A vision-capable model — check the registry entry. |
Note
Temperature and tokens#
- Temperature
- How much randomness. Near 0 for extraction, classification and anything you'll parse; 0.5–0.8 for writing. High temperature on a tool-using agent makes it erratic about which tool it calls.
- Max tokens
- A cap on the reply length. Too low truncates mid-sentence — a common cause of a JSON response that won't parse.
- Context window
- Total budget for prompt plus reply. Long retrieved context plus long history is what exhausts it; the oldest turns fall out first.
Overrides and fallback#
An agent has a saved model, which you can override per session in Agent Chat — the fastest A/B test available. If a provider errors or a model is disallowed, the platform surfaces the reason and offers to retry with an allowed model rather than failing silently.
Cost#
Every call records tokens in/out, latency and cost against the user who made it. Spend is visible in Analytics, and can be capped per user, group or credential in Budgets. On a shared instance, set a cap before handing out access rather than after the first surprise.