> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ragen.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Configuring AI Language Model Providers in Ragen AI
> Set which model answers, rephrases, summarises and embeds — and reach any provider directly or through a gateway you run.
Ragen calls language models in four roles — answering questions, rephrasing queries, summarising documents, and generating embeddings. This page is about **which model** fills each role. [Model gateway](/configuration/model-gateway) is about **how Ragen reaches it**: a route table naming the upstream for each model id.
Two things have to agree, and this is the mistake worth avoiding before you make it: the variables below name a **model id**, and that id has to exist in the route table. A name with no route is a name nothing can serve.
## Model Roles
Ragen calls language models in four distinct roles. Each role has its own environment variable and its own default, and each default points to a cloud-hosted model. Set all four when you want full control over costs, privacy, or latency.
| Role | Environment Variable | Default | Called When |
| -------------------------------- | -------------------- | ------------------------- | ------------------------------------------ |
| Chat / answering | `DEFAULT_MODEL` | `gemini-3-flash-preview` | Every question a user asks |
| Rephrase + multi-query expansion | `REPHRASE_MODEL` | `gemini-2.5-flash` | Before retrieval on every question |
| Document summary at ingest | `SUMMARY_MODEL` | `gemini-2.5-flash` | When `FEATURE_FLAG_DOC_SUMMARIES=1` is set |
| Embeddings | `EMBEDDINGS_MODEL` | `bge-multilingual-gemma2` | Every document ingest and every query |
`VECTOR_SIZE` must match the output dimensions of your `EMBEDDINGS_MODEL`. If they disagree, Qdrant will reject every vector upsert and document ingest will silently fail. The default `bge-multilingual-gemma2` model produces **3584**-dimensional vectors; `cohere-embed-multilingual-v3` produces **1024**. Check your model's specification rather than assuming.
Changing `EMBEDDINGS_MODEL` or `VECTOR_SIZE` after you have ingested documents invalidates the entire Qdrant collection. You must re-index all documents from scratch if you switch embedding models.
A complete set of model roles:
```bash theme={null}
DEFAULT_MODEL=gemini-3-flash-preview
REPHRASE_MODEL=gemini-2.5-flash
REPHRASE_TEMPERATURE=0.5
SUMMARY_MODEL=gemini-2.5-flash
EMBEDDINGS_MODEL=bge-multilingual-gemma2
# VECTOR_SIZE=3584 # must match EMBEDDINGS_MODEL output dimensions
```
## Supported Providers
Every major provider is reachable. The model is named in the [route table](/configuration/model-gateway); the credentials come from the environment, per provider family.
GPT-4o, o3, and all OpenAI models.
```bash theme={null}
OPENAI_API_KEY=sk-...
```
Claude 3.5, Claude 4, and Haiku variants.
```bash theme={null}
ANTHROPIC_API_KEY=sk-ant-...
```
Gemini models via a GCP service account.
```bash theme={null}
VERTEX_PROJECT=your-gcp-project-id
VERTEX_LOCATION=europe-west1
VERTEX_CREDENTIALS={"type":"service_account",...}
```
Claude, Cohere, and Titan models via Bedrock.
```bash theme={null}
AWS_BEDROCK_REGION=eu-central-1
# Credentials come from the AWS default chain — static keys, an instance
# role, or SSO. Static keys look like this:
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
```
IAM permissions: [`aws-iam-policy.json`](https://github.com/webamigos/RagenAI/blob/main/docs/aws-iam-policy.json).
GPT models deployed to your Azure subscription.
```bash theme={null}
AZURE_API_KEY=...
AZURE_API_BASE=https://your-instance.openai.azure.com
AZURE_API_VERSION=2024-12-01-preview
```
The deployment name is the route's `model`, not an environment variable.
Mistral, BGE embeddings, and other Scaleway Generative APIs.
```bash theme={null}
SCW_API_BASE=https://api.scaleway.ai/YOUR_PROJECT_ID/v1
SCW_API_KEY=...
```
Spend dashboards, provider fallback, caching and retries in front of any upstream. Self-host it as one container, or use the hosted service.
```bash theme={null}
LLM_PORTKEY_BASE_URL=http://localhost:8787/v1
LLM_PORTKEY_API_KEY=
LLM_PORTKEY_HEADERS={"x-portkey-provider":"openai"}
```
See [Model gateway](/configuration/model-gateway#portkey).
Hundreds of models behind one key. OpenRouter speaks OpenAI's API, so it attaches as a **connection**, not as a provider of its own:
```bash theme={null}
LLM_OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
LLM_OPENROUTER_API_KEY=sk-or-...
```
Give the route `provider: openai-compatible` and `connection: openrouter`. A bare `OPENROUTER_API_KEY` is read by nothing.
Self-hosted models via the vLLM OpenAI-compatible server.
See the [Local Models](/configuration/open-models) page for full setup instructions.
Local models running in Ollama on your own hardware. OpenAI-compatible, so it attaches as a **connection**:
```bash theme={null}
LLM_OLLAMA_BASE_URL=http://localhost:11434/v1
LLM_OLLAMA_API_KEY=unused
```
See the [Local Models](/configuration/open-models) page for full setup instructions.
## Registering a Model
A model id you set in `DEFAULT_MODEL`, `REPHRASE_MODEL` or `EMBEDDINGS_MODEL` has to appear in the route table, which says which upstream serves it and under what name *there*. The shipped table is `infra/llm-gateway/routes.yaml`; point `LLM_ROUTES_PATH` at your own file to serve a different set.
```yaml title="infra/llm-gateway/routes.yaml" theme={null}
version: 1
routes:
gemini-3-flash-preview:
provider: vertex
model: gemini-3-flash-preview
location: global
gemini-2.5-flash:
provider: vertex
model: gemini-2.5-flash
bge-multilingual-gemma2:
provider: openai-compatible
connection: scaleway
model: bge-multilingual-gemma2
```
The left-hand key is Ragen's name for the model — the one you put in an environment variable. `model` is the **upstream's** name for it, which is often different: Bedrock prefixes the region and the vendor (`eu.anthropic.claude-sonnet-5`), and an Azure deployment name is whatever you called it.
A route carries no credentials and no display name. Credentials come from the environment; display names and capability flags live in the model catalogue.
**The chat model picker shows what your credentials can actually serve.** It is the route table intersected with the credentials present in the app processes — so a route pointing at a provider you have no key for does not appear, rather than appearing and failing on the first click. A model you set in an environment variable but never routed will not appear either.
Check the whole set before you rely on it, with one real call per model:
```bash theme={null}
npm run gateway:preflight -- --probe
```
## Additional Model Settings
| Variable | Default | Description |
| ----------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `REPHRASE_TEMPERATURE` | `0.5` | Temperature for the rephrase model |
| `VECTOR_SIZE` | `3584` | Output dimensions of `EMBEDDINGS_MODEL`; must match exactly |
| `OPENAI_API_KEY` | — | Also used for content moderation (`MODERATION_ENABLED=1`), which calls the OpenAI Moderation API directly rather than through the route table |
| `OPENAI_MODERATION_KEY` | — | Dedicated key for moderation; falls back to `OPENAI_API_KEY` |
| `MULTIMODAL_TEXT_ONLY_MODELS` | — | Comma-separated model IDs that cannot process images; Ragen will swap to `MULTIMODAL_FALLBACK_MODEL` when a request includes an image |
| `MULTIMODAL_FALLBACK_MODEL` | — | Vision-capable model to use when the primary model is text-only |