Nebius Token Factory

Route agentgateway LLM traffic to models served by Nebius Token Factory.

Configure Nebius Token Factory as an LLM provider in agentgateway. The nebius provider sends Chat Completions and Embeddings requests to Nebius.

Configuration

You can add the model in the UI or in your configuration file.

To add the model in the UI, follow these steps. The steps create the same model as the Config file tab.

Set your API key in the NEBIUS_API_KEY environment variable. The UI checks that the variable exists when you save the model.
```
export NEBIUS_API_KEY='<your-api-key>'
```
Create a configuration file with an empty llm section, and start agentgateway in the same terminal. Without an llm section, the UI does not show the LLM > Models page.
```
cat > config.yaml <<'EOF'
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models: []
EOF
agentgateway -f config.yaml
```
Open LLM > Models, such as http://localhost:15000/ui/llm/models, and click Add model.
From the Provider dropdown, select Nebius Token Factory.
Keep the default nebius/* in the Incoming model match field.
For the Provider API key, select Env var, and enter NEBIUS_API_KEY.
For Outgoing model, keep Strip nebius/. This option removes the nebius/ prefix before the request is forwarded. To send every request to one model instead, select Explicit, and enter the model name. This option sets params.model.
Click Save model.

The form has no field for params.baseUrl. To override the base URL, click the Raw icon (<>) in the drawer header and add the field, or use the Config file tab.

Whether the UI can save depends on your storage mode. For more information, see Save changes from the UI.

Review the following example configuration.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: "nebius/*"
    provider: nebius
    params:
      apiKey: "$NEBIUS_API_KEY"
    transformation:
      model: llmRequest.model.stripPrefix("nebius/")

Review the following table to understand this configuration.

Setting	Description
`name`	The model name to match in incoming requests. When a client sends `"model": "<name>"`, the request is routed to this provider. The `nebius/*` pattern matches any model name that starts with `nebius/`.
`provider`	The LLM provider, set to `nebius`.
`params.model`	Optional. If set, this model is used for all requests. If not set, the request must include the model to use.
`params.apiKey`	Your Nebius API key. You can reference environment variables using the `$VAR_NAME` syntax.
`params.baseUrl`	Optional. Overrides the provider base URL. Default: `https://api.tokenfactory.nebius.com/v1`.
`transformation.model`	Removes the `nebius/` prefix from the model name before the request is forwarded, so the provider receives its own model name. Nebius model names include the vendor, such as `openai/gpt-oss-120b`.

Example requests

After running agentgateway with the configuration from the previous section, you can send an OpenAI-compatible request to the v1/chat/completions endpoint:

curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nebius/openai/gpt-oss-120b",
    "messages": [{"role": "user", "content": "Hello from Nebius!"}]
  }'

To create embeddings, send a request with a Nebius embedding model to the v1/embeddings endpoint:

curl -X POST http://localhost:4000/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nebius/Qwen/Qwen3-Embedding-8B",
    "input": "Hello from Nebius!"
  }'

Mistral AI OpenRouter

Was this page helpful?

Nebius Token Factory

Configuration

Example requests

What could be improved?