For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
Fireworks AI
Configuration and setup for Fireworks AI LLM provider
Configure Fireworks AI as an LLM provider in agentgateway.
Configuration
Review the following example configuration.# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
models:
- name: "*"
provider: fireworks
params:
apiKey: "$FIREWORKS_API_KEY"
# Optional. If omitted, agentgateway uses the default:
# baseUrl: "https://api.fireworks.ai/inference/v1"| Setting | Description |
|---|---|
name | The model name to match in incoming requests. When a client sends "model": "<name>", the request is routed to this provider. Use * to match any model name. |
provider | The LLM provider, set to fireworks. |
params.model | Optional. If set, this model is used for all requests. If not set, the request must include the model to use. |
params.apiKey | Your Fireworks API key. You can reference environment variables using the $VAR_NAME syntax. |
params.baseUrl | Optional. Overrides the provider base URL. Default: https://api.fireworks.ai/inference/v1. |
Example request
After running agentgateway with the configuration from the previous section, you can send an OpenAI-compatible request to the v1/chat/completions endpoint:
curl -X POST http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "accounts/fireworks/models/llama-v3p1-70b-instruct",
"messages": [{"role": "user", "content": "Hello from Fireworks!"}]
}'Was this page helpful?
