For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
OpenAI SDK
Use OpenAI SDK (Python and Node.js) with agentgateway
Call configured LLM providers from your application code through agentgateway with the OpenAI SDK for Python, Node.js, or Go.
Agentgateway exposes an OpenAI-compatible API. Point the SDK’s base URL at agentgateway and send OpenAI Chat Completions requests. Agentgateway forwards each request to the provider that matches the model field and translates it when the provider uses a different API, such as Anthropic, Amazon Bedrock, or Vertex AI. For per-provider API support, see About LLMs.
Before you begin
Install theagentgateway binary.Tip
If you manage models and virtual API keys in the standalone UI, Client Setup can generate the connection settings or snippet for this client. Review the gateway URL, select a model and key, choose the client from the Integration dropdown, and copy the recipe. Each recipe uses only the values that its client supports.
Client Setup generates client-side values from existing configuration. It does not create a route, model, authentication policy, virtual key, or provider credential. Follow the steps in this guide to configure those prerequisites or to set up the client manually.
Example agentgateway configuration
Python
Install the OpenAI SDK in your Python project.
pip install openaiCreate and run the following script to send a request through agentgateway.
from openai import OpenAI client = OpenAI( base_url="http://localhost:3000/v1", api_key="anything", # placeholder if gateway has no auth ) response = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": "Hello!"}], ) print(response.choices[0].message.content)
You can also configure the SDK using environment variables.
export OPENAI_BASE_URL=http://localhost:3000/v1
export OPENAI_API_KEY=anythingThen initialize the client without arguments.
from openai import OpenAI
client = OpenAI() # picks up OPENAI_BASE_URL and OPENAI_API_KEY from envNode.js
Install the OpenAI SDK in your Node.js project.
npm install openaiCreate and run the following script to send a request through agentgateway.
import OpenAI from "openai"; const client = new OpenAI({ baseURL: "http://localhost:3000/v1", apiKey: "anything", }); const response = await client.chat.completions.create({ model: "gpt-4o-mini", messages: [{ role: "user", content: "Hello!" }], }); console.log(response.choices[0].message.content);
Go
Add the OpenAI Go SDK to your module.
go get github.com/openai/openai-go/v3Create and run the following program to send a request through agentgateway.
package main import ( "context" "fmt" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/option" ) func main() { client := openai.NewClient( option.WithBaseURL("http://localhost:3000/v1"), option.WithAPIKey("anything"), ) resp, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{ Model: "gpt-4o-mini", Messages: []openai.ChatCompletionMessageParamUnion{openai.UserMessage("Hello!")}, }) if err != nil { panic(err) } fmt.Println(resp.Choices[0].Message.Content) }
Authenticate with a JWT
Configure a on agentgateway. Use mode: strict to require a valid token on every request.
Obtain a JWT access token for your gateway API from your identity provider. The token must satisfy the policy’s validation rules, including any configured issuer and audience requirements. Set AGENTGATEWAY_JWT in your application’s environment to the raw token, without the Bearer prefix.
Pass that token as the SDK API key. The SDK sends Authorization: Bearer <token>.
import os
from openai import OpenAI
jwt_token = os.environ["AGENTGATEWAY_JWT"]
client = OpenAI(
base_url="http://localhost:3000/v1",
api_key=jwt_token,
)This token authenticates the caller to agentgateway. Keep the upstream provider API keys in the agentgateway configuration. Obtain a fresh token when it expires and update the SDK client before making more requests.