Release notes

What’s new, changed, and fixed in each agentgateway on Kubernetes release.

Review the release notes for agentgateway on Kubernetes.

Note

For more details, review the GitHub release notes in the agentgateway repository.

✨ Highlights

Version 1.6 provides many updates to existing features, including the following quick highlights. Before you upgrade, review the breaking changes, because several defaults change.

AgentgatewayModel is on by default: The model-centric API is no longer experimental, so you can serve LLMs without a Helm flag.
More resilient LLM routing: Failover evicts unhealthy targets automatically, a virtual model with one broken target keeps serving, and a new idle timeout catches backends that stall mid-stream.
Authentication improvements: JWT providers are selected by issuer and key ID, requiredClaims sets which claims a token must carry, and AWS and Azure backend authentication gain externalId and scopes.
Session affinity: Send the requests that share a value, such as a session header, to the same endpoint.

🔥 Breaking changes

Anthropic Messages requests convert to the Responses format

Agentgateway now prefers the OpenAI Responses format when it converts Anthropic Messages requests. In 1.5, it preferred Chat Completions. This change applies to these providers:

OpenAI, Ollama, Groq, Huggingface, and XAI
Azure for models other than Claude
Custom providers that declare both Responses and Completions

The Responses conversion drops extended-thinking history. The Chat Completions conversion preserves it.

Actions to take: Use the OpenAI provider only for the OpenAI API. For other OpenAI-compatible servers, use a Custom provider with the formats that the server supports. If the server does not support /v1/responses, declare only Completions. Also declare only Completions if your clients need extended-thinking history.

Use AGENTGATEWAY_MESSAGES_PREFER_COMPLETIONS=true only to work around a Responses conversion bug with a provider that supports both formats. Set this environment variable in spec.env of the AgentgatewayParameters resource. The variable is removed in 1.7. For more information, see Supported formats.

Built-in model catalog for default request pricing

Agentgateway now includes a built-in model cost catalog. Requests to common public models have a cost in logs, traces, metrics, and CEL without extra configuration. In 1.5, you had to configure a catalog to calculate costs. Cost-based CEL expressions, such as rate limits that use llm.cost, now apply to these models.

Actions to take: Review your cost-based policies and CEL expressions. If you already configure a catalog, you can remove that configuration. To customize rates, add an overlay to the built-in catalog. For more information, see Model costs.

A `baseURL` with no path sets the base path to `/`

spec.baseURL on an AgentgatewayModel now always sets the full base path. A URL with no path, such as https://api.openai.com, sets the base path to /. In 1.5, Custom and Ollama providers appended /v1. Built-in providers such as OpenAI forwarded the client’s path.

Actions to take: Add the provider’s API path to every spec.baseURL that has no path. For example, change https://api.openai.com to https://api.openai.com/v1. For Ollama, change http://ollama.default.svc.cluster.local:11434 to http://ollama.default.svc.cluster.local:11434/v1. URLs that already have a path are unaffected. For more information, see Providers.

⚠️ Removed

Legacy token counts: The AGENTGATEWAY_LEGACY_LLM_USAGE_TOKEN_SEMANTICS environment variable is removed. In 1.5, it restored the earlier token counts that left out cache tokens. In logs, metrics, and CEL, input and total token counts now always include cache tokens. The token usage that agentgateway returns to clients follows the API that the client calls instead. For more information, see Other behavior changes.
Server-side defaults in the CRDs: The CRD schemas no longer declare default values, such as action: Allow or tracing.protocol: GRPC. The controller applies the same defaults at runtime, so behavior does not change. However, kubectl get -o yaml now shows only the fields that you set. If your GitOps diffs or scripts expect the default values to appear in the stored resource, update them.

🔄 Other behavior changes

Policy conflicts: When two policies with the same specificity set the same field, the oldest policy by metadata.creationTimestamp now wins consistently. In 1.5, the winner was not predictable.
AI transformation fields: The spec.backend.ai.transformations and spec.backend.ai.finalTransformations lists of an AgentgatewayPolicy now reject duplicate field values. A policy with duplicates fails validation when you next apply it.
LLM paths on a listener: A model router that is attached to a Gateway or ListenerSet listener serves only the standard LLM paths, and each path must match exactly. To serve the paths under a prefix, attach the models to an HTTPRoute. For more information, see Parent types.
Backend authentication errors: When agentgateway cannot get credentials from a provider, such as an OAuth token endpoint, AWS STS, or Azure, the request now fails with 502 instead of 500. Local failures, such as a static key that cannot be set, return 500 instead of 503.
Backend request timeout: The spec.backend.http.requestTimeout field of an AgentgatewayPolicy now also limits the time to read a buffered body. This limit applies to external authorization and CEL expressions that use response.body, for example. Streamed bodies are unaffected.
Access log level: Request log records now use error when agentgateway records a request error, and info otherwise. In 1.5, all request records used info.
Guardrail results: The guardrails CEL variable now has an entry for every guard that ran, including a new allow action. To find interventions only, filter on action != "allow".
Failover eviction: A backend or virtual model with more than one priority group now evicts a failing target by default. A health policy replaces this default. To keep failover, include spec.backend.health.eviction in the AgentgatewayPolicy. For concrete AgentgatewayModel targets, set spec.policies.health.eviction. For more information, see Model failover.
Policy service timeouts: Calls to external services default to a timeout of 2 seconds for gRPC external authorization and 10 seconds for rate limit and external processing services.
Streaming guardrails: Agentgateway now rejects content if a provider guard fails while checking a streamed response or realtime connection. To keep the 1.5 behavior, set failureMode: FailOpen on the guard. For example, set it in spec.backend.ai.promptGuard.response[].bedrockGuardrails of an AgentgatewayPolicy. For more information, see Provider failures.
Token usage in responses: The usage that agentgateway returns to clients now follows the conventions of the API that the client calls, whichever provider serves the request. Anthropic Messages responses leave cache tokens out of input_tokens and report them in cache_read_input_tokens and cache_creation_input_tokens. OpenAI Chat Completions and Responses include cache tokens in the input and total token counts. For example, a Chat Completions request to an Anthropic or Amazon Bedrock model that writes to the prompt cache now reports those cache tokens in prompt_tokens. If a client calculates usage or cost from these fields, review its counts after you upgrade.
Deployer ownership: The controller no longer overwrites an existing resource of the same name that it does not own. It reports an error instead.

🌟 New features

LLM

Meta provider: Set spec.provider: Meta on an AgentgatewayModel to use the Meta Model API preset. For the backend configuration, see OpenAI-compatible providers.
AgentgatewayModel on by default: The agentgatewayModels.enabled Helm value now defaults to true. For more information, see About models.
Wildcard models in /v1/models: For an AgentgatewayModel with a wildcard spec.match.model, /v1/models lists matching model IDs from the catalog. For example, openai/* expands to IDs such as openai/gpt-4o. For more information, see Verify model discovery.
Skip broken virtual model targets: A virtual model with one broken target skips that target instead of failing as a whole. For more information, see Fail over when a model degrades.
Partially valid backends: An AgentgatewayBackend with one invalid inline policy is accepted with a PartiallyValid status instead of being dropped. For more information, see Debug.
Response idle timeout: A response idle timeout ends a response if the backend stops sending body data for the configured time. It does not limit the duration of an active stream. Set spec.traffic.timeouts.responseIdle in an AgentgatewayPolicy. The HTTP/1 idle timeout applies to unused client connections between requests. For more information, see Timeouts.
Larger LLM buffer: LLM requests can now buffer up to 32 MiB by default, up from 2 MiB. The larger buffer supports long-context prompts. To change the limit, set spec.frontend.http.maxBufferSize in an AgentgatewayPolicy. For more information, see Buffer limits.
Per-page pricing: To price OCR and document models that bill per processed page, set rates.perPage in the model’s cost catalog entry. For more information, see Model costs.
Bedrock Mantle routing: The Bedrock provider chooses the Runtime or Mantle endpoint for each model. To set the preference, use spec.bedrock.endpointPreference on an AgentgatewayModel or spec.ai.provider.bedrock.endpointPreference on an AgentgatewayBackend. For more information, see Bedrock Mantle.
More accurate Messages conversion: Conversion between the Messages and OpenAI formats now carries citations, refusals, the strict setting of tool schemas, reasoning effort, and images in tool results. A context overflow error from the provider is translated so that Claude Code can compact and retry. For more information, see Converted replies and errors.
Guardrails: The openAIModeration, bedrockGuardrails, and googleModelArmor guards now support failureMode. Set this field on a guard in spec.backend.ai.promptGuard of an AgentgatewayPolicy. The built-in credit card pattern now checks the Luhn checksum. For more information, see Provider failures.

Security

JWT validation: With several providers in the spec.traffic.jwtAuthentication field of an AgentgatewayPolicy, agentgateway tries each provider whose issuer and JWKS key ID match the token. A provider’s validation.requiredClaims field sets the claims that a token must carry. Agentgateway also checks the nbf (not before) claim to reject tokens that are not yet valid. This check allows 60 seconds of leeway for clock skew. For more information, see JWT required claims.
Backend authorization: Kubernetes policies can now authorize requests after the destination backend is selected. Set spec.backend.authorization on an AgentgatewayPolicy, or configure authorization inline on an AgentgatewayBackend for the whole backend or an individual AI provider. For more information, see Authorize requests to a selected backend.
Backend authentication: In the spec.backend.auth field of an AgentgatewayPolicy, AWS assumeRole takes an externalId, and Azure authentication takes scopes. For more information, see AWS and Azure.
CA certificate key: You can now read the CA bundle from a key other than ca.crt, such as one that trust-manager writes. Set key in a caCertificateRefs entry under spec.backend.tls of an AgentgatewayPolicy. For more information, see Read the certificate from another key.
Network authorization by destination: Expressions in the spec.frontend.networkAuthorization field of an AgentgatewayPolicy can match on destination.address, destination.port, and the TLS SNI hostname in destination.hostname. For more information, see Restrict network access by TLS SNI.

MCP

List pagination: List responses for tools, prompts, and resources carry a nextCursor. When an endpoint federates several targets, the client gets one combined cursor that tracks every target. For more information, see List pagination.
Request size limit: An MCP request body that is larger than the buffer limit returns 413. For more information, see Buffer limits.
Method names in policies: Agentgateway automatically sets mcp.methodName from the request’s JSON-RPC method, such as tools/call. Route and backend policies can use this CEL variable for rate limits and authorization. For more information, see MCP rate limits.

Traffic management and operations

Session affinity: Session affinity sends related requests to the same endpoint. Set spec.backend.sessionAffinity in an AgentgatewayPolicy. Its CEL source expression provides the value to hash, such as a session header. For more information, see Session affinity.
Per-key local rate limits: Local rate limits can now give each user or API key a separate bucket. Set a CEL key in spec.traffic.rateLimit.local of an AgentgatewayPolicy. For more information, see Claim-level rate limits.
Stable ListenerSet ordering: ListenerSets now have a stable order. They sort by creation time, oldest first, then by namespace and name. For more information, see Listener precedence.
Telemetry: OTLP access logs use the agentgateway.access instrumentation scope. The request duration metric records failed requests with an error_type label.
Istio permissions: Set the istio.enabled=false Helm value to drop the Istio permissions from the controller ClusterRole. For more information, see Istio resource discovery.
Gateway name in proxy metrics: When monitoring.enabled is true, the Helm chart creates a PodMonitor. The PodMonitor now copies the gateway.networking.k8s.io/gateway-name pod label onto proxy metrics. This label lets the Grafana dashboard filter by Gateway. To change the copied labels, set the monitoring.proxy.podMonitor.podTargetLabels Helm value.

OpenTelemetry field names

You can now opt in to OpenTelemetry field names for stdout access logs. Set preset: Otel in spec.frontend.accessLog of an AgentgatewayPolicy. For more information, see Use OpenTelemetry field names.

🐛 Fixes

Traffic management

Connections with maxConnectionDuration close with up to 10% jitter, which reduces synchronized reconnects at the configured connection age.

LLM

Vertex AI catalog lookups resolve Anthropic model aliases such as claude-sonnet-4-5-20250929, anthropic/claude-sonnet-4-5@20250929, and publishers/anthropic/models/claude-sonnet-4-5@20250929.

MCP

Access-log CEL expressions can now read dynamic metadata that ExtMCP request-phase guardrails return through mcpGuardrails, including on resumed stateful MCP sessions. For more information, see Log MCP guardrail metadata.

Security

The controller no longer fails on a JWT authentication policy that sets jwks.remote.url without a backendRef when the AGW_BACKEND_REF_GRANT_MODE environment variable is set to route-and-policy.

Version support

Was this page helpful?