For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Buffer requests and responses for inspection or replay.

Fine-tune connection speeds for read and write operations by setting a connection buffer limit.

About buffer limits

By default, agentgateway allows up to 2 MiB of HTTP body to be buffered into memory for each gateway.

Requests that are routed to an LLM backend are the exception and use a larger default of 32 MiB, because a single large-context prompt can exceed the general-traffic limit on its own.

For example, a policy that runs in the PreRouting phase buffers before route selection, so it uses the 2 MiB limit even when the request is bound for an LLM backend.

For large requests that must be buffered and that exceed the default buffer limit, agentgateway either disconnects the connection to the downstream service if headers were already sent, or returns a 413 HTTP response code. To make sure that large requests can be sent and received, you can use maxBufferSize to specify the maximum number of bytes that can be buffered between the gateway and the downstream service. The buffer limit is configured at the Gateway level via a AgentgatewayPolicy. The value that you set replaces both defaults, so LLM requests use that value instead of 32 MiB.

Choose a buffer limit

The value you choose depends on how large a body your policies must be able to read and how much memory you are willing to spend on buffering.

• Set it at least as large as the largest body that a policy must inspect. A policy that needs a complete body cannot act on a body that exceeds the limit, so the request fails instead of being evaluated. Body-based authorization and an external processor that runs in a buffered mode are the common cases.
• Account for concurrency, not a single request. The limit applies to each buffered request, so the worst case is roughly the limit multiplied by the number of requests that buffer at the same time. A generous limit on a busy gateway is a larger commitment than it appears.
• Set a small limit when the gateway faces untrusted downstreams. When using agentgateway as an edge proxy, a small number such as 32768 bytes (32KiB) better guards against potential attacks or misconfigured downstreams that could excessively use the proxy’s resources.
• Keep the Gateway limit conservative and raise it only where it is needed. A route-level buffer policy defaults to the Gateway setting, so you can leave the shared limit low and lift it on the routes that carry large bodies.

Before you begin

1. Set up an agentgateway proxy.
2. Install the httpbin sample app.

Set up buffer limits per gateway

Use a AgentgatewayPolicy to set a buffer limit on your Gateway, which applies to all routes served by the Gateway.

1. Create an AgentgatewayPolicy that sets the maximum HTTP body buffer size.

   kubectl apply -f- <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayPolicy
   metadata:
     name: maxbuffer
     namespace: agentgateway-system
   spec:
     targetRefs:
     - kind: Gateway
       name: agentgateway-proxy
       group: gateway.networking.k8s.io
     frontend:
       http:
         maxBufferSize: 2097152
   EOF

   Setting	Description
   maxBufferSize	The maximum size of HTTP body that can be buffered into memory.

2. Port-forward the gateway proxy on port 15000.

   kubectl port-forward deployment/agentgateway-proxy -n agentgateway-system 15000

3. Get the config dump and verify that the policy is set as you configured it.

   Example jq command:

   curl -s http://localhost:15000/config_dump | jq '[.policies[] | select(.policy.frontend != null and .policy.frontend.hTTP != null and .policy.frontend.hTTP.maxBufferSize != null)] | .[0]'

   Example output:

   http://localhost:15000/config_dump

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

   { "key": "frontend/agentgateway-system/maxbuffer:frontend-http:agentgateway-system/agentgateway-proxy", "name": { "kind": "AgentgatewayPolicy", "name": "maxbuffer", "namespace": "agentgateway-system" }, "target": { "gateway": { "gatewayName": "agentgateway-proxy", "gatewayNamespace": "agentgateway-system", "listenerName": null } }, "policy": { "frontend": { "hTTP": { "maxBufferSize": 2097152, "http1MaxHeaders": null, "http1IdleTimeout": null, "http2WindowSize": null, "http2ConnectionWindowSize": null, "http2FrameSize": null, "http2KeepaliveInterval": null, "http2KeepaliveTimeout": null } } } }

Cleanup

kubectl delete AgentgatewayPolicy maxbuffer -n agentgateway-system