For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Send the requests that the agentgateway proxy makes on its own behalf, such as external authorization calls and cloud credential fetches, through a forward proxy.

In networks where outbound traffic must leave through a corporate HTTP proxy, you can send the requests that the agentgateway proxy makes on its own behalf through that forward proxy. These requests are called callouts. Callouts that use TLS go through the forward proxy in an HTTP CONNECT tunnel, and plaintext callouts go to the forward proxy as standard forward-proxy requests.

About callouts

A callout is any request that the agentgateway proxy sends for itself rather than for a client. Callouts include the following requests.

• External authorization, external processing, and remote rate limit requests.
• Guardrail requests, such as webhooks, OpenAI moderation, AWS Bedrock Guardrails, Google Model Armor, and Azure AI Content Safety.
• OpenID Connect (OIDC) requests from the agentgateway proxy.
• Virtual model callout routing requests.
• Credential fetches from cloud providers, such as AWS, Azure, and Google Cloud Platform (GCP).

The callout proxy does not apply to the following traffic.

• Client traffic to your backends: This traffic is not a callout. To send the traffic to one backend through a forward proxy, set the tunnel policy on that backend instead. For more information, see Backend tunnel proxy.
• Remote JSON Web Key Set (JWKS) fetches: In Kubernetes, the control plane fetches remote JWKS, not the agentgateway proxy, so the callout proxy settings have no effect on these fetches. To send a JWKS fetch through a forward proxy, set the tunnel policy on the AgentgatewayBackend that serves the JWKS.

Configure the callout proxy

You configure the callout proxy on the agentgateway proxy deployment through an AgentgatewayParameters resource, in either of the following ways.

• With environment variables: Set the standard proxy environment variables, such as HTTPS_PROXY, in spec.env. Use this configuration for most setups. Note that this configuration is also the only way to send GCP credential fetches through the forward proxy.
• In rawConfig: Set config.callouts.tunnel in spec.rawConfig. Use this way to keep the forward proxy settings separate from environment variables that other tools in the pod read.

If you use both, config.callouts.tunnel takes precedence, and the agentgateway proxy ignores the proxy environment variables for most callouts. GCP credential fetches are the exception. These fetches never use config.callouts.tunnel, and they still use the proxy environment variables if you set them.

Option 1: Environment variables

Set the following variables in spec.env of the AgentgatewayParameters resource. The agentgateway proxy reads them at startup. Uppercase names take precedence over lowercase names, such as HTTPS_PROXY over https_proxy.

You can set NO_PROXY together with the other variables. The following example sends TLS and plaintext callouts through the same forward proxy, and keeps callouts to internal.example.com, its subdomains, and the 10.0.0.0/8 range direct.

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayParameters
metadata:
  name: callout-proxy
  namespace: agentgateway-system
spec:
  env:
  - name: HTTPS_PROXY
    value: http://proxy.example.com:3128
  - name: HTTP_PROXY
    value: http://proxy.example.com:3128
  - name: NO_PROXY
    value: internal.example.com,10.0.0.0/8

Option 2: rawConfig

Set config.callouts.tunnel under spec.rawConfig of the AgentgatewayParameters resource. You can set rawConfig on an AgentgatewayParameters resource that a Gateway or a GatewayClass references. If both resources set rawConfig, the Gateway’s rawConfig replaces the GatewayClass’s rawConfig as a whole. The two are not merged.

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayParameters
metadata:
  name: callout-proxy
  namespace: agentgateway-system
spec:
  rawConfig:
    config:
      callouts:
        tunnel:
          url: http://proxy.example.com:3128
          noProxy:
          - internal.example.com
          - 10.0.0.0/8

Destinations that bypass the forward proxy

The following callouts go directly to their destination, even when you configure a callout proxy.

• Callouts to a Kubernetes Service that you reference with a Service backendRef, such as an external authorization server in the cluster. An AgentgatewayBackend with a static host is not a Service reference, even when the host is a cluster hostname such as ext-authz.auth.svc.cluster.local. Callouts to that backend go through the forward proxy, unless you add svc.cluster.local to NO_PROXY or noProxy.
• Destinations that match noProxy or NO_PROXY.
• Loopback and link-local IP addresses, such as 127.0.0.1 and the cloud metadata server at 169.254.169.254. A remote forward proxy cannot reach these addresses.
• Callouts to an AgentgatewayBackend that has its own tunnel policy. These callouts use that tunnel instead.
• GCP credential fetches, when you configure the callout proxy with config.callouts.tunnel. To send these fetches through a forward proxy, use the environment variables instead.
• Remote JWKS fetches. The control plane makes these fetches, not the agentgateway proxy. For more information, see About callouts.

Before you begin

1. Follow the Get started guide to install agentgateway.

2. Follow the Sample app guide to create a gateway proxy with an HTTP listener and deploy the httpbin sample app.

3. Get the external address of the gateway and save it in an environment variable.

   Cloud Provider LoadBalancerPort-forward for local testing

   Tip

   Kind cluster? Kind does not support LoadBalancer services by default. To use this option with a Kind cluster, install and run cloud-provider-kind.

   export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-proxy -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
   echo $INGRESS_GW_ADDRESS  

   kubectl port-forward deployment/agentgateway-proxy -n agentgateway-system 8080:80

Send an external authorization callout through the forward proxy

Deploy a test forward proxy, point the agentgateway proxy at it with the HTTPS_PROXY environment variable, and confirm that an external authorization callout goes through the forward proxy. To use rawConfig instead, change step 2 as that step describes. The example uses httpbin.org as a stand-in authorization server, because the server returns any status code that you ask for.

1. Deploy Squid as a test forward proxy. If you already have a forward proxy that the cluster can reach, skip this step and use its URL in the next step.

   kubectl apply -f- <<EOF
   apiVersion: v1
   kind: Namespace
   metadata:
     name: squid
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: squid
     namespace: squid
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: squid
     template:
       metadata:
         labels:
           app: squid
       spec:
         containers:
         - name: squid
           image: ubuntu/squid:latest
           ports:
           - containerPort: 3128
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: squid
     namespace: squid
   spec:
     selector:
       app: squid
     ports:
     - port: 3128
       targetPort: 3128
   EOF
   kubectl rollout status deployment/squid -n squid

2. Create an AgentgatewayParameters resource that points callouts at the forward proxy. The external authorization callout in this guide uses TLS, so set HTTPS_PROXY.

   kubectl apply -f- <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayParameters
   metadata:
     name: callout-proxy
     namespace: agentgateway-system
   spec:
     env:
     - name: HTTPS_PROXY
       value: http://squid.squid.svc.cluster.local:3128
   EOF

   To use rawConfig instead, replace spec.env with spec.rawConfig.config.callouts.tunnel.url, as shown in rawConfig.

3. Attach the AgentgatewayParameters resource to your Gateway. The patch keeps the rest of your Gateway as it is. If your Gateway already references an AgentgatewayParameters resource, add the spec.env setting to that resource instead, because a Gateway can reference one AgentgatewayParameters resource only.

   kubectl patch gateway agentgateway-proxy -n agentgateway-system --type merge \
     -p '{"spec":{"infrastructure":{"parametersRef":{"group":"agentgateway.dev","kind":"AgentgatewayParameters","name":"callout-proxy"}}}}'

4. Wait for the agentgateway proxy to roll out with the new settings.

   kubectl rollout status deployment/agentgateway-proxy -n agentgateway-system

5. Check the agentgateway proxy logs for the line that names the forward proxy.

   kubectl logs -n agentgateway-system -l gateway.networking.k8s.io/gateway-name=agentgateway-proxy --tail=-1 | grep "callouts will use proxy"

   Example output:

   info	client::callout	callouts will use proxy Matcher { https: Intercept { uri: http://squid.squid.svc.cluster.local:3128/ } }
   

6. Create an AgentgatewayBackend for the stand-in authorization server. Because the backend has a static host, callouts to it go through the forward proxy.

   kubectl apply -f- <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayBackend
   metadata:
     name: httpbin-org
     namespace: httpbin
   spec:
     static:
       host: httpbin.org
       port: 443
     policies:
       tls:
         sni: httpbin.org
   EOF

7. Create an AgentgatewayPolicy that sends an external authorization request for every request to the httpbin stand-in authorization server. The /status/200 path makes httpbin.org allow every request. The http.path value is a CEL expression, so the path is a string literal in double quotes.

   kubectl apply -f- <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayPolicy
   metadata:
     name: callout-ext-auth
     namespace: httpbin
   spec:
     targetRefs:
     - group: gateway.networking.k8s.io
       kind: HTTPRoute
       name: httpbin
     traffic:
       extAuth:
         backendRef:
           group: agentgateway.dev
           kind: AgentgatewayBackend
           name: httpbin-org
         http:
           path: '"/status/200"'
   EOF

8. Send a request to the httpbin app. The request succeeds after the external authorization callout returns 200.

   Cloud Provider LoadBalancerPort-forward for local testing

   curl -i http://$INGRESS_GW_ADDRESS:80/headers -H "host: www.example.com"

   The rollout in step 4 replaced the agentgateway proxy pod, so start the port-forward again.

   kubectl port-forward deployment/agentgateway-proxy -n agentgateway-system 8080:80 &
   curl -i localhost:8080/headers -H "host: www.example.com"

   Example output:

   HTTP/1.1 200 OK
   

9. Check the Squid access log for a CONNECT request to httpbin.org:443. This entry shows the whole callout path, from the agentgateway proxy through Squid to httpbin.org.

   kubectl exec -n squid deployment/squid -- cat /var/log/squid/access.log

   Example output:

   1791983992.466  46059 10.244.0.19 TCP_TUNNEL/200 4933 CONNECT httpbin.org:443 - HIER_DIRECT/54.159.186.149 -
   

   Field	Description
   10.244.0.19	The client of Squid, which is the agentgateway proxy pod.
   TCP_TUNNEL/200	Squid opened the CONNECT tunnel.
   CONNECT httpbin.org:443	The destination of the external authorization callout.
   HIER_DIRECT/54.159.186.149	Squid connected directly to httpbin.org at this IP address.

10. If the log has no CONNECT entry, restart the agentgateway proxy, and then check the log again. Squid writes the entry only when the tunnel closes, and the agentgateway proxy keeps the tunnel open to reuse it. The restart closes the tunnel.

    kubectl rollout restart deployment/agentgateway-proxy -n agentgateway-system
    kubectl rollout status deployment/agentgateway-proxy -n agentgateway-system
    kubectl exec -n squid deployment/squid -- cat /var/log/squid/access.log

Cleanup

1. Delete the external authorization policy and the backend.

   kubectl delete AgentgatewayPolicy callout-ext-auth -n httpbin
   kubectl delete AgentgatewayBackend httpbin-org -n httpbin

2. Remove infrastructure.parametersRef from your Gateway, and delete the AgentgatewayParameters resource.

   kubectl patch gateway agentgateway-proxy -n agentgateway-system --type json -p '[{"op":"remove","path":"/spec/infrastructure/parametersRef"}]'
   kubectl delete AgentgatewayParameters callout-proxy -n agentgateway-system

3. Delete Squid.

   kubectl delete namespace squid