Grafana

Visualize agentgateway metrics in Grafana by using the pre-built Kubernetes dashboard or custom PromQL panels for binary and Docker deployments.

Verified

Grafana is an open-source visualization platform that turns time-series data into dashboards, graphs, and alerts. It is the standard way to visualize agentgateway metrics that are collected by Prometheus, and supports correlating metrics with traces by adding Jaeger as a second data source in the same Grafana instance.

Agentgateway ships a pre-built dashboard for Kubernetes deployments that covers requests, LLM traffic, MCP traffic, and connections out of the box.

For binary and Docker deployments where the pre-built dashboard does not apply, you can use the PromQL queries that are included in this guide to help you get started with building your own Grafana panels.

Before you begin

Set up a Prometheus instance so that you can start collecting metrics and feeding them into Grafana. Do not run that guide’s cleanup step until you finish this one, because both guides use the monitoring namespace.

Note

The kube-prometheus-stack chart already installs Grafana as the kube-prometheus-stack-grafana service. If you set up Prometheus with that chart, skip steps 1 through 3 and port-forward svc/kube-prometheus-stack-grafana instead. Get its admin password with kubectl get secret -n monitoring kube-prometheus-stack-grafana -o jsonpath="{.data.admin-password}" | base64 --decode.

Use the pre-built Grafana dashboard (Kubernetes only)

The pre-built dashboard includes the following sections:

Section	Metric	Description
Overview	Memory	The working set memory that each agentgateway proxy pod consumes.
Overview	CPU	The CPU usage rate for each agentgateway proxy pod.
Requests	Requests (by Pod)	The request rate that each agentgateway proxy pod handles.
Requests	Requests (by Gateway)	The request rate for each gateway.
Requests	Requests (by Status)	The request rate grouped by HTTP response status.
Requests	Requests (by Reason)	The request rate grouped by the response reason.
LLM	Token Consumption	The rate of tokens that LLM requests consume, grouped by token type, model, and gateway.
LLM	Time To First Token	The time that it takes the LLM provider to return the first token of a response.
LLM	Request Time	The total duration of LLM requests.
LLM	Tokens Per Second	The rate at which the LLM provider returns output tokens.
MCP	MCP Calls (by method)	The rate of MCP requests grouped by JSON-RPC method.
MCP	Tool Calls (by tool)	The rate of MCP tool calls grouped by server, resource, and tool.
Latency	Latency by Route	The 50th, 95th, and 99th percentile request latency for each gateway and route.
XDS	XDS Messages by Type	The rate of xDS configuration messages that the control plane sends, grouped by resource type.
XDS	XDS Average Message Size	The average size of xDS messages, grouped by resource type.
Runtime	Cgroup Memory	The cgroup memory usage for each agentgateway proxy pod, such as working set, anonymous, file, and kernel memory.
Runtime	Process Memory	The process-level memory for each agentgateway proxy pod, such as RSS, PSS, private, shared, and swap memory.
Runtime	Tokio Runtime	The async runtime statistics for each agentgateway proxy pod, such as the worker count, number of alive tasks, and global queue depth.
Runtime	Build Versions	The agentgateway build versions that are running, grouped by tag.

Add the Grafana Helm repository and install Grafana.

helm repo add grafana https://grafana.github.io/helm-charts
helm install grafana grafana/grafana -n monitoring --create-namespace

Verify that the Grafana pod is running.
```
kubectl get pods -n monitoring
```

Get the Grafana admin password.

kubectl get secret -n monitoring grafana -o jsonpath="{.data.admin-password}" | base64 --decode

Forward the Grafana port to access the UI.

kubectl port-forward -n monitoring svc/grafana 3001:80

Access Grafana at http://localhost:3001. Log in with the admin username and the password that you created in the previous step.
Add a Prometheus data source.
1. Go to Connections → Add new connection.
2. Search for and select the Prometheus plugin, then click Add new data source.
3. Set the URL to your in-cluster Prometheus service, such as http://kube-prometheus-stack-prometheus.monitoring.svc.cluster.local:9090.
4. Click Save & Test.

Download the agentgateway dashboard JSON.

curl -L "https://raw.githubusercontent.com/agentgateway/agentgateway/main/controller/install/helm/agentgateway/files/agentgateway-dashboard.json" \
  -o agentgateway-dashboard.json

In Grafana, go to Dashboards → New → Import.
Click Upload dashboard JSON file and select the agentgateway-dashboard.json file.
Select your Prometheus data source and click Import.

When you are done, remove Grafana and the monitoring namespace.

helm uninstall grafana -n monitoring
kubectl delete namespace monitoring

Build your own Grafana panels (Binary and Docker)

Run Grafana with Docker.

docker run -d --name grafana \
  -p 3001:3000 \
  grafana/grafana:latest

Access the Grafana UI at http://localhost:3001. Use the admin username and admin password to log into Grafana.
Add a Prometheus data source.
1. Go to Connections → Add new connection.
2. Search for and select the Prometheus plugin, then click Add new data source.
3. Set the Prometheus server URL to http://host.docker.internal:9090.
4. Click Save & Test.
Create a dashboard.
1. Go to Dashboards → New → New dashboard.
2. Add a Panel and click Configure visualization.
3. Select your Prometheus data source.
4. Switch to the Code view and enter a PromQL query in the query editor. For example, to see request rate by route:
  rate(agentgateway_requests_total[5m])
5. Click Apply to save the panel, then save the dashboard.
For more queries to build out your dashboard, see Common PromQL queries.
When you are done, remove the Grafana container.
```
docker rm -f grafana
```

Common PromQL queries

Use these queries to build custom panels or alerts. To enter a raw PromQL query in Grafana, switch the query editor from Builder to Code mode by using the toggle in the query section.

Use case	PromQL query
Request rate	`rate(agentgateway_requests_total[5m])`
Error rate	`rate(agentgateway_requests_total{status=~"5.."}[5m]) / rate(agentgateway_requests_total[5m])`
LLM token usage (input)	`sum by (gen_ai_system, gen_ai_request_model) (rate(agentgateway_gen_ai_client_token_usage_sum{gen_ai_token_type="input"}[5m]))`
Time to first token (p95)	`histogram_quantile(0.95, rate(agentgateway_gen_ai_server_time_to_first_token_bucket[5m]))`
MCP tool call rate by tool	`sum by (server, resource) (rate(agentgateway_mcp_requests_total{method="tools/call"}[5m]))`

Add a Jaeger data source for traces

To correlate metrics with traces in the same Grafana instance:

Go to Connections → Add new connection.
Search for and select the Jaeger plugin, then click Add new data source.
Set the URL to your Jaeger instance, such as http://jaeger:16686.
Click Save & Test.

Prometheus Metrics reference

Was this page helpful?