For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
Observability
Send LLM telemetry to Axiom, Langfuse, LangSmith, and other observability platforms.
Agentgateway can send LLM telemetry to specialized observability platforms for prompt logging, cost tracking, audit trail, and performance monitoring.
How it works
Agentgateway exports LLM telemetry via OpenTelemetry, which can be forwarded to LLM-specific observability platforms. These platforms provide the following.
- Prompt/response logging - Full request and response capture
- Token usage tracking - Monitor costs across models and users
- Latency analytics - Track response times and identify bottlenecks
- Evaluation - Score and evaluate LLM outputs
- Prompt management - Version and manage prompts
Configuration
Enable OpenTelemetry tracing with LLM-specific attributes.
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
config:
tracing:
otlpEndpoint: http://localhost:4317
randomSampling: true
gateways:
default:
port: 3000
routes:
- backends:
- ai:
name: openai
provider:
openAI:
model: gpt-4o-mini
policies:
backendAuth:
key: "$OPENAI_API_KEY"Agentgateway automatically includes these LLM-specific trace attributes:
| Attribute | Description |
|---|---|
gen_ai.operation.name | Operation type (chat, completion, embedding) |
gen_ai.request.model | Requested model name |
gen_ai.response.model | Actual model used |
gen_ai.usage.input_tokens | Input token count, including the tokens read from or written to the prompt cache. See Token usage fields. |
gen_ai.usage.output_tokens | Output token count |
gen_ai.provider.name | LLM provider (openai, anthropic, etc.) |
Axiom
Export standalone agentgateway traces, access logs, and metrics to Axiom.
Langfuse
Integrate agentgateway with Langfuse for LLM analytics and prompt management
LangSmith
Integrate agentgateway with LangSmith for LLM debugging and monitoring
Arize Phoenix
Integrate agentgateway with Arize Phoenix for LLM tracing and evaluation
