For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Route OpenAI Decisions and TypeSafe SystemOne requests through agentgateway and track token usage.

Route OpenAI Decisions and TypeSafe SystemOne API requests through agentgateway to get token usage tracking and observability.

About

The OpenAI Decisions API (/v1/decisions) answers a list of questions about an input. Each question is a predicate, a choice, or a score. The response returns one answer per question.

The TypeSafe SystemOne API (/v1/systemone) uses a different format for the same question types. Agentgateway translates requests and replies between the two formats to match the client and provider.

Review the following behavior of Decisions and SystemOne requests.

Supported providers

These providers accept Decisions and SystemOne requests. Other providers reject them.

If a custom provider declares both formats, each request uses its original format.

Custom provider requirements

A custom provider must declare decisions or systemOne in formats. If formats is omitted or contains only other formats, both endpoints return 400. The error names the request format and provider:

failed to process LLM request: unsupported conversion: from Decisions to provider custom

Version 1.6 had no Decisions or SystemOne route type. With the llm configuration, it passed both paths through to the provider unchanged. After upgrading, add the format your upstream server supports to formats to keep serving these requests.

Map the Decisions route type

With the simplified llm configuration, no route setup is needed. Agentgateway maps /v1/decisions to decisions and /v1/systemone to systemOne.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: "*"
    provider: openAI
    params:
      apiKey: "$OPENAI_API_KEY"

For a custom provider, declare the upstream format as decisions or systemOne. Agentgateway appends /decisions or /systemone to baseUrl. This example uses decisions.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: decisions-model
    provider:
      custom:
        formats:
        - type: decisions
    params:
      apiKey: "$CUSTOM_PROVIDER_API_KEY"
      baseUrl: "https://api.example.com/v1"

For explicit routing, use the gateways and routes configuration. Map /v1/decisions to decisions and /v1/systemone to systemOne in policies.ai.routes.

Without this map, requests default to completions. A Decisions request then fails with failed to parse Completions request.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
gateways:
  default:
    port: 4000
routes:
- backends:
  - ai:
      name: openai
      provider:
        openAI: {}
  policies:
    ai:
      routes:
        "/v1/decisions": "decisions"
        "/v1/systemone": "systemOne"
        "/v1/chat/completions": "completions"
        "*": "passthrough"
    backendAuth:
      key: "$OPENAI_API_KEY"

Review the following table to understand this configuration.

Send a Decisions request

Send a request with an input and a list of questions to the /v1/decisions endpoint. Replace <decisions-model> with an OpenAI model that supports the Decisions API.

curl http://localhost:4000/v1/decisions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "<decisions-model>",
    "input": [{"role": "user", "content": [{"type": "input_text", "text": "The package was left in the rain and the box is soaked through."}]}],
    "questions": [{"type": "predicate", "name": "visible_damage", "instructions": "Does the customer report visible damage to the package?"}]
  }'

Example output:

{
  "model": "<decisions-model>",
  "answers": [
    {"type": "predicate", "name": "visible_damage", "probability": 0.64}
  ],
  "usage": {"input_tokens": 156, "output_tokens": 0, "total_tokens": 156}
}

To ask the same question in SystemOne, use state for the input and a map keyed by name for the questions.

The SystemOne noul question type matches the Decisions predicate type. Both return the probability that a statement is true.

curl http://localhost:4000/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "<decisions-model>",
    "state": [{"role": "user", "content": "The package was left in the rain and the box is soaked through."}],
    "questions": {"visible_damage": {"type": "noul", "instructions": "Does the customer report visible damage to the package?"}}
  }'

Example output:

{
  "model": "<decisions-model>",
  "answers": {"visible_damage": {"type": "noul", "noul": 0.64}},
  "usage": {"input_tokens": 156, "output_tokens": 0}
}

Verify token tracking

After the request completes, check that agentgateway recorded the token usage in its metrics.

curl -s http://localhost:15020/metrics | grep 'agentgateway_gen_ai_client_token_usage_sum.*gen_ai_operation_name="decisions"'

Example output:

agentgateway_gen_ai_client_token_usage_sum{gen_ai_token_type="input",gen_ai_operation_name="decisions",gen_ai_system="openai",gen_ai_request_model="<decisions-model>",...} 156.0
agentgateway_gen_ai_client_token_usage_sum{gen_ai_token_type="output",gen_ai_operation_name="decisions",gen_ai_system="openai",gen_ai_request_model="<decisions-model>",...} 0.0

For more information about LLM metrics, see Observe traffic.