Backends

Agentgateway backends Backend A destination service that receives traffic from agentgateway. Backends can be static hosts, MCP servers, LLM providers, or other services. control where traffic is routed to. Agentgateway supports a variety of backends, such as simple hostnames and IP addresses, LLM providers Provider A service that provides LLM capabilities, such as OpenAI, Anthropic, or Azure. Agentgateway supports multiple LLM providers and can route to different providers based on configuration. , and MCP servers.

Static Hosts

The simplest form of backend is a static hostname or IP address. For example:

binds:
- port: 3000
  listeners:
  - protocol: HTTP
    routes:
    - backends:
      - host: example.com:8080
        weight: 1
      - host: 127.0.0.1:80
        weight: 9

MCP Servers

The MCP backend allows you to connect to an MCP server. Below shows a simple example, exposing a local and remote MCP server. See the MCP connectivity guide for more information.

backends:
- mcp:
    targets:
    - name: stdio-server
      




  stdio
  
    STDIO (Standard Input/Output)
    A connection method for MCP servers that communicates through standard input and output streams. STDIO is commonly used for local MCP server processes.
    
  



:
        cmd: npx
        args: ["@modelcontextprotocol/server-everything"]
    - name: http-server
      mcp:
        host: https://example.com/mcp

LLM Providers

Agentgateway natively supports connecting to LLM providers, such as OpenAI and Anthropic. Below shows a simple example, connecting to OpenAI. See the LLM consumption guide for more information.

backends:
- ai:
    provider:
      openAI:
        model: gpt-3.5-turbo
policies:
  backendAuth:
    key: "$OPENAI_API_KEY"