LibreChat

Front LibreChat with agentgateway to centralize LLM credentials, audit, and rate limits.

LibreChat is an open-source, multi-user chat interface that supports many LLM providers. Configuring LibreChat with a single OpenAI-compatible custom endpoint pointed at agentgateway lets you keep provider keys server-side and apply policies once for all chats.

What you get

LibreChat sees one endpoint; agentgateway routes to the actual upstream model.
API keys held by agentgateway, not by every LibreChat instance.
Per-request metrics, traces, and audit logs across providers.

Architecture

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│    LibreChat    │────▶│   agentgateway  │────▶│  LLM provider   │
│   (port 3080)   │     │   (port 3000)   │     │ (OpenAI, etc.)  │
└─────────────────┘     └─────────────────┘     └─────────────────┘

Before you begin

Install the agentgateway binary.
Have an LLM provider API key, such as an OpenAI API key.
Set up LibreChat via Docker Compose with MongoDB.

Steps

Step 1: Configure agentgateway

Create a config.yaml:

# yaml-language-server: $schema=https://agentgateway.dev/schema/config

llm:
  port: 3000
  models:
  - name: "*"
    provider: openAI
    params:
      apiKey: "$OPENAI_API_KEY"

Step 2: Start agentgateway

export OPENAI_API_KEY='<your-api-key>'
agentgateway -f config.yaml

Step 3: Add agentgateway as a custom endpoint in LibreChat

LibreChat’s custom endpoints feature lets you add any OpenAI-compatible backend. Edit librechat.yaml:

version: 1.2.1
cache: true

endpoints:
  custom:
    - name: "agentgateway"
      apiKey: "placeholder"
      baseURL: "http://localhost:3000/v1"
      models:
        default: ["gpt-4o", "gpt-4o-mini"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-4o-mini"
      modelDisplayLabel: "agentgateway"

apiKey must be set, but it is not used to call the upstream provider — agentgateway holds the real key. Set it to any non-empty value or to a key you require LibreChat clients to present (and validate at the gateway).

Set fetch: false and list your models explicitly in models.default. When agentgateway is configured with a wildcard (*) model, the /v1/models endpoint returns only the wildcard entry, which LibreChat cannot use to populate its model list.

If LibreChat runs in Docker and agentgateway runs on your host machine, replace localhost with host.docker.internal:

baseURL: "http://host.docker.internal:3000/v1"

Step 4: Enable registration and restart LibreChat

By default LibreChat disables new user registration. Add ALLOW_REGISTRATION=true to your LibreChat .env file so you can create an account:

ALLOW_REGISTRATION=true

Restart your LibreChat stack so it picks up the updated librechat.yaml and .env. In the UI, select agentgateway from the endpoint switcher and start a conversation. Confirm in the agentgateway logs that the request was proxied upstream. You should see a log entry similar to:

info  request gateway=default/default listener=llm route=internal/model:* endpoint=api.openai.com:443 http.method=POST http.path=/v1/chat/completions http.status=200 protocol=llm gen_ai.operation.name=chat gen_ai.provider.name=openai gen_ai.request.model=gpt-4o gen_ai.usage.input_tokens=4419 gen_ai.usage.output_tokens=10 duration=2195ms

Next steps

Control spendingApply rate limits and token budgets across providers. LLM observabilityMetrics, traces, and access logs for every LLM call. LLM providersAdd additional upstream providers behind the same endpoint.

Goose Open WebUI

Was this page helpful?