Argo Rollouts

Argo Rollouts is a Kubernetes controller that provides advanced deployment capabilities such as blue-green, canary, canary analysis, experimentation, and progressive delivery features to Kubernetes. Because Argo Rollouts supports the Kubernetes Gateway API, you can use Argo Rollouts to control how traffic is split and forwarded from the proxies that agentgateway manages to the apps in your cluster.

Before you begin

Follow the Get started guide to install agentgateway.
Follow the Sample app guide to create a gateway proxy with an HTTP listener and deploy the httpbin sample app.

Get the external address of the gateway and save it in an environment variable.

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system http -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS

kubectl port-forward deployment/agentgateway-proxy -n agentgateway-system 8080:80

Install Argo Rollouts

Create the argo-rollouts namespace and deploy the Argo Rollouts components into it.

kubectl create namespace argo-rollouts
kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml

Change the config map for the Argo Rollouts pod to install the Argo Rollouts Gateway API plug-in as shown in the following example. Alternatively, you could install the Argo Rollouts Helm chart instead to use init containers to achieve the same thing. For more information about either method, refer to the Argo Rollouts docs.

ℹ️

This configuration is only an example. Ensure you use the correct plugin binary for your platform, such as amd64 or arm64. For more platform and version options, refer to the releases of the Argo rollouts traffic router plugin for the Gateway API.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: argo-rollouts-config
  namespace: argo-rollouts
data:
  trafficRouterPlugins: |-
    - name: "argoproj-labs/gatewayAPI"
      # example uses amd64 and v0.11.0
      # for other builds and versions,
      # see https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases
      location: "https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases/download/v0.11.0/gatewayapi-plugin-amd64"
EOF

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: argo-rollouts-config
  namespace: argo-rollouts
data:
  trafficRouterPlugins: |-
    - name: "argoproj-labs/gatewayAPI"
      # example uses arm64 and v0.11.0
      # for other builds and versions,
      # see https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases
      location: "https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases/download/v0.11.0/gatewayapi-plugin-linux-arm64"
EOF

Restart the Argo Rollouts pod to pick up the latest configuration changes.
```
kubectl rollout restart deployment -n argo-rollouts argo-rollouts
```

Create RBAC rules for Argo

Create a cluster role to allow the Argo Rollouts pod to manage HTTPRoute resources.

⚠️

The following cluster role allows the Argo Rollouts pod to access and work with any resources in the cluster. Use this configuration with caution and only in test environments.

kubectl apply -f- <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: gateway-controller-role
  namespace: argo-rollouts
rules:
  - apiGroups:
      - "*"
    resources:
      - "*"
    verbs:
      - "*"
EOF

Create a cluster role binding to give the Argo Rollouts service account the permissions from the cluster role.

kubectl apply -f- <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: gateway-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: gateway-controller-role
subjects:
  - namespace: argo-rollouts
    kind: ServiceAccount
    name: argo-rollouts
EOF

Set up a rollout

Create a stable and canary service for the httpbun pod that you deploy in the next step.

kubectl apply -f- <<EOF
apiVersion: v1
kind: Service
metadata:
  name: httpbun-stable-service
  namespace: default
  labels:
    app: httpbun
    version: v1
spec:
  selector:
    app: httpbun
  ports:
    - protocol: TCP
      port: 3090
      targetPort: 3090
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: httpbun-canary-service
  namespace: default
  labels:
    app: httpbun
spec:
  selector:
    app: httpbun
  ports:
    - protocol: TCP
      port: 3090
      targetPort: 3090
  type: ClusterIP
EOF

Create an Argo Rollout that deploys the httpbun pod. Add your stable and canary services to the spec.strategy.canary section.

kubectl apply -f- <<EOF
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: httpbun-rollout
  namespace: default
  labels:
    app: httpbun
spec:
  replicas: 2
  selector:
    matchLabels:
      app: httpbun
  template:
    metadata:
      labels:
        app: httpbun
    spec:
      containers:
        - name: httpbun
          image: sharat87/httpbun
          env:
            - name: HTTPBUN_BIND
              value: "0.0.0.0:3090"
          ports:
            - containerPort: 3090
  strategy:
    canary:
      stableService: httpbun-stable-service
      canaryService: httpbun-canary-service
      trafficRouting:
        plugins:
          argoproj-labs/gatewayAPI:
            httpRoute: httpbun-http-route
            namespace: default
      steps:
        - setWeight: 30
        - pause: {}
        - setWeight: 60
        - pause: { duration: 30s }
        - setWeight: 100
EOF

Create an HTTPRoute resource to expose the httpbun pod on the HTTP gateway that you created as part of the Get started guide. The HTTP resource can serve both the stable and canary versions of your app.

kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbun-http-route
  namespace: default
spec:
  parentRefs:
    - name: agentgateway-proxy
      namespace: agentgateway-system
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /llm/chat/completions
      backendRefs:
        - name: httpbun-stable-service
          kind: Service
          port: 3090
        - name: httpbun-canary-service
          kind: Service
          port: 3090
EOF

Send a request to the httpbun app and verify your CLI output.

Get the external address of the gateway and save it in an environment variable.

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-proxy \
  -o=jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS

Send a standard chat completion Non-streaming request.

curl -s http://$INGRESS_GW_ADDRESS/llm/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [
      {"role": "user", "content": "Explain agentgateway in one sentence."}
    ]
  }' | jq

Example output:

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1748000000,
  "model": "gpt-4",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "This is a mock response from httpbun."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 8,
    "total_tokens": 18
  }
}

Port-forward the gateway proxy http pod on port 8080.

kubectl port-forward deployment/agentgateway-proxy -n agentgateway-system 8080:80

Send a standard chat completion Non-streaming request.

curl -s http://localhost:8080/llm/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [
      {"role": "user", "content": "Explain agentgateway in one sentence."}
    ]
  }' | jq

Example output:

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1748000000,
  "model": "gpt-4",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "This is a mock response from httpbun."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 8,
    "total_tokens": 18
  }
}

Test the promotion

ℹ️

Make sure you have installed the argo-rollouts extension for kubectl.

Verify httpbun image.

kubectl get pod httpbun -o yaml | grep image:

Example output:

image: sharat87/httpbun
image: docker.io/sharat87/httpbun

Check the weight difference between the stable and canary service.
```
kubectl get httproute httpbun-http-route -o yaml
```
Or use the following command to watch the promotion progress in real time.
```
kubectl argo rollouts get rollout httpbun-rollout --watch
```
Change the manifest to use a other image to start a rollout of your app. Argo Rollouts automatically starts splitting traffic between version 1 and version 2 of the app for the duration of the rollout.
```
kubectl argo rollouts set image httpbun-rollout httpbun=sharat87/httpbun:latest
```

Promote the rollout.

kubectl argo rollouts promote httpbun-rollout

Check the weight difference between the stable and canary services again to verify the traffic split.

kubectl get httproute httpbun-http-route -o jsonpath='{.spec.rules[0].backendRefs}' | jq

Example Output:

[
  {
    "group": "",
    "kind": "Service",
    "name": "httpbun-stable-service",
    "port": 3090,
    "weight": 70
  },
  {
    "group": "",
    "kind": "Service",
    "name": "httpbun-canary-service",
    "port": 3090,
    "weight": 30
  }
]

Check that the httpbun Pod is running the latest image.

k get pod httpbun -o yaml | grep image:

Example output:

image: sharat87/httpbun:latest
image: docker.io/sharat87/httpbun:latest

Congratulations, you successfully rolled out a new version of your app without downtime by using the HTTP gateway that is managed by agentgateway. After a rollout, you typically perform tasks such as the following:

Testing: Conduct thorough testing of your app to ensure that it functions correctly after the rollout.
Monitoring: Monitor your application to detect any issues that may arise after the rollout.
Documentation: Update documentation or runbooks to reflect any changes in your application.
User Validation: Have users validate that the app functions correctly and meets their requirements.
Performance Testing: Depending on your app, consider conducting performance testing to ensure that your app can handle the expected load without issues.
Resource Cleanup: If the rollout included changes to infrastructure or other resources, ensure that any temporary or unused resources are cleaned up to avoid incurring unnecessary costs.
Communication: Communicate with your team and stakeholders to update them on the status of the rollout and any issues that were encountered and resolved.
Security Audit: If your application has undergone significant changes, consider conducting a security audit to ensure that no security vulnerabilities have been introduced.

Cleanup

You can remove the resources that you created in this guide.

Remove the HTTPRoute.

kubectl delete httproute httpbun-http-route

Remove the Argo Rollout.
```
kubectl delete rollout httpbun-rollout
```

Remove the stable and canary services.

kubectl delete services httpbun-canary-service httpbun-stable-service

Remove the cluster role for the Argo Rollouts pod.

kubectl delete clusterrole gateway-controller-role -n argo-rollouts

Remove the cluster role binding.

kubectl delete clusterrolebinding gateway-admin

Remove Argo Rollouts.

kubectl delete -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
kubectl delete namespace argo-rollouts

Argo Rollouts

Before you begin

Install Argo Rollouts

Create RBAC rules for Argo

Set up a rollout

Test the promotion

Cleanup

What could be improved?