Skip to content
kube-agents

Examples

Self-contained example stacks live in examples/. Each is a plain Kubernetes manifest bundle you deploy with kubectl apply -f (see the README.md in each directory) — no framework, no rendering.

inference-replay

Section titled “inference-replay”

examples/inference-replay/

A proxy that sits between the Platform Agent and LiteLLM. Requests are keyed by prompt hash; hits return cached responses, misses forward to LiteLLM and cache the reply. Backed by a PersistentVolumeClaim.

When to use: deterministic demos, cheap CI tests against the agent’s tool loop, cost containment during development.

Modes: off (passthrough), on (cache hits, forward misses). Toggle via ConfigMap patch — see Inference gateway → Inference replay.

Deploy (development only): make -C k8s-operator deploy-inference-replay — the replay proxy is a dev tool and is never part of the installer.

litellm-gemini

Section titled “litellm-gemini”

examples/litellm-gemini/

LiteLLM Deployment + Service + ConfigMap fronting Gemini, plus a Secret, PodDisruptionBudget, NetworkPolicy, and PodMonitoring. Reads GEMINI_API_KEY from the Secret. The default install path (the chart’s litellm.* values) deploys an equivalent LiteLLM + Gemini config rather than this example directory; the dev copy lives at k8s-operator/config/integrations/litellm/base.

When to use: the default install path; anything except explicit local-inference or subscription-based demos.

litellm-chatgpt-subscription

Section titled “litellm-chatgpt-subscription”

examples/litellm-chatgpt-subscription/

LiteLLM configured to proxy a personal ChatGPT subscription via OAuth device flow. No per-token cost — useful for demos where you don’t want to burn API credit.

When to use: demos, education, hackathons. Not for production.

vllm-gemma

Section titled “vllm-gemma”

examples/vllm-gemma/

vLLM serving Gemma (gemma-4-e2b-it) on GKE GPU nodes, based on GKE’s official inference tutorial. Ships the vLLM Deployment, Service, PodDisruptionBudget, NetworkPolicy, and PodMonitoring. It does not include a node pool spec or GPU driver installer — a cluster with GPU nodes is a prerequisite.

When to use: data-locality, air-gapped, or open-model requirements. Provision a GPU node pool first (or use the gke-compute-classes skill to spec one).

Layering

Section titled “Layering”

Both LiteLLM examples and vllm-gemma speak OpenAI-compatible Completions. You can layer LiteLLM in front of vLLM to get routing and observability across a mix of hosted and local models — that’s the pattern for “one config for many providers”.

Not shipped as examples (but reference-worthy)

Section titled “Not shipped as examples (but reference-worthy)”
  • k8s-operator/examples/platformagent.yaml — a working sample PlatformAgent custom resource.
  • k8s-operator/testing/staging_workloads/ — a multi-cluster GKE staging PoC with workloads and traffic simulators.

Where to go next

Section titled “Where to go next”