This hardens MCP degraded-mode behavior so multiplexed backends can keep serving healthy targets without hanging, looping, or corrupting resumable session state. The original `failureMode` work was meant to cover three concrete failure cases: - `#1162`: zero-target MCP backends should not trap clients in a reconnect loop - `#1029`: `failOpen` should skip bad targets, but still fail if every configured target is unusable - `#962`: one bad upstream should not take down the rest of a multiplexed MCP session This change makes that behavior consistent across init, runtime fanout, and session resume. At init time, `failOpen` now allows partial success but still returns `NoBackends` when all configured targets fail. Zero configured targets remain a special case so we do not regress the zero-target handling behind `#1162`. The `failureMode` setting is also threaded through the controller, local config, schema, and MCP runtime so the configured behavior is preserved end to end. At runtime, multiplexed fanout now degrades correctly when individual upstreams fail. That includes the legacy SSE path, which was the sharp edge here: a dead legacy SSE transport could leave pending requests orphaned and hang the whole downstream request. This change makes wrapped transports fail pending work, fail fast once dead, and treat dead legacy SSE upstreams as unavailable for the rest of that downstream session instead of trying to silently rejoin mid-session. That keeps healthy stdio, SSE, and streamable HTTP targets serving without introducing questionable session semantics for non-resumable transports. For resumable streamable HTTP MCP sessions, persisted upstream session state is now keyed by target name. New session handles include an optional target field, and resume restores by target identity instead of positional zip. Old handles without target names still use the legacy positional path, so existing session IDs remain readable. This avoids misbinding persisted sessions when `failOpen` changes the initialized target set. Most of the diff is test coverage plus generated/schema updates from wiring the config surface through. The behavior changes themselves are concentrated in the MCP relay, upstream transport wrappers, and resume logic. Fixes #1162 Fixes #1029 Fixes #962 --------- Co-authored-by: John Howard <john.howard@solo.io>
APIs for Agentgateway Controller
This directory contains Go types for Agentgateway Controller APIs & custom resources.
Adding a new API / CRD
These are the steps required to add a new CRD to be used in the Kubernetes Gateway integration:
- If creating a new API version (e.g.
v1,v2alpha1), create a new directory for the version and create adoc.gofile with the// +kubebuilder:object:generate=trueannotation, so that Go types in that directory will be converted into CRDs when codegen is run.- The
groupNamemarker specifies the API group name for the generated CRD. - RBAC rules are defined via the
+kubebuilder:rbacannotation (note: this annotation should not belong to the type, but rather the file or package).
- The
- Create a
_types.gofile in the API version directory. Following agentgateway_parameters_types.go as an example:- Define a struct for the resource (containing the metadata fields,
Spec, andStatus). Follow the API guidelines below. - Define a struct for the resource list (containing the metadata fields and
Items)
- Define a struct for the resource (containing the metadata fields,
- Run codegen via
make generated-code -B. This will invoke thecontroller-gencommand specified in generate.go, which should result in the following:- A
zz_generated.deepcopy.gofile is created in the same directory as the Go types. - A
zz_generated.register.gofile is created in the same directory as the Go types, to help with registering the Go types with the scheme. - CRDs are generated in the CRD helm chart template dir: install/helm/agentgateway-crds/templates
- RBAC roles are generated in controller/install/helm/templates/role.yaml
- Updates the api/applyconfiguration, pkg/generated and pkg/client folders with kube clients. These are used in plugin initialization and the fake client is used in tests.
- A
API guidelines
- Include documentation as well as any appropriate json and kubebuilder annotations on all fields.
- Document the default value for each field, if applicable.
- For optional fields:
- Use the
+optionalmarker. - Use the
omitemptyjson struct tag. - Use pointer types (e.g.
*string), unless the type has a nil zero value (e.g. slices/maps). An exception is if the field has a default value (+kubebuilder:default=...); then it it acceptable to use a non-pointer type.
- Use the
- For required fields:
- Use the
+requiredmarker. - Required fields MUST NOT set the
omitemptyjson struct tag.
- Use the
- Avoid using slices with pointers (e.g. use
[]stringinstead of[]*string). See: https://github.com/kubernetes/code-generator/issues/166 - For time duration fields, use the
metav1.Durationtype and use CEL validation rules to ensure it is within the correct range.
Custom validation markers
The CRD generation pipeline supports custom kubebuilder validation markers, implemented in
controller/hack/crdgen.
+kubebuilder:validation:AtLeastOneFieldSetrequires at least one field on the struct to be set.+kubebuilder:validation:ExactlyOneFieldSetrequires exactly one field on the struct to be set.+kubebuilder:validation:IfThenOnlyFields:if="...",fields=a;b;c,message="..."requires that, whenifis true, only the listed fields may be set.
Optional arguments for AtLeastOneFieldSet and ExactlyOneFieldSet:
fields=a;b;cto explicitly pick fields to count.exclude=a;bto drop fields from the counted set.message="..."to override the generated validation message.
These markers are intended for struct/type comments (markers.DescribesType).
Replicating Gateway API policies in TrafficPolicy API
Gateway API policies may be replicated as a part of the TrafficPolicy API to enable policy attachment at different levels in the config hierarchy, such as at the Gateway, Gateway's listener, or route level. The following guidelines should be considered when doing so:
- When the Gateway API types are considered sufficient to meet the requirements, they can be embedded as is in the TrafficPolicy API. TrafficPolicy's
corsis an example where the Gateway API typeHTTPCORSFilteris embedded directly. - When embedding the Gateway API type, it is important to consider whether the type is marked as
<gateway:experimental>, as experimental types may introduce breaking changes and should be noted similarly in the TrafficPolicy API. It is discouraged to embed experimental types in the TrafficPolicy API. However, if there is a breaking change in the Gateway API type, it is recommended to replicate the previous version of that type into the TrafficPolicy API and not propagate the breaking change to the TrafficPolicy API. - When the Gateway API types are not sufficient and a more advanced API is required, a new type should be created in the TrafficPolicy API instead of embedding the Gateway API type. TrafficPolicy's
retryandtimeoutsare examples that define new types instead of reusing theHTTPRouteRetryandHTTPRouteTimeoutstypes from the Gateway API.