apexlnc 5798b5acaf
mcp: harden fail-open degraded mode (#1232)
This hardens MCP degraded-mode behavior so multiplexed backends can keep
serving healthy targets without hanging, looping, or corrupting
resumable session state.

The original `failureMode` work was meant to cover three concrete
failure cases:
- `#1162`: zero-target MCP backends should not trap clients in a
reconnect loop
- `#1029`: `failOpen` should skip bad targets, but still fail if every
configured target is unusable
- `#962`: one bad upstream should not take down the rest of a
multiplexed MCP session

This change makes that behavior consistent across init, runtime fanout,
and session resume.

At init time, `failOpen` now allows partial success but still returns
`NoBackends` when all configured targets fail. Zero configured targets
remain a special case so we do not regress the zero-target handling
behind `#1162`. The `failureMode` setting is also threaded through the
controller, local config, schema, and MCP runtime so the configured
behavior is preserved end to end.

At runtime, multiplexed fanout now degrades correctly when individual
upstreams fail. That includes the legacy SSE path, which was the sharp
edge here: a dead legacy SSE transport could leave pending requests
orphaned and hang the whole downstream request. This change makes
wrapped transports fail pending work, fail fast once dead, and treat
dead legacy SSE upstreams as unavailable for the rest of that downstream
session instead of trying to silently rejoin mid-session. That keeps
healthy stdio, SSE, and streamable HTTP targets serving without
introducing questionable session semantics for non-resumable transports.

For resumable streamable HTTP MCP sessions, persisted upstream session
state is now keyed by target name. New session handles include an
optional target field, and resume restores by target identity instead of
positional zip. Old handles without target names still use the legacy
positional path, so existing session IDs remain readable. This avoids
misbinding persisted sessions when `failOpen` changes the initialized
target set.

Most of the diff is test coverage plus generated/schema updates from
wiring the config surface through. The behavior changes themselves are
concentrated in the MCP relay, upstream transport wrappers, and resume
logic.

Fixes #1162
Fixes #1029
Fixes #962

---------

Co-authored-by: John Howard <john.howard@solo.io>
2026-03-16 15:57:45 -07:00
..

APIs for Agentgateway Controller

This directory contains Go types for Agentgateway Controller APIs & custom resources.

Adding a new API / CRD

These are the steps required to add a new CRD to be used in the Kubernetes Gateway integration:

  1. If creating a new API version (e.g. v1, v2alpha1), create a new directory for the version and create a doc.go file with the // +kubebuilder:object:generate=true annotation, so that Go types in that directory will be converted into CRDs when codegen is run.
    • The groupName marker specifies the API group name for the generated CRD.
    • RBAC rules are defined via the +kubebuilder:rbac annotation (note: this annotation should not belong to the type, but rather the file or package).
  2. Create a _types.go file in the API version directory. Following agentgateway_parameters_types.go as an example:
    • Define a struct for the resource (containing the metadata fields, Spec, and Status). Follow the API guidelines below.
    • Define a struct for the resource list (containing the metadata fields and Items)
  3. Run codegen via make generated-code -B. This will invoke the controller-gen command specified in generate.go, which should result in the following:

API guidelines

  • Include documentation as well as any appropriate json and kubebuilder annotations on all fields.
  • Document the default value for each field, if applicable.
  • For optional fields:
    • Use the +optional marker.
    • Use the omitempty json struct tag.
    • Use pointer types (e.g. *string), unless the type has a nil zero value (e.g. slices/maps). An exception is if the field has a default value (+kubebuilder:default=...); then it it acceptable to use a non-pointer type.
  • For required fields:
    • Use the +required marker.
    • Required fields MUST NOT set the omitempty json struct tag.
  • Avoid using slices with pointers (e.g. use []string instead of []*string). See: https://github.com/kubernetes/code-generator/issues/166
  • For time duration fields, use the metav1.Duration type and use CEL validation rules to ensure it is within the correct range.

Custom validation markers

The CRD generation pipeline supports custom kubebuilder validation markers, implemented in controller/hack/crdgen.

  • +kubebuilder:validation:AtLeastOneFieldSet requires at least one field on the struct to be set.
  • +kubebuilder:validation:ExactlyOneFieldSet requires exactly one field on the struct to be set.
  • +kubebuilder:validation:IfThenOnlyFields:if="...",fields=a;b;c,message="..." requires that, when if is true, only the listed fields may be set.

Optional arguments for AtLeastOneFieldSet and ExactlyOneFieldSet:

  • fields=a;b;c to explicitly pick fields to count.
  • exclude=a;b to drop fields from the counted set.
  • message="..." to override the generated validation message.

These markers are intended for struct/type comments (markers.DescribesType).

Replicating Gateway API policies in TrafficPolicy API

Gateway API policies may be replicated as a part of the TrafficPolicy API to enable policy attachment at different levels in the config hierarchy, such as at the Gateway, Gateway's listener, or route level. The following guidelines should be considered when doing so:

  • When the Gateway API types are considered sufficient to meet the requirements, they can be embedded as is in the TrafficPolicy API. TrafficPolicy's cors is an example where the Gateway API type HTTPCORSFilter is embedded directly.
  • When embedding the Gateway API type, it is important to consider whether the type is marked as <gateway:experimental>, as experimental types may introduce breaking changes and should be noted similarly in the TrafficPolicy API. It is discouraged to embed experimental types in the TrafficPolicy API. However, if there is a breaking change in the Gateway API type, it is recommended to replicate the previous version of that type into the TrafficPolicy API and not propagate the breaking change to the TrafficPolicy API.
  • When the Gateway API types are not sufficient and a more advanced API is required, a new type should be created in the TrafficPolicy API instead of embedding the Gateway API type. TrafficPolicy's retry and timeouts are examples that define new types instead of reusing the HTTPRouteRetry and HTTPRouteTimeouts types from the Gateway API.