Add `cacheMessageOffset` field to `PromptCachingConfig`. When set,
shifts the Bedrock message cache point further back from the default
second-to-last position.
- `cacheMessageOffset: 0` (default): no change in behavior
- `cacheMessageOffset: N`: cache point moves N additional messages
toward the start, clamped to bounds
This gives callers control over which portion of the conversation is
covered by the cache prefix — useful for agentic workloads where
mid-conversation compaction may mutate recent messages.
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
## Changes
1. **Split `AgwPolicyCollection` into two phases** (authored by
@alexeldeib):
- `CollectPolicyReferences`: collects `PolicyAttachments` from all
plugins first
- `BuildPolicies`: builds policies with the fully-populated
`ReferenceIndex`
2. **Use `LookupGatewaysForBackend` in `TranslateAgentgatewayPolicy`**:
traverses `PolicyAttachments` to resolve gateway associations for
indirectly-referenced targets.
3. **Add golden test**: validates that a Service-targeted backend TLS
policy is correctly associated with a gateway when the Service is only
referenced via an ext_proc policy's `backendRef`.
---------
Co-authored-by: Ace Eldeib <aeldeib@coreweave.com>
Co-authored-by: John Howard <john.howard@solo.io>
With -race, this took 53s on my fast machine before. now it takes 3.
The problem was we parsed all CRDs each time, now we do it once (and
ruse more code)
Found an issue with the helm version `make run` when checking
https://github.com/agentgateway/agentgateway/issues/1115 was fixed.
```
Error: path "agentgateway/controller/_test/agentgateway-crds-v1.0.1-dev.tgz" not found
make: *** [Makefile:500: deploy-agentgateway-crd-chart] Error 1
```
The deploy targets were using `$(VERSION)` (which is `v1.0.1-dev`) but
the package targets create files using `$(HELM_CHART_VERSION_NO_V)`
(which is `1.0.1-dev` - without the "v" prefix). `make run` now uses the
correct version.
Signed-off-by: npolshakova <nina.polshakova@solo.io>
We did not add a helm tag before, and if we did use latest-dev it
wouldn't work anyways
fixes https://github.com/agentgateway/agentgateway/issues/1316
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Currently, mcpAuthentication is a backend level policy. This sort of
made sense at the time, as its only applicable for MCP, and "is MCP" is
a property of the backend. However, in practice this causes issues.
McpAuth is a glorified JWT validator with some additional special path
handling. One of the things you might want to use JWT/MCPAuth for is
apply other policies based on the claims. Authorization, rate limit,
transformations, etc
all based on the jwt claims are quite handy and common.
The alternative to this is to start copying every single policy to an
mcpXYZ policy which sucks.
----
This would be the first time the project faces a serious "deprecating an
API" so warrants some thinking. In this case I propose:
* The old backend policy is kept around in Kubernetes and marked as
deprecated. We can keep the field around for a very long time (forever?)
without too much burden on the project as its pretty much the same
translation regardless of where it is
* Standalone is not impacted; we already had users set it in the route
level and just translated it down to a backend policy.
* If a user sets both, we reject the backend one (TODO: this is not done
in this PR)
* If a user sets JWT and MCP auth, we prefer whichever is more precise.
(TODO: this is not done in this PR)
Fixes https://github.com/agentgateway/agentgateway/issues/870
Per GW API spec we should send a weight-proportional amount of traffic
to invalid backends. We did not and instead sent a backend `{}` which
implies `weight=0` which implies no traffic.
This policy is equivalent to the HTTP authorization policy but runs at
the L4 level. This means you can use it for non-HTTP traffic, and you
can layer it on top of HTTP traffic to have L4+L7 rules. For example,
you might deny banned CIDR ranges at L4 and then have L7 rules on top of
that.
These are independant rule sets, so an allow in L4 does not allow at L7.
The policy runs _after_ TLS handshake and de-tunneling. This ensures the
properties of the handshakes can be used in policy decision making. In
theory, you may want to block users earlier to avoid the cost of a TLS
handshake, but IMO this is the wrong tradeoff here; if you are trying to
prevent things like a DOS you want something at L3 in front of the
gateway that prevents the cost of the TCP handshake too.
`Deny` mode is a bad API with a useful goal: to ensure that policies
MUST match the rule, essentialy adding AND semantics on top of the ALLOW
policies OR semantics.
Deny is trick, because you often end up having double negatives.
For example `deny: !(port in [443, 8443])`. Or worse, somethign like
`deny: jwt.group != 'eng'`; if the jwt doesn't have a `group` this rule
will not deny.
Instead, this proposes adding a new Require mode which is like deny but
inverted.
The above rules would be `require: port in [443, 8443]` or `require:
jwt.group == 'eng'`.
These are useful for admin level rules that apply on top of user level
rules like `require: source.address not in [banned country cidr ranges]`
etc
https://github.com/agentgateway/agentgateway/pull/1352 could set this
ServiceKey to attach routes directly to services
* add serviceKey to the Route and TCPRoute protos
* plumb serviceKey from ParentInfo into those protos as routes are
attached to their parents
* refactor route selection in the proxy to share some code between paths
that use service or listener attached routes
most of the diff is proto regen + tests
A plugin that add routes can re-use the main route generation logic but
re-implement or wrap this interface to allow changing the parentRef
lookup behavior.
We can add routes and backends, but without sending the ancestor
information the backends will not be sent to the proxy because they
don't seem to be referenced by any route.
* Make status able to share the StatusSync logic
* Now that target refs are extensible, we also need the presence of the
target object to be; extend that interface slightly
- Handle 'global' region explicitly in get_host(), returning
aiplatform.googleapis.com instead of global-aiplatform.googleapis.com
- Make VertexAI Region field optional with default 'global' in CRD
- Add unit tests for get_host() covering all region cases
---------
Signed-off-by: Rajat Vig <rvig@etsy.com>
Before we had some type-specific status handling. Now, we consistently
report up the Gateway that the target is attached to as part of the
status with deduping.
The dedupe is concerning since translatePolicyToAgw is called
per-target, which means in theory they could return different errors.
and, its computed redundantly. This is refactored to just call translate
once and then build a per-target condition.
This code is used for things like referencing ext rate limit, ext auth,
ext proc, etc servers. Pointing to an InferencePool makes no ense at
all.
This does NOT prevent attaching policies or routes to inferencePool
This PR attempts to document and rationalize our failure modes during
translation. The idea is to pick a logical best option for each type of
policy, that generally results in "fails at runtime" and NOT "policy is
silently ignored". For policies where there is a 'fail open' mode that
is respected.
As part of this, the PR fixes a number of bad behaviors. Below lists the
_current behaviors_
# Failure modes
**External Authorization (ExtAuth)**: backend ref can fail, making it
"invalid". Proxy fails or doesn't based on failure mode *which is not
exposed in CRD so is always fail-closed today*.
**External Processor (ExtProc)**: Same as above; always fail-closed
**JWT Authentication**: invalid provider is skipped and thus cannot
match
**Basic Authentication**: Keys we cannot fetch are elided, so no keys
will match
**API Key Authentication**: Keys we cannot fetch are elided, so no keys
will match. If we fetch multiple keys and some are invalid, the valid
keys will still work.
**Authorization (RBAC)**: invalid expressions will be treated as runtime
errors. "Allow" never matches (so will never allow), "Deny" also never
matches (so will never deny!!); "Require" mode would be a safer "Deny"
**Global Rate Limit**: same as ext authz. Failure mode exists but NOT
exposed in CRD
**Transformation (Headers/Body)**: invalid expressions will be treated
as runtime errors. These will strip the header we processing or empty
the body.
**Retry**: backoff: disabled (should be impossible), attempts: clamped
to 0..255
**Local Rate Limit**: cannot fail
**CORS**: cannot fail
**CSRF**: cannot fail
**Header Modifiers**: cannot fail
**Hostname Rewrite**: cannot fail
**Timeouts**: cannot fail
**Direct Response**: cannot fail
**Frontend TCP**: cannot fail
**Frontend TLS**: cannot fail
**Frontend HTTP**: cannot fail
**Access Logging**: backend ref can fail, making it "invalid". Calls
will fail to export which will do.... ???. Missing status for CEL
expressions. Invalid CEL expressions will make the attribute not log,
filter not match, etc.
**Tracing**: backend ref can fail, making it "invalid". Calls will fail
to export which will do.... ???. Missing status for CEL expressions.
Invalid CEL expressions will make the attribute not log, sample never
match, etc.
**Backend TLS**: MtlsCertificateRef not found, does not use any client
auth. This seems right, since then the server will probably reject which
is fine. CACertificateRefs missing will not include those; also seems
right as it will (probably) make them reject.
**Backend HTTP**: cannot fail
**Backend TCP**: does nothing! TODO.
**Backend Tunnel (HTTPS Proxy)**: invalid backend ref will make the
backend invalid and thus block requests.
**Backend Auth (API Keys, AWS, Azure, GCP)**: For missing key, always
set to an empty value. AWS: sets empty values which I assume proxy fails
at runtime with. Azure: same as AWS. GCP: cannot fail.
**Backend Health (Eviction)**: cannot fail
**AI Policy (Prompts, Guards, Aliases)**: Invalid defaults or overrides
are ignored, but should not be possible. Invalid transformation CEL
expressions are reported as status and sent to the data plane, where it
will wipe out the field if it fails. Invalid prompt guards are ignored
entirely
**MCP Authorization**: Same as Authorization; CEL expressions not
checked for status either
**MCP Authentication**: If we cannot resolve JWKS we will get no JWKS
which will never validate a request. an invalid keys in ResourceMetadata
(should be impossible) are stripped
1. Avoid special handling of InternalName where we need to cut it and
assume knowledge. Instead just pass what we need
2. Avoid excessive nesting of functions for route generation.
refactor the core wds collection init to make it easier to
change/override when wee want to work around istio changes
additionally, removing some unused
WithWorkloadAddressProviderFunc/WithServiceAddressProviderFunc
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This hardens MCP degraded-mode behavior so multiplexed backends can keep
serving healthy targets without hanging, looping, or corrupting
resumable session state.
The original `failureMode` work was meant to cover three concrete
failure cases:
- `#1162`: zero-target MCP backends should not trap clients in a
reconnect loop
- `#1029`: `failOpen` should skip bad targets, but still fail if every
configured target is unusable
- `#962`: one bad upstream should not take down the rest of a
multiplexed MCP session
This change makes that behavior consistent across init, runtime fanout,
and session resume.
At init time, `failOpen` now allows partial success but still returns
`NoBackends` when all configured targets fail. Zero configured targets
remain a special case so we do not regress the zero-target handling
behind `#1162`. The `failureMode` setting is also threaded through the
controller, local config, schema, and MCP runtime so the configured
behavior is preserved end to end.
At runtime, multiplexed fanout now degrades correctly when individual
upstreams fail. That includes the legacy SSE path, which was the sharp
edge here: a dead legacy SSE transport could leave pending requests
orphaned and hang the whole downstream request. This change makes
wrapped transports fail pending work, fail fast once dead, and treat
dead legacy SSE upstreams as unavailable for the rest of that downstream
session instead of trying to silently rejoin mid-session. That keeps
healthy stdio, SSE, and streamable HTTP targets serving without
introducing questionable session semantics for non-resumable transports.
For resumable streamable HTTP MCP sessions, persisted upstream session
state is now keyed by target name. New session handles include an
optional target field, and resume restores by target identity instead of
positional zip. Old handles without target names still use the legacy
positional path, so existing session IDs remain readable. This avoids
misbinding persisted sessions when `failOpen` changes the initialized
target set.
Most of the diff is test coverage plus generated/schema updates from
wiring the config surface through. The behavior changes themselves are
concentrated in the MCP relay, upstream transport wrappers, and resume
logic.
Fixes#1162Fixes#1029Fixes#962
---------
Co-authored-by: John Howard <john.howard@solo.io>
We associate Backends with Gateways to avoid sending Backends that are
not relevant. To do this, we need to know all relevant backends.
This was working only for backends from routes, not policies. This adds
policies.
A tricky thing is policies can be inline on backend, and in a few
places. You can also have cycles and chains (!) of backend --> backend
--> backend, etc.
The DumpStatus was mostly intended to avoid the churn this has of
regenerating all policies, but has a major limit that we can only report
1 status object which I want to break in some tests. Switching over
## Summary
- honor when translating synthetic inference services
- map to while keeping the default protocol as
- add translator golden coverage for an h2c-backed InferencePool
Closes#1194.
## Validation
- go test ./controller/pkg/kgateway/agentgatewaysyncer/...
- go test ./controller/pkg/agentgateway/translator/...
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
When a Gateway or GatewayParameters is updated or deleted such that a
previously-created PodDisruptionBudget, HorizontalPodAutoscaler, or
VerticalPodAutoscaler is no longer required, the controller does not
currently remove those resources. This is a resource leak.
ownerReferences-based garbage collection is insufficient here because it
only fires when the owner is deleted — if the owner (e.g. the Gateway)
still exists but has been updated to no longer require a PDB/HPA/VPA,
Kubernetes will not clean up the now-orphaned resource.
The `PruneRemovedResources` method on the `Deployer` collects
pdb/hpa/vpa resources owned by the reconciling gateway and checks them
against the list of desired objects and prunes those which are
undesired.