Compare commits

...

47 Commits

Author SHA1 Message Date
Rajat Vig
34871c59e3
fix: handle missing fields in Vertex AI Gemini completions responses (#1465)
## Summary
Gemini's OpenAI-compatible endpoint intermittently omits `message` from
choices and `completion_tokens`/`prompt_tokens` from usage. Add instead
of returning 503. Follows the same pattern as #977.

- Add `#[serde(default)]` to `Choice.message` and `Usage` token count
fields in `completions.rs`
- Derive `Default` on `ResponseMessage` and `Usage` so missing fields
deserialize to zero/empty values
- Add test that verifies deserialization succeeds when Gemini omits
these fields

Fixes #1464

---------

Signed-off-by: Rajat Vig <rvig@etsy.com>
2026-04-08 14:22:24 -07:00
Filinto Duran
0d4b0a9f90
fix(openai): passthrough responses input (#1454)
Problem:

without this relaxation passthrough openai through agw would fail on the
second and subsequent requests

## Summary

- Use raw JSON (`serde_json::Value`) for Responses API input items so
assistant
history and unknown item types (e.g. `phase`, `refusal`) pass through
without
    deserialization failure
- Best-effort message extraction for policies/token counting still works
via
`RawInputItem::as_simple_message()`; strict validation deferred to
provider
    or Bedrock conversion layer
2026-04-08 14:12:06 -07:00
dependabot[bot]
b7bb1a5a7c
build(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp from 0.14.0 to 0.19.0 in the go_modules group across 1 directory (#1495)
Bumps the go_modules group with 1 update in the / directory:
[go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp](https://github.com/open-telemetry/opentelemetry-go).

Updates `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`
from 0.14.0 to 0.19.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/open-telemetry/opentelemetry-go/releases">go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp's
releases</a>.</em></p>
<blockquote>
<h2>Release v0.19.0</h2>
<h3>Added</h3>
<ul>
<li>Added <code>Marshaler</code> config option to <code>otlphttp</code>
to enable otlp over json or protobufs. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1586">#1586</a>)</li>
<li>A <code>ForceFlush</code> method to the
<code>&quot;go.opentelemetry.io/otel/sdk/trace&quot;.TracerProvider</code>
to flush all registered <code>SpanProcessor</code>s. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1608">#1608</a>)</li>
<li>Added <code>WithSampler</code> and <code>WithSpanLimits</code> to
tracer provider. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1633">#1633</a>,
<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1702">#1702</a>)</li>
<li><code>&quot;go.opentelemetry.io/otel/trace&quot;.SpanContext</code>
now has a <code>remote</code> property, and <code>IsRemote()</code>
predicate, that is true when the <code>SpanContext</code> has been
extracted from remote context data. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1701">#1701</a>)</li>
<li>A <code>Valid</code> method to the
<code>&quot;go.opentelemetry.io/otel/attribute&quot;.KeyValue</code>
type. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1703">#1703</a>)</li>
</ul>
<h3>Changed</h3>
<ul>
<li><code>trace.SpanContext</code> is now immutable and has no exported
fields. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1573">#1573</a>)
<ul>
<li><code>trace.NewSpanContext()</code> can be used in conjunction with
the <code>trace.SpanContextConfig</code> struct to initialize a new
<code>SpanContext</code> where all values are known.</li>
</ul>
</li>
<li>Update the <code>ForceFlush</code> method signature to the
<code>&quot;go.opentelemetry.io/otel/sdk/trace&quot;.SpanProcessor</code>
to accept a <code>context.Context</code> and return an error. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1608">#1608</a>)</li>
<li>Update the <code>Shutdown</code> method to the
<code>&quot;go.opentelemetry.io/otel/sdk/trace&quot;.TracerProvider</code>
return an error on shutdown failure. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1608">#1608</a>)</li>
<li>The SimpleSpanProcessor will now shut down the enclosed
<code>SpanExporter</code> and gracefully ignore subsequent calls to
<code>OnEnd</code> after <code>Shutdown</code> is called. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1612">#1612</a>)</li>

<li><code>&quot;go.opentelemetry.io/sdk/metric/controller.basic&quot;.WithPusher</code>
is replaced with <code>WithExporter</code> to provide consistent naming
across project. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1656">#1656</a>)</li>
<li>Added non-empty string check for trace <code>Attribute</code> keys.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1659">#1659</a>)</li>
<li>Add <code>description</code> to SpanStatus only when
<code>StatusCode</code> is set to error. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1662">#1662</a>)</li>
<li>Jaeger exporter falls back to <code>resource.Default</code>'s
<code>service.name</code> if the exported Span does not have one. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1673">#1673</a>)</li>
<li>Jaeger exporter populates Jaeger's Span Process from Resource. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1673">#1673</a>)</li>
<li>Renamed the <code>LabelSet</code> method of
<code>&quot;go.opentelemetry.io/otel/sdk/resource&quot;.Resource</code>
to <code>Set</code>. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1692">#1692</a>)</li>
<li>Changed <code>WithSDK</code> to <code>WithSDKOptions</code> to
accept variadic arguments of <code>TracerProviderOption</code> type in
<code>go.opentelemetry.io/otel/exporters/trace/jaeger</code> package.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1693">#1693</a>)</li>
<li>Changed <code>WithSDK</code> to <code>WithSDKOptions</code> to
accept variadic arguments of <code>TracerProviderOption</code> type in
<code>go.opentelemetry.io/otel/exporters/trace/zipkin</code> package.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1693">#1693</a>)</li>

<li><code>&quot;go.opentelemetry.io/otel/sdk/resource&quot;.NewWithAttributes</code>
will now drop any invalid attributes passed. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1703">#1703</a>)</li>

<li><code>&quot;go.opentelemetry.io/otel/sdk/resource&quot;.StringDetector</code>
will now error if the produced attribute is invalid. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1703">#1703</a>)</li>
</ul>
<h3>Removed</h3>
<ul>
<li>Removed <code>serviceName</code> parameter from Zipkin exporter and
uses resource instead. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1549">#1549</a>)</li>
<li>Removed <code>WithConfig</code> from tracer provider to avoid
overriding configuration. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1633">#1633</a>)</li>
<li>Removed the exported <code>SimpleSpanProcessor</code> and
<code>BatchSpanProcessor</code> structs.
These are now returned as a SpanProcessor interface from their
respective constructors. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1638">#1638</a>)</li>
<li>Removed <code>WithRecord()</code> from <code>trace.SpanOption</code>
when creating a span. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1660">#1660</a>)</li>
<li>Removed setting status to <code>Error</code> while recording an
error as a span event in <code>RecordError</code>. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1663">#1663</a>)</li>
<li>Removed <code>jaeger.WithProcess</code> configuration option. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1673">#1673</a>)</li>
<li>Removed <code>ApplyConfig</code> method from
<code>&quot;go.opentelemetry.io/otel/sdk/trace&quot;.TracerProvider</code>
and the now unneeded <code>Config</code> struct. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1693">#1693</a>)</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>Jaeger Exporter: Ensure mapping between OTEL and Jaeger span data
complies with the specification. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1626">#1626</a>)</li>
<li><code>SamplingResult.TraceState</code> is correctly propagated to a
newly created span's <code>SpanContext</code>. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1655">#1655</a>)</li>
<li>The <code>otel-collector</code> example now correctly flushes metric
events prior to shutting down the exporter. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1678">#1678</a>)</li>
<li>Do not set span status message in
<code>SpanStatusFromHTTPStatusCode</code> if it can be inferred from
<code>http.status_code</code>. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1681">#1681</a>)</li>
<li>Synchronization issues in global trace delegate implementation. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1686">#1686</a>)</li>
<li>Reduced excess memory usage by global <code>TracerProvider</code>.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1687">#1687</a>)</li>
</ul>
<hr />
<h2>Raw changes made between v0.18.0 and v0.19.0</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md">go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp's
changelog</a>.</em></p>
<blockquote>
<h2>[1.43.0/0.65.0/0.19.0] 2026-04-02</h2>
<h3>Added</h3>
<ul>
<li>Add <code>IsRandom</code> and <code>WithRandom</code> on
<code>TraceFlags</code>, and <code>IsRandom</code> on
<code>SpanContext</code> in <code>go.opentelemetry.io/otel/trace</code>
for <a
href="https://www.w3.org/TR/trace-context-2/#random-trace-id-flag">W3C
Trace Context Level 2 Random Trace ID Flag</a> support. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8012">#8012</a>)</li>
<li>Add service detection with <code>WithService</code> in
<code>go.opentelemetry.io/otel/sdk/resource</code>. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/7642">#7642</a>)</li>
<li>Add <code>DefaultWithContext</code> and
<code>EnvironmentWithContext</code> in
<code>go.opentelemetry.io/otel/sdk/resource</code> to support plumbing
<code>context.Context</code> through default and environment detectors.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8051">#8051</a>)</li>
<li>Support attributes with empty value (<code>attribute.EMPTY</code>)
in
<code>go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc</code>.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li>
<li>Support attributes with empty value (<code>attribute.EMPTY</code>)
in
<code>go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc</code>.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li>
<li>Support attributes with empty value (<code>attribute.EMPTY</code>)
in
<code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc</code>.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li>
<li>Support attributes with empty value (<code>attribute.EMPTY</code>)
in
<code>go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp</code>.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li>
<li>Support attributes with empty value (<code>attribute.EMPTY</code>)
in
<code>go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp</code>.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li>
<li>Support attributes with empty value (<code>attribute.EMPTY</code>)
in
<code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp</code>.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li>
<li>Support attributes with empty value (<code>attribute.EMPTY</code>)
in
<code>go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest</code>.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li>
<li>Add support for per-series start time tracking for cumulative
metrics in <code>go.opentelemetry.io/otel/sdk/metric</code>.
Set <code>OTEL_GO_X_PER_SERIES_START_TIMESTAMPS=true</code> to enable.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8060">#8060</a>)</li>
<li>Add <code>WithCardinalityLimitSelector</code> for metric reader for
configuring cardinality limits specific to the instrument kind. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/7855">#7855</a>)</li>
</ul>
<h3>Changed</h3>
<ul>
<li>Introduce the <code>EMPTY</code> Type in
<code>go.opentelemetry.io/otel/attribute</code> to reflect that an empty
value is now a valid value, with <code>INVALID</code> remaining as a
deprecated alias of <code>EMPTY</code>. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li>
<li>Improve slice handling in
<code>go.opentelemetry.io/otel/attribute</code> to optimize short slice
values with fixed-size fast paths. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8039">#8039</a>)</li>
<li>Improve performance of span metric recording in
<code>go.opentelemetry.io/otel/sdk/trace</code> by returning early if
self-observability is not enabled. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8067">#8067</a>)</li>
<li>Improve formatting of metric data diffs in
<code>go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest</code>.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8073">#8073</a>)</li>
</ul>
<h3>Deprecated</h3>
<ul>
<li>Deprecate <code>INVALID</code> in
<code>go.opentelemetry.io/otel/attribute</code>. Use <code>EMPTY</code>
instead. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>Return spec-compliant <code>TraceIdRatioBased</code> description.
This is a breaking behavioral change, but it is necessary to
make the implementation <a
href="https://opentelemetry.io/docs/specs/otel/trace/sdk/#traceidratiobased">spec-compliant</a>.
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8027">#8027</a>)</li>
<li>Fix a race condition in
<code>go.opentelemetry.io/otel/sdk/metric</code> where the lastvalue
aggregation could collect the value 0 even when no zero-value
measurements were recorded. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8056">#8056</a>)</li>
<li>Limit HTTP response body to 4 MiB in
<code>go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp</code>
to mitigate excessive memory usage caused by a misconfigured or
malicious server.
Responses exceeding the limit are treated as non-retryable errors. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li>
<li>Limit HTTP response body to 4 MiB in
<code>go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp</code>
to mitigate excessive memory usage caused by a misconfigured or
malicious server.
Responses exceeding the limit are treated as non-retryable errors. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li>
<li>Limit HTTP response body to 4 MiB in
<code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp</code>
to mitigate excessive memory usage caused by a misconfigured or
malicious server.
Responses exceeding the limit are treated as non-retryable errors. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li>
<li><code>WithHostID</code> detector in
<code>go.opentelemetry.io/otel/sdk/resource</code> to use full path for
<code>kenv</code> command on BSD. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8113">#8113</a>)</li>
<li>Fix missing <code>request.GetBody</code> in
<code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp</code>
to correctly handle HTTP2 GOAWAY frame. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8096">#8096</a>)</li>
</ul>
<h2>[1.42.0/0.64.0/0.18.0/0.0.16] 2026-03-06</h2>
<h3>Added</h3>
<ul>
<li>Add <code>go.opentelemetry.io/otel/semconv/v1.40.0</code> package.
The package contains semantic conventions from the <code>v1.40.0</code>
version of the OpenTelemetry Semantic Conventions.
See the <a
href="https://github.com/open-telemetry/opentelemetry-go/blob/main/semconv/v1.40.0/MIGRATION.md">migration
documentation</a> for information on how to upgrade from
<code>go.opentelemetry.io/otel/semconv/v1.39.0</code>. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/7985">#7985</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="2b4fa9681b"><code>2b4fa96</code></a>
Release v0.19.0 (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1710">#1710</a>)</li>
<li><a
href="4beb70416e"><code>4beb704</code></a>
sdk/trace: removing ApplyConfig and Config (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1693">#1693</a>)</li>
<li><a
href="1d42be1601"><code>1d42be1</code></a>
Rename WithDefaultSampler TracerProvider option to WithSampler and
update doc...</li>
<li><a
href="860d5d86e7"><code>860d5d8</code></a>
Add flag to determine whether SpanContext is remote (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1701">#1701</a>)</li>
<li><a
href="0fe65e6bd2"><code>0fe65e6</code></a>
Comply with OpenTelemetry attributes specification (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1703">#1703</a>)</li>
<li><a
href="888843519d"><code>8888435</code></a>
Bump google.golang.org/api from 0.40.0 to 0.41.0 in
/exporters/trace/jaeger (...</li>
<li><a
href="345f264a13"><code>345f264</code></a>
breaking(zipkin): removes servicName from zipkin exporter. (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1697">#1697</a>)</li>
<li><a
href="62cbf0f240"><code>62cbf0f</code></a>
Populate Jaeger's Span.Process from Resource (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1673">#1673</a>)</li>
<li><a
href="28eaaa9a91"><code>28eaaa9</code></a>
Add a test to prove the Tracer is safe for concurrent calls (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1665">#1665</a>)</li>
<li><a
href="8b1be11a54"><code>8b1be11</code></a>
Rename resource pkg label vars and methods (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/1692">#1692</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/open-telemetry/opentelemetry-go/compare/v0.14.0...v0.19.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp&package-manager=go_modules&previous-version=0.14.0&new-version=0.19.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/agentgateway/agentgateway/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-08 13:03:27 -07:00
markdingram-anaplan
39c98f335b
feat: support OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES env vars (#1452)
- slight tweak of #1396 

- check xds address is set before overwriting env var values, otherwise
OTEL_SERVICE_NAME & OTEL_RESOURCE_ATTRIBUTES values are overidden with
"default" -
<cf5ade8a49/crates/agentgateway/src/config.rs (L84)>
2026-04-08 12:10:29 -07:00
Steven Landow
edbb343c98
allow expanding references in traffic plugin (#1482)
Allow plugins which resolve policy targets to expand a single
group+kind+section to expand to multiple actual targets in terms of
proxy configuration.
2026-04-08 10:53:33 -07:00
Rajat Vig
a9491f891a
test: add MCP level tests for extauth and ratelimit policies (#1421)
## Summary
- Adds `mcp_local_ratelimit` test to verify local rate limiting works
with MCP backends
- Adds `mcp_extauth_deny` test to verify external authorization denial
works with MCP backends
- Adds `mcp_remote_ratelimit_deny` test to verify remote rate limiting
works with MCP backends

All three tests use mock gRPC servers where applicable and validate that
policy enforcement correctly propagates through the MCP protocol layer.

Closes #554

---------

Signed-off-by: Rajat Vig <rvig@etsy.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: John Howard <john.howard@solo.io>
2026-04-08 09:10:21 -07:00
John Howard
ee83b193f7
fixup release post-blacksmith (#1493)
Release failed due to the blacksmith auto-migration not properly
handling our matrix
2026-04-08 08:02:04 -07:00
Copilot
0b1cb95c9d
restrict nightly workflow to upstream repo only (#1466)
The nightly release workflow runs on forks, which is unnecessary and can
cause failures due to missing secrets/permissions.

## Changes

- **`.github/workflows/nightly.yml`**: Add `if: github.repository ==
'agentgateway/agentgateway'` to the `prepare` job, gating the entire
workflow since `release` depends on `prepare`.

```yaml
prepare:
  if: github.repository == 'agentgateway/agentgateway'
  runs-on: ubuntu-24.04
```

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: howardjohn <623453+howardjohn@users.noreply.github.com>
2026-04-08 06:21:06 -07:00
John Howard
ac0005fb0d
mcp: omit expected 405 error logs (#1484)
Ref https://github.com/agentgateway/agentgateway/issues/1475
2026-04-08 06:20:43 -07:00
dependabot[bot]
ead514fffd
build(deps): bump the npm_and_yarn group across 1 directory with 2 updates (#1489)
Bumps the npm_and_yarn group with 2 updates in the /ui directory:
[@hono/node-server](https://github.com/honojs/node-server) and
[hono](https://github.com/honojs/hono).

Updates `@hono/node-server` from 1.19.10 to 1.19.13
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/honojs/node-server/releases"><code>@​hono/node-server</code>'s
releases</a>.</em></p>
<blockquote>
<h2>v1.19.13</h2>
<h2>Security Fix</h2>
<p>Fixed an issue in Serve Static Middleware where inconsistent handling
of repeated slashes (<code>//</code>) between the router and static file
resolution could allow middleware to be bypassed. Users of Serve Static
Middleware are encouraged to upgrade to this version.</p>
<p>See GHSA-92pp-h63x-v22m for details.</p>
<h2>v1.19.12</h2>
<h2>What's Changed</h2>
<ul>
<li>chore: ignore claude setting by <a
href="https://github.com/yusukebe"><code>@​yusukebe</code></a> in <a
href="https://redirect.github.com/honojs/node-server/pull/314">honojs/node-server#314</a></li>
<li>fix: request draining for early 413 responses by <a
href="https://github.com/usualoma"><code>@​usualoma</code></a> in <a
href="https://redirect.github.com/honojs/node-server/pull/329">honojs/node-server#329</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/honojs/node-server/compare/v1.19.11...v1.19.12">https://github.com/honojs/node-server/compare/v1.19.11...v1.19.12</a></p>
<h2>v1.19.11</h2>
<h2>What's Changed</h2>
<ul>
<li>fix: do not overwrite Content-Length in the fast path pattern if
Content-Length already exists. by <a
href="https://github.com/usualoma"><code>@​usualoma</code></a> in <a
href="https://redirect.github.com/honojs/node-server/pull/309">honojs/node-server#309</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/honojs/node-server/compare/v1.19.10...v1.19.11">https://github.com/honojs/node-server/compare/v1.19.10...v1.19.11</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="fd64e659a3"><code>fd64e65</code></a>
1.19.13</li>
<li><a
href="025c30f55d"><code>025c30f</code></a>
Merge commit from fork</li>
<li><a
href="6cdb5a7249"><code>6cdb5a7</code></a>
1.19.12</li>
<li><a
href="70250f780e"><code>70250f7</code></a>
fix: request draining for early 413 responses (<a
href="https://redirect.github.com/honojs/node-server/issues/329">#329</a>)</li>
<li><a
href="cfc08b330a"><code>cfc08b3</code></a>
chore: ignore claude setting (<a
href="https://redirect.github.com/honojs/node-server/issues/314">#314</a>)</li>
<li><a
href="ecd4d6be55"><code>ecd4d6b</code></a>
1.19.11</li>
<li><a
href="c94489955e"><code>c944899</code></a>
fix: do not overwrite Content-Length in the fast path pattern if
Content-Leng...</li>
<li>See full diff in <a
href="https://github.com/honojs/node-server/compare/v1.19.10...v1.19.13">compare
view</a></li>
</ul>
</details>
<br />

Updates `hono` from 4.12.7 to 4.12.12
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/honojs/hono/releases">hono's
releases</a>.</em></p>
<blockquote>
<h2>v4.12.12</h2>
<h2>Security fixes</h2>
<p>This release includes fixes for the following security issues:</p>
<h3>Middleware bypass via repeated slashes in serveStatic</h3>
<p>Affects: Serve Static middleware. Fixes a path normalization
inconsistency where repeated slashes (<code>//</code>) could bypass
route-based middleware protections and allow access to protected static
files. GHSA-wmmm-f939-6g9c</p>
<h3>Path traversal in toSSG() allows writing files outside the output
directory</h3>
<p>Affects: <code>toSSG()</code> for Static Site Generation. Fixes a
path traversal issue where crafted <code>ssgParams</code> values could
write files outside the configured output directory.
GHSA-xf4j-xp2r-rqqx</p>
<h3>Incorrect IP matching in ipRestriction() for IPv4-mapped IPv6
addresses</h3>
<p>Affects: IP Restriction Middleware. Fixes improper handling of
IPv4-mapped IPv6 addresses (e.g. <code>::ffff:127.0.0.1</code>) that
could cause allow/deny rules to be bypassed. GHSA-xpcf-pg52-r92g</p>
<h3>Missing validation of cookie name on write path in setCookie()</h3>
<p>Affects: <code>setCookie()</code>, <code>serialize()</code>, and
<code>serializeSigned()</code> from <code>hono/cookie</code>. Fixes
missing validation of cookie names on the write path, preventing
inconsistent handling between parsing and serialization.
GHSA-26pp-8wgv-hjvm</p>
<h3>Non-breaking space prefix bypass in cookie name handling in
getCookie()</h3>
<p>Affects: <code>getCookie()</code> from <code>hono/cookie</code>.
Fixes a discrepancy in cookie name handling that could allow
attacker-controlled cookies to override legitimate ones and bypass
prefix protections. GHSA-r5rp-j6wh-rvv4</p>
<hr />
<p>Users who use Serve Static, Static Site Generation, Cookie utilities,
or IP restriction middleware are strongly encouraged to upgrade to this
version.</p>
<h2>v4.12.11</h2>
<h2>What's Changed</h2>
<ul>
<li>feat(css): add classNameSlug option to createCssContext by <a
href="https://github.com/flow-pie"><code>@​flow-pie</code></a> in <a
href="https://redirect.github.com/honojs/hono/pull/4834">honojs/hono#4834</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/flow-pie"><code>@​flow-pie</code></a>
made their first contribution in <a
href="https://redirect.github.com/honojs/hono/pull/4834">honojs/hono#4834</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/honojs/hono/compare/v4.12.10...v4.12.11">https://github.com/honojs/hono/compare/v4.12.10...v4.12.11</a></p>
<h2>v4.12.10</h2>
<h2>What's Changed</h2>
<ul>
<li>test(router): fix <code>Simple capturing group</code> test by <a
href="https://github.com/yusukebe"><code>@​yusukebe</code></a> in <a
href="https://redirect.github.com/honojs/hono/pull/4838">honojs/hono#4838</a></li>
<li>docs: fix impaired -&gt; inspired typo in benchmark READMEs by <a
href="https://github.com/Abhi3975"><code>@​Abhi3975</code></a> in <a
href="https://redirect.github.com/honojs/hono/pull/4843">honojs/hono#4843</a></li>
<li>fix(jsx/dom): apply select value after children are rendered by <a
href="https://github.com/usualoma"><code>@​usualoma</code></a> in <a
href="https://redirect.github.com/honojs/hono/pull/4847">honojs/hono#4847</a></li>
<li>fix(compress): convert strong ETag to weak ETag when compressing by
<a href="https://github.com/usualoma"><code>@​usualoma</code></a> in <a
href="https://redirect.github.com/honojs/hono/pull/4848">honojs/hono#4848</a></li>
<li>docs(ip-restriction): add clear JSDoc examples and param types by <a
href="https://github.com/VISHNU7KASIREDDY"><code>@​VISHNU7KASIREDDY</code></a>
in <a
href="https://redirect.github.com/honojs/hono/pull/4851">honojs/hono#4851</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/Abhi3975"><code>@​Abhi3975</code></a>
made their first contribution in <a
href="https://redirect.github.com/honojs/hono/pull/4843">honojs/hono#4843</a></li>
<li><a
href="https://github.com/VISHNU7KASIREDDY"><code>@​VISHNU7KASIREDDY</code></a>
made their first contribution in <a
href="https://redirect.github.com/honojs/hono/pull/4851">honojs/hono#4851</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c37ba26da9"><code>c37ba26</code></a>
4.12.12</li>
<li><a
href="cc067c8559"><code>cc067c8</code></a>
Merge commit from fork</li>
<li><a
href="a586cd72e3"><code>a586cd7</code></a>
Merge commit from fork</li>
<li><a
href="48fa2233bc"><code>48fa223</code></a>
Merge commit from fork</li>
<li><a
href="b470278920"><code>b470278</code></a>
Merge commit from fork</li>
<li><a
href="9aff14bd72"><code>9aff14b</code></a>
Merge commit from fork</li>
<li><a
href="2c403c67eb"><code>2c403c6</code></a>
4.12.11</li>
<li><a
href="f82aba8e8e"><code>f82aba8</code></a>
feat(css): add classNameSlug option to createCssContext (<a
href="https://redirect.github.com/honojs/hono/issues/4834">#4834</a>)</li>
<li><a
href="9f374a55b2"><code>9f374a5</code></a>
4.12.10</li>
<li><a
href="a8c56a6620"><code>a8c56a6</code></a>
docs(ip-restriction): add clear JSDoc examples and param types (<a
href="https://redirect.github.com/honojs/hono/issues/4851">#4851</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/honojs/hono/compare/v4.12.7...v4.12.12">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/agentgateway/agentgateway/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-08 06:20:22 -07:00
syn-zhu
70042851d6
fix(ui): parse namespace from service name when matching A2A policies (#1487)
Fixes #1486

## Summary

`NamespacedHostname` always serializes as `"namespace/hostname"` (via
its `Display` impl), but `routeBackendMatchesA2aPolicy()` compared the
full `"namespace/hostname"` string against just the `hostname` from the
A2A policy target. This caused A2A routes to show as "Service" in the
Playground instead of "A2A".

## Change

Split `service.name` on `/` to extract namespace and hostname before
comparing against the policy target. Also removed the dead object-format
branch (`name?.hostname`) since `NamespacedHostname` never serializes as
an object.

## Regression of #978 / #979

The original fix in #979 handled the string case but didn't account for
the `namespace/` prefix in the serialized format.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 19:14:23 -07:00
John Howard
4cf848a9e9
api: documented that backend MCP auth is deprecated (#1437)
The removals are deadcode, not API changes.
2026-04-07 16:28:07 -07:00
Ákos Tóth
0418247e7e
feat: add cacheMessageOffset to PromptCachingConfig (#1412)
Add `cacheMessageOffset` field to `PromptCachingConfig`. When set,
shifts the Bedrock message cache point further back from the default
second-to-last position.

- `cacheMessageOffset: 0` (default): no change in behavior
- `cacheMessageOffset: N`: cache point moves N additional messages
toward the start, clamped to bounds

This gives callers control over which portion of the conversation is
covered by the cache prefix — useful for agentic workloads where
mid-conversation compaction may mutate recent messages.

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-04-07 16:25:36 -07:00
dependabot[bot]
28ead632fe
build(deps): bump aiohttp from 3.13.3 to 3.13.4 in /examples/a2a/strands-agents in the uv group across 1 directory (#1427)
[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=aiohttp&package-manager=uv&previous-version=3.13.3&new-version=3.13.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/agentgateway/agentgateway/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-07 15:30:58 -07:00
Filinto Duran
ccc31c040b
fix(llm/bedrock): correct output_index in Responses API stream events (#1455)
Summary

- Fix output_index in Bedrock streaming Responses API events to reflect
position in the response.output[] array instead of
  Bedrock's internal content_block_index
- Text content events now correctly use output_index: 0 (the message
item), tool calls get sequential indices starting at 1
- Add streaming tool-call snapshot tests (tool.bin) to prevent
regressions

  Problem

The old code used Bedrock's content_block_index as output_index. This
collided with the message item (always at index 0) for
tool-only or multi-tool responses — e.g., a tool-only stream would emit
both the message and the first tool call at output_index:
  0.

  Test plan

  - Existing from_bedrock snapshot tests pass
- New streaming tool-call snapshots verify correct output_index values
(message=0, tool1=1, tool2=2)
  - Verified against live Bedrock with a multi-tool streaming request
2026-04-07 15:09:58 -07:00
Filinto Duran
3bf346eb6b
update async-openai with new types (#1453)
https://github.com/64bit/async-openai/releases/tag/async-openai-v0.34.0

feat: sync upstream spec by @64bit in
https://github.com/64bit/async-openai/pull/529
added video charater, edit apis
updated types

---------

Co-authored-by: John Howard <john.howard@solo.io>
2026-04-07 13:42:32 -07:00
John Howard
c1d05ae952
fix: make index lookup nil-safe after Istio bump (#1483)
https://github.com/agentgateway/agentgateway/pull/1477 triggers a new
race, as Index can now be nil. To avoid adding a bunch of nil-checks
everywhere, introduce a common helper
2026-04-07 13:20:59 -07:00
Sai Teja Battula
4e9a4bdc4f
Fix #900: system-assigned managed identity crashes with NoopClient panic (#1433)
## Summary

Fix system-assigned managed identity crash caused by NoopClient panic.

## Problem

When `managedIdentity: {}` is configured (system-assigned), the gateway
panics on the first IMDS request:

**Root cause:** The `.map()` call short-circuits to `None` when
`user_assigned_identity` is `None`, so
`ManagedIdentityCredential::new(None)` is called. The Azure SDK then
falls back to its default `NoopClient` instead of the custom
reqwest-backed transport.

## Fix

Always construct `ManagedIdentityCredentialOptions` with the custom
transport, making `user_assigned_id` the only optional field. When
`user_assigned_id` is `None`, the SDK treats it as system-assigned —
same behaviour as before, but now with the correct HTTP client.

## Behaviour after fix

| Configuration | Before | After |
|---|---|---|
| `managedIdentity: {}` (system-assigned) | 💥 Panic | ✅ Works |
| `managedIdentity: { clientId: "..." }` | ✅ Works | ✅ Works |
| `managedIdentity: { objectId: "..." }` | ✅ Works | ✅ Works |
| `managedIdentity: { resourceId: "..." }` | ✅ Works | ✅ Works |

## Files changed

- `crates/agentgateway/src/http/auth.rs` — fix `ManagedIdentity` arm in
`build_credential()`

Fixes #900

---------

Co-authored-by: John Howard <john.howard@solo.io>
2026-04-07 13:06:26 -07:00
dependabot[bot]
661c1b0d99
build(deps): bump github.com/go-jose/go-jose/v4 from 4.1.3 to 4.1.4 in the go_modules group across 1 directory (#1481)
Bumps the go_modules group with 1 update in the / directory:
[github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose).

Updates `github.com/go-jose/go-jose/v4` from 4.1.3 to 4.1.4
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/go-jose/go-jose/releases">github.com/go-jose/go-jose/v4's
releases</a>.</em></p>
<blockquote>
<h2>v4.1.4</h2>
<h2>What's Changed</h2>
<p>Fixes Panic in JWE decryption. See <a
href="https://github.com/go-jose/go-jose/security/advisories/GHSA-78h2-9frx-2jm8">https://github.com/go-jose/go-jose/security/advisories/GHSA-78h2-9frx-2jm8</a></p>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/go-jose/go-jose/compare/v4.1.3...v4.1.4">https://github.com/go-jose/go-jose/compare/v4.1.3...v4.1.4</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="0e59876635"><code>0e59876</code></a>
Merge commit from fork</li>
<li><a
href="ddffdbcec8"><code>ddffdbc</code></a>
Bump actions/checkout from 5 to 6 (<a
href="https://redirect.github.com/go-jose/go-jose/issues/213">#213</a>)</li>
<li>See full diff in <a
href="https://github.com/go-jose/go-jose/compare/v4.1.3...v4.1.4">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/go-jose/go-jose/v4&package-manager=go_modules&previous-version=4.1.3&new-version=4.1.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/agentgateway/agentgateway/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-07 12:13:24 -07:00
Rui Gu
d3db9106b8
fix(controller): Service-targeted policies dropped due to init ordering and missing PolicyAttachment traversal (#1474)
## Changes
1. **Split `AgwPolicyCollection` into two phases** (authored by
@alexeldeib):
- `CollectPolicyReferences`: collects `PolicyAttachments` from all
plugins first
- `BuildPolicies`: builds policies with the fully-populated
`ReferenceIndex`
2. **Use `LookupGatewaysForBackend` in `TranslateAgentgatewayPolicy`**:
traverses `PolicyAttachments` to resolve gateway associations for
indirectly-referenced targets.
3. **Add golden test**: validates that a Service-targeted backend TLS
policy is correctly associated with a gateway when the Service is only
referenced via an ext_proc policy's `backendRef`.

---------

Co-authored-by: Ace Eldeib <aeldeib@coreweave.com>
Co-authored-by: John Howard <john.howard@solo.io>
2026-04-07 11:30:52 -07:00
John Howard
8b96e4da5c
controller: faster deployer tests (#1469)
With -race, this took 53s on my fast machine before. now it takes 3.

The problem was we parsed all CRDs each time, now we do it once (and
ruse more code)
2026-04-07 10:46:48 -07:00
Yossi Mesika
143ee77d86
Bump Istio dependency (#1477)
Bump Istio dependency and adapt to changes in the ambient.Builder API.
2026-04-07 10:46:35 -07:00
John Howard
ebed6743b7
make sponsor icon bigger (#1480) 2026-04-07 10:45:30 -07:00
John Howard
b35cc7d561
Add sponsors to readme (#1479) 2026-04-07 10:44:06 -07:00
blacksmith-sh[bot]
5c6c7f10e2
.github/workflows: Migrate workflows to Blacksmith runners (#1473)
To whomever may be reviewing this PR,

**[Blacksmith](https://www.blacksmith.sh/) is the fastest way to run
your GitHub Actions.**

## What does this PR change?

This PR has been automatically generated by a team member in your GitHub
organization using Blacksmith's [Migration
Wizard](https://docs.blacksmith.sh/introduction/quickstart), or MigWiz
for short. This PR changes the following:

1. Your selected workflows will now run on Blacksmith's 2x faster
hardware (e.g., `runs-on: blacksmith-4vcpu-ubuntu-2204`). Learn more
about the [different instances available to choose
from](https://docs.blacksmith.sh/blacksmith-runners/overview#runner-tags).
2. Your jobs running on Blacksmith will now have all official GitHub and
popular third-party cache actions automatically interact with our 4x
faster, colocated cache. Learn more about [Blacksmith's actions
cache](https://docs.blacksmith.sh/blacksmith-caching/dependencies-actions).
3. Your GitHub Actions will now actually be observable. Learn more about
Blacksmith's
[logging](https://docs.blacksmith.sh/blacksmith-observability/logs) and
other
[observability](https://docs.blacksmith.sh/blacksmith-observability/dashboard)
features.
4. Your Docker builds will now automatically share their Docker layer
cache, resulting in up to 40x faster builds. Learn more about
[Blacksmith's Docker layer
caching](https://docs.blacksmith.sh/blacksmith-caching/docker-builds).

## FAQ

- Is this free? The first 3,000 minutes per month are free.
- Who uses Blacksmith? Clerk, Ashby, VEED, and 600+ others.
- What's the catch? There is none. Merge this thing already.

Co-authored-by: blacksmith-sh[bot] <157653362+blacksmith-sh[bot]@users.noreply.github.com>
2026-04-07 10:43:46 -07:00
Copilot
6deb9d38ae
Move proxy-test CI config validation to Rust integration tests (#1468)
- [x] Created `crates/agentgateway/tests/validate_examples.rs`:
integration test validating all `examples/**/config.yaml` files
- Fixed Windows path issue: `strip_prefix` operates on raw `PathBuf`
before string conversion
  - Removed `dbg!()` noise from CI logs
- Fixed header comment: `examples/**/config.yaml` (recursive walk, not
just one level)
- Converted single looping test to 17 per-config `#[rstest]` cases for
better CI failure granularity
  - Ran `rustfmt`
- [x] Updated `tools/manage-validation-deps.sh`:
- Removed `docker logs -f keycloak` (would hang indefinitely); replaced
with `docker logs --tail 50` only on failure
- [x] Updated `.github/workflows/pull_request.yml`:
  - Removed redundant `make validate` step
- Added "Start validation dependencies" step (ubuntu only) before tests
  - Set `KEYCLOAK_AVAILABLE=1` in env for the Test step on ubuntu
  - Added "Stop validation dependencies" step (ubuntu only, always runs)

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: howardjohn <623453+howardjohn@users.noreply.github.com>
Co-authored-by: John Howard <john.howard@solo.io>
2026-04-07 10:19:06 -07:00
John Howard
cf5ade8a49
xds: cleanup error paths that cannot fail (#1432) 2026-04-02 14:41:08 -07:00
Steven Landow
076fa978a1
proxy: apply service targeted policy (#1441)
* waypoint routes will either have a service_key (either in the explicit
route over xds _or_ in the implicit/default route)
* service_key will be used to lookup policies as a
`PolicyTarget::Backend(BackendTarget::Service(...))`

also adds some tests on the general hbone path + various policies on
that path
2026-04-02 11:17:36 -07:00
Nina Polshakova
fff0ec3e85
Use agentgateway.dev annotations (#1439)
Clean up to remove kgateway.dev annotations, but keep backwards
compatibility.

Signed-off-by: npolshakova <nina.polshakova@solo.io>
2026-04-02 12:37:23 -04:00
John Howard
a0cdfbeeb6
Enhance waypoint support in proxy (#1185)
This is like setting HTTP_PROXY=xyz but done with explicit policies not
implicit.

We support a subset of policies only on the tunnel connection: HTTP/1.1
only, authentication, and TLS.

TODO before merging: auth is hardcoded in a weird way right now pending
the backend auth customization
2026-04-02 08:43:28 -07:00
John Howard
4d04dea960
Add tests for reference grant (#1428) 2026-04-01 19:38:56 -07:00
John Howard
badc180a79
ext_proc: better handling of immediate response and fail_open (#1430)
Fixes https://github.com/agentgateway/agentgateway/issues/1400

1. Handle immediate response in various phases of the lifecycle
2. Remove complex task message passing
3. Adopt Envoy's new fail_open strategy which is that it only triggers
   on the initial request failing unless there is no body.
2026-04-01 19:38:47 -07:00
John Howard
9e726dbd69
cleanup misc hyper-util cruft that is unused (#1431) 2026-04-01 19:38:27 -07:00
John Howard
be0c3128ac
cleanup logging: executor is now cheap (#1425)
Before it was super expensive to make this; now its ~free so we don't
need hacks to skip it.

Verified with a quick perf test + flamegraph
2026-04-01 09:29:31 -07:00
John Howard
eeb246b9fd
Misc refactoring after oidc (#1424) 2026-04-01 08:35:29 -07:00
John Howard
8f0e00850d
controller: stricter controller securityContext by default (#1406) 2026-04-01 10:34:45 -05:00
Nina Polshakova
304e2c3073
Fix make run: no v prefix (#1422)
Found an issue with the helm version `make run` when checking
https://github.com/agentgateway/agentgateway/issues/1115 was fixed.

```
Error: path "agentgateway/controller/_test/agentgateway-crds-v1.0.1-dev.tgz" not found                                                                                                
make: *** [Makefile:500: deploy-agentgateway-crd-chart] Error 1 
```

The deploy targets were using `$(VERSION)` (which is `v1.0.1-dev`) but
the package targets create files using `$(HELM_CHART_VERSION_NO_V)`
(which is `1.0.1-dev` - without the "v" prefix). `make run` now uses the
correct version.

Signed-off-by: npolshakova <nina.polshakova@solo.io>
2026-04-01 08:20:19 -07:00
John Howard
8a08a7e431
api: do not allow invalid MCP policies (#1374)
This must be applied per virtual MCP group, not per individual target.
If set, today, these do absolutely nothing.
2026-04-01 08:17:06 -07:00
John Howard
570f9b18b3
nightly: add a usable helm tag (#1376)
We did not add a helm tag before, and if we did use latest-dev it
wouldn't work anyways

fixes https://github.com/agentgateway/agentgateway/issues/1316

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-01 08:16:25 -07:00
apexlnc
225ca63127
feat(auth): Add built-in OIDC browser auth flow (#1233)
Add built-in `oidc` browser authentication to the Rust runtime.

This PR introduces a self-contained `oidc` traffic policy for local
config. The gateway can start the authorization code flow with PKCE,
handle the callback itself, exchange the code for tokens, validate the
returned ID token, and persist an encrypted browser session cookie. That
gives users a built-in browser auth path without requiring an external
`oauth2-proxy` sidecar.

The feature is route-owned rather than globally dispatched. A route is
protected by attaching `oidc` to it, and the route that matches the
callback path is the route whose effective OIDC policy completes the
flow and owns the resulting browser session. Listener-level OIDC config
is lowered into route-phase policy attachments, but callback handling
still happens through normal route selection rather than a separate
callback registry or listener interception path.

Provider setup supports the two expected local-config modes:
- issuer-based discovery
- fully explicit `authorizationEndpoint`, `tokenEndpoint`, and `jwks`
configuration

At the config boundary, local OIDC compilation owns discovery, JWKS
loading, endpoint validation, and policy construction. The runtime OIDC
surface is kept focused on browser-flow behavior: redirects, callback
processing, token exchange, ID-token validation, and browser session
handling. The stored post-login redirect target is sanitized when
captured so callback completion only reflects local in-gateway
destinations.

This PR also:
- wires `OIDC_COOKIE_SECRET` through core config/bootstrap
- updates the generated schema and docs for the new policy
- adds a local Keycloak-backed `examples/oidc` example
- keeps `examples/oauth2-proxy` updated as the documented external
sidecar alternative

This is local-config only. Built-in logout, post-logout redirects,
refresh or access token persistence, and xDS/controller-managed OIDC
config remain out of scope for this first cut.
2026-04-01 08:05:40 -07:00
John Howard
88d3ac4e45
client: better error messages (#1416)
Before/after
```
upstream call failed: SendRequest: user body write aborted
upstream call failed: SendRequest: user body write aborted: early end, expected 5 more bytes
```
2026-03-31 14:03:33 -07:00
John Howard
d95d555d5b
llm: support serviceTier, image and audio tokens (#1414)
Fixes https://github.com/agentgateway/agentgateway/issues/1409
2026-03-31 10:50:53 -07:00
Anders Schwartz
e5eea3ebd8
fix(mcp): propagate maxBufferSize to MCP upstream HTTP requests (#1413)
Fixes #1410

The MCP upstream path (IncomingRequestContext) does not propagate the
BufferLimit extension from incoming frontend requests to outgoing
backend requests. This causes the HTTP client to fall back to the 2MB
default when reading MCP backend responses, ignoring maxBufferSize.

The LLM path was fixed in #881 and the Bedrock path in #1378. This
applies the same pattern to the MCP path by preserving BufferLimit
through IncomingRequestContext, matching how Claims is already handled.

Co-authored-by: FlowState Bot <flowstate-bot@users.noreply.github.com>
2026-03-31 08:59:57 -07:00
John Howard
a1b70fa595
Wait for bind before becoming ready (#1407) 2026-03-30 14:56:51 -07:00
kyle
db5e9598b7
fix: Vertex AI compatibility for Claude Code proxy requests (#1359)
# fix: Vertex AI compatibility for Claude Code proxy requests

## Summary

When using agentgateway as a proxy for **Claude Code** with the **Vertex
AI Anthropic provider**, requests fail due to two issues:

1. **Serde deserialization** — `#[serde(tag = "type")]` on passthrough
content block enums fails on unknown content types like `thinking`,
`tool_use`, etc.
2. **Unsupported body fields** — Vertex rejects fields like
`output_config` and `cache_control.scope` that Claude Code sends via
beta features

This PR fixes both in code.

> **Note:** To use agentgateway as a Claude Code proxy, you also need an
`AgentgatewayPolicy` to rewrite the `anthropic-beta` request header to
only include Vertex-supported values. Vertex hard-resets TCP connections
on unrecognized beta header values. See the "Beta header policy" section
below for the recommended configuration.

## Changes

### `messages.rs` — Fix content block passthrough

Switch `ContentPart` and `TextPart` from internally-tagged to untagged
serde enums:

```rust
// Before: requires "type" field on every content block
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ContentPart {
    Text { text: String, ... },
    #[serde(untagged)]
    Unknown(serde_json::Value),
}

// After: tries Text first (needs type + text fields), falls back to Unknown
#[serde(untagged)]
pub enum ContentPart {
    Text { r#type: String, text: String, ... },
    Unknown(serde_json::Value),
}
```

This allows `thinking`, `tool_use`, `tool_result`, and other content
block types to pass through without deserialization errors.

### `vertex.rs` — Strip unsupported fields and deduplicate body
preparation

Extract a shared `prepare_anthropic_body` pipeline that both
`prepare_anthropic_message_body` and
`prepare_anthropic_count_tokens_body` use. The pipeline: parse → inject
`anthropic_version` → apply caller-specific model handling → strip
unsupported fields → serialize.

Unsupported field removal:
- `output_config` and `output_format` — removed at top level only (not
recursively)
- `cache_control.scope` — removed recursively via `remove_nested_field`,
since `cache_control` can appear on content blocks at any depth

Fields that Vertex **does** support (verified empirically) are
intentionally kept:
- `metadata` (user attribution)
- `context_management` (compaction, context editing)
- `thinking` (including `"type": "adaptive"`)

### Required: Beta header policy

Vertex rejects unknown `anthropic-beta` header values with a TCP
connection reset (not a clean HTTP error). An `AgentgatewayPolicy` is
required to rewrite the header:

```yaml
traffic:
  headerModifiers:
    request:
      set:
      - name: anthropic-beta
        value: "interleaved-thinking-2025-05-14,output-128k-2025-02-19,context-management-2025-06-27,compact-2026-01-12"
```

---

## Testing Evidence

### 1. `missing field 'type'` — fixed

**Before:**
```
error="processing failed: failed to marshal request: missing field `type`"
```

**After:** All content block types pass through as `Unknown(Value)`. No
deserialization errors.

---

### 2. `output_config` rejection — fixed

**Before:**
```json
{"type":"error","error":{"type":"invalid_request_error",
  "message":"output_config: Extra inputs are not permitted"}}
```

**After:** `output_config` stripped before reaching Vertex. Returns `200
OK`.

---

### 3. `cache_control.scope` rejection — fixed

Claude Code sends `"cache_control": {"type": "ephemeral", "scope":
"global"}` via the `prompt-caching-scope` beta.

**Before:**
```json
{"type":"error","error":{"type":"invalid_request_error",
  "message":"system.2.cache_control.ephemeral.scope: Extra inputs are not permitted"}}
```

**After:** `scope` stripped recursively. Caching still works:
```
cache_creation_input_tokens: 31962
```

---

### 4. Unsupported beta headers — fixed via policy

Claude Code sends an `anthropic-beta` header with all its enabled betas.
Several of these are not supported on Vertex AI and cause a hard **TCP
connection reset** (not a clean 400 error):

| Beta header value | Vertex support |
|---|---|
| `interleaved-thinking-2025-05-14` | Supported |
| `output-128k-2025-02-19` | Supported |
| `context-management-2025-06-27` | Supported |
| `compact-2026-01-12` | Supported |
| `prompt-caching-scope-2026-01-05` | **Rejected** (connection reset) |
| `structured-outputs-2025-12-15` | **Rejected** (connection reset) |
| `effort-2025-11-24` | **Rejected** (connection reset) |
| `redact-thinking-2026-02-12` | **Rejected** (400 error) |
| `claude-code-20250219` | **Rejected** (connection reset) |

**Before:**
```bash
$ curl -s ... -H "anthropic-beta: interleaved-thinking-2025-05-14,prompt-caching-scope-2026-01-05"
upstream connect error or disconnect/reset before headers. reset reason: connection termination
```

**After:** An `AgentgatewayPolicy` with `headerModifiers.request.set`
rewrites the header to only include Vertex-supported values. The
original header from the client is replaced entirely:
```bash
# Client sends: anthropic-beta: claude-code-20250219,interleaved-thinking-2025-05-14,...,prompt-caching-scope-2026-01-05
# Policy rewrites to: interleaved-thinking-2025-05-14,output-128k-2025-02-19,context-management-2025-06-27,compact-2026-01-12
# Result: 200 OK
```

---

### 5. `context_management` and `metadata` — verified supported

```bash
curl -s -o /dev/null -w "%{http_code}" ... -d '{"context_management": {"edits": [...]}, ...}'
# 200

curl -s -o /dev/null -w "%{http_code}" ... -d '{"metadata": {"user_id": "test"}, ...}'
# 200
```

These are intentionally **not** stripped.

---

### 6. End-to-end: Claude Code via agentgateway

```bash
$ ANTHROPIC_BASE_URL=https://agentgateway.example.com/claude ANTHROPIC_AUTH_TOKEN=dummy claude
> hi
Hi! What are you working on today?
```

```
http.status=200 gen_ai.request.model=claude-haiku-4-5-20251001 duration=774ms
http.status=200 gen_ai.request.model=claude-opus-4-6 duration=1681ms
```
2026-03-30 21:54:24 +00:00
John Howard
85d5e1ade5
Prune unused deps in cargo (#1404) 2026-03-30 14:50:23 -07:00
Steven Landow
2fd260be35
fix finding listener for HBONE path (#1403)
This fixes two issues:

1. the same thing #1392 fixes - avoids ListenerNotFound when we get to
proxy_http
2. passing the synthetic listener with a bogus listener name and empty
gateway name breaks on the TCP path would break policy lookup
  * get_backend_policies gets a route path that contains a listener_name
  * internal_backend_policy gets a gateway with empty names
* as_gateway_target_ref produces a PolicyTargetRef with empty stuff, so
we miss in policies_by_target
2026-03-30 12:44:57 -07:00
209 changed files with 9707 additions and 2708 deletions

View File

@ -12,6 +12,7 @@ permissions:
jobs:
prepare:
if: github.repository == 'agentgateway/agentgateway'
runs-on: ubuntu-24.04
outputs:
has_commits: ${{ steps.check.outputs.has_commits }}
@ -50,4 +51,6 @@ jobs:
with:
version: ${{ needs.prepare.outputs.version }}
extra_tag: latest-dev
# Helm is special and requires a semver.
extra_helm_tag: 0.0.0-latest-dev
secrets: inherit

View File

@ -16,7 +16,7 @@ env:
CARGO_TERM_COLOR: always
CI: true
RUST_VERSION: "1.93.1"
GO_VERSION: "1.26.1"
GO_VERSION: "1.26.2"
jobs:
#### Proxy jobs ####
@ -24,13 +24,14 @@ jobs:
strategy:
matrix:
include:
- os: ubuntu-24.04
validate: true
- os: blacksmith-4vcpu-ubuntu-2404
name: "Proxy Test"
- os: windows-2025
validate: false
name: "Proxy Test (Windows)"
- os: macos-15
validate: false
name: "Proxy Test (Mac)"
runs-on: ${{ matrix.os }}
name: ${{ matrix.name }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Install Rust
@ -39,20 +40,25 @@ jobs:
toolchain: "${{ env.RUST_VERSION }}"
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
# Github cache is too tiny to handle the full matrix :-(
if: ${{ startsWith(matrix.os, 'ubuntu-') }}
if: ${{ startsWith(matrix.os, 'blacksmith-') }}
with:
# Github cache will restore from 'main' OR the current branch.
# If we cache PR branches, we may evict the 'main' cache and end up with no cache hits at all.
# Ideally, we would have main be 'high priority' and not be evicted but I don't think thats possible.
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Start validation dependencies
if: ${{ startsWith(matrix.os, 'blacksmith-') }}
run: tools/manage-validation-deps.sh start
- name: Test
run: make test
- name: Validate
if: ${{ matrix.validate }}
run: make validate
env:
KEYCLOAK_AVAILABLE: ${{ startsWith(matrix.os, 'blacksmith-') && '1' || '' }}
- name: Stop validation dependencies
if: ${{ always() && startsWith(matrix.os, 'blacksmith-') }}
run: tools/manage-validation-deps.sh stop
proxy-lint:
runs-on: ubuntu-24.04
runs-on: blacksmith-4vcpu-ubuntu-2404
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Install Rust
@ -88,7 +94,7 @@ jobs:
#### Controller jobs ####
controller-test:
runs-on: ubuntu-24.04
runs-on: blacksmith-4vcpu-ubuntu-2404
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6
@ -101,7 +107,7 @@ jobs:
run: go test -race ./...
controller-lint:
runs-on: ubuntu-24.04
runs-on: blacksmith-4vcpu-ubuntu-2404
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6
@ -121,7 +127,7 @@ jobs:
run: make generate-apis check-clean-repo
controller-e2e:
runs-on: ubuntu-24.04
runs-on: blacksmith-4vcpu-ubuntu-2404
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Check Runner Info
@ -173,7 +179,7 @@ jobs:
cat controller/_test/ci-step-timings.log >> $GITHUB_STEP_SUMMARY
controller-conformance:
runs-on: ubuntu-24.04
runs-on: blacksmith-4vcpu-ubuntu-2404
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Install Rust

View File

@ -13,6 +13,11 @@ on:
required: false
default: false
type: boolean
extra_helm_tag:
description: 'Additional Helm chart version to push (e.g. 0.0.0-latest-dev)'
required: false
default: ''
type: string
workflow_call:
inputs:
version:
@ -24,12 +29,17 @@ on:
required: false
default: ''
type: string
extra_helm_tag:
description: 'Additional Helm chart version to push (e.g. 0.0.0-latest-dev)'
required: false
default: ''
type: string
env:
CARGO_TERM_COLOR: always
CI: true
RUST_VERSION: "1.93.1"
GO_VERSION: "1.26.1"
GO_VERSION: "1.26.2"
REGISTRY_IMAGE: ghcr.io/${{ github.repository_owner }}/agentgateway
CONTROLLER_REGISTRY_IMAGE: ghcr.io/${{ github.repository_owner }}/controller
# Note: /charts is added to this
@ -70,9 +80,9 @@ jobs:
matrix:
include:
- platform: linux/amd64
os: ubuntu-24.04
os: blacksmith-4vcpu-ubuntu-2404
- platform: linux/arm64
os: ubuntu-24.04-arm
os: blacksmith-4vcpu-ubuntu-2404-arm
env:
VERSION: ${{ needs.setup.outputs.version }}
steps:
@ -195,9 +205,9 @@ jobs:
matrix:
include:
- platform: linux/amd64
os: ubuntu-24.04
os: blacksmith-4vcpu-ubuntu-2404
- platform: linux/arm64
os: ubuntu-24.04-arm
os: blacksmith-4vcpu-ubuntu-2404-arm
env:
VERSION: ${{ needs.setup.outputs.version }}
steps:
@ -325,7 +335,7 @@ jobs:
os: ubuntu-24.04
goarch: amd64
- platform: linux/arm64
os: ubuntu-24.04-arm
os: blacksmith-4vcpu-ubuntu-2404-arm
goarch: arm64
defaults:
run:
@ -489,6 +499,7 @@ jobs:
run: make -C controller release-charts
env:
IMAGE_REGISTRY: ${{ env.CHART_REGISTRY_IMAGE_BASE }}
EXTRA_HELM_TAG: ${{ inputs.extra_helm_tag }}
build:
if: ${{ github.event_name != 'workflow_call' && (github.event_name == 'push' || ! github.event.inputs.skip_gh_release) }}
@ -502,20 +513,24 @@ jobs:
strategy:
matrix:
include:
- os: ubuntu-24.04
- os: blacksmith-4vcpu-ubuntu-2404
target: x86_64-unknown-linux-musl
# Performance is horrendous on musl without jemalloc
features: jemalloc
- os: ubuntu-24.04-arm
name: linux
- os: blacksmith-4vcpu-ubuntu-2404-arm
target: aarch64-unknown-linux-musl
# TODO: arm64 build fails with jemalloc for some reason? https://github.com/tikv/jemallocator/issues/146
features: default
name: linux-arm
- os: macos-26
target: aarch64-apple-darwin
features: default
name: mac
- os: windows-2025
target: x86_64-pc-windows-msvc
features: default
name: windows
steps:
- name: Checkout Repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
@ -533,8 +548,8 @@ jobs:
cd ui
npm install
npm run build
- name: Install musl-tools
if: ${{ startsWith(matrix.os, 'ubuntu-') }}
- name: Prepare Linux target
if: ${{ matrix.target == 'x86_64-unknown-linux-musl' || matrix.target == 'aarch64-unknown-linux-musl' }}
run: |
sudo apt-get update
sudo apt-get install -y musl-tools
@ -549,7 +564,7 @@ jobs:
- name: Upload Artifact
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: release-binary-${{ matrix.os }}
name: release-binary-${{ matrix.name }}
path: |
target/${{ matrix.target }}/release/agentgateway
target/${{ matrix.target }}/release/agentgateway.exe
@ -563,7 +578,7 @@ jobs:
- push-controller-image
- push-helm-charts
- build
runs-on: ubuntu-24.04
runs-on: blacksmith-4vcpu-ubuntu-2404
permissions:
contents: write
env:
@ -578,13 +593,13 @@ jobs:
run: |
ls -R
mkdir outputs
mv release-binary-macos-26/agentgateway outputs/agentgateway-darwin-arm64
mv release-binary-mac/agentgateway outputs/agentgateway-darwin-arm64
sha256sum outputs/agentgateway-darwin-arm64 > outputs/agentgateway-darwin-arm64.sha256
mv release-binary-ubuntu-24.04/agentgateway outputs/agentgateway-linux-amd64
mv release-binary-linux/agentgateway outputs/agentgateway-linux-amd64
sha256sum outputs/agentgateway-linux-amd64 > outputs/agentgateway-linux-amd64.sha256
mv release-binary-ubuntu-24.04-arm/agentgateway outputs/agentgateway-linux-arm64
mv release-binary-linux-arm/agentgateway outputs/agentgateway-linux-arm64
sha256sum outputs/agentgateway-linux-arm64 > outputs/agentgateway-linux-arm64.sha256
mv release-binary-windows-2025/agentgateway.exe outputs/agentgateway-windows-amd64.exe
mv release-binary-windows/agentgateway.exe outputs/agentgateway-windows-amd64.exe
sha256sum outputs/agentgateway-windows-amd64.exe > outputs/agentgateway-windows-amd64.exe.sha256
- name: Create GitHub Release
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2

316
Cargo.lock generated
View File

@ -26,11 +26,9 @@ dependencies = [
"base64",
"cel",
"divan",
"hashbrown 0.16.1",
"hex",
"ipnet",
"md-5 0.11.0",
"once_cell",
"rand 0.10.0",
"regex",
"serde",
@ -59,8 +57,6 @@ dependencies = [
"itertools 0.14.0",
"itoa",
"once_cell",
"opentelemetry",
"opentelemetry-http",
"opentelemetry-otlp",
"opentelemetry_sdk",
"pin-project-lite",
@ -72,7 +68,6 @@ dependencies = [
"time",
"tokio",
"tracing",
"tracing-appender",
"tracing-core",
"tracing-log",
"tracing-subscriber",
@ -113,14 +108,12 @@ dependencies = [
"http 1.4.0",
"prometheus-client",
"prost",
"prost-types",
"prost-wkt-types",
"protos",
"split-iter",
"thiserror 2.0.18",
"tokio",
"tonic",
"tonic-prost",
"tower",
"tracing",
]
@ -182,7 +175,7 @@ dependencies = [
"hyper-util",
"hyper-util-fork",
"include_dir",
"indexmap 2.13.0",
"indexmap 2.13.1",
"insta",
"ipnet",
"itertools 0.14.0",
@ -248,7 +241,6 @@ dependencies = [
"tokio-util",
"tokio_sse_codec",
"tonic",
"tonic-prost",
"tower",
"tower-http",
"tower-serve-static",
@ -273,7 +265,6 @@ dependencies = [
"anyhow",
"clap",
"fs-err",
"lazy_static",
"libc",
"tikv-jemallocator",
"tokio",
@ -431,9 +422,9 @@ dependencies = [
[[package]]
name = "arc-swap"
version = "1.9.0"
version = "1.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a07d1f37ff60921c83bdfc7407723bdefe89b44b98a9b772f225c8f9d67141a6"
checksum = "6a3a1fd6f75306b68087b831f025c712524bcb19aad54e557b1129cfa0a2b207"
dependencies = [
"rustversion",
]
@ -527,8 +518,9 @@ dependencies = [
[[package]]
name = "async-openai"
version = "0.33.0"
source = "git+https://github.com/64bit/async-openai?rev=6e482db7185e3e186fe49bae0ec2f214fc822e4a#6e482db7185e3e186fe49bae0ec2f214fc822e4a"
version = "0.34.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec08254d61379df136135d3d1ac04301be7699fd7d9e57655c63ac7d650a6922"
dependencies = [
"bytes",
"derive_builder",
@ -650,9 +642,9 @@ dependencies = [
[[package]]
name = "aws-lc-sys"
version = "0.39.0"
version = "0.39.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fa7e52a4c5c547c741610a2c6f123f3881e409b714cd27e6798ef020c514f0a"
checksum = "83a25cf98105baa966497416dbd42565ce3a8cf8dbfd59803ec9ad46f3126399"
dependencies = [
"cc",
"cmake",
@ -1295,9 +1287,9 @@ checksum = "37b2a672a2cb129a2e41c10b1224bb368f9f37a2b16b612598138befd7b37eb5"
[[package]]
name = "cc"
version = "1.2.57"
version = "1.2.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a0dd1ca384932ff3641c8718a02769f1698e7563dc6974ffd03346116310423"
checksum = "b7a4d3ec6524d28a329fc53654bbadc9bdd7b0431f5d65f1a56ffb28a1ee5283"
dependencies = [
"find-msvc-tools",
"jobserver",
@ -1487,9 +1479,9 @@ dependencies = [
[[package]]
name = "cmake"
version = "0.1.57"
version = "0.1.58"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75443c44cd6b379beb8c5b45d85d0773baf31cce901fe7bb252f4eff3008ef7d"
checksum = "c0f78a02292a74a88ac736019ab962ece0bc380e3f977bf72e376c5d78ff0678"
dependencies = [
"cc",
]
@ -2271,9 +2263,9 @@ dependencies = [
[[package]]
name = "fastrand"
version = "2.3.0"
version = "2.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be"
checksum = "9f1f227452a390804cdb637b74a86990f2a7d7ba4b7d5693aac9b4dd6defd8d6"
[[package]]
name = "file-id"
@ -2680,7 +2672,7 @@ dependencies = [
"futures-core",
"futures-sink",
"http 1.4.0",
"indexmap 2.13.0",
"indexmap 2.13.1",
"slab",
"tokio",
"tokio-util",
@ -2954,18 +2946,18 @@ checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
[[package]]
name = "hybrid-array"
version = "0.4.8"
version = "0.4.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8655f91cd07f2b9d0c24137bd650fe69617773435ee5ec83022377777ce65ef1"
checksum = "3944cf8cf766b40e2a1a333ee5e9b563f854d5fa49d6a8ca2764e97c6eddb214"
dependencies = [
"typenum",
]
[[package]]
name = "hyper"
version = "1.8.1"
version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2ab2d4f250c3d7b1c9fcdff1cece94ea4e2dfbec68614f7b87cb205f24ca9d11"
checksum = "6299f016b246a94207e63da54dbe807655bf9e00044f73ded42c3ac5305fbcca"
dependencies = [
"atomic-waker",
"bytes",
@ -2978,7 +2970,6 @@ dependencies = [
"httpdate",
"itoa",
"pin-project-lite",
"pin-utils",
"smallvec",
"tokio",
"want",
@ -3083,12 +3074,13 @@ dependencies = [
[[package]]
name = "icu_collections"
version = "2.1.1"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4c6b649701667bbe825c3b7e6388cb521c23d88644678e83c0c4d0a621a34b43"
checksum = "2984d1cd16c883d7935b9e07e44071dca8d917fd52ecc02c04d5fa0b5a3f191c"
dependencies = [
"displaydoc",
"potential_utf",
"utf8_iter",
"yoke",
"zerofrom",
"zerovec",
@ -3096,9 +3088,9 @@ dependencies = [
[[package]]
name = "icu_locale_core"
version = "2.1.1"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "edba7861004dd3714265b4db54a3c390e880ab658fec5f7db895fae2046b5bb6"
checksum = "92219b62b3e2b4d88ac5119f8904c10f8f61bf7e95b640d25ba3075e6cac2c29"
dependencies = [
"displaydoc",
"litemap",
@ -3109,9 +3101,9 @@ dependencies = [
[[package]]
name = "icu_normalizer"
version = "2.1.1"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5f6c8828b67bf8908d82127b2054ea1b4427ff0230ee9141c54251934ab1b599"
checksum = "c56e5ee99d6e3d33bd91c5d85458b6005a22140021cc324cea84dd0e72cff3b4"
dependencies = [
"icu_collections",
"icu_normalizer_data",
@ -3123,15 +3115,15 @@ dependencies = [
[[package]]
name = "icu_normalizer_data"
version = "2.1.1"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7aedcccd01fc5fe81e6b489c15b247b8b0690feb23304303a9e560f37efc560a"
checksum = "da3be0ae77ea334f4da67c12f149704f19f81d1adf7c51cf482943e84a2bad38"
[[package]]
name = "icu_properties"
version = "2.1.2"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "020bfc02fe870ec3a66d93e677ccca0562506e5872c650f893269e08615d74ec"
checksum = "bee3b67d0ea5c2cca5003417989af8996f8604e34fb9ddf96208a033901e70de"
dependencies = [
"icu_collections",
"icu_locale_core",
@ -3143,15 +3135,15 @@ dependencies = [
[[package]]
name = "icu_properties_data"
version = "2.1.2"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "616c294cf8d725c6afcd8f55abc17c56464ef6211f9ed59cccffe534129c77af"
checksum = "8e2bbb201e0c04f7b4b3e14382af113e17ba4f63e2c9d2ee626b720cbce54a14"
[[package]]
name = "icu_provider"
version = "2.1.1"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85962cf0ce02e1e0a629cc34e7ca3e373ce20dda4c4d7294bbd0bf1fdb59e614"
checksum = "139c4cf31c8b5f33d7e199446eff9c1e02decfc2f0eec2c8d71f65befa45b421"
dependencies = [
"displaydoc",
"icu_locale_core",
@ -3227,9 +3219,9 @@ dependencies = [
[[package]]
name = "indexmap"
version = "2.13.0"
version = "2.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7714e70437a7dc3ac8eb7e6f8df75fd8eb422675fc7678aff7364301092b1017"
checksum = "45a8a2b9cb3e0b0c1803dbb0758ffac5de2f425b23c28f518faabd9d805342ff"
dependencies = [
"equivalent",
"hashbrown 0.16.1",
@ -3268,9 +3260,9 @@ dependencies = [
[[package]]
name = "insta"
version = "1.47.1"
version = "1.47.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "99322078b2c076829a1db959d49da554fabc4342257fc0ba5a070a1eb3a01cd8"
checksum = "7b4a6248eb93a4401ed2f37dfe8ea592d3cf05b7cf4f8efa867b6895af7e094e"
dependencies = [
"console",
"once_cell",
@ -3284,9 +3276,9 @@ dependencies = [
[[package]]
name = "inventory"
version = "0.3.22"
version = "0.3.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "009ae045c87e7082cb72dab0ccd01ae075dd00141ddc108f43a0ea150a9e7227"
checksum = "a4f0c30c76f2f4ccee3fe55a2435f691ca00c0e4bd87abe4f4a851b1d4dac39b"
dependencies = [
"rustversion",
]
@ -3315,9 +3307,9 @@ dependencies = [
[[package]]
name = "iri-string"
version = "0.7.11"
version = "0.7.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d8e7418f59cc01c88316161279a7f665217ae316b388e58a0d10e29f54f1e5eb"
checksum = "25e659a4bb38e810ebc252e53b5814ff908a8c58c2a9ce2fae1bbec24cbf4e20"
dependencies = [
"memchr",
"serde",
@ -3437,10 +3429,12 @@ dependencies = [
[[package]]
name = "js-sys"
version = "0.3.91"
version = "0.3.94"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b49715b7073f385ba4bc528e5747d02e66cb39c6146efb66b781f131f0fb399c"
checksum = "2e04e2ef80ce82e13552136fabeef8a5ed1f985a96805761cbb9a2c34e7664d9"
dependencies = [
"cfg-if",
"futures-util",
"once_cell",
"wasm-bindgen",
]
@ -3496,9 +3490,9 @@ checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
[[package]]
name = "libc"
version = "0.2.183"
version = "0.2.184"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b5b646652bf6661599e1da8901b3b9522896f01e736bad5f723fe7a3a27f899d"
checksum = "48f5d2a454e16a5ea0f4ced81bd44e4cfc7bd3a507b61887c99fd3538b28e4af"
[[package]]
name = "libredox"
@ -3529,9 +3523,9 @@ checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
[[package]]
name = "litemap"
version = "0.8.1"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6373607a59f0be73a39b6fe456b8192fcc3585f602af20751600e974dd455e77"
checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0"
[[package]]
name = "lock_api"
@ -3788,9 +3782,9 @@ checksum = "c505b3e17ed6b70a7ed2e67fbb2c560ee327353556120d6e72f5232b6880d536"
[[package]]
name = "mio"
version = "1.1.1"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a69bcab0ad47271a0234d9422b131806bf3968021e5dc9328caf2d4cd58557fc"
checksum = "50b7e5b27aa02a74bac8c3f23f448f8d87ff11f92d3aac1a6ed369ee08cc56c1"
dependencies = [
"libc",
"log",
@ -4090,7 +4084,7 @@ version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c8d427828b22ae1fff2833a03d8486c2c881367f1c336349f307f321e7f4d05"
dependencies = [
"indexmap 2.13.0",
"indexmap 2.13.1",
"serde",
"serde_json",
]
@ -4299,7 +4293,7 @@ checksum = "8701b58ea97060d5e5b155d383a69952a60943f0e6dfe30b04c287beb0b27455"
dependencies = [
"fixedbitset",
"hashbrown 0.15.5",
"indexmap 2.13.0",
"indexmap 2.13.1",
]
[[package]]
@ -4437,9 +4431,9 @@ dependencies = [
[[package]]
name = "potential_utf"
version = "0.1.4"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b73949432f5e2a09657003c25bca5e19a0e9c84f8058ca374f49e0ebe605af77"
checksum = "0103b1cef7ec0cf76490e969665504990193874ea05c85ff9bab8b911d0a0564"
dependencies = [
"zerovec",
]
@ -4562,7 +4556,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2e842efad9119158434d193c6682e2ebee4b44d6ad801d7b349623b3f57cdf55"
dependencies = [
"futures",
"indexmap 2.13.0",
"indexmap 2.13.1",
"nix 0.31.2",
"tokio",
"tracing",
@ -4571,9 +4565,9 @@ dependencies = [
[[package]]
name = "prometheus-client"
version = "0.24.0"
version = "0.24.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e4500adecd7af8e0e9f4dbce15cfee07ce913fbf6ad605cc468b83f2d531ee94"
checksum = "cca3d75b4566b9a29fe1ed623587fb058e826eb329a0be4b7c4da1ebb2d7a6ca"
dependencies = [
"dtoa",
"itoa",
@ -4752,7 +4746,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4aeaa1f2460f1d348eeaeed86aea999ce98c1bded6f089ff8514c9d9dbdc973"
dependencies = [
"anyhow",
"indexmap 2.13.0",
"indexmap 2.13.1",
"log",
"protobuf",
"protobuf-support",
@ -4878,7 +4872,7 @@ dependencies = [
"pin-project-lite",
"quinn-proto",
"quinn-udp",
"rustc-hash 2.1.1",
"rustc-hash 2.1.2",
"rustls",
"socket2",
"thiserror 2.0.18",
@ -4899,7 +4893,7 @@ dependencies = [
"lru-slab",
"rand 0.9.2",
"ring",
"rustc-hash 2.1.1",
"rustc-hash 2.1.2",
"rustls",
"rustls-pki-types",
"slab",
@ -5430,9 +5424,9 @@ checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
[[package]]
name = "rustc-hash"
version = "2.1.1"
version = "2.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "357703d41365b4b27c590e3ed91eabb1b663f07c4c084095e60cbed4362dff0d"
checksum = "94300abf3f1ae2e2b8ffb7b58043de3d399c73fa6f4b73826402a5c457614dbe"
[[package]]
name = "rustc_version"
@ -5679,9 +5673,9 @@ checksum = "689224d06523904ebcc9b482c6a3f4f7fb396096645c4cd10c0d2ff7371a34d3"
[[package]]
name = "semver"
version = "1.0.27"
version = "1.0.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d767eb0aabc880b29956c35734170f26ed551a859dbd361d140cdbeca61ab1e2"
checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd"
[[package]]
name = "serde"
@ -5770,7 +5764,7 @@ version = "1.0.149"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
dependencies = [
"indexmap 2.13.0",
"indexmap 2.13.1",
"itoa",
"memchr",
"serde",
@ -5812,9 +5806,9 @@ dependencies = [
[[package]]
name = "serde_spanned"
version = "1.1.0"
version = "1.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "876ac351060d4f882bb1032b6369eb0aef79ad9df1ea8bc404874d8cc3d0cd98"
checksum = "6662b5879511e06e8999a8a235d848113e942c9124f211511b16466ee2995f26"
dependencies = [
"serde_core",
]
@ -5841,7 +5835,7 @@ dependencies = [
"chrono",
"hex",
"indexmap 1.9.3",
"indexmap 2.13.0",
"indexmap 2.13.1",
"schemars 0.9.0",
"schemars 1.0.4",
"serde_core",
@ -5868,7 +5862,7 @@ version = "0.9.34+deprecated"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a8b1a1a2ebf674015cc02edccce75287f1a0130d394307b36743c2f5d504b47"
dependencies = [
"indexmap 2.13.0",
"indexmap 2.13.1",
"itoa",
"ryu",
"serde",
@ -5990,9 +5984,9 @@ dependencies = [
[[package]]
name = "simd-adler32"
version = "0.3.8"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e320a6c5ad31d271ad523dcf3ad13e2767ad8b1cb8f047f75a8aeaf8da139da2"
checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214"
[[package]]
name = "similar"
@ -6125,15 +6119,15 @@ checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "sval"
version = "2.17.0"
version = "2.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c1aaf178a50bbdd86043fce9bf0a5867007d9b382db89d1c96ccae4601ff1ff9"
checksum = "2eb9318255ebd817902d7e279d8f8e39b35b1b9954decd5eb9ea0e30e5fd2b6a"
[[package]]
name = "sval_buffer"
version = "2.17.0"
version = "2.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f89273e48f03807ebf51c4d81c52f28d35ffa18a593edf97e041b52de143df89"
checksum = "12571299185e653fdb0fbfe36cd7f6529d39d4e747a60b15a3f34574b7b97c61"
dependencies = [
"sval",
"sval_ref",
@ -6141,18 +6135,18 @@ dependencies = [
[[package]]
name = "sval_dynamic"
version = "2.17.0"
version = "2.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0430f4e18e7eba21a49d10d25a8dec3ce0e044af40b162347e99a8e3c3ced864"
checksum = "39526f24e997706c0de7f03fb7371f7f5638b66a504ded508e20ad173d0a3677"
dependencies = [
"sval",
]
[[package]]
name = "sval_fmt"
version = "2.17.0"
version = "2.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "835f51b9d7331b9d7fc48fc716c02306fa88c4a076b1573531910c91a525882d"
checksum = "933dd3bb26965d682280fcc49400ac2a05036f4ee1e6dbd61bf8402d5a5c3a54"
dependencies = [
"itoa",
"ryu",
@ -6161,9 +6155,9 @@ dependencies = [
[[package]]
name = "sval_json"
version = "2.17.0"
version = "2.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13cbfe3ef406ee2366e7e8ab3678426362085fa9eaedf28cb878a967159dced3"
checksum = "a0cda08f6d5c9948024a6551077557b1fdcc3880ff2f20ae839667d2ec2d87ed"
dependencies = [
"itoa",
"ryu",
@ -6172,9 +6166,9 @@ dependencies = [
[[package]]
name = "sval_nested"
version = "2.17.0"
version = "2.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b20358af4af787c34321a86618c3cae12eabdd0e9df22cd9dd2c6834214c518"
checksum = "88d49d5e6c1f9fd0e53515819b03a97ca4eb1bff5c8ee097c43391c09ecfb19f"
dependencies = [
"sval",
"sval_buffer",
@ -6183,18 +6177,18 @@ dependencies = [
[[package]]
name = "sval_ref"
version = "2.17.0"
version = "2.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb5e500f8eb2efa84f75e7090f7fc43f621b9f8b6cde571c635b3855f97b332a"
checksum = "14f876c5a78405375b4e19cbb9554407513b59c93dea12dc6a4af4e1d30899ca"
dependencies = [
"sval",
]
[[package]]
name = "sval_serde"
version = "2.17.0"
version = "2.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca2032ae39b11dcc6c18d5fbc50a661ea191cac96484c59ccf49b002261ca2c1"
checksum = "5f9ccd3b7f7200239a655e517dd3fd48d960b9111ad24bd6a5e055bef17607c7"
dependencies = [
"serde_core",
"sval",
@ -6448,9 +6442,9 @@ dependencies = [
[[package]]
name = "tinystr"
version = "0.8.2"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42d3e9c45c09de15d06dd8acf5f4e0e399e85927b7f00711024eb7ae10fa4869"
checksum = "c8323304221c2a851516f22236c5722a72eaa19749016521d6dff0824447d96d"
dependencies = [
"displaydoc",
"zerovec",
@ -6483,9 +6477,9 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
[[package]]
name = "tokio"
version = "1.50.0"
version = "1.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "27ad5e34374e03cfffefc301becb44e9dc3c17584f414349ebe29ed26661822d"
checksum = "2bd1c4c0fc4a7ab90fc15ef6daaa3ec3b893f004f915f2392557ed23237820cd"
dependencies = [
"backtrace",
"bytes",
@ -6501,9 +6495,9 @@ dependencies = [
[[package]]
name = "tokio-macros"
version = "2.6.1"
version = "2.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c55a2eff8b69ce66c84f85e1da1c233edc36ceb85a2058d11b0d6a3c7e7569c"
checksum = "385a6cb71ab9ab790c5fe8d67f1645e6c450a7ce006a33de03daa956cf70a496"
dependencies = [
"proc-macro2",
"quote",
@ -6560,11 +6554,11 @@ dependencies = [
[[package]]
name = "toml"
version = "1.1.0+spec-1.1.0"
version = "1.1.2+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8195ca05e4eb728f4ba94f3e3291661320af739c4e43779cbdfae82ab239fcc"
checksum = "81f3d15e84cbcd896376e6730314d59fb5a87f31e4b038454184435cd57defee"
dependencies = [
"indexmap 2.13.0",
"indexmap 2.13.1",
"serde_core",
"serde_spanned",
"toml_datetime",
@ -6575,20 +6569,20 @@ dependencies = [
[[package]]
name = "toml_datetime"
version = "1.1.0+spec-1.1.0"
version = "1.1.1+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97251a7c317e03ad83774a8752a7e81fb6067740609f75ea2b585b569a59198f"
checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7"
dependencies = [
"serde_core",
]
[[package]]
name = "toml_edit"
version = "0.25.8+spec-1.1.0"
version = "0.25.10+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "16bff38f1d86c47f9ff0647e6838d7bb362522bdf44006c7068c2b1e606f1f3c"
checksum = "a82418ca169e235e6c399a84e395ab6debeb3bc90edc959bf0f48647c6a32d1b"
dependencies = [
"indexmap 2.13.0",
"indexmap 2.13.1",
"toml_datetime",
"toml_parser",
"winnow",
@ -6596,18 +6590,18 @@ dependencies = [
[[package]]
name = "toml_parser"
version = "1.1.0+spec-1.1.0"
version = "1.1.2+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2334f11ee363607eb04df9b8fc8a13ca1715a72ba8662a26ac285c98aabb4011"
checksum = "a2abe9b86193656635d2411dc43050282ca48aa31c2451210f4202550afb7526"
dependencies = [
"winnow",
]
[[package]]
name = "toml_writer"
version = "1.1.0+spec-1.1.0"
version = "1.1.1+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d282ade6016312faf3e41e57ebbba0c073e4056dab1232ab1cb624199648f8ed"
checksum = "756daf9b1013ebe47a8776667b466417e2d4c5679d441c26230efd9ef78692db"
[[package]]
name = "tonic"
@ -6685,7 +6679,7 @@ checksum = "ebe5ef63511595f1344e2d5cfa636d973292adc0eec1f0ad45fae9f0851ab1d4"
dependencies = [
"futures-core",
"futures-util",
"indexmap 2.13.0",
"indexmap 2.13.1",
"pin-project-lite",
"slab",
"sync_wrapper",
@ -6759,18 +6753,6 @@ dependencies = [
"tracing-core",
]
[[package]]
name = "tracing-appender"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "786d480bce6247ab75f005b14ae1624ad978d3029d9113f0a22fa1ac773faeaf"
dependencies = [
"crossbeam-channel",
"thiserror 2.0.18",
"time",
"tracing-subscriber",
]
[[package]]
name = "tracing-attributes"
version = "0.1.31"
@ -7144,9 +7126,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen"
version = "0.2.114"
version = "0.2.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6532f9a5c1ece3798cb1c2cfdba640b9b3ba884f5db45973a6f442510a87d38e"
checksum = "0551fc1bb415591e3372d0bc4780db7e587d84e2a7e79da121051c5c4b89d0b0"
dependencies = [
"cfg-if",
"once_cell",
@ -7157,23 +7139,19 @@ dependencies = [
[[package]]
name = "wasm-bindgen-futures"
version = "0.4.64"
version = "0.4.67"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e9c5522b3a28661442748e09d40924dfb9ca614b21c00d3fd135720e48b67db8"
checksum = "03623de6905b7206edd0a75f69f747f134b7f0a2323392d664448bf2d3c5d87e"
dependencies = [
"cfg-if",
"futures-util",
"js-sys",
"once_cell",
"wasm-bindgen",
"web-sys",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.114"
version = "0.2.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "18a2d50fcf105fb33bb15f00e7a77b772945a2ee45dcf454961fd843e74c18e6"
checksum = "7fbdf9a35adf44786aecd5ff89b4563a90325f9da0923236f6104e603c7e86be"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
@ -7181,9 +7159,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.114"
version = "0.2.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03ce4caeaac547cdf713d280eda22a730824dd11e6b8c3ca9e42247b25c631e3"
checksum = "dca9693ef2bab6d4e6707234500350d8dad079eb508dca05530c85dc3a529ff2"
dependencies = [
"bumpalo",
"proc-macro2",
@ -7194,9 +7172,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.114"
version = "0.2.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75a326b8c223ee17883a4251907455a2431acc2791c98c26279376490c378c16"
checksum = "39129a682a6d2d841b6c429d0c51e5cb0ed1a03829d8b3d1e69a011e62cb3d3b"
dependencies = [
"unicode-ident",
]
@ -7218,7 +7196,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb0e353e6a2fbdc176932bbaab493762eb1255a7900fe0fea1a2f96c296cc909"
dependencies = [
"anyhow",
"indexmap 2.13.0",
"indexmap 2.13.1",
"wasm-encoder",
"wasmparser",
]
@ -7257,15 +7235,15 @@ checksum = "47b807c72e1bac69382b3a6fb3dbe8ea4c0ed87ff5629b8685ae6b9a611028fe"
dependencies = [
"bitflags 2.11.0",
"hashbrown 0.15.5",
"indexmap 2.13.0",
"indexmap 2.13.1",
"semver",
]
[[package]]
name = "web-sys"
version = "0.3.91"
version = "0.3.94"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "854ba17bb104abfb26ba36da9729addc7ce7f06f5c0f90f3c391f8461cca21f9"
checksum = "cd70027e39b12f0849461e08ffc50b9cd7688d942c1c8e3c7b22273236b4dd0a"
dependencies = [
"js-sys",
"wasm-bindgen",
@ -7721,9 +7699,9 @@ checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650"
[[package]]
name = "winnow"
version = "1.0.0"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a90e88e4667264a994d34e6d1ab2d26d398dcdca8b7f52bec8668957517fc7d8"
checksum = "09dac053f1cd375980747450bfc7250c264eaae0583872e845c0c7cd578872b5"
dependencies = [
"memchr",
]
@ -7781,7 +7759,7 @@ checksum = "b7c566e0f4b284dd6561c786d9cb0142da491f46a9fbed79ea69cdad5db17f21"
dependencies = [
"anyhow",
"heck",
"indexmap 2.13.0",
"indexmap 2.13.1",
"prettyplease",
"syn",
"wasm-metadata",
@ -7812,7 +7790,7 @@ checksum = "9d66ea20e9553b30172b5e831994e35fbde2d165325bec84fc43dbf6f4eb9cb2"
dependencies = [
"anyhow",
"bitflags 2.11.0",
"indexmap 2.13.0",
"indexmap 2.13.1",
"log",
"serde",
"serde_derive",
@ -7831,7 +7809,7 @@ checksum = "ecc8ac4bc1dc3381b7f59c34f00b67e18f910c2c0f50015669dde7def656a736"
dependencies = [
"anyhow",
"id-arena",
"indexmap 2.13.0",
"indexmap 2.13.1",
"log",
"semver",
"serde",
@ -7843,9 +7821,9 @@ dependencies = [
[[package]]
name = "writeable"
version = "0.6.2"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9edde0db4769d2dc68579893f2306b26c6ecfbe0ef499b013d731b7b9247e0b9"
checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4"
[[package]]
name = "x509-parser"
@ -7893,9 +7871,9 @@ dependencies = [
[[package]]
name = "yoke"
version = "0.8.1"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72d6e5c6afb84d73944e5cedb052c4680d5657337201555f9f2a16b7406d4954"
checksum = "abe8c5fda708d9ca3df187cae8bfb9ceda00dd96231bed36e445a1a48e66f9ca"
dependencies = [
"stable_deref_trait",
"yoke-derive",
@ -7904,9 +7882,9 @@ dependencies = [
[[package]]
name = "yoke-derive"
version = "0.8.1"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b659052874eb698efe5b9e8cf382204678a0086ebf46982b79d6ca3182927e5d"
checksum = "de844c262c8848816172cef550288e7dc6c7b7814b4ee56b3e1553f275f1858e"
dependencies = [
"proc-macro2",
"quote",
@ -7916,18 +7894,18 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.47"
version = "0.8.48"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "efbb2a062be311f2ba113ce66f697a4dc589f85e78a4aea276200804cea0ed87"
checksum = "eed437bf9d6692032087e337407a86f04cd8d6a16a37199ed57949d415bd68e9"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.47"
version = "0.8.48"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0e8bc7269b54418e7aeeef514aa68f8690b8c0489a06b0136e5f57c4c5ccab89"
checksum = "70e3cd084b1788766f53af483dd21f93881ff30d7320490ec3ef7526d203bad4"
dependencies = [
"proc-macro2",
"quote",
@ -7936,18 +7914,18 @@ dependencies = [
[[package]]
name = "zerofrom"
version = "0.1.6"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "50cc42e0333e05660c3587f3bf9d0478688e15d870fab3346451ce7f8c9fbea5"
checksum = "69faa1f2a1ea75661980b013019ed6687ed0e83d069bc1114e2cc74c6c04c4df"
dependencies = [
"zerofrom-derive",
]
[[package]]
name = "zerofrom-derive"
version = "0.1.6"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d71e5d6e06ab090c67b5e44993ec16b72dcbaabc526db883a360057678b48502"
checksum = "11532158c46691caf0f2593ea8358fed6bbf68a0315e80aae9bd41fbade684a1"
dependencies = [
"proc-macro2",
"quote",
@ -7963,9 +7941,9 @@ checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0"
[[package]]
name = "zerotrie"
version = "0.2.3"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a59c17a5562d507e4b54960e8569ebee33bee890c70aa3fe7b97e85a9fd7851"
checksum = "0f9152d31db0792fa83f70fb2f83148effb5c1f5b8c7686c3459e361d9bc20bf"
dependencies = [
"displaydoc",
"yoke",
@ -7974,9 +7952,9 @@ dependencies = [
[[package]]
name = "zerovec"
version = "0.11.5"
version = "0.11.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c28719294829477f525be0186d13efa9a3c602f7ec202ca9e353d310fb9a002"
checksum = "90f911cbc359ab6af17377d242225f4d75119aec87ea711a880987b18cd7b239"
dependencies = [
"yoke",
"zerofrom",
@ -7985,9 +7963,9 @@ dependencies = [
[[package]]
name = "zerovec-derive"
version = "0.11.2"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eadce39539ca5cb3985590102671f2567e659fca9666581ad3411d59207951f3"
checksum = "625dc425cab0dca6dc3c3319506e6593dcb08a9f387ea3b284dbd52a92c40555"
dependencies = [
"proc-macro2",
"quote",

View File

@ -50,7 +50,7 @@ anyhow = "1.0"
arc-swap = "1.9"
arcstr = { version = "1.2", features = ["serde"] }
assert_matches = "1.5.0"
async-openai = { git = "https://github.com/64bit/async-openai", rev = "6e482db7185e3e186fe49bae0ec2f214fc822e4a", default-features = false, features = ["chat-completion-types", "moderation-types", "response-types", "realtime-types"] }
async-openai = { version = "0.34", default-features = false, features = ["chat-completion-types", "moderation-types", "response-types", "realtime-types"] }
async-stream = "0.3"
async-trait = "0.1"
aws-config = { version = "1.8", default-features = false, features = ["default-https-client", "rt-tokio"] }

View File

@ -80,6 +80,23 @@ Agentgateway has a built-in UI for you to explore agentgateway connecting agent-
<img alt="agentgateway UI" src="img/UI-homepage.png">
</div>
## Sponsors
<table>
<tr>
<td align="center" width="33.3%">
<a href="https://blacksmith.sh/">
<picture>
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/agentgateway/agentgateway/main/img/blacksmith-light.svg">
<img src="https://raw.githubusercontent.com/agentgateway/agentgateway/main/img/blacksmith.svg" alt="Blacksmith" height="48">
</picture>
</a>
</td>
</tr>
</table>
Thank you to our sponsors for helping to fund the development of agentgateway!
## Contributing
For instructions on how to contribute to the agentgateway project, see the [CONTRIBUTION.md](CONTRIBUTION.md) file.

View File

@ -10461,13 +10461,14 @@ func (x *BackendPolicySpec_Ai_PromptGuard) GetResponse() []*BackendPolicySpec_Ai
// Default JSON key-value pairs to add to the LLM request if the key is not set in the request.
type BackendPolicySpec_Ai_PromptCaching struct {
state protoimpl.MessageState `protogen:"open.v1"`
CacheSystem bool `protobuf:"varint,1,opt,name=cache_system,json=cacheSystem,proto3" json:"cache_system,omitempty"`
CacheMessages bool `protobuf:"varint,2,opt,name=cache_messages,json=cacheMessages,proto3" json:"cache_messages,omitempty"`
CacheTools bool `protobuf:"varint,3,opt,name=cache_tools,json=cacheTools,proto3" json:"cache_tools,omitempty"`
MinTokens *uint32 `protobuf:"varint,4,opt,name=min_tokens,json=minTokens,proto3,oneof" json:"min_tokens,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
state protoimpl.MessageState `protogen:"open.v1"`
CacheSystem bool `protobuf:"varint,1,opt,name=cache_system,json=cacheSystem,proto3" json:"cache_system,omitempty"`
CacheMessages bool `protobuf:"varint,2,opt,name=cache_messages,json=cacheMessages,proto3" json:"cache_messages,omitempty"`
CacheTools bool `protobuf:"varint,3,opt,name=cache_tools,json=cacheTools,proto3" json:"cache_tools,omitempty"`
MinTokens *uint32 `protobuf:"varint,4,opt,name=min_tokens,json=minTokens,proto3,oneof" json:"min_tokens,omitempty"`
CacheMessageOffset *uint32 `protobuf:"varint,5,opt,name=cache_message_offset,json=cacheMessageOffset,proto3,oneof" json:"cache_message_offset,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *BackendPolicySpec_Ai_PromptCaching) Reset() {
@ -10528,6 +10529,13 @@ func (x *BackendPolicySpec_Ai_PromptCaching) GetMinTokens() uint32 {
return 0
}
func (x *BackendPolicySpec_Ai_PromptCaching) GetCacheMessageOffset() uint32 {
if x != nil && x.CacheMessageOffset != nil {
return *x.CacheMessageOffset
}
return 0
}
type BackendPolicySpec_McpAuthentication_ResourceMetadata struct {
state protoimpl.MessageState `protogen:"open.v1"`
Extra map[string]*structpb.Value `protobuf:"bytes,1,rep,name=extra,proto3" json:"extra,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
@ -11793,7 +11801,7 @@ const file_resource_proto_rawDesc = "" +
"\vPolicyPhase\x12\t\n" +
"\x05ROUTE\x10\x00\x12\v\n" +
"\aGATEWAY\x10\x01B\x06\n" +
"\x04kind\"\xb6B\n" +
"\x04kind\"\x86C\n" +
"\x11BackendPolicySpec\x12D\n" +
"\x03a2a\x18\x01 \x01(\v20.agentgateway.dev.resource.BackendPolicySpec.A2aH\x00R\x03a2a\x12l\n" +
"\x11inference_routing\x18\x02 \x01(\v2=.agentgateway.dev.resource.BackendPolicySpec.InferenceRoutingH\x00R\x10inferenceRouting\x12Z\n" +
@ -11813,7 +11821,7 @@ const file_resource_proto_rawDesc = "" +
"backendTcp\x12k\n" +
"\x0etransformation\x18\x0e \x01(\v2A.agentgateway.dev.resource.TrafficPolicySpec.TransformationPolicyH\x00R\x0etransformation\x12M\n" +
"\x06health\x18\x0f \x01(\v23.agentgateway.dev.resource.BackendPolicySpec.HealthH\x00R\x06health\x12c\n" +
"\x0ebackend_tunnel\x18\x10 \x01(\v2:.agentgateway.dev.resource.BackendPolicySpec.BackendTunnelH\x00R\rbackendTunnel\x1a\x93\"\n" +
"\x0ebackend_tunnel\x18\x10 \x01(\v2:.agentgateway.dev.resource.BackendPolicySpec.BackendTunnelH\x00R\rbackendTunnel\x1a\xe3\"\n" +
"\x02Ai\x12^\n" +
"\fprompt_guard\x18\x01 \x01(\v2;.agentgateway.dev.resource.BackendPolicySpec.Ai.PromptGuardR\vpromptGuard\x12Y\n" +
"\bdefaults\x18\x02 \x03(\v2=.agentgateway.dev.resource.BackendPolicySpec.Ai.DefaultsEntryR\bdefaults\x12\\\n" +
@ -11880,15 +11888,17 @@ const file_resource_proto_rawDesc = "" +
"\x04kind\x1a\xc0\x01\n" +
"\vPromptGuard\x12V\n" +
"\arequest\x18\x01 \x03(\v2<.agentgateway.dev.resource.BackendPolicySpec.Ai.RequestGuardR\arequest\x12Y\n" +
"\bresponse\x18\x02 \x03(\v2=.agentgateway.dev.resource.BackendPolicySpec.Ai.ResponseGuardR\bresponse\x1a\xad\x01\n" +
"\bresponse\x18\x02 \x03(\v2=.agentgateway.dev.resource.BackendPolicySpec.Ai.ResponseGuardR\bresponse\x1a\xfd\x01\n" +
"\rPromptCaching\x12!\n" +
"\fcache_system\x18\x01 \x01(\bR\vcacheSystem\x12%\n" +
"\x0ecache_messages\x18\x02 \x01(\bR\rcacheMessages\x12\x1f\n" +
"\vcache_tools\x18\x03 \x01(\bR\n" +
"cacheTools\x12\"\n" +
"\n" +
"min_tokens\x18\x04 \x01(\rH\x00R\tminTokens\x88\x01\x01B\r\n" +
"\v_min_tokens\x1a;\n" +
"min_tokens\x18\x04 \x01(\rH\x00R\tminTokens\x88\x01\x01\x125\n" +
"\x14cache_message_offset\x18\x05 \x01(\rH\x01R\x12cacheMessageOffset\x88\x01\x01B\r\n" +
"\v_min_tokensB\x17\n" +
"\x15_cache_message_offset\x1a;\n" +
"\rDefaultsEntry\x12\x10\n" +
"\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" +
"\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\x1a<\n" +

View File

@ -461,6 +461,7 @@ HELM ?= helm
HELM_CHART_VERSION_NO_V ?= $(patsubst v%,%,$(VERSION))
HELM_CHART_VERSION_V ?= v$(HELM_CHART_VERSION_NO_V)
HELM_PACKAGE_ARGS ?= --version $(HELM_CHART_VERSION_NO_V) --app-version $(HELM_CHART_VERSION_NO_V)
EXTRA_HELM_TAG ?=
HELM_CHART_DIR_AGW=install/helm/agentgateway
HELM_CHART_DIR_AGW_CRD=install/helm/agentgateway-crds
HELM_ADDITIONAL_VALUES ?= hack/helm/dev.yaml
@ -490,18 +491,29 @@ release-charts: package-agentgateway-charts
mkdir -p $(TEST_ASSET_DIR); \
$(HELM) package --version $(HELM_CHART_VERSION_V) --app-version $(HELM_CHART_VERSION_V) --destination $(TEST_ASSET_DIR) $(HELM_CHART_DIR_AGW_CRD); \
$(HELM) repo index $(TEST_ASSET_DIR);
@if [ -n "$(EXTRA_HELM_TAG)" ]; then \
mkdir -p $(TEST_ASSET_DIR); \
$(HELM) package --version $(EXTRA_HELM_TAG) --app-version $(VERSION) --destination $(TEST_ASSET_DIR) $(HELM_CHART_DIR_AGW); \
$(HELM) repo index $(TEST_ASSET_DIR); \
mkdir -p $(TEST_ASSET_DIR); \
$(HELM) package --version $(EXTRA_HELM_TAG) --app-version $(VERSION) --destination $(TEST_ASSET_DIR) $(HELM_CHART_DIR_AGW_CRD); \
$(HELM) repo index $(TEST_ASSET_DIR); \
fi
@for chart in $(CHART_NAMES); do \
$(HELM) push $(TEST_ASSET_DIR)/$$chart-$(HELM_CHART_VERSION_NO_V).tgz oci://$(IMAGE_REGISTRY)/charts; \
$(HELM) push $(TEST_ASSET_DIR)/$$chart-$(HELM_CHART_VERSION_V).tgz oci://$(IMAGE_REGISTRY)/charts; \
if [ -n "$(EXTRA_HELM_TAG)" ]; then \
$(HELM) push $(TEST_ASSET_DIR)/$$chart-$(EXTRA_HELM_TAG).tgz oci://$(IMAGE_REGISTRY)/charts; \
fi; \
done;
.PHONY: deploy-agentgateway-crd-chart
deploy-agentgateway-crd-chart: ## Deploy the agentgateway crd chart
$(HELM) upgrade --install agentgateway-crds $(TEST_ASSET_DIR)/agentgateway-crds-$(VERSION).tgz --namespace $(INSTALL_NAMESPACE) --create-namespace
$(HELM) upgrade --install agentgateway-crds $(TEST_ASSET_DIR)/agentgateway-crds-$(HELM_CHART_VERSION_NO_V).tgz --namespace $(INSTALL_NAMESPACE) --create-namespace
.PHONY: deploy-agentgateway-chart
deploy-agentgateway-chart: ## Deploy the agentgateway chart
$(HELM) upgrade --install agentgateway $(TEST_ASSET_DIR)/agentgateway-$(VERSION).tgz \
$(HELM) upgrade --install agentgateway $(TEST_ASSET_DIR)/agentgateway-$(HELM_CHART_VERSION_NO_V).tgz \
--namespace $(INSTALL_NAMESPACE) --create-namespace \
--set image.registry=$(IMAGE_REGISTRY) \
--set image.tag=$(VERSION) \

View File

@ -1,4 +1,7 @@
package annotations
// LegacyMCPServiceHTTPPath is the legacy annotation used to specify the HTTP path for the MCP service. Users should switch to MCPServiceHTTPPath.
const LegacyMCPServiceHTTPPath = "kgateway.dev/mcp-path"
// MCPServiceHTTPPath is the annotation used to specify the HTTP path for the MCP service
const MCPServiceHTTPPath = "kgateway.dev/mcp-path"
const MCPServiceHTTPPath = "agentgateway.dev/mcp-path"

View File

@ -41,6 +41,28 @@ spec:
matchExpressions:
- 'true'
---
_err: unknown fields
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
name: mcp-static-target-authentication
spec:
mcp:
targets:
- name: target
static:
host: example.com
port: 443
policies:
mcp:
authentication:
issuer: https://issuer.example.com/
jwks:
backendRef:
name: jwks
port: 443
jwksPath: /.well-known/jwks.json
---
_err: path and pathPrefix are mutually exclusive
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend

View File

@ -63,6 +63,20 @@ kind: AgentgatewayBackend
metadata:
name: mcp-full
spec:
policies:
mcp:
authorization:
action: Deny
policy:
matchExpressions:
- 'mcp.tool.name == "hello"'
authentication:
issuer: https://issuer.example.com/
jwks:
backendRef:
name: jwks
port: 443
jwksPath: /.well-known/jwks.json
mcp:
targets:
- name: everything
@ -71,12 +85,6 @@ spec:
port: 443
policies:
tls: {}
mcp:
authorization:
action: Deny
policy:
matchExpressions:
- 'mcp.tool.name == "hello"'
- name: k8s
selector:
namespaces:
@ -137,13 +145,13 @@ spec:
static:
host: example.com
port: 443
policies:
mcp:
authorization:
action: Require
policy:
matchExpressions:
- 'mcp.tool.name == "hello"'
policies:
mcp:
authorization:
action: Require
policy:
matchExpressions:
- 'mcp.tool.name == "hello"'
---
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend

View File

@ -455,7 +455,7 @@ type McpTarget struct {
// This field may only be used with host-based static targets, not
// `backendRef`.
// +optional
Policies *BackendWithMCP `json:"policies,omitempty"`
Policies *BackendSimple `json:"policies,omitempty"`
}
// MCPProtocol defines the protocol to use for the `MCPBackend` target.

View File

@ -35,7 +35,7 @@ type AgentgatewayPolicy struct {
// status defines the current state of AgentgatewayPolicy.
// +optional
Status gwv1.PolicyStatus `json:"status,omitempty"`
Status gwv1.PolicyStatus `json:"status,omitzero"`
// TODO: embed this into a typed Status field when
// https://github.com/kubernetes/kubernetes/issues/131533 is resolved
}
@ -220,16 +220,6 @@ type BackendEviction struct {
HealthThreshold *int32 `json:"healthThreshold,omitempty"`
}
// +kubebuilder:validation:AtLeastOneFieldSet
type BackendWithMCP struct {
BackendSimple `json:",inline"`
// `mcp` specifies settings for MCP workloads. This is only applicable when
// connecting to a `Backend` of type `mcp`.
// +optional
MCP *BackendMCP `json:"mcp,omitempty"`
}
// +kubebuilder:validation:AtLeastOneFieldSet
type BackendWithAI struct {
BackendSimple `json:",inline"`
@ -251,6 +241,10 @@ type BackendFull struct {
// `mcp` specifies settings for MCP workloads. This is only applicable when
// connecting to a `Backend` of type `mcp`.
//
// This field is deprecated; prefer to use traffic policy `jwtAuthentication.mcp`, which ensures authentication runs before
// other policies such as transformation and rate limiting.
//
// +optional
MCP *BackendMCP `json:"mcp,omitempty"`
}

View File

@ -401,4 +401,13 @@ type PromptCachingConfig struct {
// +kubebuilder:validation:Minimum=0
// +kubebuilder:default=1024
MinTokens int `json:"minTokens,omitempty"`
// CacheMessageOffset shifts the message cache point further back in the
// conversation. 0 (default) places it at the second-to-last message.
// Higher values move it N additional messages towards the start, clamped
// to bounds.
// +optional
// +kubebuilder:validation:Minimum=0
// +kubebuilder:default=0
CacheMessageOffset int `json:"cacheMessageOffset,omitempty"`
}

View File

@ -1218,27 +1218,6 @@ func (in *BackendWithAI) DeepCopy() *BackendWithAI {
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *BackendWithMCP) DeepCopyInto(out *BackendWithMCP) {
*out = *in
in.BackendSimple.DeepCopyInto(&out.BackendSimple)
if in.MCP != nil {
in, out := &in.MCP, &out.MCP
*out = new(BackendMCP)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackendWithMCP.
func (in *BackendWithMCP) DeepCopy() *BackendWithMCP {
if in == nil {
return nil
}
out := new(BackendWithMCP)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *BasicAuthentication) DeepCopyInto(out *BasicAuthentication) {
*out = *in
@ -2201,7 +2180,7 @@ func (in *McpTarget) DeepCopyInto(out *McpTarget) {
}
if in.Policies != nil {
in, out := &in.Policies, &out.Policies
*out = new(BackendWithMCP)
*out = new(BackendSimple)
(*in).DeepCopyInto(*out)
}
}

View File

@ -4,7 +4,7 @@ Name|Version|License
[anthropics/anthropic-sdk-go](https://github.com/anthropics/anthropic-sdk-go)|v1.27.1|MIT License
[retry-go/v4](https://github.com/avast/retry-go)|v4.7.0|MIT License
[xds/go](https://github.com/cncf/xds)|v0.0.0-20260202195803-dba9d589def2|Apache License 2.0
[go-control-plane/envoy](https://github.com/envoyproxy/go-control-plane)|v1.37.1-0.20260313105501-0df655d8a214|Apache License 2.0
[go-control-plane/envoy](https://github.com/envoyproxy/go-control-plane)|v1.37.1-0.20260402053555-4e9843ddb8d0|Apache License 2.0
[ghodss/yaml](https://github.com/ghodss/yaml)|v1.0.1-0.20190212211648-25d852aebe32|MIT License
[go-jose/v4](https://github.com/go-jose/go-jose)|v4.1.3|Apache License 2.0
[go-logr/logr](https://github.com/go-logr/logr)|v1.4.3|Apache License 2.0
@ -37,7 +37,7 @@ Name|Version|License
[helm/v3](https://helm.sh/helm/v3)|v3.20.1|Apache License 2.0
[istio.io/api](https://istio.io/api)|v1.29.0-alpha.0.0.20260315093121-ce7e56c13e1d|Apache License 2.0
[istio.io/client-go](https://istio.io/client-go)|v1.29.0-alpha.0.0.20260315093321-a99807642da7|Apache License 2.0
[istio.io/istio](https://istio.io/istio)|v0.0.0-20260320151437-efc2b41130da|Apache License 2.0
[istio.io/istio](https://istio.io/istio)|v0.0.0-20260407051556-b9f70d130eb6|Apache License 2.0
[k8s.io/api](https://k8s.io/api)|v0.35.3|Apache License 2.0
[k8s.io/apiextensions-apiserver](https://k8s.io/apiextensions-apiserver)|v0.35.3|Apache License 2.0
[k8s.io/apimachinery](https://k8s.io/apimachinery)|v0.35.3|Apache License 2.0

View File

@ -7042,223 +7042,6 @@ spec:
- HTTP2
type: string
type: object
mcp:
description: |-
`mcp` specifies settings for MCP workloads. This is only applicable when
connecting to a `Backend` of type `mcp`.
properties:
authentication:
description: '`authentication` defines `MCPBackend`-specific
authentication rules.'
properties:
audiences:
description: |-
`audiences` specifies the list of allowed audiences that are allowed
access. This corresponds to the `aud` claim
(https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.3).
If unset, any audience is allowed.
items:
type: string
maxItems: 64
minItems: 1
type: array
issuer:
description: |-
`issuer` identifies the IdP that issued the JWT. This corresponds to the
`iss` claim (https://tools.ietf.org/html/rfc7519#section-4.1.1).
maxLength: 256
minLength: 1
type: string
jwks:
description: |-
`jwks` defines the remote JSON Web Key used to validate the signature of
the JWT.
properties:
backendRef:
description: |-
`backendRef` references the remote JWKS server to reach.
Supported types are `Service` and static `Backend`. An
`AgentgatewayPolicy` containing backend TLS config can then be attached
to the `Service` or `Backend` in order to set TLS options for a
connection to the remote `jwks` source.
properties:
group:
default: ""
description: |-
Group is the group of the referent. For example, "gateway.networking.k8s.io".
When unspecified or empty string, core API group is inferred.
maxLength: 253
pattern: ^$|^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$
type: string
kind:
default: Service
description: |-
Kind is the Kubernetes resource kind of the referent. For example
"Service".
Defaults to "Service" when not specified.
ExternalName services can refer to CNAME DNS records that may live
outside of the cluster and as such are difficult to reason about in
terms of conformance. They also may not be safe to forward to (see
CVE-2021-25740 for more information). Implementations SHOULD NOT
support ExternalName Services.
Support: Core (Services with a type other than ExternalName)
Support: Implementation-specific (Services with type ExternalName)
maxLength: 63
minLength: 1
pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$
type: string
name:
description: Name is the name of
the referent.
maxLength: 253
minLength: 1
type: string
namespace:
description: |-
Namespace is the namespace of the backend. When unspecified, the local
namespace is inferred.
Note that when a namespace different than the local namespace is specified,
a ReferenceGrant object is required in the referent namespace to allow that
namespace's owner to accept the reference. See the ReferenceGrant
documentation for details.
Support: Core
maxLength: 63
minLength: 1
pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$
type: string
port:
description: |-
Port specifies the destination port number to use for this resource.
Port is required when the referent is a Kubernetes Service. In this
case, the port number is the service port number, not the target port.
For other resources, destination port might be derived from the referent
resource or this field.
format: int32
maximum: 65535
minimum: 1
type: integer
required:
- name
type: object
x-kubernetes-validations:
- message: Must have port for Service
reference
rule: '(size(self.group) == 0 && self.kind
== ''Service'') ? has(self.port)
: true'
cacheDuration:
default: 5m
type: string
x-kubernetes-validations:
- message: invalid duration value
rule: matches(self, '^([0-9]{1,5}(h|m|s|ms)){1,4}$')
- message: cacheDuration must be at
least 5m.
rule: duration(self) >= duration('5m')
jwksPath:
description: |-
Path to the IdP `jwks` endpoint, relative to the root, commonly
`".well-known/jwks.json"`.
maxLength: 2000
minLength: 1
type: string
required:
- backendRef
- jwksPath
type: object
mode:
default: Strict
description: '`mode` is the validation mode
for JWT authentication.'
enum:
- Strict
- Optional
- Permissive
type: string
provider:
description: '`provider` specifies the identity
provider to use for authentication.'
enum:
- Auth0
- Keycloak
type: string
resourceMetadata:
additionalProperties:
x-kubernetes-preserve-unknown-fields: true
description: ResourceMetadata defines the
metadata to use for MCP resources.
type: object
required:
- jwks
type: object
authorization:
description: |-
authorization defines MCPBackend level authorization. Unlike authorization at the HTTP level, which will reject
unauthorized requests with a `403` error, this policy works at the
`MCPBackend` level.
List operations, such as `list_tools`, will have each item evaluated.
Items that do not meet the rule will be filtered.
Get or call operations, such as `call_tool`, will evaluate the specific
item and reject requests that do not meet the rule.
properties:
action:
default: Allow
description: |-
`action` defines whether the rule allows, denies, or requires the request if
matched. If unspecified, the default is `Allow`.
Require policies are conjunctive across merged policies: all require policies must match.
enum:
- Allow
- Deny
- Require
type: string
policy:
description: |-
`policy` specifies the authorization rule to evaluate.
* For `Allow` rules: any policy allows the request.
* For `Require` rules: all policies must match for the request to be allowed.
* For `Deny` rules: any matching policy denies the request. Note: a CEL expression that fails to evaluate is not
considered to match, making this a risky policy; prefer to use `Require`.
The presence of at least one `Allow` rule triggers a deny-by-default policy, requiring at least 1 match to allow.
With no rules, all requires are allowed.
properties:
matchExpressions:
description: |-
MatchExpressions defines a set of conditions that must be satisfied for the rule to match.
These expressions should be in the form of a Common Expression Language
(`CEL`) expression.
items:
description: CELExpression represents
a Common Expression Language (CEL)
expression.
maxLength: 16384
minLength: 1
type: string
maxItems: 256
minItems: 1
type: array
required:
- matchExpressions
type: object
required:
- policy
type: object
type: object
x-kubernetes-validations:
- message: at least one of the fields in [authentication
authorization] must be set
rule: '[has(self.authentication),has(self.authorization)].filter(x,x==true).size()
>= 1'
tcp:
description: tcp defines settings for managing TCP
connections to the backend.
@ -7775,12 +7558,6 @@ spec:
- backendRef
type: object
type: object
x-kubernetes-validations:
- message: at least one of the fields in [auth health
http mcp tcp tls transformation tunnel] must be
set
rule: '[has(self.auth),has(self.health),has(self.http),has(self.mcp),has(self.tcp),has(self.tls),has(self.transformation),has(self.tunnel)].filter(x,x==true).size()
>= 1'
port:
description: Port is the port number of the MCP target.
format: int32
@ -12781,6 +12558,9 @@ spec:
description: |-
`mcp` specifies settings for MCP workloads. This is only applicable when
connecting to a `Backend` of type `mcp`.
This field is deprecated; prefer to use traffic policy `jwtAuthentication.mcp`, which ensures authentication runs before
other policies such as transformation and rate limiting.
properties:
authentication:
description: '`authentication` defines `MCPBackend`-specific

View File

@ -5030,6 +5030,9 @@ spec:
description: |-
`mcp` specifies settings for MCP workloads. This is only applicable when
connecting to a `Backend` of type `mcp`.
This field is deprecated; prefer to use traffic policy `jwtAuthentication.mcp`, which ensures authentication runs before
other policies such as transformation and rate limiting.
properties:
authentication:
description: '`authentication` defines `MCPBackend`-specific

View File

@ -44,6 +44,14 @@ spec:
{{- with .Values.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- else }}
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
{{- end }}
image: "{{ .Values.controller.image.registry | default .Values.image.registry }}/{{ .Values.controller.image.repository }}:{{ include "agentgateway.imageTag" (.Values.controller.image.tag | default .Values.image.tag | default .Chart.AppVersion) }}"
imagePullPolicy: {{ .Values.controller.image.pullPolicy | default .Values.image.pullPolicy }}

View File

@ -17,7 +17,8 @@ import (
)
const (
a2aProtocol = "kgateway.dev/a2a"
legacyA2aProtocol = "kgateway.dev/a2a"
a2aProtocol = "agentgateway.dev/a2a"
)
// NewA2APlugin creates a new A2A policy plugin
@ -48,7 +49,8 @@ func translatePoliciesForService(krtctx krt.HandlerContext, svc *corev1.Service,
}).UnsortedList()
for _, port := range svc.Spec.Ports {
if port.AppProtocol != nil && *port.AppProtocol == a2aProtocol {
// support legacy kgateway.dev/a2a and agentgateway.dev/a2a
if port.AppProtocol != nil && (*port.AppProtocol == a2aProtocol || *port.AppProtocol == legacyA2aProtocol) {
logger.Debug("found A2A service", "service", svc.Name, "namespace", svc.Namespace, "port", port.Port)
hostname := fmt.Sprintf("%s.%s.svc.%s", svc.Name, svc.Namespace, clusterDomain)
policy := &api.Policy{

View File

@ -622,6 +622,9 @@ func translateBackendAI(ctx PolicyCtx, agwPolicy *agentgateway.AgentgatewayPolic
CacheTools: aiSpec.PromptCaching.CacheTools,
}
translatedAIPolicy.PromptCaching.MinTokens = ptr.Of(uint32(aiSpec.PromptCaching.MinTokens)) //nolint:gosec // G115: MinTokens is validated by kubebuilder to be >= 0
if aiSpec.PromptCaching.CacheMessageOffset > 0 {
translatedAIPolicy.PromptCaching.CacheMessageOffset = ptr.Of(uint32(aiSpec.PromptCaching.CacheMessageOffset)) //nolint:gosec // G115: CacheMessageOffset is validated by kubebuilder to be >= 0
}
}
if aiSpec.Routes != nil {

View File

@ -20,6 +20,7 @@ import (
"github.com/agentgateway/agentgateway/api"
"github.com/agentgateway/agentgateway/controller/pkg/agentgateway/utils"
"github.com/agentgateway/agentgateway/controller/pkg/pluginsdk/krtutil"
"github.com/agentgateway/agentgateway/controller/pkg/wellknown"
)
@ -129,16 +130,14 @@ func translatePoliciesForBackendTLS(
}
}
backendTLSPoliciesForThisTarget := krt.FetchOne(krtctx, targetIndex, krt.FilterKey(tgtRef.String()))
if backendTLSPoliciesForThisTarget != nil {
if err := checkConflicted(btls, target, backendTLSPoliciesForThisTarget); err != nil {
conds[string(gwv1.PolicyConditionAccepted)].error = &ConfigError{
Reason: string(gwv1.PolicyReasonConflicted),
Message: err.Error(),
}
// We cannot send this policy to agentgateway, as it would not know the priority logic.
continue
backendTLSPoliciesForThisTarget := krtutil.FetchIndexObjects(krtctx, targetIndex, tgtRef)
if err := checkConflicted(btls, target, backendTLSPoliciesForThisTarget); err != nil {
conds[string(gwv1.PolicyConditionAccepted)].error = &ConfigError{
Reason: string(gwv1.PolicyReasonConflicted),
Message: err.Error(),
}
// We cannot send this policy to agentgateway, as it would not know the priority logic.
continue
}
switch string(target.Kind) {
@ -253,9 +252,9 @@ func translatePoliciesForBackendTLS(
func checkConflicted(
btls *gwv1.BackendTLSPolicy,
target gwv1.LocalPolicyTargetReferenceWithSectionName,
allMatches *krt.IndexObject[utils.TypedNamespacedName, *gwv1.BackendTLSPolicy],
allMatches []*gwv1.BackendTLSPolicy,
) error {
for _, m := range allMatches.Objects {
for _, m := range allMatches {
if m.UID == btls.UID {
// This is ourself, skip it
continue

View File

@ -13,12 +13,13 @@ import (
"github.com/agentgateway/agentgateway/api"
"github.com/agentgateway/agentgateway/controller/pkg/agentgateway/utils"
"github.com/agentgateway/agentgateway/controller/pkg/pluginsdk/krtutil"
"github.com/agentgateway/agentgateway/controller/pkg/utils/kubeutils"
"github.com/agentgateway/agentgateway/controller/pkg/wellknown"
)
type ReferenceTypes struct {
PolicyTargets func(krtctx krt.HandlerContext, namespace string, name gwv1.ObjectName, gk schema.GroupKind, sectionName *gwv1.SectionName) (*api.PolicyTarget, bool)
PolicyTargets func(krtctx krt.HandlerContext, namespace string, name gwv1.ObjectName, gk schema.GroupKind, sectionName *gwv1.SectionName) ([]*api.PolicyTarget, bool)
PolicyBackend func(krtctx krt.HandlerContext, defaultNamespace string, gk schema.GroupKind, name gwv1.ObjectName, namespace *gwv1.Namespace, port *gwv1.PortNumber) (*api.BackendReference, error)
RouteBackend func(krtctx krt.HandlerContext, defaultNamespace string, gk schema.GroupKind, name gwv1.ObjectName, namespace *gwv1.Namespace, port *gwv1.PortNumber) (*api.BackendReference, error)
}
@ -43,29 +44,29 @@ func (e *BackendReferenceError) Error() string {
func DefaultReferenceTypes(agw *AgwCollections) ReferenceTypes {
return ReferenceTypes{
// AgentgatewayPolicy targets
PolicyTargets: func(krtctx krt.HandlerContext, namespace string, name gwv1.ObjectName, gk schema.GroupKind, sectionName *gwv1.SectionName) (*api.PolicyTarget, bool) {
PolicyTargets: func(krtctx krt.HandlerContext, namespace string, name gwv1.ObjectName, gk schema.GroupKind, sectionName *gwv1.SectionName) ([]*api.PolicyTarget, bool) {
key := namespace + "/" + string(name)
switch gk {
case wellknown.GatewayGVK.GroupKind():
return &api.PolicyTarget{
return []*api.PolicyTarget{{
Kind: utils.GatewayTarget(namespace, string(name), sectionName),
}, ptr.Flatten(krt.FetchOne(krtctx, agw.Gateways, krt.FilterKey(key))) != nil
}}, ptr.Flatten(krt.FetchOne(krtctx, agw.Gateways, krt.FilterKey(key))) != nil
case wellknown.HTTPRouteGVK.GroupKind():
return &api.PolicyTarget{
return []*api.PolicyTarget{{
Kind: utils.RouteTarget(namespace, string(name), wellknown.HTTPRouteGVK.Kind, sectionName),
}, ptr.Flatten(krt.FetchOne(krtctx, agw.HTTPRoutes, krt.FilterKey(key))) != nil
}}, ptr.Flatten(krt.FetchOne(krtctx, agw.HTTPRoutes, krt.FilterKey(key))) != nil
case wellknown.GRPCRouteGVK.GroupKind():
return &api.PolicyTarget{
return []*api.PolicyTarget{{
Kind: utils.RouteTarget(namespace, string(name), wellknown.GRPCRouteGVK.Kind, sectionName),
}, ptr.Flatten(krt.FetchOne(krtctx, agw.GRPCRoutes, krt.FilterKey(key))) != nil
}}, ptr.Flatten(krt.FetchOne(krtctx, agw.GRPCRoutes, krt.FilterKey(key))) != nil
case wellknown.AgentgatewayBackendGVK.GroupKind():
return &api.PolicyTarget{
return []*api.PolicyTarget{{
Kind: utils.BackendTarget(namespace, string(name), sectionName),
}, ptr.Flatten(krt.FetchOne(krtctx, agw.Backends, krt.FilterKey(key))) != nil
}}, ptr.Flatten(krt.FetchOne(krtctx, agw.Backends, krt.FilterKey(key))) != nil
case wellknown.ServiceGVK.GroupKind():
return &api.PolicyTarget{
return []*api.PolicyTarget{{
Kind: utils.ServiceTarget(namespace, string(name), sectionName),
}, ptr.Flatten(krt.FetchOne(krtctx, agw.Services, krt.FilterKey(key))) != nil
}}, ptr.Flatten(krt.FetchOne(krtctx, agw.Services, krt.FilterKey(key))) != nil
}
return nil, false
},
@ -275,13 +276,13 @@ func (p ReferenceIndex) LookupGatewaysForTarget(ctx krt.HandlerContext, object u
return sets.New(object.NamespacedName)
case wellknown.HTTPRouteGVK.Kind, wellknown.GRPCRouteGVK.Kind, wellknown.TCPRouteGVK.Kind, wellknown.TLSRouteGVK.Kind:
gateways := sets.New[types.NamespacedName]()
for _, ancestor := range krt.FetchOne(ctx, p.attachments, krt.FilterKey(object.String())).Objects {
for _, ancestor := range krtutil.FetchIndexObjects(ctx, p.attachments, object) {
gateways.Insert(ancestor.Gateway)
}
return gateways
default:
gateways := sets.New[types.NamespacedName]()
for _, ancestor := range krt.FetchOne(ctx, p.Ancestors, krt.FilterKey(object.String())).Objects {
for _, ancestor := range krtutil.FetchIndexObjects(ctx, p.Ancestors, object) {
gateways.Insert(ancestor.Gateway)
}
return gateways
@ -289,21 +290,19 @@ func (p ReferenceIndex) LookupGatewaysForTarget(ctx krt.HandlerContext, object u
}
func (p ReferenceIndex) LookupGatewaysForBackend(ctx krt.HandlerContext, object utils.TypedNamespacedName) sets.Set[types.NamespacedName] {
return p.lookupGatewaysForBackend(ctx, object, map[string]struct{}{})
return p.lookupGatewaysForBackend(ctx, object, sets.New[utils.TypedNamespacedName]())
}
func (p ReferenceIndex) lookupGatewaysForBackend(ctx krt.HandlerContext, object utils.TypedNamespacedName, seen map[string]struct{}) sets.Set[types.NamespacedName] {
key := object.String()
if _, ok := seen[key]; ok {
func (p ReferenceIndex) lookupGatewaysForBackend(ctx krt.HandlerContext, object utils.TypedNamespacedName, seen sets.Set[utils.TypedNamespacedName]) sets.Set[types.NamespacedName] {
if seen.InsertContains(object) {
return sets.New[types.NamespacedName]()
}
seen[key] = struct{}{}
base := p.LookupGatewaysForTarget(ctx, object)
if p.PolicyAttachments == nil {
return base
}
for _, pref := range krt.FetchOne(ctx, p.PolicyAttachments, krt.FilterKey(key)).Objects {
for _, pref := range krtutil.FetchIndexObjects(ctx, p.PolicyAttachments, object) {
base = base.Union(p.lookupGatewaysForBackend(ctx, pref.Target, seen))
}
return base
@ -314,7 +313,7 @@ func (p ReferenceIndex) WithPolicyAttachments(references krt.IndexCollection[uti
return p
}
func (p ReferenceIndex) PolicyTarget(krtctx krt.HandlerContext, namespace string, name gwv1.ObjectName, gk schema.GroupKind, sectionName *gwv1.SectionName) (*api.PolicyTarget, bool) {
func (p ReferenceIndex) PolicyTarget(krtctx krt.HandlerContext, namespace string, name gwv1.ObjectName, gk schema.GroupKind, sectionName *gwv1.SectionName) ([]*api.PolicyTarget, bool) {
return p.explicitReferences.PolicyTargets(krtctx, namespace, name, gk, sectionName)
}

View File

@ -135,25 +135,27 @@ func TranslateAgentgatewayPolicy(ctx krt.HandlerContext, policy *agentgateway.Ag
for _, target := range policy.Spec.TargetRefs {
gk := schema.GroupKind{Group: string(target.Group), Kind: string(target.Kind)}
policyTarget, targetExists := references.PolicyTarget(ctx, policy.Namespace, target.Name, gk, target.SectionName)
if policyTarget == nil {
policyTargets, targetExists := references.PolicyTarget(ctx, policy.Namespace, target.Name, gk, target.SectionName)
if len(policyTargets) == 0 {
// This should be impossible, verified by CEL validation
logger.Warn("unsupported target kind", "kind", target.Kind, "policy", policy.Name)
continue
}
gatewayTargets := references.LookupGatewaysForTarget(ctx, utils.TypedNamespacedName{
gatewayTargets := references.LookupGatewaysForBackend(ctx, utils.TypedNamespacedName{
NamespacedName: types.NamespacedName{Namespace: policy.Namespace, Name: string(target.Name)},
Kind: gk.Kind,
}).UnsortedList()
translatedPolicies := clonePoliciesForTarget(baseTranslatedPolicies, policyTarget)
for _, translatedPolicy := range translatedPolicies {
for _, gatewayTarget := range gatewayTargets {
agwPolicies = append(agwPolicies, AgwPolicy{
Gateway: ptr.Of(gatewayTarget),
Policy: translatedPolicy,
})
for _, policyTarget := range policyTargets {
translatedPolicies := clonePoliciesForTarget(baseTranslatedPolicies, policyTarget)
for _, translatedPolicy := range translatedPolicies {
for _, gatewayTarget := range gatewayTargets {
agwPolicies = append(agwPolicies, AgwPolicy{
Gateway: ptr.Of(gatewayTarget),
Policy: translatedPolicy,
})
}
}
}
@ -302,7 +304,7 @@ func resolvePolicyAncestorRefs(
NamespacedName: types.NamespacedName{Namespace: policyNamespace, Name: string(targetName)},
Kind: targetGK.Kind,
}
gatewayTargets := references.LookupGatewaysForTarget(ctx, object).UnsortedList()
gatewayTargets := references.LookupGatewaysForBackend(ctx, object).UnsortedList()
if len(gatewayTargets) == 0 {
return nil, fmt.Sprintf("Policy is not attached: %s %s/%s is not attached to any Gateway", targetGK.Kind, policyNamespace, targetName)
}

View File

@ -11,7 +11,7 @@ spec:
- protocol: TCP
port: 9090
targetPort: 9090
appProtocol: kgateway.dev/a2a
appProtocol: agentgateway.dev/a2a
---
# Output
output:

View File

@ -10,11 +10,11 @@ spec:
test: test
ports:
- protocol: TCP
appProtocol: kgateway.dev/mcp
appProtocol: agentgateway.dev/mcp
port: 80
targetPort: test
- protocol: TCP
appProtocol: kgateway.dev/mcp-sse
appProtocol: agentgateway.dev/mcp-sse
port: 90
targetPort: test-2
---

View File

@ -21,7 +21,7 @@ spec:
ports:
- name: mcp-http
port: 9090
appProtocol: kgateway.dev/mcp
appProtocol: agentgateway.dev/mcp
---
# Output
output:

View File

@ -21,12 +21,12 @@ metadata:
labels:
app: mcp-server
annotations:
kgateway.dev/mcp-http-path: /custom-path
agentgateway.dev/mcp-http-path: /custom-path
spec:
ports:
- name: mcp
port: 8080
appProtocol: kgateway.dev/mcp-sse
appProtocol: agentgateway.dev/mcp-sse
---
# Output
output:

View File

@ -23,7 +23,7 @@ spec:
ports:
- name: mcp-http
port: 9090
appProtocol: kgateway.dev/mcp
appProtocol: agentgateway.dev/mcp
---
# Output
output:

View File

@ -6,13 +6,13 @@ metadata:
labels:
app: mcp-app
annotations:
kgateway.dev/mcp-path: /test
agentgateway.dev/mcp-path: /test
spec:
selector:
test: test
ports:
- protocol: TCP
appProtocol: kgateway.dev/mcp
appProtocol: agentgateway.dev/mcp
port: 80
targetPort: test
---

View File

@ -0,0 +1,104 @@
apiVersion: v1
kind: Service
metadata:
name: processor
namespace: default
spec:
selector:
app: processor
ports:
- protocol: TCP
port: 9002
targetPort: 9002
---
apiVersion: v1
kind: Secret
metadata:
name: client-tls
namespace: default
type: kubernetes.io/tls
data:
tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJiakNDQVJPZ0F3SUJBZ0lVTFpDLzFkMEhpMlRFUGk5MUtwamVZbzgzZ1pBd0NnWUlLb1pJemowRUF3SXcKRERFS01BZ0dBMVVFQXd3QllUQWVGdzB5TlRFeE1qVXlNRFE0TURLYUZ3MHlOVEV4TWpZeU1EUTRNREphTUF3eApDakFJQmdOVkJBTU1BV0V3V1RBVEJnY3Foa2pPUFFJQkJnZ3Foa2pPUFFNQkJ3TkNBQVIvc1UybWRnbkhab2Z6CmR0b0tRZm5FMWdrVE9HY2JJaWFUeDdXUGpZcXF1c1p5ZWYybTBpbkgrcnJVRFBJY2tTRlQ5dnlzRVJFVTBKcGMKL3JTZmVVOG9vMU13VVRBZEJnTlZIUTRFRmdRVWdNQ01qYnRKZnQ2VjAyT1JWMzZQcmpBWWNPd3dId1lEVlIwagpCQmd3Rm9BVWdNQ01qYnRKZnQ2VjAyT1JWMzZQcmpBWWNPd3dEd1lEVlIwVEFRSC9CQVV3QXdFQi96QUtCZ2dxCmhrak9QUVFEQWdOSkFEQkdBaUVBcW9ISTVENUt2VFR4YVVubW1mc0Nzem1vazZWR0dsZ3hpVm5HVmpKNHJ5VUMKSVFENExVcTRZMjVPbG4zQnNlbE1nT0M5VWNFS3dLRGlOTDZlMkc4S09EcU0wZz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
tls.key: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUxRblhESm5qZFRianhhV2NKQkJOK0FDVjh3UDRxR3RZdVA0VjdZSXdJdGRvQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFZjdGTnBuWUp4MmFIODNiYUNrSDV4TllKRXpobkd5SW1rOGUxajQyS3Fyckdjbm45cHRJcAp4L3E2MUF6eUhKRWhVL2I4ckJFUkZOQ2FYUDYwbjNsUEtBPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo=
---
# ext-proc policy targets Gateway, with backendRef to processor Service
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
name: ext-proc
namespace: default
spec:
targetRefs:
- kind: Gateway
name: test
group: gateway.networking.k8s.io
traffic:
phase: PreRouting
extProc:
backendRef:
name: processor
port: 9002
---
# backend TLS policy targets the processor Service (not the Gateway)
# This should still be associated with the gateway via the ext-proc policy's backendRef
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
name: processor-backend-tls
namespace: default
spec:
targetRefs:
- kind: Service
name: processor
group: ""
backend:
tls:
mtlsCertificateRef:
- name: client-tls
sni: processor.default.svc.cluster.local
---
# Output
output:
- gateway:
Name: test
Namespace: default
resource:
policy:
backend:
backendTls:
cert: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJiakNDQVJPZ0F3SUJBZ0lVTFpDLzFkMEhpMlRFUGk5MUtwamVZbzgzZ1pBd0NnWUlLb1pJemowRUF3SXcKRERFS01BZ0dBMVVFQXd3QllUQWVGdzB5TlRFeE1qVXlNRFE0TURLYUZ3MHlOVEV4TWpZeU1EUTRNREphTUF3eApDakFJQmdOVkJBTU1BV0V3V1RBVEJnY3Foa2pPUFFJQkJnZ3Foa2pPUFFNQkJ3TkNBQVIvc1UybWRnbkhab2Z6CmR0b0tRZm5FMWdrVE9HY2JJaWFUeDdXUGpZcXF1c1p5ZWYybTBpbkgrcnJVRFBJY2tTRlQ5dnlzRVJFVTBKcGMKL3JTZmVVOG9vMU13VVRBZEJnTlZIUTRFRmdRVWdNQ01qYnRKZnQ2VjAyT1JWMzZQcmpBWWNPd3dId1lEVlIwagpCQmd3Rm9BVWdNQ01qYnRKZnQ2VjAyT1JWMzZQcmpBWWNPd3dEd1lEVlIwVEFRSC9CQVV3QXdFQi96QUtCZ2dxCmhrak9QUVFEQWdOSkFEQkdBaUVBcW9ISTVENUt2VFR4YVVubW1mc0Nzem1vazZWR0dsZ3hpVm5HVmpKNHJ5VUMKSVFENExVcTRZMjVPbG4zQnNlbE1nT0M5VWNFS3dLRGlOTDZlMkc4S09EcU0wZz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
hostname: processor.default.svc.cluster.local
key: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUxRblhESm5qZFRianhhV2NKQkJOK0FDVjh3UDRxR3RZdVA0VjdZSXdJdGRvQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFZjdGTnBuWUp4MmFIODNiYUNrSDV4TllKRXpobkd5SW1rOGUxajQyS3Fyckdjbm45cHRJcAp4L3E2MUF6eUhKRWhVL2I4ckJFUkZOQ2FYUDYwbjNsUEtBPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo=
key: default/processor-backend-tls:tls:default/processor.default.svc.cluster.local
name:
kind: AgentgatewayPolicy
name: processor-backend-tls
namespace: default
target:
service:
hostname: processor.default.svc.cluster.local
namespace: default
- gateway:
Name: test
Namespace: default
resource:
policy:
key: traffic/default/ext-proc:extproc:default/test
name:
kind: AgentgatewayPolicy
name: ext-proc
namespace: default
target:
gateway:
name: test
namespace: default
traffic:
extProc:
target:
port: 9002
service:
hostname: processor.default.svc.cluster.local
namespace: default
phase: GATEWAY
status: []

View File

@ -0,0 +1,78 @@
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: test-route
namespace: default
spec:
parentRefs:
- name: test-gateway
hostnames:
- example.com
rules:
- matches:
- path:
type: PathPrefix
value: /api
backendRefs:
- name: be
namespace: other
kind: AgentgatewayBackend
group: agentgateway.dev
---
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
name: be
namespace: other
spec:
static:
host: primary.example.com
port: 80
---
# Output
output:
- gateway:
Name: test-gateway
Namespace: default
resource:
route:
backends:
- weight: 1
hostnames:
- example.com
key: default/test-route.00.http
listenerKey: default/test-gateway.http
matches:
- path:
pathPrefix: /api
name:
kind: HTTPRoute
name: test-route
namespace: default
status:
- apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: test-route
namespace: default
spec: null
status:
parents:
- conditions:
- lastTransitionTime: fake
message: ""
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: fake
message: backendRef be/other not accessible to a HTTPRoute in namespace "default"
(missing a ReferenceGrant?)
reason: RefNotPermitted
status: "False"
type: ResolvedRefs
controllerName: agentgateway.dev/agentgateway
parentRef:
group: gateway.networking.k8s.io
kind: Gateway
name: test-gateway
namespace: default

View File

@ -0,0 +1,78 @@
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: test-route
namespace: default
spec:
parentRefs:
- name: test-gateway
hostnames:
- example.com
rules:
- matches:
- path:
type: PathPrefix
value: /api
backendRefs:
- name: be
kind: AgentgatewayBackend
group: agentgateway.dev
---
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
name: be
namespace: default
spec:
static:
host: primary.example.com
port: 80
---
# Output
output:
- gateway:
Name: test-gateway
Namespace: default
resource:
route:
backends:
- backend:
backend: default/be
weight: 1
hostnames:
- example.com
key: default/test-route.00.http
listenerKey: default/test-gateway.http
matches:
- path:
pathPrefix: /api
name:
kind: HTTPRoute
name: test-route
namespace: default
status:
- apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: test-route
namespace: default
spec: null
status:
parents:
- conditions:
- lastTransitionTime: fake
message: ""
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: fake
message: ""
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
controllerName: agentgateway.dev/agentgateway
parentRef:
group: gateway.networking.k8s.io
kind: Gateway
name: test-gateway
namespace: default

View File

@ -0,0 +1,23 @@
package krtutil
import (
"fmt"
"istio.io/istio/pkg/kube/krt"
)
func FetchIndexObjects[K comparable, O any](ctx krt.HandlerContext, index krt.IndexCollection[K, O], name K) []O {
res := krt.FetchOne(ctx, index, krt.FilterKey(toString(name)))
if res == nil {
return nil
}
return res.Objects
}
func toString(rk any) string {
tk, ok := rk.(string)
if !ok {
return rk.(fmt.Stringer).String()
}
return tk
}

View File

@ -17,7 +17,7 @@ import (
"github.com/agentgateway/agentgateway/controller/pkg/wellknown"
)
const fakeCondition = "kgateway.dev/SomeCondition"
const fakeCondition = "agentgateway.dev/SomeCondition"
var ctx = context.Background()
@ -41,7 +41,7 @@ func TestBuildGatewayStatus(t *testing.T) {
t.Run("preserve conditions set externally", func(t *testing.T) {
gw := gw()
gw.Status.Conditions = append(gw.Status.Conditions, metav1.Condition{
Type: "gateway.kgateway.dev/SomeCondition",
Type: "gateway.agentgateway.dev/SomeCondition",
Status: metav1.ConditionFalse,
})
rm := reports.NewReportMap()

View File

@ -910,7 +910,7 @@ spec:
- protocol: TCP
port: 3001
targetPort: 3001
appProtocol: kgateway.dev/mcp
appProtocol: agentgateway.dev/mcp
type: ClusterIP
EOF
```
@ -1002,7 +1002,7 @@ spec:
ports:
- port: 80
targetPort: 8000
appProtocol: kgateway.dev/mcp
appProtocol: agentgateway.dev/mcp
EOF
```
@ -1044,11 +1044,11 @@ spec:
- protocol: TCP
port: 9090
targetPort: 9090
appProtocol: kgateway.dev/a2a
appProtocol: agentgateway.dev/a2a
EOF
```
Note, you must use `kgateway.dev/a2a` as the app protocol for the kgateway control plane to configure agentgateway to use a2a.
Note, you must use `agentgateway.dev/a2a` as the app protocol for the kgateway control plane to configure agentgateway to use a2a.
Apply the routing config:
```shell
@ -1539,7 +1539,7 @@ spec:
- protocol: TCP
port: 3001
targetPort: 3001
appProtocol: kgateway.dev/mcp
appProtocol: agentgateway.dev/mcp
type: ClusterIP
EOF
```
@ -1764,7 +1764,7 @@ EOF
```shell
kubectl apply -f- <<EOF
apiVersion: gateway.kgateway.dev/v1alpha1
apiVersion: gateway.agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
name: token-rate-limit
@ -1901,7 +1901,7 @@ EOF
```
#### Bedrock prompt caching
Enable Bedrock prompt caching via `AgentgatewayPolicy.spec.backend.ai.promptCaching` to reduce costs on repeated prompts and tool specs. Set `minTokens` to avoid caching short prompts; only Bedrock models support this feature.
Enable Bedrock prompt caching via `AgentgatewayPolicy.spec.backend.ai.promptCaching` to reduce costs on repeated prompts and tool specs. Set `minTokens` to avoid caching short prompts; only Bedrock models support this feature. Use `cacheMessageOffset` to shift the message cache point further back in the conversation (0 = default second-to-last message; higher values move it N additional messages toward the start).
```shell
kubectl apply -f- <<EOF

View File

@ -103,6 +103,12 @@ func TranslateMCPSelectorTargets(
targetName = service.Name + "-" + port.Name
}
path := service.Annotations[apiannotations.MCPServiceHTTPPath]
// use legacy annotation for backwards compatibility
if path == "" && service.Annotations[apiannotations.LegacyMCPServiceHTTPPath] != "" {
path = service.Annotations[apiannotations.LegacyMCPServiceHTTPPath]
}
svcHostname := kubeutils.ServiceFQDN(service.ObjectMeta)
mcpTargets = append(mcpTargets, &api.MCPTarget{
Name: targetName,
@ -116,7 +122,7 @@ func TranslateMCPSelectorTargets(
Port: uint32(port.Port), //nolint:gosec // G115: Kubernetes service ports are always positive
},
Protocol: toMCPProtocol(appProtocol),
Path: service.Annotations[apiannotations.MCPServiceHTTPPath],
Path: path,
})
}
}

View File

@ -0,0 +1,14 @@
- key: test-ns/service-mcp-backend-legacy
mcp:
targets:
- backend:
port: 8080
service:
hostname: mcp-service-legacy.test-ns.svc.cluster.local
namespace: test-ns
name: mcp-service-legacy-mcp
path: /legacy/mcp/path
protocol: STREAMABLE_HTTP
name:
name: service-mcp-backend-legacy
namespace: test-ns

View File

@ -0,0 +1,14 @@
- key: test-ns/service-mcp-backend-precedence
mcp:
targets:
- backend:
port: 8080
service:
hostname: mcp-service-both.test-ns.svc.cluster.local
namespace: test-ns
name: mcp-service-both-mcp
path: /new/path
protocol: STREAMABLE_HTTP
name:
name: service-mcp-backend-precedence
namespace: test-ns

View File

@ -92,11 +92,8 @@ func BuildAgwBackendReferences(
if mcp := backend.Spec.MCP; mcp != nil {
for _, r := range mcp.Targets {
if r.Static != nil && r.Static.Policies != nil {
p := r.Static.Policies
plugins.BackendReferencesFromBackendPolicy(&agentgateway.BackendFull{
BackendSimple: p.BackendSimple,
MCP: p.MCP,
AI: nil,
BackendSimple: *r.Static.Policies,
}, app)
}
}
@ -247,11 +244,7 @@ func TranslateMCPBackends(ctx plugins.PolicyCtx, be *agentgateway.AgentgatewayBa
}
staticBackendRef := utils.InternalMCPStaticBackendName(be.Namespace, be.Name, string(target.Name))
pol, err := TranslateMCPBackendPolicies(ctx, be.Namespace, s.Policies)
if err != nil {
logger.Error("failed to translate static MCP backend policies", "err", err)
errs = append(errs, err)
}
staticBackend := &api.Backend{
Key: staticBackendRef,
Name: plugins.ResourceName(be),
@ -261,7 +254,17 @@ func TranslateMCPBackends(ctx plugins.PolicyCtx, be *agentgateway.AgentgatewayBa
Port: s.Port,
},
},
InlinePolicies: pol,
}
if s.Policies != nil {
polt, err := TranslateBackendPolicies(ctx, be.Namespace, &agentgateway.BackendFull{
BackendSimple: *s.Policies,
})
if err != nil {
logger.Error("failed to translate static MCP backend policies", "err", err)
errs = append(errs, err)
}
staticBackend.InlinePolicies = polt
}
backends = append(backends, staticBackend)
@ -378,19 +381,6 @@ func TranslateBackendPolicies(
return plugins.TranslateInlineBackendPolicy(ctx, namespace, policies)
}
func TranslateMCPBackendPolicies(
ctx plugins.PolicyCtx,
namespace string, policies *agentgateway.BackendWithMCP,
) ([]*api.BackendPolicySpec, error) {
if policies == nil {
return nil, nil
}
return TranslateBackendPolicies(ctx, namespace, &agentgateway.BackendFull{
BackendSimple: policies.BackendSimple,
MCP: policies.MCP,
})
}
func translateAIBackendPolicies(
ctx plugins.PolicyCtx,
namespace string, policies *agentgateway.BackendWithAI,

View File

@ -132,6 +132,56 @@ func TestBuildMCP(t *testing.T) {
},
inputs: []any{createMockMCPServiceWithProtocol("test-ns", "mcp-service", "app=mcp-server", "agentgateway.dev/mcp")},
},
{
name: "Service selector MCPBackend backend - legacy annotation path",
backend: &agentgateway.AgentgatewayBackend{
ObjectMeta: metav1.ObjectMeta{
Name: "service-mcp-backend-legacy",
Namespace: "test-ns",
},
Spec: agentgateway.AgentgatewayBackendSpec{
MCP: &agentgateway.MCPBackend{
Targets: []agentgateway.McpTargetSelector{
{
Selector: &agentgateway.McpSelector{
Service: &metav1.LabelSelector{
MatchLabels: map[string]string{
"app": "mcp-server-legacy",
},
},
},
},
},
},
},
},
inputs: []any{createMockMCPServiceWithLegacyAnnotation("test-ns", "mcp-service-legacy", "app=mcp-server-legacy", "/legacy/mcp/path")},
},
{
name: "Service selector MCPBackend backend - new annotation takes precedence",
backend: &agentgateway.AgentgatewayBackend{
ObjectMeta: metav1.ObjectMeta{
Name: "service-mcp-backend-precedence",
Namespace: "test-ns",
},
Spec: agentgateway.AgentgatewayBackendSpec{
MCP: &agentgateway.MCPBackend{
Targets: []agentgateway.McpTargetSelector{
{
Selector: &agentgateway.McpSelector{
Service: &metav1.LabelSelector{
MatchLabels: map[string]string{
"app": "mcp-server-both",
},
},
},
},
},
},
},
},
inputs: []any{createMockMCPServiceWithBothAnnotations("test-ns", "mcp-service-both", "app=mcp-server-both", "/new/path", "/legacy/path")},
},
{
name: "Service backendRef MCPBackend backend - same namespace",
backend: &agentgateway.AgentgatewayBackend{
@ -774,6 +824,81 @@ func createMockMCPServiceWithProtocol(namespace, serviceName, _ /* labels */, ap
}
}
// createMockMCPServiceWithLegacyAnnotation creates a mock service with the legacy kgateway.dev/mcp-path annotation
func createMockMCPServiceWithLegacyAnnotation(namespace, serviceName, labels, legacyPath string) *corev1.Service {
// Parse labels
labelsMap := make(map[string]string)
if labels != "" {
// Simple parsing for "key=value" format
for _, label := range []string{labels} {
if len(label) > 0 {
parts := []string{"app", "mcp-server-legacy"} // hardcoded for test
if len(parts) == 2 {
labelsMap[parts[0]] = parts[1]
}
}
}
}
return &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: serviceName,
Namespace: namespace,
Labels: labelsMap,
Annotations: map[string]string{
"kgateway.dev/mcp-path": legacyPath,
},
},
Spec: corev1.ServiceSpec{
Ports: []corev1.ServicePort{
{
Name: "mcp",
Port: 8080,
AppProtocol: ptr.Of("agentgateway.dev/mcp"),
},
},
},
}
}
// createMockMCPServiceWithBothAnnotations creates a mock service with both new and legacy annotations to test precedence
func createMockMCPServiceWithBothAnnotations(namespace, serviceName, labels, newPath, legacyPath string) *corev1.Service {
// Parse labels
labelsMap := make(map[string]string)
if labels != "" {
// Simple parsing for "key=value" format
for _, label := range []string{labels} {
if len(label) > 0 {
parts := []string{"app", "mcp-server-both"} // hardcoded for test
if len(parts) == 2 {
labelsMap[parts[0]] = parts[1]
}
}
}
}
return &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: serviceName,
Namespace: namespace,
Labels: labelsMap,
Annotations: map[string]string{
"agentgateway.dev/mcp-path": newPath,
"kgateway.dev/mcp-path": legacyPath,
},
},
Spec: corev1.ServiceSpec{
Ports: []corev1.ServicePort{
{
Name: "mcp",
Port: 8080,
AppProtocol: ptr.Of("agentgateway.dev/mcp"),
},
},
},
}
}
// createMockMCPService creates a mock service collection with a specific MCPBackend service
func createMockMCPService(namespace, serviceName, labels string) *corev1.Service {
// Parse labels
@ -801,7 +926,7 @@ func createMockMCPService(namespace, serviceName, labels string) *corev1.Service
{
Name: "mcp",
Port: 8080,
AppProtocol: ptr.Of("kgateway.dev/mcp"),
AppProtocol: ptr.Of("agentgateway.dev/mcp"),
},
},
},
@ -825,7 +950,7 @@ func createMockMultipleNamespaceServices() []any {
{
Name: "mcp",
Port: 8080,
AppProtocol: ptr.Of("kgateway.dev/mcp"),
AppProtocol: ptr.Of("agentgateway.dev/mcp"),
},
},
},
@ -843,7 +968,7 @@ func createMockMultipleNamespaceServices() []any {
{
Name: "mcp",
Port: 8080,
AppProtocol: ptr.Of("kgateway.dev/mcp"),
AppProtocol: ptr.Of("agentgateway.dev/mcp"),
},
},
},
@ -861,7 +986,7 @@ func createMockMultipleNamespaceServices() []any {
{
Name: "mcp",
Port: 8080,
AppProtocol: ptr.Of("kgateway.dev/mcp"),
AppProtocol: ptr.Of("agentgateway.dev/mcp"),
},
},
},

View File

@ -14,22 +14,32 @@ import (
type PolicyStatusCollections = map[schema.GroupKind]krt.StatusCollection[controllers.Object, any]
func AgwPolicyCollection(agwPlugins plugins.AgwPlugin, references plugins.ReferenceIndex, krtopts krtutil.KrtOptions) (krt.Collection[ir.AgwResource], krt.Collection[*plugins.PolicyAttachment], PolicyStatusCollections) {
var allPolicies []krt.Collection[plugins.AgwPolicy]
// CollectPolicyReferences collects backend references from all plugins without
// building policies. This allows the reference index to be fully populated
// (including PolicyAttachments from e.g. ext_proc backendRefs) before policies
// like BackendTLSPolicy run and need to look up gateways for backends.
func CollectPolicyReferences(agwPlugins plugins.AgwPlugin, references plugins.ReferenceIndex, krtopts krtutil.KrtOptions) krt.Collection[*plugins.PolicyAttachment] {
var allReferences []krt.Collection[*plugins.PolicyAttachment]
policyStatusMap := PolicyStatusCollections{}
// Collect all policies from registered plugins.
// Note: Only one plugin should be used per source GVK.
// Avoid joining collections per-GVK before passing them to a plugin.
for gvk, plugin := range agwPlugins.ContributesPolicies {
policy, policyStatus, refs := plugin.ApplyPolicies(plugins.PolicyPluginInput{References: references})
if refs != nil {
allReferences = append(allReferences, refs)
for _, plugin := range agwPlugins.ContributesPolicies {
if plugin.BuildReferences != nil {
refs := plugin.BuildReferences(plugins.PolicyPluginInput{References: references})
if refs != nil {
allReferences = append(allReferences, refs)
}
}
allPolicies = append(allPolicies, policy)
if policyStatus != nil {
// some plugins may not have a status collection (a2a services, etc.)
policyStatusMap[gvk] = policyStatus
}
return krt.JoinCollection(allReferences, krtopts.ToOptions("PolicyReferences")...)
}
// BuildPolicies builds all policies using the provided (fully-populated) reference index.
func BuildPolicies(agwPlugins plugins.AgwPlugin, references plugins.ReferenceIndex, krtopts krtutil.KrtOptions) (krt.Collection[ir.AgwResource], PolicyStatusCollections) {
var allPolicies []krt.Collection[plugins.AgwPolicy]
policyStatusMap := PolicyStatusCollections{}
for gvk, plugin := range agwPlugins.ContributesPolicies {
status, col := plugin.Build(plugins.PolicyPluginInput{References: references})
allPolicies = append(allPolicies, col)
if status != nil {
policyStatusMap[gvk] = status
}
}
joinPolicies := krt.JoinCollection(allPolicies, krtopts.ToOptions("JoinPolicies")...)
@ -38,7 +48,5 @@ func AgwPolicyCollection(agwPlugins plugins.AgwPlugin, references plugins.Refere
return ptr.Of(translator.ToResourceForGateway(*i.Gateway, i))
}, krtopts.ToOptions("AllPolicies")...)
allRefsCol := krt.JoinCollection(allReferences, krtopts.ToOptions("PolicyReferences")...)
return allPoliciesCol, allRefsCol, policyStatusMap
return allPoliciesCol, policyStatusMap
}

View File

@ -12,7 +12,6 @@ import (
"istio.io/istio/pkg/cluster"
"istio.io/istio/pkg/config/mesh"
"istio.io/istio/pkg/kube/krt"
"istio.io/istio/pkg/network"
"istio.io/istio/pkg/ptr"
"istio.io/istio/pkg/slices"
"istio.io/istio/pkg/util/sets"
@ -259,18 +258,18 @@ func (s *Syncer) buildFinalListenerSetStatus(
counts[r.ListenerName]++
}
for idx, l := range i.Obj.Spec.Listeners {
gatewayListener := krt.FetchOne(ctx, gatewayIndex, krt.FilterKey(utils.SectionedNamespacedName{
gatewayListeners := krtutil.FetchIndexObjects(ctx, gatewayIndex, utils.SectionedNamespacedName{
NamespacedName: types.NamespacedName{
Namespace: i.Obj.Namespace,
Name: i.Obj.Name,
},
SectionName: l.Name,
}.String()))
if len(gatewayListener.Objects) == 0 {
})
if len(gatewayListeners) == 0 {
continue
}
obj := gatewayListener.Objects[0]
obj := gatewayListeners[0]
if !obj.Valid {
invalidListenerCount++
} else {
@ -480,14 +479,12 @@ func (s *Syncer) buildAgwResources(gateways krt.Collection[*translator.GatewayLi
})
ancestorCollection := ancestorsIndex.AsCollection(append(krtopts.ToOptions("AncestorBackend"), utils.TypedNamespacedNameIndexCollectionFunc)...)
// First, make the policies with access to the route-level attachment references.
referenceIndex := plugins.BuildReferenceIndex(ancestorCollection, routeAttachmentsIndex, referenceTypes)
agwPolicies, policyReferences, policyStatuses := AgwPolicyCollection(s.agwPlugins, referenceIndex, krtopts)
for _, col := range policyStatuses {
status.RegisterStatus(s.statusCollections, col, translator.GetStatus)
}
// Next, build backend references.
// Phase 1: Collect policy references (e.g. ext_proc backendRefs) BEFORE building
// policies. This ensures BackendTLSPolicy can look up gateways for backends that
// are only reachable via PolicyAttachments (like ext_proc processor Services).
policyReferences := CollectPolicyReferences(s.agwPlugins, referenceIndex, krtopts)
backendPolicyReferences := AgwBackendReferencesCollection(s.agwPlugins, krtopts)
joinedPolicyReferences := krt.JoinCollection([]krt.Collection[*plugins.PolicyAttachment]{policyReferences, backendPolicyReferences}, krtopts.ToOptions("JoinPolicyAttachment")...)
policyReferencesIndex := krt.NewIndex(joinedPolicyReferences, "policyReferences", func(o *plugins.PolicyAttachment) []utils.TypedNamespacedName {
@ -496,7 +493,13 @@ func (s *Syncer) buildAgwResources(gateways krt.Collection[*translator.GatewayLi
policyReferencesIndexCollection := policyReferencesIndex.AsCollection(append(krtopts.ToOptions("PolicyReferencesIndex"), utils.TypedNamespacedNameIndexCollectionFunc)...)
referenceIndex = referenceIndex.WithPolicyAttachments(policyReferencesIndexCollection)
// Finally, build the backend collection with backend+route references
// Phase 2: Build policies with the fully-populated reference index.
agwPolicies, policyStatuses := BuildPolicies(s.agwPlugins, referenceIndex, krtopts)
for _, col := range policyStatuses {
status.RegisterStatus(s.statusCollections, col, translator.GetStatus)
}
// Build the backend collection with backend+route references
agwBackends, agwBackendStatus := AgwBackendCollection(s.agwPlugins, referenceIndex, krtopts)
for _, col := range agwBackendStatus {
status.RegisterStatus(s.statusCollections, col, translator.GetStatus)
@ -601,17 +604,13 @@ func defaultBuildAddressCollections(cols *plugins.AgwCollections, krtopts krtuti
ClusterID: clusterId,
}, opts)
builder := ambient.Builder{
DomainSuffix: kubeutils.GetClusterDomainName(),
ClusterID: clusterId,
NetworkGateways: Networks.NetworkGateways,
GatewaysByNetwork: Networks.GatewaysByNetwork,
DomainSuffix: kubeutils.GetClusterDomainName(),
ClusterID: clusterId,
Networks: Networks,
Flags: ambient.FeatureFlags{
EnableK8SServiceSelectWorkloadEntries: true,
EnableMtlsTransportProtocol: true,
},
Network: func(ctx krt.HandlerContext) network.ID {
return ""
},
}
meshConfigMapName := GetMeshConfigMapName(cols.IstioRevision)

View File

@ -16,6 +16,7 @@ import (
gwv1 "sigs.k8s.io/gateway-api/apis/v1"
"sigs.k8s.io/yaml"
apitests "github.com/agentgateway/agentgateway/controller/api/tests"
"github.com/agentgateway/agentgateway/controller/api/v1alpha1/agentgateway"
"github.com/agentgateway/agentgateway/controller/pkg/agentgateway/plugins"
"github.com/agentgateway/agentgateway/controller/pkg/apiclient"
@ -138,33 +139,22 @@ func (dt DeployerTester) GetObjects(
tt HelmTestCase,
scheme *runtime.Scheme,
dir string,
crdDir string,
) []client.Object {
filePath := filepath.Join(dir, "testdata/", tt.InputFile)
inputFile := filePath + ".yaml"
gvkToStructuralSchema, err := testutils.GetStructuralSchemas(crdDir)
require.NoError(t, err, "error getting structural schemas")
validator := apitests.NewAgentgatewayValidatorSkipMissing(t)
objs, err := testutils.LoadFromFiles(inputFile, scheme, gvkToStructuralSchema)
objs, err := testutils.LoadFromFiles(inputFile, scheme, validator)
require.NoError(t, err, "error loading files from input file")
return objs
}
func (dt DeployerTester) RunHelmChartTest(
t *testing.T,
tt HelmTestCase,
scheme *runtime.Scheme,
dir string,
crdDir string,
fakeClient apiclient.Client,
) {
func (dt DeployerTester) RunHelmChartTest(t *testing.T, tt HelmTestCase, scheme *runtime.Scheme, dir string, fakeClient apiclient.Client, objs []client.Object) {
filePath := filepath.Join(dir, "testdata/", tt.InputFile)
outputFile := filePath + "-out.yaml"
objs := dt.GetObjects(t, tt, scheme, dir, crdDir)
_, gtw := ExtractCommonObjs(t, objs)
if gtw == nil {
t.Log("No Gateway found in test files, failing...")

View File

@ -35,7 +35,6 @@ import (
"github.com/agentgateway/agentgateway/controller/pkg/utils/fsutils"
"github.com/agentgateway/agentgateway/controller/pkg/version"
"github.com/agentgateway/agentgateway/controller/pkg/wellknown"
"github.com/agentgateway/agentgateway/controller/test/testutils"
)
func mockVersion(t *testing.T) {
@ -447,14 +446,14 @@ wIDAQABMA0GCSqGSIb3DQEBCwUAA4IBAQBtestcertdata
dir := fsutils.MustGetThisDir()
scheme := schemes.GatewayScheme()
crdDir := filepath.Join(testutils.ControllerRootDirectory(), testutils.AgwCRDPath)
VerifyAllYAMLFilesReferenced(t, filepath.Join(dir, "testdata"), tests)
for _, tt := range tests {
t.Run(tt.Name, func(t *testing.T) {
fakeClient := fake.NewClient(t, tester.GetObjects(t, tt, scheme, dir, crdDir)...)
tester.RunHelmChartTest(t, tt, scheme, dir, crdDir, fakeClient)
objs := tester.GetObjects(t, tt, scheme, dir)
fakeClient := fake.NewClient(t, objs...)
tester.RunHelmChartTest(t, tt, scheme, dir, fakeClient, objs)
})
}
}

View File

@ -14,21 +14,21 @@ import (
"github.com/Masterminds/semver/v3"
"github.com/stretchr/testify/suite"
"istio.io/istio/pkg/config/crd"
"istio.io/istio/pkg/maps"
"istio.io/istio/pkg/test/util/assert"
"istio.io/istio/pkg/test/util/yml"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
apiserverschema "k8s.io/apiextensions-apiserver/pkg/apiserver/schema"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/runtime/serializer/yaml"
"sigs.k8s.io/controller-runtime/pkg/client"
gwv1 "sigs.k8s.io/gateway-api/apis/v1"
apitests "github.com/agentgateway/agentgateway/controller/api/tests"
"github.com/agentgateway/agentgateway/controller/test/e2e"
"github.com/agentgateway/agentgateway/controller/test/e2e/defaults"
"github.com/agentgateway/agentgateway/controller/test/testutils"
@ -82,7 +82,7 @@ var (
)
// selfManagedGatewayAnnotation is the annotation used to mark a Gateway as self-managed in e2e tests
const selfManagedGatewayAnnotation = "e2e.kgateway.dev/self-managed"
const selfManagedGatewayAnnotation = "e2e.agentgateway.dev/self-managed"
// TestCase defines the manifests and resources used by a test or test suite.
type TestCase struct {
@ -131,7 +131,7 @@ type BaseTestingSuite struct {
CrdPath string
// used internally to parse the manifest files
gvkToStructuralSchema map[schema.GroupVersionKind]*apiserverschema.Structural
validator *crd.Validator
// gwApiVersion stores the detected Gateway API version (detected once and cached)
gwApiVersion *semver.Version
@ -471,9 +471,7 @@ func (s *BaseTestingSuite) setupHelpers() {
if s.CrdPath == "" {
s.CrdPath = testutils.AgwCRDPath
}
var err error
s.gvkToStructuralSchema, err = testutils.GetStructuralSchemasForBothCharts()
s.Require().NoError(err)
s.validator = apitests.NewAgentgatewayValidatorSkipMissing(s.T())
}
// loadManifestResources populates the `manifestResources` for the given test case, by parsing each
@ -486,14 +484,14 @@ func (s *BaseTestingSuite) loadManifestResources(testCase *TestCase) {
var resources []client.Object
for _, manifest := range testCase.Manifests {
objs, err := testutils.LoadFromFiles(manifest, s.TestInstallation.ClusterContext.Client.Scheme(), s.gvkToStructuralSchema)
objs, err := testutils.LoadFromFiles(manifest, s.TestInstallation.ClusterContext.Client.Scheme(), s.validator)
s.Require().NoError(err)
resources = append(resources, objs...)
}
for manifest, transform := range testCase.ManifestsWithTransform {
// we don't need to transform the resource since the transformation applies to the spec and not object metadata,
// which ensures that parsed Go objects in manifestResources can be used normally
objs, err := testutils.LoadFromFileWithTransform(manifest, s.TestInstallation.ClusterContext.Client.Scheme(), s.gvkToStructuralSchema, transform)
objs, err := testutils.LoadFromFileWithTransform(manifest, s.TestInstallation.ClusterContext.Client.Scheme(), s.validator, transform)
s.Require().NoError(err)
resources = append(resources, objs...)
}

View File

@ -188,7 +188,7 @@ spec:
- protocol: TCP
port: 9999
targetPort: 9999
appProtocol: kgateway.dev/a2a
appProtocol: agentgateway.dev/a2a
---
apiVersion: v1
kind: Service
@ -231,7 +231,7 @@ spec:
ports:
- port: 80
targetPort: 8000
appProtocol: kgateway.dev/mcp
appProtocol: agentgateway.dev/mcp
---
apiVersion: v1
kind: Service
@ -246,7 +246,7 @@ spec:
- name: http
port: 80
targetPort: 3001
appProtocol: kgateway.dev/mcp
appProtocol: agentgateway.dev/mcp
type: ClusterIP
---
apiVersion: v1

View File

@ -294,6 +294,13 @@ spec:
serviceAccountName: test-release-agentgateway
containers:
- name: controller
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
image: "cr.agentgateway.dev/controller:v0.0.1"
imagePullPolicy: IfNotPresent
ports:

View File

@ -288,6 +288,13 @@ spec:
serviceAccountName: test-release-agentgateway
containers:
- name: controller
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
image: "cr.agentgateway.dev/controller:v0.0.1"
imagePullPolicy: IfNotPresent
ports:

View File

@ -288,6 +288,13 @@ spec:
serviceAccountName: test-release-agentgateway
containers:
- name: controller
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
image: "cr.agentgateway.dev/controller:v0.0.1"
imagePullPolicy: IfNotPresent
ports:

View File

@ -288,6 +288,13 @@ spec:
serviceAccountName: test-release-agentgateway
containers:
- name: controller
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
image: "cr.agentgateway.dev/controller:v0.0.1"
imagePullPolicy: IfNotPresent
ports:

View File

@ -309,6 +309,13 @@ spec:
serviceAccountName: test-release-agentgateway
containers:
- name: controller
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
image: "cr.agentgateway.dev/controller:v0.0.1"
imagePullPolicy: IfNotPresent
ports:

View File

@ -309,6 +309,13 @@ spec:
serviceAccountName: test-release-agentgateway
containers:
- name: controller
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
image: "cr.agentgateway.dev/controller:v0.0.1"
imagePullPolicy: IfNotPresent
ports:

View File

@ -289,6 +289,13 @@ spec:
serviceAccountName: test-release-agentgateway
containers:
- name: controller
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
image: "cr.agentgateway.dev/controller:v0.0.1"
imagePullPolicy: IfNotPresent
ports:

View File

@ -316,6 +316,13 @@ spec:
serviceAccountName: test-release-agentgateway
containers:
- name: controller
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
image: "cr.agentgateway.dev/controller:v0.0.1"
imagePullPolicy: IfNotPresent
ports:

View File

@ -288,6 +288,13 @@ spec:
serviceAccountName: test-release-agentgateway
containers:
- name: controller
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
image: "cr.agentgateway.dev/controller:v0.0.1"
imagePullPolicy: IfNotPresent
ports:

View File

@ -11,8 +11,7 @@ import (
"strings"
"github.com/ghodss/yaml"
apiserverschema "k8s.io/apiextensions-apiserver/pkg/apiserver/schema"
apiextensionsvalidation "k8s.io/apiextensions-apiserver/pkg/apiserver/validation"
"istio.io/istio/pkg/config/crd"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/schema"
@ -25,18 +24,14 @@ var ErrNoFilesFound = errors.New("no k8s files found")
// FileContentTransformer is a function that transforms a file's contents
type FileContentTransformer func(content string) string
func LoadFromFiles(
filename string,
scheme *runtime.Scheme,
gvkToStructuralSchema map[schema.GroupVersionKind]*apiserverschema.Structural,
) ([]client.Object, error) {
return LoadFromFileWithTransform(filename, scheme, gvkToStructuralSchema, nil)
func LoadFromFiles(filename string, scheme *runtime.Scheme, validator *crd.Validator) ([]client.Object, error) {
return LoadFromFileWithTransform(filename, scheme, validator, nil)
}
func LoadFromFileWithTransform(
filename string,
scheme *runtime.Scheme,
gvkToStructuralSchema map[schema.GroupVersionKind]*apiserverschema.Structural,
validator *crd.Validator,
transformer FileContentTransformer,
) ([]client.Object, error) {
fileOrDir, err := os.Stat(filename)
@ -68,7 +63,7 @@ func LoadFromFileWithTransform(
var resources []client.Object
for _, file := range yamlFiles {
objs, err := parseFile(file, scheme, gvkToStructuralSchema, transformer)
objs, err := parseFile(file, scheme, validator, transformer)
if err != nil {
return nil, err
}
@ -93,7 +88,7 @@ func LoadFromFileWithTransform(
func parseFile(
filename string,
scheme *runtime.Scheme,
gvkToStructuralSchema map[schema.GroupVersionKind]*apiserverschema.Structural,
validator *crd.Validator,
transformer FileContentTransformer,
) ([]runtime.Object, error) {
file, err := os.ReadFile(filename)
@ -150,21 +145,8 @@ func parseFile(
)
continue
}
if structuralSchema, ok := gvkToStructuralSchema[gvk]; ok {
unstructuredObj, objYamlWithDefaults, err := ApplyDefaults(objYaml, structuralSchema)
if err != nil {
return nil, fmt.Errorf("failed to apply defaults for %s: %w", gvk, err)
}
validator := apiextensionsvalidation.NewSchemaValidatorFromOpenAPI(structuralSchema.ToKubeOpenAPI())
validationErrs := apiextensionsvalidation.ValidateCustomResource(nil, unstructuredObj.UnstructuredContent(), validator)
if len(validationErrs) > 0 {
agg := validationErrs.ToAggregate()
return nil, fmt.Errorf("failed to validate %s: %w", gvk, agg)
}
if err := yaml.Unmarshal(objYamlWithDefaults, obj); err != nil {
return nil, fmt.Errorf("failed to unmarshal object with defaults for %s: %w", gvk, err)
}
if err := validator.ValidateCustomResource(obj); err != nil {
return nil, err
}
genericResources = append(genericResources, obj)

View File

@ -21,7 +21,6 @@ agentgateway.workspace = true
anyhow.workspace = true
clap.workspace = true
fs-err = { workspace = true, features = ["tokio"] }
lazy_static.workspace = true
libc.workspace = true
tokio.workspace = true
tracing.workspace = true

View File

@ -130,7 +130,6 @@ tokio-rustls.workspace = true
tokio-stream.workspace = true
tokio-util.workspace = true
tokio_sse_codec.workspace = true
tonic-prost.workspace = true
tonic.workspace = true
tower-http.workspace = true
tower-serve-static.workspace = true

View File

@ -172,19 +172,7 @@ impl Bound {
b.all_shutdown_policies()
};
for p in sdp {
if let Some(t) = p.tracer.get() {
t.shutdown()
}
}
let access_log_policies = {
let b = self.stores.binds.read();
b.all_access_log_policies()
};
for p in access_log_policies {
if let Some(logger) = p.logger.get() {
logger.shutdown()
}
p();
}
// Start a drain; this will attempt to end all connections

View File

@ -312,13 +312,13 @@ impl<'a> Executor<'a> {
}
pub fn new_tcp_logger(
source_context: Option<&'a SourceContext>,
end_time: Option<&'a RequestTime>,
end_time: &'a RequestTime,
) -> Self {
let mut this = Self::new_empty();
// For TCP connections, set the source context directly
this.source = ExtensionOrDirect::Direct(source_context);
if let Some(f) = this.request.as_mut() {
f.end_time = end_time;
f.end_time = Some(end_time);
}
this
}
@ -788,6 +788,20 @@ pub struct LLMContext {
#[dynamic(rename = "inputTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
pub input_tokens: Option<u64>,
/// The number of image tokens in the input/prompt.
#[dynamic(rename = "inputImageTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
pub input_image_tokens: Option<u64>,
/// The number of text tokens in the input/prompt.
/// Note: this field is only set in multi-modal calls where the total token count is split out by
/// text/image/audio; for standard all-text calls, this is unset.
#[dynamic(rename = "inputTextTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
pub input_text_tokens: Option<u64>,
/// The number of audio tokens in the input/prompt.
#[dynamic(rename = "inputAudioTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
pub input_audio_tokens: Option<u64>,
/// The number of tokens in the input/prompt read from cache (savings)
#[dynamic(rename = "cachedInputTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
@ -801,6 +815,20 @@ pub struct LLMContext {
#[dynamic(rename = "outputTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
pub output_tokens: Option<u64>,
/// The number of image tokens in the output/completion.
#[dynamic(rename = "outputImageTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
pub output_image_tokens: Option<u64>,
/// The number of text tokens in the output/completion.
#[dynamic(rename = "outputTextTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
pub output_text_tokens: Option<u64>,
/// The number of audio tokens in the output/completion.
/// Note: this field is only set in multi-modal calls where the total token count is split out by
/// text/image/audio; for standard all-text calls, this is unset.
#[dynamic(rename = "outputAudioTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
pub output_audio_tokens: Option<u64>,
/// The number of reasoning tokens in the output/completion.
#[dynamic(rename = "reasoningTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
@ -809,6 +837,10 @@ pub struct LLMContext {
#[dynamic(rename = "totalTokens")]
#[serde(skip_serializing_if = "Option::is_none")]
pub total_tokens: Option<u64>,
/// The service tier the provider served the request under.
#[dynamic(rename = "serviceTier")]
#[serde(skip_serializing_if = "Option::is_none")]
pub service_tier: Option<Strng>,
// For now, not exposed to CEL; only used to piggy-back this field for metrics.
#[serde(skip)]
#[dynamic(skip)]
@ -833,12 +865,19 @@ impl From<llm::LLMInfo> for LLMContext {
let resp = value.response;
let mut base = LLMContext {
output_tokens: resp.output_tokens,
output_image_tokens: resp.output_image_tokens,
output_text_tokens: resp.output_text_tokens,
output_audio_tokens: resp.output_audio_tokens,
count_tokens: resp.count_tokens,
total_tokens: resp.total_tokens,
first_token: resp.first_token,
reasoning_tokens: resp.reasoning_tokens,
input_image_tokens: resp.input_image_tokens,
input_text_tokens: resp.input_text_tokens,
input_audio_tokens: resp.input_audio_tokens,
cached_input_tokens: resp.cached_input_tokens,
cache_creation_input_tokens: resp.cache_creation_input_tokens,
service_tier: resp.service_tier.clone(),
response_model: resp.provider_model.clone(),
// Not always set
completion: resp.completion.clone(),
@ -875,11 +914,18 @@ impl From<llm::LLMRequest> for LLMContext {
count_tokens: None,
response_model: None,
output_tokens: None,
output_image_tokens: None,
output_text_tokens: None,
output_audio_tokens: None,
total_tokens: None,
completion: None,
reasoning_tokens: None,
input_image_tokens: None,
input_text_tokens: None,
input_audio_tokens: None,
cached_input_tokens: None,
cache_creation_input_tokens: None,
service_tier: None,
}
}
}
@ -1446,11 +1492,18 @@ pub fn full_example_executor() -> ExecutorSerde {
response_model: Some("gpt-4-turbo".into()),
provider: "fake-ai".into(),
input_tokens: Some(100),
input_image_tokens: Some(60),
input_text_tokens: Some(40),
input_audio_tokens: Some(5),
cached_input_tokens: Some(20),
cache_creation_input_tokens: Some(10),
output_tokens: Some(50),
output_image_tokens: Some(30),
output_text_tokens: Some(20),
output_audio_tokens: Some(3),
reasoning_tokens: Some(30),
total_tokens: Some(150),
service_tier: Some("default".into()),
first_token: None,
count_tokens: Some(10),

View File

@ -52,8 +52,15 @@ fn build_test_request() -> crate::http::Request {
response_model: Some("gpt-4-turbo".into()),
provider: "openai".into(),
input_tokens: Some(100),
input_image_tokens: None,
input_text_tokens: None,
input_audio_tokens: None,
output_tokens: Some(50),
output_image_tokens: None,
output_text_tokens: None,
output_audio_tokens: None,
total_tokens: Some(150),
service_tier: None,
first_token: None,
count_tokens: None,
reasoning_tokens: None,

View File

@ -53,14 +53,7 @@ impl azure_core::http::HttpClient for Client {
.call(Call {
req: request,
target: match url.host().expect("url must have a host") {
url::Host::Domain(h) => Target::try_from((h, url.port_or_known_default().unwrap_or(80)))
.map_err(|e| {
azure_core::Error::with_error(
azure_core::error::ErrorKind::Other,
e,
"failed to parse host for `agentgateway::client::Client` request",
)
})?,
url::Host::Domain(h) => Target::from((h, url.port_or_known_default().unwrap_or(80))),
url::Host::Ipv4(ip) => Target::Address(SocketAddr::from((
ip,
url.port_or_known_default().unwrap_or(80),

View File

@ -389,7 +389,7 @@ impl Client {
backend_config: Arc::new(backend_config),
metrics,
};
let client = b.build_with_pool_key(connector.clone());
let client = b.build(connector.clone());
Client { client, connector }
}
@ -412,8 +412,7 @@ impl Client {
} else {
ApplicationTransport::Plaintext.into()
};
let target = Target::try_from((host, port))
.map_err(|e| ProxyError::ProcessingString(format!("failed to parse host: {e}")))?;
let target = Target::from((host, port));
self
.call(Call {
req,

View File

@ -249,6 +249,10 @@ pub fn parse_config(contents: String, filename: Option<PathBuf>) -> anyhow::Resu
Some(session) => crate::http::sessionpersistence::Encoder::aes(session.key.expose_secret())?,
}
};
// Browser OIDC cookie crypto is core gateway runtime config, not per-policy input.
let oidc_cookie_encoder = parse::<String>("OIDC_COOKIE_SECRET")?
.map(|key| crate::http::sessionpersistence::Encoder::aes(key.trim()))
.transpose()?;
Ok(crate::Config {
ipv6_enabled,
@ -405,6 +409,7 @@ pub fn parse_config(contents: String, filename: Option<PathBuf>) -> anyhow::Resu
node_id: ENV.node_id.clone(),
},
session_encoder,
oidc_cookie_encoder,
hbone: Arc::new(agent_hbone::Config {
// window size: per-stream limit
window_size: parse("HTTP2_STREAM_WINDOW_SIZE")?

View File

@ -184,7 +184,7 @@ impl WorkloadCertificate {
pub fn hbone_termination(&self) -> Result<ServerConfig, Error> {
let Identity::Spiffe { trust_domain, .. } = &self.identity;
// TODO: this istoo expensive to build per request
// TODO: this is too expensive to build per request
let roots = self.roots.clone();
let raw_client_cert_verifier = rustls::server::WebPkiClientVerifier::builder_with_provider(
roots.clone(),
@ -195,7 +195,7 @@ impl WorkloadCertificate {
raw_client_cert_verifier,
Some(trust_domain.clone()),
);
let sc = ServerConfig::builder_with_provider(transport::tls::provider())
let mut sc = ServerConfig::builder_with_provider(transport::tls::provider())
.with_protocol_versions(transport::tls::ALL_TLS_VERSIONS)
.expect("server config must be valid")
.with_client_cert_verifier(client_cert_verifier)
@ -203,6 +203,7 @@ impl WorkloadCertificate {
self.chain.iter().map(|c| c.der.clone()).collect(),
self.private_key.clone_key(),
)?;
sc.alpn_protocols = vec![b"h2".into()];
Ok(sc)
}
}

View File

@ -747,24 +747,27 @@ mod azure {
AzureAuthCredentialSource::ManagedIdentity {
user_assigned_identity,
} => {
let options: Option<azure_identity::ManagedIdentityCredentialOptions> =
user_assigned_identity.as_ref().map(|uami| {
azure_identity::ManagedIdentityCredentialOptions {
user_assigned_id: match uami {
AzureUserAssignedIdentity::ClientId(cid) => {
Some(UserAssignedId::ClientId(cid.to_string()))
},
AzureUserAssignedIdentity::ObjectId(oid) => {
Some(UserAssignedId::ObjectId(oid.to_string()))
},
AzureUserAssignedIdentity::ResourceId(rid) => {
Some(UserAssignedId::ResourceId(rid.to_string()))
},
},
client_options,
}
});
Ok(azure_identity::ManagedIdentityCredential::new(options)?)
// Always construct ManagedIdentityCredentialOptions so that the
// custom reqwest-backed transport is injected regardless of whether
// a user-assigned identity is specified.
//
// Before this fix, .map() short-circuited to None for system-assigned
// identity (managedIdentity: {}), causing ManagedIdentityCredential::new(None)
// to fall back to NoopClient which panics on every IMDS request.
// See: https://github.com/agentgateway/agentgateway/issues/900
let options = azure_identity::ManagedIdentityCredentialOptions {
user_assigned_id: user_assigned_identity.as_ref().map(|uami| match uami {
AzureUserAssignedIdentity::ClientId(cid) => UserAssignedId::ClientId(cid.to_string()),
AzureUserAssignedIdentity::ObjectId(oid) => UserAssignedId::ObjectId(oid.to_string()),
AzureUserAssignedIdentity::ResourceId(rid) => {
UserAssignedId::ResourceId(rid.to_string())
},
}),
client_options,
};
Ok(azure_identity::ManagedIdentityCredential::new(Some(
options,
))?)
},
AzureAuthCredentialSource::WorkloadIdentity {} => {
Ok(azure_identity::WorkloadIdentityCredential::new(Some(

View File

@ -2,6 +2,7 @@ use serde::ser::SerializeMap;
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use crate::cel::{ContextBuilder, Executor};
use crate::proxy::ProxyError;
use crate::*;
#[derive(Clone, Debug)]
@ -34,13 +35,14 @@ impl NetworkAuthorizationSet {
Self(rs)
}
pub fn apply(&self, source: &crate::cel::SourceContext) -> anyhow::Result<()> {
pub fn apply(&self, source: &crate::cel::SourceContext) -> Result<(), ProxyError> {
let exec = Executor::new_source(source);
let allowed = self.0.validate(&exec);
if !allowed {
anyhow::bail!("network authorization denied");
Err(ProxyError::AuthorizationFailed)
} else {
Ok(())
}
Ok(())
}
pub fn register(&self, cel: &mut ContextBuilder) {

View File

@ -1,5 +1,4 @@
use std::convert::Infallible;
use std::sync::atomic::{AtomicBool, Ordering};
use ::http::HeaderMap;
use anyhow::anyhow;
@ -18,6 +17,7 @@ use tokio_stream::wrappers::ReceiverStream;
use crate::cel::{Executor, Expression, RequestSnapshot};
use crate::client::ResolvedDestination;
use crate::http;
use crate::http::envoy_proto_common;
use crate::http::ext_proc::proto::{
BodyMutation, BodyResponse, HeaderMutation, HeaderValueOption, HeadersResponse, HttpBody,
@ -243,7 +243,7 @@ impl ExtProcRequest {
#[derive(Debug)]
struct ExtProcInstance {
failure_mode: FailureMode,
skipped: Arc<AtomicBool>,
skipped: bool,
tx_req: Sender<ProcessingRequest>,
rx_resp_for_request: Option<Receiver<ProcessingResponse>>,
rx_resp_for_response: Option<Receiver<ProcessingResponse>>,
@ -293,7 +293,6 @@ impl ExtProcInstance {
let (tx_resp_for_response, rx_resp_for_response) = tokio::sync::mpsc::channel(1);
tokio::task::spawn(async move {
while let Some(item) = rx_resp.recv().await {
trace!("received response item {item:?}");
match &item.response {
Some(processing_response::Response::ResponseBody(_))
| Some(processing_response::Response::ResponseHeaders(_))
@ -336,7 +335,6 @@ impl ExtProcInstance {
req: http::Request,
) -> Result<(http::Request, Option<PolicyResponse>), Error> {
let headers = req_to_header_map(&req);
let buffer = http::buffer_limit(&req);
let exec = cel::Executor::new_request(&req);
// request_attributes should only be sent on first ProcessingRequest
@ -362,23 +360,11 @@ impl ExtProcInstance {
})
.unwrap_or_default();
let (parts, body) = req.into_parts();
// For fail open we need a copy of the body. There is definitely a better way to do this, but for
// now its good enough?
let (body_copy, body) = if self.failure_mode == FailureMode::FailOpen {
let buffered = http::read_body_with_limit(body, buffer)
.await
.map_err(|e| Error::BodyBuffer(e.to_string()))?;
(Some(buffered.clone()), http::Body::from(buffered))
} else {
(None, body)
};
let end_of_stream = body.is_end_stream();
let had_body = !end_of_stream;
let failure_mode = self.failure_mode;
let end_of_stream = req.body().is_end_stream();
// Send the request headers to ext_proc.
self
if let Err(e) = self
.send_request(ProcessingRequest {
request: Some(Request::RequestHeaders(HttpHeaders {
headers,
@ -389,140 +375,140 @@ impl ExtProcInstance {
protocol_config: Default::default(),
observability_mode: false,
})
.await?;
// The EPP will await for our headers and body. The body is going to be streaming in.
// We will spin off a task that is going to pipe the body to the ext_proc server as we read it.
let tx = self.tx_req.clone();
if had_body {
tokio::task::spawn(async move {
let mut stream = BodyStream::new(body);
while let Some(Ok(frame)) = stream.next().await {
let request = Some(if frame.is_data() {
let frame = frame.into_data().expect("already checked");
trace!("sending request body chunk...",);
Request::RequestBody(HttpBody {
body: frame.into(),
end_of_stream: false,
})
} else if frame.is_trailers() {
let frame = frame.into_trailers().expect("already checked");
Request::RequestTrailers(HttpTrailers {
trailers: to_header_map(&frame),
})
} else {
// http_body::Frame only has data and trailers variants
unreachable!("Frame is neither data nor trailers")
});
trace!("sending request body chunk...");
let Ok(()) = tx
.send(ProcessingRequest {
request,
metadata_context: metadata_context.as_deref().cloned(),
attributes: Default::default(),
protocol_config: Default::default(),
observability_mode: false,
})
.await
else {
// TODO: on error here we need a way to signal to the outer task to fail fast
return;
};
}
// Send end of stream marker - try to unwrap Arc to avoid final clone
let final_metadata = metadata_context.and_then(Arc::into_inner);
let _ = tx
.send(ProcessingRequest {
request: Some(Request::RequestBody(HttpBody {
body: Default::default(),
end_of_stream: true,
})),
metadata_context: final_metadata,
attributes: Default::default(),
protocol_config: Default::default(),
observability_mode: false,
})
.await;
trace!("body request done");
});
.await
{
if failure_mode == FailureMode::FailOpen {
trace!("fail open triggered");
self.skipped = true;
return Ok((req, None));
}
return Err(e);
}
// Now we need to build the new body. This is going to be streamed in from the ext_proc server.
let (tx_chunk, rx_chunk) = tokio::sync::mpsc::channel(1);
let body = http_body_util::StreamBody::new(ReceiverStream::new(rx_chunk));
let mut req = http::Request::from_parts(parts, http::Body::new(body));
// At this point, we start body handling. Fail open + streaming bodies is a disaster,
// as we could silently corrupt data.
// We behave approximately like Envoy here (https://github.com/envoyproxy/envoy/pull/41276); after
// the headers are sent we drop fail_open.
// In practice, this means that the server was running fine at the start of the request which covers
// all but edge cases around the server dying mid-request.
let (parts, body) = req.into_parts();
let had_body = !end_of_stream;
let tx = self.tx_req.clone();
if had_body {
tokio::task::spawn(Self::handle_body_stream(
metadata_context,
body,
tx,
Request::RequestBody,
Request::RequestTrailers,
));
}
// Now we need to build the new body. This is going to be streamed in from the ext_proc server.
let (mut tx_chunk, rx_chunk) = tokio::sync::mpsc::channel(1);
let upstream_body = http_body_util::StreamBody::new(ReceiverStream::new(rx_chunk));
let mut req = http::Request::from_parts(parts, http::Body::new(upstream_body));
req.headers_mut().remove(http::header::CONTENT_LENGTH);
let (tx_done, rx_done) = tokio::sync::oneshot::channel();
let mut rx = self
.rx_resp_for_request
.take()
.expect("mutate_request called twice");
let failure_mode = self.failure_mode;
let skipped = self.skipped.clone();
tokio::task::spawn(async move {
let mut req = Some(req);
let mut tx_done = Some(tx_done);
let mut tx_chunkh = Some(tx_chunk);
loop {
// Loop through all the ext_proc responses and process them
let Some(presp) = rx.recv().await else {
trace!("done receiving request");
if failure_mode == FailureMode::FailOpen
&& let Some(req) = req.take()
&& let Some(tx_done) = tx_done.take()
{
trace!("fail open triggered");
skipped.store(true, Ordering::SeqCst);
let (parts, _) = req.into_parts();
let new_req = http::Request::from_parts(parts, http::Body::from(body_copy.unwrap()));
let _ = tx_done.send(Ok((new_req, None)));
tx_chunkh.take();
}
return;
};
if let Some(resp) = to_immediate_response(&presp) {
trace!("got immediate response in request handler");
let _ = tx_done
.take()
.unwrap()
.send(Ok((http::Request::default(), Some(resp))));
tx_chunkh.take();
return;
}
if let Some(dm) = &presp.dynamic_metadata
&& let Err(e) = Self::extract_dynamic_metadata(req.as_mut(), dm)
{
warn!("Failed to extract ext_proc dynamic metadata: {}", e);
}
let Some(tx_chunk) = tx_chunkh.as_mut() else {
return;
};
let r = handle_response_for_request_mutation(had_body, req.as_mut(), tx_chunk, presp).await;
match r {
Ok((headers_done, eos)) => {
if headers_done
&& let Some(req) = req.take()
&& let Some(tx_done) = tx_done.take()
{
trace!("request complete!");
let _ = tx_done.send(Ok((req, None)));
}
if eos || !had_body {
trace!("request EOS!");
tx_chunkh.take();
}
},
Err(e) => {
warn!("error {e:?}");
return;
},
loop {
let Some(presp) = rx.recv().await else {
if !had_body && failure_mode == FailureMode::FailOpen {
trace!("fail open triggered");
self.skipped = true;
return Ok((req, None));
}
trace!("done receiving request");
return Err(Error::NoMoreResponses);
};
if let Some(resp) = to_immediate_response(&presp) {
trace!("got immediate response in request handler");
return Ok((req, Some(resp)));
}
});
rx_done.await.map_err(|_| Error::ResponseDropped)?
let (headers_done, eos) =
handle_response_for_request_mutation(had_body, Some(&mut req), &mut tx_chunk, presp).await;
if headers_done {
if !eos {
trace!("spawn body!");
// Moving rest of body handling to async
tokio::task::spawn(async move {
loop {
let Some(presp) = rx.recv().await else {
trace!("done receiving request");
return;
};
let (_, eos) =
handle_response_for_request_mutation(had_body, None, &mut tx_chunk, presp).await;
if eos || !had_body {
trace!("request EOS!");
drop(tx_chunk);
return;
}
}
});
}
return Ok((req, None));
}
}
}
async fn handle_body_stream(
metadata_context: Option<Arc<Metadata>>,
body: http::Body,
tx: Sender<ProcessingRequest>,
body_fn: fn(HttpBody) -> Request,
trail_fn: fn(HttpTrailers) -> Request,
) {
let mut stream = BodyStream::new(body);
while let Some(Ok(frame)) = stream.next().await {
let request = Some(if frame.is_data() {
let frame = frame.into_data().expect("already checked");
trace!("sending body chunk...",);
body_fn(HttpBody {
body: frame.into(),
end_of_stream: false,
})
} else if frame.is_trailers() {
let frame = frame.into_trailers().expect("already checked");
trail_fn(HttpTrailers {
trailers: to_header_map(&frame),
})
} else {
// http_body::Frame only has data and trailers variants
unreachable!("Frame is neither data nor trailers")
});
let Ok(()) = tx
.send(ProcessingRequest {
request,
metadata_context: metadata_context.as_deref().cloned(),
attributes: Default::default(),
protocol_config: Default::default(),
observability_mode: false,
})
.await
else {
return;
};
}
// Send end of stream marker - try to unwrap Arc to avoid final clone
let final_metadata = metadata_context.and_then(Arc::into_inner);
let _ = tx
.send(ProcessingRequest {
request: Some(body_fn(HttpBody {
body: Default::default(),
end_of_stream: true,
})),
metadata_context: final_metadata,
attributes: Default::default(),
protocol_config: Default::default(),
observability_mode: false,
})
.await;
trace!("body request done");
}
pub async fn mutate_response(
@ -531,7 +517,7 @@ impl ExtProcInstance {
request: Option<&RequestSnapshot>,
resolved_destination_metadata: Option<SocketAddr>,
) -> Result<(http::Response, Option<PolicyResponse>), Error> {
if self.skipped.load(Ordering::SeqCst) {
if self.skipped {
return Ok((req, None));
}
let headers = resp_to_header_map(&req);
@ -575,6 +561,7 @@ impl ExtProcInstance {
let had_body = !end_of_stream;
// Send the response headers to ext_proc.
// No response side fail_open handling.
self
.send_request(ProcessingRequest {
request: Some(Request::ResponseHeaders(HttpHeaders {
@ -591,156 +578,61 @@ impl ExtProcInstance {
// The EPP will await for our headers and body. The body is going to be streaming in.
// We will spin off a task that is going to pipe the body to the ext_proc server as we read it.
let tx = self.tx_req.clone();
if had_body {
tokio::task::spawn(async move {
let mut stream = BodyStream::new(body);
while let Some(Ok(frame)) = stream.next().await {
let request = Some(if frame.is_data() {
let frame = frame.into_data().expect("already checked");
Request::ResponseBody(HttpBody {
body: frame.into(),
end_of_stream: false,
})
} else if frame.is_trailers() {
let frame = frame.into_trailers().expect("already checked");
Request::ResponseTrailers(HttpTrailers {
trailers: to_header_map(&frame),
})
} else {
// http_body::Frame only has data and trailers variants
unreachable!("Frame is neither data nor trailers")
});
trace!("sending response body chunk...");
let Ok(()) = tx
.send(ProcessingRequest {
request,
metadata_context: metadata_context.as_deref().cloned(),
attributes: HashMap::new(),
protocol_config: Default::default(),
observability_mode: false,
})
.await
else {
// TODO: on error here we need a way to signal to the outer task to fail fast
return;
};
}
// Send end of stream marker - try to unwrap Arc to avoid final clone
let final_metadata = metadata_context.and_then(Arc::into_inner);
let _ = tx
.send(ProcessingRequest {
request: Some(Request::ResponseBody(HttpBody {
body: Default::default(),
end_of_stream: true,
})),
metadata_context: final_metadata,
attributes: HashMap::new(),
protocol_config: Default::default(),
observability_mode: false,
})
.await;
trace!("body response done");
});
tokio::task::spawn(Self::handle_body_stream(
metadata_context,
body,
tx,
Request::ResponseBody,
Request::ResponseTrailers,
));
}
// Now we need to build the new body. This is going to be streamed in from the ext_proc server.
let (tx_chunk, rx_chunk) = tokio::sync::mpsc::channel(1);
let (mut tx_chunk, rx_chunk) = tokio::sync::mpsc::channel(1);
let body = http_body_util::StreamBody::new(ReceiverStream::new(rx_chunk));
let mut resp = http::Response::from_parts(parts, http::Body::new(body));
resp.headers_mut().remove(http::header::CONTENT_LENGTH);
let (tx_done, rx_done) = tokio::sync::oneshot::channel();
let mut rx = self
.rx_resp_for_response
.take()
.expect("mutate_request called twice");
tokio::task::spawn(async move {
let mut resp = Some(resp);
let mut tx_done = Some(tx_done);
let mut tx_chunkh = Some(tx_chunk);
loop {
// Loop through all the ext_proc responses and process them
let Some(presp) = rx.recv().await else {
trace!("done receiving response");
return;
};
if let Some(resp) = to_immediate_response(&presp) {
trace!("got immediate response in response handler");
let _ = tx_done
.take()
.unwrap()
.send(Ok((http::Response::default(), Some(resp))));
tx_chunkh.take();
return;
}
let Some(tx_chunk) = tx_chunkh.as_mut() else {
trace!("body done, skipping");
return;
};
let r =
handle_response_for_response_mutation(had_body, resp.as_mut(), tx_chunk, presp).await;
match r {
Ok((headers_done, eos)) => {
if headers_done
&& let Some(resp) = resp.take()
&& let Some(tx_done) = tx_done.take()
{
trace!("response complete!");
let _ = tx_done.send(Ok((resp, None)));
}
if eos || !had_body {
trace!("response EOS!");
tx_chunkh.take();
}
},
Err(e) => {
warn!("error {e:?}");
return;
// return tx_done.take().expect("must be called once").send(Err(e));
},
}
.expect("mutate_response called twice");
loop {
let Some(presp) = rx.recv().await else {
trace!("done receiving response");
return Err(Error::NoMoreResponses);
};
if let Some(dr) = to_immediate_response(&presp) {
trace!("got immediate response in request handler");
return Ok((resp, Some(dr)));
}
});
rx_done.await.map_err(|_| Error::ResponseDropped)?
}
pub(crate) fn extract_dynamic_metadata(
req: Option<&mut http::Request>,
metadata: &prost_wkt_types::Struct,
) -> Result<(), Error> {
let Some(req) = req else {
// Warn when metadata is sent after headers are processed (body/trailer phase)
if !metadata.fields.is_empty() {
warn!(
"ext_proc server sent dynamic_metadata after headers were processed; \
metadata cannot be attached and will be ignored. Consider sending \
metadata in the RequestHeaders response instead."
);
let (headers_done, eos) =
handle_response_for_response_mutation(had_body, Some(&mut resp), &mut tx_chunk, presp)
.await;
if headers_done {
if !eos {
trace!("spawn body!");
// Moving rest of body handling to async
tokio::task::spawn(async move {
loop {
let Some(presp) = rx.recv().await else {
trace!("done receiving response");
return;
};
let (_, eos) =
handle_response_for_response_mutation(had_body, None, &mut tx_chunk, presp).await;
if eos || !had_body {
trace!("response EOS!");
drop(tx_chunk);
return;
}
}
});
}
return Ok((resp, None));
}
return Ok(());
};
// Get or create metadata container, merging with existing metadata
let mut dynamic_metadata = req
.extensions_mut()
.remove::<Arc<ExtProcDynamicMetadata>>()
.map(|arc| (*arc).clone())
.unwrap_or_default();
// Merge new fields into existing metadata
for (key, value) in &metadata.fields {
let json_val = envoy_proto_common::prost_value_to_json(value).map_err(|e| {
Error::MetadataConversion(format!("failed to convert key '{}': {}", key, e))
})?;
dynamic_metadata.0.insert(key.clone(), json_val);
}
if !dynamic_metadata.0.is_empty() {
req.extensions_mut().insert(Arc::new(dynamic_metadata));
}
Ok(())
}
}
@ -758,7 +650,7 @@ fn to_immediate_response(rp: &ProcessingResponse) -> Option<PolicyResponse> {
::http::response::Builder::new().status(status.map(|s| s.code).unwrap_or(200) as u16);
if let Some(hm) = rb.headers_mut() {
let _ = apply_header_mutations(hm, headers.as_ref());
apply_header_mutations(hm, headers.as_ref());
}
let resp = rb
.body(http::Body::from(body.to_string()))
@ -776,15 +668,29 @@ fn to_immediate_response(rp: &ProcessingResponse) -> Option<PolicyResponse> {
// handle_response_for_request_mutation handles a single ext_proc response. If it returns 'true' we are done processing.
async fn handle_response_for_request_mutation(
had_body: bool,
req: Option<&mut http::Request>,
mut req: Option<&mut http::Request>,
body_tx: &mut Sender<Result<Frame<Bytes>, Infallible>>,
presp: ProcessingResponse,
) -> Result<(bool, bool), Error> {
) -> (bool, bool) {
if let Some(dm) = &presp.dynamic_metadata {
if let Some(req) = req.as_mut() {
if let Err(e) = extract_dynamic_metadata(req, dm) {
warn!("Failed to extract ext_proc dynamic metadata: {}", e);
}
} else if !dm.fields.is_empty() {
warn!(
"ext_proc server sent dynamic_metadata after headers were processed; \
metadata cannot be attached and will be ignored. Consider sending \
metadata in the RequestHeaders response instead."
);
}
}
let res = matches!(presp.response, Some(Response::RequestHeaders(_)));
let cr = match presp.response {
Some(Response::RequestHeaders(HeadersResponse { response: None })) => {
trace!("no headers");
return Ok((res, false));
return (true, !had_body);
},
Some(Response::RequestHeaders(HeadersResponse { response: Some(cr) })) => {
trace!("got request headers back");
@ -792,20 +698,27 @@ async fn handle_response_for_request_mutation(
},
Some(Response::RequestBody(BodyResponse { response: None })) => {
trace!("got empty request body back");
return Ok((res, true));
return (false, true);
},
Some(Response::RequestBody(BodyResponse { response: Some(cr) })) => {
trace!("got request body back");
cr
},
Some(Response::ImmediateResponse(_)) => {
if req.is_none() {
trace!("immediate response received after request sent; will apply only on the response");
}
// Handled out of this function.
return (true, true);
},
msg => {
// In theory, there can trailers too. EPP never sends them
warn!("ignoring response during request {msg:?}");
return Ok((res, false));
return (false, false);
},
};
if let Some(req) = req {
apply_header_mutations_request(req, cr.header_mutation.as_ref())?;
apply_header_mutations_request(req, cr.header_mutation.as_ref());
}
if let Some(BodyMutation { mutation: Some(b) }) = cr.body_mutation {
match b {
@ -815,7 +728,7 @@ async fn handle_response_for_request_mutation(
let _ = body_tx.send(Ok(Frame::data(by.clone()))).await;
trace!(eos, "got stream request body");
return Ok((res, eos));
return (res, eos);
},
Mutation::Body(_) => {
warn!("Body() not valid for streaming mode, skipping...");
@ -826,53 +739,44 @@ async fn handle_response_for_request_mutation(
}
} else if !had_body {
trace!("got headers back and do not expect body; we are done");
return Ok((res, true));
return (res, true);
}
trace!("still waiting for response...");
Ok((res, false))
(res, false)
}
fn apply_header_with_action(
headers: &mut HeaderMap,
hk: &HeaderName,
hvo: &HeaderValueOption,
) -> Result<(), Error> {
fn apply_header_with_action(headers: &mut HeaderMap, hk: &HeaderName, hvo: &HeaderValueOption) {
let Some(_) = hvo.header else {
return Ok(());
return;
};
// Skip content-length as the EPP sets it to invalid values
// https://github.com/kubernetes-sigs/gateway-api-inference-extension/issues/943
if hk == http::header::CONTENT_LENGTH {
debug!("skipping invalid content-length");
return Ok(());
return;
}
let _ = envoy_proto_common::apply_header_value_option(headers, hk, hvo);
Ok(())
}
fn apply_header_mutations(
headers: &mut HeaderMap,
h: Option<&HeaderMutation>,
) -> Result<(), Error> {
fn apply_header_mutations(headers: &mut HeaderMap, h: Option<&HeaderMutation>) {
if let Some(hm) = h {
for rm in &hm.remove_headers {
headers.remove(rm);
}
for set in &hm.set_headers {
let Some(h) = &set.header else { continue };
let hk = HeaderName::try_from(h.key.as_str())?;
apply_header_with_action(headers, &hk, set)?;
let Ok(hk) = HeaderName::try_from(h.key.as_str()) else {
warn!("invalid header key: {}", h.key);
continue;
};
apply_header_with_action(headers, &hk, set);
}
}
Ok(())
}
fn apply_header_mutations_request(
req: &mut http::Request,
h: Option<&HeaderMutation>,
) -> Result<(), Error> {
fn apply_header_mutations_request(req: &mut http::Request, h: Option<&HeaderMutation>) {
if let Some(hm) = h {
for rm in &hm.remove_headers {
req.headers_mut().remove(rm);
@ -881,23 +785,21 @@ fn apply_header_mutations_request(
let Some(h) = &set.header else { continue };
match HeaderOrPseudo::try_from(h.key.as_str()) {
Ok(HeaderOrPseudo::Header(hk)) => {
apply_header_with_action(req.headers_mut(), &hk, set)?;
apply_header_with_action(req.headers_mut(), &hk, set);
},
Ok(_) => {
let mut rr = crate::http::RequestOrResponse::Request(req);
let _ = envoy_proto_common::apply_pseudo_header_option(&mut rr, set);
},
Err(_) => {},
Err(e) => {
warn!("invalid header key: {} {e}", h.key);
},
}
}
}
Ok(())
}
fn apply_header_mutations_response(
resp: &mut http::Response,
h: Option<&HeaderMutation>,
) -> Result<(), Error> {
fn apply_header_mutations_response(resp: &mut http::Response, h: Option<&HeaderMutation>) {
if let Some(hm) = h {
for rm in &hm.remove_headers {
resp.headers_mut().remove(rm);
@ -906,17 +808,18 @@ fn apply_header_mutations_response(
let Some(h) = &set.header else { continue };
match crate::http::HeaderOrPseudo::try_from(h.key.as_str()) {
Ok(crate::http::HeaderOrPseudo::Header(hk)) => {
apply_header_with_action(resp.headers_mut(), &hk, set)?;
apply_header_with_action(resp.headers_mut(), &hk, set);
},
Ok(_) => {
let mut rr = crate::http::RequestOrResponse::Response(resp);
let _ = envoy_proto_common::apply_pseudo_header_option(&mut rr, set);
},
Err(_) => {},
Err(e) => {
warn!("invalid header key: {} {e}", h.key);
},
}
}
}
Ok(())
}
// handle_response_for_response_mutation handles a single ext_proc response. If it returns 'true' we are done processing.
@ -925,27 +828,27 @@ async fn handle_response_for_response_mutation(
resp: Option<&mut http::Response>,
body_tx: &mut Sender<Result<Frame<Bytes>, Infallible>>,
presp: ProcessingResponse,
) -> Result<(bool, bool), Error> {
) -> (bool, bool) {
let res = matches!(presp.response, Some(Response::ResponseHeaders(_)));
let cr = match presp.response {
Some(Response::ResponseHeaders(HeadersResponse { response: None })) => {
trace!("no headers");
return Ok((res, false));
return (res, false);
},
Some(Response::ResponseHeaders(HeadersResponse { response: Some(cr) })) => cr,
Some(Response::ResponseBody(BodyResponse { response: Some(cr) })) => cr,
Some(Response::ResponseBody(BodyResponse { response: None })) => {
trace!("got empty response body back");
return Ok((res, true));
return (res, true);
},
msg => {
// In theory, there can trailers too. EPP never sends them
warn!("ignoring {msg:?}");
return Ok((res, false));
return (res, false);
},
};
if let Some(resp) = resp {
apply_header_mutations_response(resp, cr.header_mutation.as_ref())?;
apply_header_mutations_response(resp, cr.header_mutation.as_ref());
}
if let Some(BodyMutation { mutation: Some(b) }) = cr.body_mutation {
match b {
@ -954,7 +857,7 @@ async fn handle_response_for_response_mutation(
let by = bytes::Bytes::from(bb.body);
let _ = body_tx.send(Ok(Frame::data(by.clone()))).await;
trace!(%eos, "got body chunk");
return Ok((res, eos));
return (res, eos);
},
Mutation::Body(_) => {
warn!("Body() not valid for streaming mode, skipping...");
@ -965,10 +868,10 @@ async fn handle_response_for_response_mutation(
}
} else if !had_body {
trace!("got headers back and do not expect body; we are done");
return Ok((res, true));
return (res, true);
}
trace!("still waiting for response for response...");
Ok((res, false))
(res, false)
}
fn req_to_header_map(req: &http::Request) -> Option<proto::HeaderMap> {
@ -1047,6 +950,31 @@ fn eval_to_struct(
})
}
pub(crate) fn extract_dynamic_metadata(
req: &mut http::Request,
metadata: &prost_wkt_types::Struct,
) -> Result<(), Error> {
// Get or create metadata container, merging with existing metadata
let mut dynamic_metadata = req
.extensions_mut()
.remove::<Arc<ExtProcDynamicMetadata>>()
.map(|arc| (*arc).clone())
.unwrap_or_default();
// Merge new fields into existing metadata
for (key, value) in &metadata.fields {
let json_val = envoy_proto_common::prost_value_to_json(value)
.map_err(|e| Error::MetadataConversion(format!("failed to convert key '{}': {}", key, e)))?;
dynamic_metadata.0.insert(key.clone(), json_val);
}
if !dynamic_metadata.0.is_empty() {
req.extensions_mut().insert(Arc::new(dynamic_metadata));
}
Ok(())
}
#[derive(Clone, Debug)]
pub struct GrpcReferenceChannel {
pub target: Arc<SimpleBackendReference>,

View File

@ -17,9 +17,10 @@ use crate::http::ext_proc::proto::{
};
use crate::http::ext_proc::{ExtProcDynamicMetadata, proto};
use crate::http::{Body, ext_proc};
use crate::test_helpers::MockInstance;
use crate::test_helpers::extprocmock::{
ExtProcMock, ExtProcMockInstance, Handler, immediate_response, request_body_response,
request_header_response, response_body_response, response_header_response,
ExtProcMock, Handler, immediate_response, request_body_response, request_header_response,
response_body_response, response_header_response,
};
use crate::test_helpers::proxymock::*;
use crate::*;
@ -121,6 +122,22 @@ async fn immediate_response_request() {
assert_eq!(body.as_ref(), b"immediate");
}
#[tokio::test]
async fn immediate_response_request_body_is_deferred_to_response() {
let mock = simple_mock().await;
let (_mock, _ext_proc, _bind, io) = setup_ext_proc_mock(
mock,
ext_proc::FailureMode::FailClosed,
ExtProcMock::new(ImmediateResponseRequestBodyExtProc::default),
"{}",
)
.await;
let res = send_request_body(io, Method::POST, "http://lo", b"request").await;
assert_eq!(res.status(), 403);
let body = read_body_raw(res.into_body()).await;
assert_eq!(body.as_ref(), b"Access denied");
}
#[tokio::test]
async fn immediate_response_response() {
let mock = simple_mock().await;
@ -153,6 +170,22 @@ async fn failure_fail_closed() {
assert!(body.as_ref().starts_with(b"ext_proc failed:"));
}
#[tokio::test]
async fn failure_fail_open_body() {
let mock = simple_mock().await;
let (_mock, _ext_proc, _bind, io) = setup_ext_proc_mock(
mock,
ext_proc::FailureMode::FailOpen,
ExtProcMock::new(FailureExtProcResponse::default),
"{}",
)
.await;
// If we have a body, we should NOT fail open
let res = send_request_body(io, Method::POST, "http://lo", b"request").await;
assert_eq!(res.status(), 500);
}
#[tokio::test]
async fn failure_fail_open() {
let mock = simple_mock().await;
@ -163,10 +196,9 @@ async fn failure_fail_open() {
"{}",
)
.await;
let res = send_request_body(io, Method::POST, "http://lo", b"request").await;
let res = send_request(io, Method::POST, "http://lo").await;
assert_eq!(res.status(), 200);
let body = read_body(res.into_body()).await;
assert_eq!(body.body.as_ref(), b"request");
}
pub async fn setup_ext_proc_mock<T: Handler + Send + Sync + 'static>(
@ -176,7 +208,7 @@ pub async fn setup_ext_proc_mock<T: Handler + Send + Sync + 'static>(
config: &str,
) -> (
MockServer,
ExtProcMockInstance,
MockInstance,
TestBind,
Client<MemoryConnector, Body>,
) {
@ -193,7 +225,7 @@ pub async fn setup_ext_proc_mock_with_meta<T: Handler + Send + Sync + 'static>(
response_attributes: Option<HashMap<String, Arc<Expression>>>,
) -> (
MockServer,
ExtProcMockInstance,
MockInstance,
TestBind,
Client<MemoryConnector, Body>,
) {
@ -388,6 +420,45 @@ impl Handler for ImmediateResponseExtProc {
}
}
#[derive(Debug, Default)]
struct ImmediateResponseRequestBodyExtProc {
sent: bool,
}
#[async_trait::async_trait]
impl Handler for ImmediateResponseRequestBodyExtProc {
async fn handle_request_headers(
&mut self,
_: &HttpHeaders,
sender: &mpsc::Sender<Result<ProcessingResponse, Status>>,
) -> Result<(), Status> {
let _ = sender.send(request_header_response(None)).await;
Ok(())
}
async fn handle_request_body(
&mut self,
_: &proto::HttpBody,
sender: &mpsc::Sender<Result<ProcessingResponse, Status>>,
) -> Result<(), Status> {
if !self.sent {
self.sent = true;
let _ = sender
.send(immediate_response(proto::ImmediateResponse {
status: Some(proto::HttpStatus {
code: proto::StatusCode::Forbidden as i32,
}),
body: "Access denied".to_string(),
headers: None,
grpc_status: None,
details: "".to_string(),
}))
.await;
}
Ok(())
}
}
#[derive(Debug, Default)]
struct ImmediateResponseExtProcResponse {
sent_req_body: bool,
@ -471,7 +542,7 @@ fn test_default_append_action_overwrite() {
}],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
let values: Vec<_> = headers.get_all("existing").iter().collect();
assert_eq!(values.len(), 1);
@ -507,7 +578,7 @@ fn test_append_if_exists_or_add() {
],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
let values: Vec<_> = headers.get_all("existing").iter().collect();
assert_eq!(values.len(), 2);
@ -545,7 +616,7 @@ fn test_add_if_absent() {
],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
let values: Vec<_> = headers.get_all("existing").iter().collect();
assert_eq!(values.len(), 1);
@ -582,7 +653,7 @@ fn test_overwrite_if_exists_or_add() {
],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
let values: Vec<_> = headers.get_all("existing").iter().collect();
assert_eq!(values.len(), 1);
@ -619,7 +690,7 @@ fn test_overwrite_if_exists() {
],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
let values: Vec<_> = headers.get_all("existing").iter().collect();
assert_eq!(values.len(), 1);
@ -638,7 +709,7 @@ fn test_remove_headers() {
set_headers: vec![],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
assert!(headers.get("to-remove").is_none());
assert_eq!(headers.get("keep").unwrap(), "value");
@ -665,7 +736,7 @@ fn test_apply_header_mutations_request() {
}],
});
super::apply_header_mutations_request(&mut req, mutation.as_ref()).unwrap();
super::apply_header_mutations_request(&mut req, mutation.as_ref());
let headers = req.headers();
assert!(headers.get("to-remove").is_none());
@ -726,7 +797,7 @@ fn test_apply_pseudo_headers_request_with_raw_value() {
],
});
super::apply_header_mutations_request(&mut req, mutation.as_ref()).unwrap();
super::apply_header_mutations_request(&mut req, mutation.as_ref());
// Verify pseudo-headers were applied
assert_eq!(req.method(), "POST");
@ -767,7 +838,7 @@ fn test_apply_pseudo_headers_request_with_value_field() {
],
});
super::apply_header_mutations_request(&mut req, mutation.as_ref()).unwrap();
super::apply_header_mutations_request(&mut req, mutation.as_ref());
// Verify pseudo-headers from value field were applied
assert_eq!(req.method(), "PUT");
@ -795,7 +866,7 @@ fn test_pseudo_headers_request_raw_value_precedence() {
}],
});
super::apply_header_mutations_request(&mut req, mutation.as_ref()).unwrap();
super::apply_header_mutations_request(&mut req, mutation.as_ref());
// raw_value should take precedence
assert_eq!(req.method(), "DELETE");
@ -822,7 +893,7 @@ fn test_apply_header_mutations_response() {
}],
});
super::apply_header_mutations_response(&mut resp, mutation.as_ref()).unwrap();
super::apply_header_mutations_response(&mut resp, mutation.as_ref());
let headers = resp.headers();
assert!(headers.get("to-remove").is_none());
@ -854,7 +925,7 @@ fn test_apply_pseudo_headers_response_with_raw_value() {
}],
});
super::apply_header_mutations_response(&mut resp, mutation.as_ref()).unwrap();
super::apply_header_mutations_response(&mut resp, mutation.as_ref());
// Verify :status pseudo-header was applied
assert_eq!(resp.status(), 404);
@ -882,7 +953,7 @@ fn test_apply_pseudo_headers_response_with_value_field() {
}],
});
super::apply_header_mutations_response(&mut resp, mutation.as_ref()).unwrap();
super::apply_header_mutations_response(&mut resp, mutation.as_ref());
// Verify :status pseudo-header from value field was applied
assert_eq!(resp.status(), 201);
@ -908,7 +979,7 @@ fn test_pseudo_headers_response_raw_value_precedence() {
}],
});
super::apply_header_mutations_response(&mut resp, mutation.as_ref()).unwrap();
super::apply_header_mutations_response(&mut resp, mutation.as_ref());
// raw_value should take precedence
assert_eq!(resp.status(), 403);
@ -956,7 +1027,7 @@ fn test_apply_mixed_headers_and_pseudo_headers_request() {
],
});
super::apply_header_mutations_request(&mut req, mutation.as_ref()).unwrap();
super::apply_header_mutations_request(&mut req, mutation.as_ref());
// Verify pseudo-header was applied
assert_eq!(req.method(), "POST");
@ -1006,7 +1077,7 @@ fn test_apply_mixed_headers_and_pseudo_headers_response() {
],
});
super::apply_header_mutations_response(&mut resp, mutation.as_ref()).unwrap();
super::apply_header_mutations_response(&mut resp, mutation.as_ref());
// Verify pseudo-header was applied
assert_eq!(resp.status(), 201);
@ -1044,7 +1115,7 @@ fn test_deprecated_append_true() {
],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
let values: Vec<_> = headers.get_all("existing").iter().collect();
assert_eq!(values.len(), 2);
@ -1071,7 +1142,7 @@ fn test_deprecated_append_false() {
}],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
let values: Vec<_> = headers.get_all("existing").iter().collect();
assert_eq!(values.len(), 1);
@ -1107,7 +1178,7 @@ fn test_value_field_instead_of_raw_value() {
],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
let values: Vec<_> = headers.get_all("existing").iter().collect();
assert_eq!(values.len(), 2);
@ -1133,7 +1204,7 @@ fn test_raw_value_takes_precedence_over_value() {
}],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
assert_eq!(headers.get("test").unwrap(), "raw-value-wins");
}
@ -1156,7 +1227,7 @@ fn test_append_action_priority_over_deprecated_append() {
}],
});
super::apply_header_mutations(&mut headers, mutation.as_ref()).unwrap();
super::apply_header_mutations(&mut headers, mutation.as_ref());
let values: Vec<_> = headers.get_all("existing").iter().collect();
assert_eq!(values.len(), 1);
@ -1276,8 +1347,8 @@ mod extract_dynamic_metadata_tests {
use prost_wkt_types::value::Kind;
use prost_wkt_types::{Struct, Value};
use super::super::extract_dynamic_metadata;
use super::*;
use crate::http::ext_proc::ExtProcInstance;
#[test]
fn test_extract_creates_extension() {
@ -1295,7 +1366,7 @@ mod extract_dynamic_metadata_tests {
.body(Body::empty())
.unwrap();
ExtProcInstance::extract_dynamic_metadata(Some(&mut req), &metadata).unwrap();
extract_dynamic_metadata(&mut req, &metadata).unwrap();
let extracted = req
.extensions()
@ -1330,7 +1401,7 @@ mod extract_dynamic_metadata_tests {
)]
.into(),
};
ExtProcInstance::extract_dynamic_metadata(Some(&mut req), &metadata).unwrap();
extract_dynamic_metadata(&mut req, &metadata).unwrap();
let extracted = req
.extensions()
@ -1370,7 +1441,7 @@ mod extract_dynamic_metadata_tests {
)]
.into(),
};
ExtProcInstance::extract_dynamic_metadata(Some(&mut req), &metadata).unwrap();
extract_dynamic_metadata(&mut req, &metadata).unwrap();
let extracted = req
.extensions()
@ -1383,21 +1454,6 @@ mod extract_dynamic_metadata_tests {
);
}
#[test]
fn test_extract_none_request_ok() {
let metadata = Struct {
fields: [(
"key".to_string(),
Value {
kind: Some(Kind::StringValue("value".to_string())),
},
)]
.into(),
};
let result = ExtProcInstance::extract_dynamic_metadata(None, &metadata);
assert!(result.is_ok());
}
#[test]
fn test_extract_empty_metadata_no_extension() {
let metadata = Struct {
@ -1408,7 +1464,7 @@ mod extract_dynamic_metadata_tests {
.body(Body::empty())
.unwrap();
ExtProcInstance::extract_dynamic_metadata(Some(&mut req), &metadata).unwrap();
extract_dynamic_metadata(&mut req, &metadata).unwrap();
assert!(
req
@ -1449,7 +1505,7 @@ mod extract_dynamic_metadata_tests {
.body(Body::empty())
.unwrap();
ExtProcInstance::extract_dynamic_metadata(Some(&mut req), &metadata).unwrap();
extract_dynamic_metadata(&mut req, &metadata).unwrap();
let extracted = req
.extensions()
@ -1484,7 +1540,7 @@ mod extract_dynamic_metadata_tests {
)]
.into(),
};
ExtProcInstance::extract_dynamic_metadata(Some(&mut req), &metadata1).unwrap();
extract_dynamic_metadata(&mut req, &metadata1).unwrap();
let metadata2 = Struct {
fields: [(
@ -1495,7 +1551,7 @@ mod extract_dynamic_metadata_tests {
)]
.into(),
};
ExtProcInstance::extract_dynamic_metadata(Some(&mut req), &metadata2).unwrap();
extract_dynamic_metadata(&mut req, &metadata2).unwrap();
let extracted = req
.extensions()

View File

@ -19,6 +19,8 @@ pub mod csrf;
pub mod envoy_proto_common;
pub mod ext_authz;
pub mod ext_proc;
pub(crate) mod oauth;
pub mod oidc;
pub mod outlierdetection;
mod peekbody;
pub mod remoteratelimit;
@ -32,6 +34,45 @@ pub type Body = axum_core::body::Body;
pub type Request = ::http::Request<Body>;
pub type Response = ::http::Response<Body>;
pub(crate) fn iter_request_cookies(
req: &Request,
) -> impl Iterator<Item = cookie::Cookie<'static>> + '_ {
req
.headers()
.get_all(header::COOKIE)
.into_iter()
.filter_map(|value| value.to_str().ok())
.flat_map(|header_value| {
cookie::Cookie::split_parse(header_value.to_owned())
.filter_map(Result::ok)
.map(cookie::Cookie::into_owned)
})
}
pub(crate) fn read_request_cookie(req: &Request, name: &str) -> Option<String> {
let mut matched = None;
for cookie in iter_request_cookies(req) {
if cookie.name() == name {
matched = Some(cookie.value().to_owned());
}
}
matched
}
pub(crate) fn strip_request_cookies_by_prefix(req: &mut Request, prefix: &str) {
let preserved: Vec<String> = iter_request_cookies(req)
.filter(|cookie| !cookie.name().starts_with(prefix))
.map(|cookie| cookie.to_string())
.collect();
req.headers_mut().remove(header::COOKIE);
if !preserved.is_empty() {
let hv =
HeaderValue::from_str(&preserved.join("; ")).expect("re-joined cookie header is valid");
req.headers_mut().insert(header::COOKIE, hv);
}
}
// SendDirectResponse is a Response that has been buffered so that it is Send.
pub struct SendDirectResponse(pub ::http::Response<Bytes>);

View File

@ -0,0 +1,95 @@
#[derive(
Debug, Clone, Copy, serde::Serialize, serde::Deserialize, Default, PartialEq, Eq, PartialOrd, Ord,
)]
#[serde(rename_all = "camelCase")]
#[cfg_attr(feature = "schema", derive(schemars::JsonSchema))]
pub enum TokenEndpointAuth {
#[default]
ClientSecretBasic,
ClientSecretPost,
}
impl TokenEndpointAuth {
pub fn as_str(self) -> &'static str {
match self {
Self::ClientSecretBasic => "clientSecretBasic",
Self::ClientSecretPost => "clientSecretPost",
}
}
}
pub(crate) fn openid_configuration_metadata_url(issuer: &str) -> String {
format!(
"{}/.well-known/openid-configuration",
issuer.trim_end_matches('/')
)
}
pub(crate) fn authorization_server_metadata_url(issuer: &str) -> String {
match url::Url::parse(issuer) {
Ok(parsed) => {
let origin = parsed.origin().ascii_serialization();
let path = parsed.path();
if path == "/" {
format!("{origin}/.well-known/oauth-authorization-server")
} else {
format!("{origin}/.well-known/oauth-authorization-server{path}")
}
},
Err(_) => {
let normalized = issuer.trim_end_matches('/');
format!("{normalized}/.well-known/oauth-authorization-server")
},
}
}
pub(crate) fn parse_token_endpoint_auth_methods(
methods: Option<Vec<String>>,
) -> Result<TokenEndpointAuth, String> {
let methods = methods.unwrap_or_else(|| vec!["client_secret_basic".into()]);
if methods.iter().any(|method| method == "client_secret_basic") {
Ok(TokenEndpointAuth::ClientSecretBasic)
} else if methods.iter().any(|method| method == "client_secret_post") {
Ok(TokenEndpointAuth::ClientSecretPost)
} else {
Err("token endpoint auth methods must include clientSecretBasic or clientSecretPost".into())
}
}
#[cfg(test)]
mod tests {
use super::{
TokenEndpointAuth, authorization_server_metadata_url, parse_token_endpoint_auth_methods,
};
#[test]
fn authorization_server_metadata_url_supports_path_based_issuers() {
assert_eq!(
authorization_server_metadata_url("https://idp.example.com/application/o/myapp"),
"https://idp.example.com/.well-known/oauth-authorization-server/application/o/myapp"
);
}
#[test]
fn parse_token_endpoint_auth_methods_prefers_basic() {
let method = parse_token_endpoint_auth_methods(Some(vec![
"private_key_jwt".into(),
"client_secret_post".into(),
"client_secret_basic".into(),
]))
.expect("supported auth method");
assert_eq!(method, TokenEndpointAuth::ClientSecretBasic);
}
#[test]
fn parse_token_endpoint_auth_methods_rejects_missing_supported_values() {
let err =
parse_token_endpoint_auth_methods(Some(vec!["private_key_jwt".into(), "none".into()]));
assert_eq!(
err.unwrap_err(),
"token endpoint auth methods must include clientSecretBasic or clientSecretPost"
);
}
}

View File

@ -0,0 +1,183 @@
use aws_lc_rs::constant_time::verify_slices_are_equal;
use base64::Engine;
use secrecy::SecretString;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use crate::http::Request;
use crate::proxy::httpproxy::PolicyClient;
use tracing::debug;
use super::provider;
use super::session::{
BrowserSession, TransactionState, generate_nonce, generate_pkce_verifier, generate_state,
generate_transaction_id, normalize_original_uri,
};
use super::{Error, OidcPolicy, build_redirect_response, cap_session_expiry, now_unix};
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct CallbackTransactionState {
pub transaction_id: String,
pub csrf_state: String,
}
pub(super) struct CallbackRequestContext {
pub code: String,
pub callback_state: CallbackTransactionState,
pub transaction_cookie_name: String,
pub transaction_cookie: String,
}
impl CallbackTransactionState {
pub fn encode(&self) -> String {
let json =
serde_json::to_vec(self).expect("serializing callback transaction state is infallible");
base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(json)
}
pub fn decode(value: &str) -> Result<Self, Error> {
let raw = base64::engine::general_purpose::URL_SAFE_NO_PAD
.decode(value)
.map_err(|_| Error::InvalidCallback)?;
serde_json::from_slice(&raw).map_err(|_| Error::InvalidCallback)
}
}
pub(super) fn start_login(
policy: &OidcPolicy,
req: &Request,
) -> Result<crate::http::PolicyResponse, Error> {
let transaction_id = generate_transaction_id();
let csrf_state = generate_state();
let nonce = generate_nonce();
let pkce_verifier = generate_pkce_verifier();
let code_challenge = {
let digest = aws_lc_rs::digest::digest(&aws_lc_rs::digest::SHA256, pkce_verifier.as_bytes());
base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(digest.as_ref())
};
let original_uri = normalize_original_uri(req.uri().path_and_query());
let transaction = TransactionState {
policy_id: policy.policy_id.clone(),
transaction_id: transaction_id.clone(),
csrf_state: csrf_state.clone(),
nonce: nonce.clone(),
pkce_verifier: SecretString::new(pkce_verifier.into_boxed_str()),
original_uri,
expires_at_unix: now_unix().saturating_add(policy.session.transaction_ttl.as_secs()),
};
let callback_state = CallbackTransactionState {
transaction_id,
csrf_state,
};
let state = callback_state.encode();
let encoded = policy.session.encode_transaction(&transaction)?;
let transaction_cookie_name = policy
.session
.transaction_cookie_name(&callback_state.transaction_id);
let cookie = policy.session.set_cookie(
&transaction_cookie_name,
&encoded,
policy.redirect_uri.https,
policy.session.transaction_ttl,
);
let location = with_query(
&policy.provider.authorization_endpoint,
&[
("response_type", "code".into()),
("client_id", policy.client.client_id.clone()),
("redirect_uri", policy.redirect_uri.redirect_uri.clone()),
("scope", policy.scopes.join(" ")),
("state", state),
("nonce", nonce),
("code_challenge", code_challenge),
("code_challenge_method", "S256".into()),
],
);
let response = build_redirect_response(&location, &[cookie])?;
Ok(crate::http::PolicyResponse::default().with_response(response))
}
pub(super) async fn handle_callback(
policy: &OidcPolicy,
context: CallbackRequestContext,
client: PolicyClient,
) -> Result<crate::http::PolicyResponse, Error> {
let transaction = policy
.session
.decode_transaction(&context.transaction_cookie)?;
if transaction.policy_id != policy.policy_id {
debug!("oidc callback rejected due to policy mismatch");
return Err(Error::PolicyMismatch);
}
if !constant_time_str_eq(
&transaction.transaction_id,
&context.callback_state.transaction_id,
) {
debug!("oidc callback rejected due to transaction mismatch");
return Err(Error::InvalidTransaction);
}
if !constant_time_str_eq(&transaction.csrf_state, &context.callback_state.csrf_state) {
debug!("oidc callback rejected due to csrf mismatch");
return Err(Error::CsrfMismatch);
}
let token = provider::exchange_code(
client,
&policy.provider,
&policy.client,
&policy.redirect_uri.redirect_uri,
&context.code,
&transaction.pkce_verifier,
)
.await?;
let id_token = token.id_token.ok_or(Error::MissingIdToken)?;
let claims = policy
.provider
.id_token_validator
.validate_claims(&id_token)
.map_err(Error::InvalidIdToken)?;
let nonce = claims
.inner
.get("nonce")
.and_then(Value::as_str)
.ok_or(Error::NonceMismatch)?;
if !constant_time_str_eq(nonce, &transaction.nonce) {
debug!("oidc callback rejected due to nonce mismatch");
return Err(Error::NonceMismatch);
}
// TODO: Revisit whether browser sessions should persist access_token / refresh_token.
// The current stateless cookie only stores the validated id_token because that is what
// the runtime uses today, and larger token payloads can exceed browser cookie limits.
let session = BrowserSession {
policy_id: policy.policy_id.clone(),
raw_id_token: SecretString::new(id_token.into_boxed_str()),
expires_at_unix: Some(cap_session_expiry(
now_unix(),
policy.session.ttl,
&claims.inner,
)),
};
let encoded = policy.session.encode_browser_session(&session)?;
let session_cookie = policy.session.set_cookie(
&policy.session.cookie_name,
&encoded,
policy.redirect_uri.https,
policy.session.ttl,
);
let clear_transaction = policy
.session
.clear_cookie(&context.transaction_cookie_name, policy.redirect_uri.https);
let location = transaction.original_uri;
let response = build_redirect_response(&location, &[session_cookie, clear_transaction])?;
Ok(crate::http::PolicyResponse::default().with_response(response))
}
fn with_query(uri: &super::ProviderEndpoint, params: &[(&str, String)]) -> String {
uri.with_query(params)
}
fn constant_time_str_eq(expected: &str, actual: &str) -> bool {
verify_slices_are_equal(expected.as_bytes(), actual.as_bytes()).is_ok()
}

View File

@ -0,0 +1,351 @@
use std::sync::Arc;
use jsonwebtoken::jwk::JwkSet;
use macro_rules_attribute::apply;
use secrecy::SecretString;
use super::{
ClientConfig, CookieSecureMode, Error, OidcPolicy, PolicyId, Provider, ProviderEndpoint,
RedirectUri, SameSiteMode, SessionConfig, dedupe_scopes, session,
};
use crate::client::Client;
use crate::http::oauth::{
TokenEndpointAuth, openid_configuration_metadata_url, parse_token_endpoint_auth_methods,
};
use crate::schema_de;
use crate::serdes::FileInlineOrRemote;
#[derive(Debug, serde::Deserialize)]
struct OidcDiscoveryDocument {
issuer: String,
authorization_endpoint: String,
token_endpoint: String,
jwks_uri: String,
#[serde(default)]
token_endpoint_auth_methods_supported: Option<Vec<String>>,
}
struct PreparedOidcProvider {
issuer: String,
authorization_endpoint: ProviderEndpoint,
token_endpoint: ProviderEndpoint,
token_endpoint_auth: TokenEndpointAuth,
id_token_jwks: JwkSet,
}
struct PreparedOidcPolicy {
provider: PreparedOidcProvider,
client_id: String,
client_secret: SecretString,
redirect_uri: RedirectUri,
scopes: Vec<String>,
}
/// Browser-based OIDC authentication policy.
///
/// Explicit mode is still OIDC: it supplies provider metadata manually instead of using discovery.
/// Unauthenticated non-callback requests always redirect to the provider login flow. Routes that
/// need non-redirect authentication behavior should use a different auth policy.
#[apply(schema_de!)]
pub struct LocalOidcConfig {
/// Issuer used for discovery and ID token validation.
pub issuer: String,
/// Optional discovery document override. If omitted, discovery uses
/// `${issuer}/.well-known/openid-configuration`.
#[serde(default)]
pub discovery: Option<FileInlineOrRemote>,
/// Authorization endpoint used to start the browser login flow.
#[serde(default)]
#[cfg_attr(feature = "schema", schemars(with = "Option<String>"))]
pub authorization_endpoint: Option<ProviderEndpoint>,
/// Token endpoint used to exchange the authorization code.
#[serde(default)]
#[cfg_attr(feature = "schema", schemars(with = "Option<String>"))]
pub token_endpoint: Option<ProviderEndpoint>,
/// Token endpoint client authentication method for explicit provider configuration.
///
/// Discovery mode derives this from provider metadata. Explicit mode defaults to
/// `clientSecretBasic` when omitted.
#[serde(default)]
pub token_endpoint_auth: Option<TokenEndpointAuth>,
/// JWKS source used to validate returned ID tokens.
#[serde(default)]
pub jwks: Option<FileInlineOrRemote>,
/// OAuth2 client identifier used for authorization and token exchange.
pub client_id: String,
/// OAuth2 client secret used for token exchange.
#[serde(serialize_with = "crate::serdes::ser_redact")]
#[cfg_attr(feature = "schema", schemars(with = "String"))]
pub client_secret: SecretString,
/// Absolute callback URI handled by the gateway.
/// This policy always redirects unauthenticated non-callback requests back through this login
/// flow.
#[serde(rename = "redirectURI")]
pub redirect_uri: String,
/// Additional OAuth2 scopes to request. `openid` is always included.
#[serde(default)]
pub scopes: Vec<String>,
}
struct DiscoveredProviderMetadata {
authorization_endpoint: ProviderEndpoint,
token_endpoint: ProviderEndpoint,
token_endpoint_auth: TokenEndpointAuth,
jwks: FileInlineOrRemote,
}
impl LocalOidcConfig {
pub(crate) async fn compile(
self,
client: Client,
policy_id: PolicyId,
oidc_cookie_encoder: &crate::http::sessionpersistence::Encoder,
) -> Result<OidcPolicy, Error> {
self
.resolve(client)
.await?
.compile(policy_id, oidc_cookie_encoder)
}
async fn resolve(self, client: Client) -> Result<PreparedOidcPolicy, Error> {
let LocalOidcConfig {
issuer,
discovery,
authorization_endpoint,
token_endpoint,
token_endpoint_auth,
jwks,
client_id,
client_secret,
redirect_uri,
scopes,
} = self;
let redirect_uri = RedirectUri::parse(redirect_uri)?;
let explicit_field_count = usize::from(authorization_endpoint.is_some())
+ usize::from(token_endpoint.is_some())
+ usize::from(jwks.is_some());
if token_endpoint_auth.is_some() && explicit_field_count != 3 {
return Err(Error::Config(
"tokenEndpointAuth must be omitted unless authorizationEndpoint, tokenEndpoint, and jwks are configured explicitly".into(),
));
}
let provider = match explicit_field_count {
0 => {
let discovery = match discovery {
Some(discovery) => discovery,
None => FileInlineOrRemote::Remote {
url: default_discovery_url(&issuer)?,
},
};
let discovered = discover_provider_metadata(client.clone(), &issuer, discovery).await?;
let id_token_jwks = load_jwks(client, discovered.jwks, "discovered jwks source").await?;
PreparedOidcProvider {
issuer,
authorization_endpoint: discovered.authorization_endpoint,
token_endpoint: discovered.token_endpoint,
token_endpoint_auth: discovered.token_endpoint_auth,
id_token_jwks,
}
},
3 => {
if discovery.is_some() {
return Err(Error::Config(
"oidc discovery must be omitted when authorizationEndpoint, tokenEndpoint, and jwks are configured explicitly".into(),
));
}
resolve_explicit_provider(
client,
issuer,
authorization_endpoint.expect("checked above"),
token_endpoint.expect("checked above"),
token_endpoint_auth.unwrap_or_default(),
jwks.expect("checked above"),
)
.await?
},
_ => {
return Err(Error::Config(
"authorizationEndpoint, tokenEndpoint, and jwks must either all be set or all be omitted"
.into(),
));
},
};
Ok(PreparedOidcPolicy {
provider,
client_id,
client_secret,
redirect_uri,
scopes,
})
}
}
async fn discover_provider_metadata(
client: Client,
issuer: &str,
discovery: FileInlineOrRemote,
) -> Result<DiscoveredProviderMetadata, Error> {
let document = discovery
.load::<OidcDiscoveryDocument>(client)
.await
.map_err(|e| {
Error::Config(format!(
"failed to decode oidc discovery response from {}: {e}",
describe_file_inline_or_remote(&discovery)
))
})?;
if document.issuer != issuer {
return Err(Error::Config(format!(
"oidc discovery issuer mismatch: expected {issuer}, got {}",
document.issuer
)));
}
let token_endpoint_auth =
parse_token_endpoint_auth_methods(document.token_endpoint_auth_methods_supported)
.map_err(Error::Config)?;
let jwks = FileInlineOrRemote::Remote {
url: document
.jwks_uri
.parse()
.map_err(|e| Error::Config(format!("invalid jwks uri: {e}")))?,
};
Ok(DiscoveredProviderMetadata {
authorization_endpoint: document
.authorization_endpoint
.parse()
.map_err(|e| Error::Config(format!("invalid authorization endpoint: {e}")))?,
token_endpoint: document
.token_endpoint
.parse()
.map_err(|e| Error::Config(format!("invalid token endpoint: {e}")))?,
token_endpoint_auth,
jwks,
})
}
async fn resolve_explicit_provider(
client: Client,
issuer: String,
authorization_endpoint: ProviderEndpoint,
token_endpoint: ProviderEndpoint,
token_endpoint_auth: TokenEndpointAuth,
jwks: FileInlineOrRemote,
) -> Result<PreparedOidcProvider, Error> {
let id_token_jwks = load_jwks(client, jwks, "explicit jwks source").await?;
Ok(PreparedOidcProvider {
issuer,
authorization_endpoint,
token_endpoint,
token_endpoint_auth,
id_token_jwks,
})
}
fn default_discovery_url(issuer: &str) -> Result<http::Uri, Error> {
openid_configuration_metadata_url(issuer)
.parse()
.map_err(|e| {
Error::Config(format!(
"invalid discovery uri derived from issuer '{issuer}': {e}"
))
})
}
async fn load_jwks(
client: Client,
jwks: FileInlineOrRemote,
source: &'static str,
) -> Result<JwkSet, Error> {
let jwks = jwks.load::<JwkSet>(client).await.map_err(|e| {
Error::Config(format!(
"failed to load oidc jwks from {} {}: {e}",
source,
describe_file_inline_or_remote(&jwks)
))
})?;
Ok(jwks)
}
impl PreparedOidcProvider {
fn compile(self, client_id: String) -> Result<Provider, Error> {
let provider = crate::http::jwt::Provider::from_jwks(
self.id_token_jwks,
self.issuer.clone(),
Some(vec![client_id]),
crate::http::jwt::JWTValidationOptions::default(),
)
.map_err(|e| Error::Config(format!("failed to create id token validator: {e}")))?;
Ok(Provider {
issuer: self.issuer,
authorization_endpoint: self.authorization_endpoint,
token_endpoint: self.token_endpoint,
id_token_validator: crate::http::jwt::Jwt::from_providers(
vec![provider],
crate::http::jwt::Mode::Strict,
),
})
}
}
impl PreparedOidcPolicy {
fn compile(
self,
policy_id: PolicyId,
oidc_cookie_encoder: &crate::http::sessionpersistence::Encoder,
) -> Result<OidcPolicy, Error> {
let (cookie_name, transaction_cookie_prefix) = session::derive_cookie_names(&policy_id);
let PreparedOidcPolicy {
provider,
client_id,
client_secret,
redirect_uri,
scopes,
} = self;
let scopes = dedupe_scopes(scopes);
let token_endpoint_auth = provider.token_endpoint_auth;
let provider = Arc::new(provider.compile(client_id.clone())?);
Ok(OidcPolicy {
policy_id,
provider,
client: ClientConfig {
client_id,
client_secret,
token_endpoint_auth,
},
redirect_uri,
session: SessionConfig {
cookie_name,
transaction_cookie_prefix,
same_site: SameSiteMode::Lax,
secure: CookieSecureMode::Auto,
ttl: session::default_session_ttl(),
transaction_ttl: session::default_transaction_ttl(),
encoder: oidc_cookie_encoder.clone(),
},
scopes,
})
}
}
fn describe_file_inline_or_remote(source: &FileInlineOrRemote) -> String {
match source {
FileInlineOrRemote::File { file } => format!("file '{}'", file.display()),
FileInlineOrRemote::Inline(_) => "inline configuration".into(),
FileInlineOrRemote::Remote { url } => format!("uri '{url}'"),
}
}

View File

@ -0,0 +1,362 @@
use std::collections::HashSet;
use std::fmt;
use std::sync::Arc;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use ::http::{HeaderValue, StatusCode, header};
use secrecy::ExposeSecret;
use secrecy::SecretString;
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use serde_json::{Map, Value};
use tracing::debug;
use crate::http::jwt;
use crate::http::{Body, PolicyResponse, Request, Response};
use crate::proxy::httpproxy::PolicyClient;
use crate::telemetry::log::RequestLog;
mod callback;
mod local;
mod provider;
mod redirect;
mod session;
#[cfg(test)]
mod tests;
pub use crate::http::oauth::TokenEndpointAuth;
pub use local::LocalOidcConfig;
pub use redirect::RedirectUri;
pub use session::{
BrowserSession, CookieSecureMode, RESERVED_COOKIE_PREFIX, SameSiteMode, SessionConfig,
TransactionState,
};
#[derive(
Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, serde::Serialize, serde::Deserialize,
)]
#[serde(transparent)]
pub struct PolicyId(String);
impl PolicyId {
pub fn as_str(&self) -> &str {
&self.0
}
pub fn route(route_key: impl std::fmt::Display) -> Self {
Self(format!("route/{route_key}"))
}
pub fn policy(policy_key: impl std::fmt::Display) -> Self {
Self(format!("policy/{policy_key}"))
}
}
/// Validated absolute HTTP(S) endpoint used by an OIDC provider.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProviderEndpoint(url::Url);
impl ProviderEndpoint {
pub fn as_str(&self) -> &str {
self.0.as_ref()
}
pub fn with_query(&self, params: &[(&str, String)]) -> String {
let mut url = self.0.clone();
{
let mut query = url.query_pairs_mut();
for (key, value) in params {
query.append_pair(key, value);
}
}
url.to_string()
}
}
impl TryFrom<&str> for ProviderEndpoint {
type Error = String;
fn try_from(value: &str) -> Result<Self, Self::Error> {
let url =
url::Url::parse(value).map_err(|e| format!("must be an absolute http(s) URL: {e}"))?;
if !matches!(url.scheme(), "http" | "https") {
return Err(format!(
"must use an http or https scheme, got '{}'",
url.scheme()
));
}
Ok(Self(url))
}
}
impl std::str::FromStr for ProviderEndpoint {
type Err = String;
fn from_str(value: &str) -> Result<Self, Self::Err> {
Self::try_from(value)
}
}
impl fmt::Display for ProviderEndpoint {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
self.0.fmt(f)
}
}
impl Serialize for ProviderEndpoint {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
self.to_string().serialize(serializer)
}
}
impl<'de> Deserialize<'de> for ProviderEndpoint {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
let value = String::deserialize(deserializer)?;
Self::try_from(value.as_str()).map_err(serde::de::Error::custom)
}
}
#[derive(Debug, Clone, serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub struct OidcPolicy {
pub policy_id: PolicyId,
pub provider: Arc<Provider>,
pub client: ClientConfig,
pub redirect_uri: RedirectUri,
pub session: SessionConfig,
pub scopes: Vec<String>,
}
#[derive(Debug, Clone, serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub struct Provider {
pub issuer: String,
pub authorization_endpoint: ProviderEndpoint,
pub token_endpoint: ProviderEndpoint,
pub id_token_validator: jwt::Jwt,
}
#[derive(Debug, Clone, serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub struct ClientConfig {
pub client_id: String,
#[serde(serialize_with = "crate::serdes::ser_redact")]
pub client_secret: SecretString,
pub token_endpoint_auth: TokenEndpointAuth,
}
#[derive(Debug, thiserror::Error)]
pub enum Error {
#[error("missing session")]
MissingSession,
#[error("invalid session")]
InvalidSession,
#[error("authentication required")]
AuthenticationRequired,
#[error("encoded browser session exceeds cookie size budget")]
SessionCookieTooLarge,
#[error("missing transaction")]
MissingTransaction,
#[error("invalid transaction")]
InvalidTransaction,
#[error("policy mismatch")]
PolicyMismatch,
#[error("csrf mismatch")]
CsrfMismatch,
#[error("token exchange failed")]
TokenExchangeFailed(#[source] anyhow::Error),
#[error("missing id token")]
MissingIdToken,
#[error("invalid id token: {0}")]
InvalidIdToken(jwt::TokenError),
#[error("nonce mismatch")]
NonceMismatch,
#[error("invalid callback")]
InvalidCallback,
#[error("oidc provider returned callback error '{0}'")]
ProviderCallback(String),
#[error("{0}")]
Config(String),
#[error("{0}")]
Http(#[from] anyhow::Error),
}
struct CallbackQuery {
state: String,
code: Option<String>,
error: Option<String>,
}
impl OidcPolicy {
pub async fn apply(
&self,
log: Option<&mut RequestLog>,
req: &mut Request,
client: PolicyClient,
) -> Result<PolicyResponse, Error> {
if let Some(response) = self.maybe_handle_callback(req, client.clone()).await? {
return Ok(response);
}
if is_cors_preflight(req) {
return Ok(PolicyResponse::default());
}
if let Some(cookie) = read_cookie(req, &self.session.cookie_name) {
match self.session.decode_browser_session(&cookie) {
Ok(browser_session) => {
if browser_session.policy_id == self.policy_id
&& let Ok(claims) = self
.provider
.id_token_validator
.validate_claims(browser_session.raw_id_token.expose_secret())
{
if let Some(Value::String(sub)) = claims.inner.get("sub")
&& let Some(log) = log
{
log.jwt_sub = Some(sub.clone());
}
req.extensions_mut().insert(claims);
return Ok(PolicyResponse::default());
}
},
Err(err) => {
debug!(error=%err, "failed to decode oidc browser session cookie");
},
}
}
// OIDC is an interactive browser policy: unauthenticated non-callback requests enter login.
callback::start_login(self, req)
}
async fn maybe_handle_callback(
&self,
req: &mut Request,
client: PolicyClient,
) -> Result<Option<PolicyResponse>, Error> {
if req.method() != ::http::Method::GET
|| req.uri().path() != self.redirect_uri.callback_path.path()
{
return Ok(None);
}
let Some(query) = CallbackQuery::parse(req) else {
return Ok(None);
};
let callback_state = callback::CallbackTransactionState::decode(&query.state)?;
let transaction_cookie_name = self
.session
.transaction_cookie_name(&callback_state.transaction_id);
let transaction_cookie =
read_cookie(req, &transaction_cookie_name).ok_or(Error::MissingTransaction)?;
if let Some(error) = query.error {
return Err(Error::ProviderCallback(error));
}
let code = query.code.ok_or(Error::InvalidCallback)?;
let response = callback::handle_callback(
self,
callback::CallbackRequestContext {
code,
callback_state,
transaction_cookie_name,
transaction_cookie,
},
client,
)
.await?;
Ok(Some(response))
}
}
fn is_cors_preflight(req: &Request) -> bool {
req.method() == ::http::Method::OPTIONS
&& req.headers().contains_key(header::ORIGIN)
&& req
.headers()
.get(header::ACCESS_CONTROL_REQUEST_METHOD)
.map(|value| !value.as_bytes().is_empty())
.unwrap_or(false)
}
impl CallbackQuery {
/// Parse callback query parameters from the request in a single pass.
/// Returns `None` if the query does not contain `state` + (`code` | `error`),
/// meaning this request is not an OAuth2 callback.
fn parse(req: &Request) -> Option<Self> {
let mut state = None;
let mut code = None;
let mut error = None;
for (key, value) in
url::form_urlencoded::parse(req.uri().query().unwrap_or_default().as_bytes())
{
match key.as_ref() {
"state" => state = Some(value.into_owned()),
"code" => code = Some(value.into_owned()),
"error" => error = Some(value.into_owned()),
_ => {},
}
}
let state = state?;
if code.is_none() && error.is_none() {
return None;
}
Some(CallbackQuery { state, code, error })
}
}
fn read_cookie(req: &Request, name: &str) -> Option<String> {
crate::http::read_request_cookie(req, name)
}
pub(crate) fn build_redirect_response(
location: &str,
set_cookies: &[String],
) -> Result<Response, Error> {
let mut response = ::http::Response::builder()
.status(StatusCode::FOUND)
.header(header::LOCATION, location);
let headers = response
.headers_mut()
.ok_or_else(|| Error::Config("failed to build redirect response".into()))?;
for cookie in set_cookies {
headers.append(
header::SET_COOKIE,
HeaderValue::from_str(cookie)
.map_err(|e| Error::Config(format!("invalid set-cookie header: {e}")))?,
);
}
response
.body(Body::empty())
.map_err(|e| Error::Config(format!("failed to finalize redirect response: {e}")))
}
pub(crate) fn now_unix() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or(Duration::ZERO)
.as_secs()
}
pub(crate) fn dedupe_scopes(mut scopes: Vec<String>) -> Vec<String> {
scopes.insert(0, "openid".into());
let mut seen = HashSet::new();
scopes.retain(|scope| seen.insert(scope.clone()));
scopes
}
pub(crate) fn cap_session_expiry(now: u64, ttl: Duration, claims: &Map<String, Value>) -> u64 {
let ttl_exp = now.saturating_add(ttl.as_secs());
match claims.get("exp").and_then(Value::as_u64) {
Some(exp) => exp.min(ttl_exp),
None => ttl_exp,
}
}

View File

@ -0,0 +1,134 @@
use std::time::Duration;
use ::http::{Method, StatusCode, header};
use anyhow::Context;
use base64::Engine;
use secrecy::{ExposeSecret, SecretString};
use crate::http::Body;
use crate::http::filters::BackendRequestTimeout;
use crate::proxy::httpproxy::PolicyClient;
use super::{Error, Provider, TokenEndpointAuth};
#[derive(Debug, serde::Deserialize)]
pub(crate) struct TokenResponse {
#[serde(default)]
pub id_token: Option<String>,
}
const DEFAULT_TOKEN_EXCHANGE_TIMEOUT: Duration = Duration::from_secs(10);
const TOKEN_RESPONSE_BODY_LIMIT: usize = 64 * 1024;
pub(crate) async fn exchange_code(
client: PolicyClient,
provider: &Provider,
client_config: &super::ClientConfig,
redirect_uri: &str,
code: &str,
pkce_verifier: &SecretString,
) -> Result<TokenResponse, Error> {
exchange_code_with_timeout(
client,
provider,
client_config,
redirect_uri,
code,
pkce_verifier,
DEFAULT_TOKEN_EXCHANGE_TIMEOUT,
)
.await
}
pub(crate) async fn exchange_code_with_timeout(
client: PolicyClient,
provider: &Provider,
client_config: &super::ClientConfig,
redirect_uri: &str,
code: &str,
pkce_verifier: &SecretString,
timeout: Duration,
) -> Result<TokenResponse, Error> {
let mut form = vec![
("grant_type", "authorization_code".to_string()),
("code", code.to_string()),
("redirect_uri", redirect_uri.to_string()),
("code_verifier", pkce_verifier.expose_secret().to_string()),
];
let mut req = ::http::Request::builder()
.method(Method::POST)
.uri(provider.token_endpoint.as_str())
.header(header::CONTENT_TYPE, "application/x-www-form-urlencoded")
.header(header::ACCEPT, "application/json");
match client_config.token_endpoint_auth {
TokenEndpointAuth::ClientSecretBasic => {
let encoded_client_id = form_urlencode_component(&client_config.client_id);
let encoded_client_secret =
form_urlencode_component(client_config.client_secret.expose_secret());
let auth = format!(
"Basic {}",
base64::engine::general_purpose::STANDARD
.encode(format!("{}:{}", encoded_client_id, encoded_client_secret))
);
req = req.header(header::AUTHORIZATION, auth);
},
TokenEndpointAuth::ClientSecretPost => {
form.push(("client_id", client_config.client_id.clone()));
form.push((
"client_secret",
client_config.client_secret.expose_secret().to_string(),
));
},
}
let body = serde_urlencoded::to_string(form).map_err(anyhow::Error::from)?;
let mut req = req
.body(Body::from(body))
.map_err(|e| Error::Config(format!("failed to build token exchange request: {e}")))?;
req.extensions_mut().insert(BackendRequestTimeout(timeout));
let resp = client
.simple_call(req)
.await
.map_err(anyhow::Error::from)
.map_err(Error::TokenExchangeFailed)?;
let status = resp.status();
let (_, body) = {
let (parts, body) = resp.into_parts();
let body = crate::http::read_body_with_limit(body, TOKEN_RESPONSE_BODY_LIMIT)
.await
.map_err(anyhow::Error::from)
.map_err(Error::TokenExchangeFailed)?;
(parts, body)
};
if status != StatusCode::OK {
return Err(Error::TokenExchangeFailed(anyhow::anyhow!(
"token endpoint returned {status}: {}",
format_token_endpoint_error_body(&body)
)));
}
serde_json::from_slice::<TokenResponse>(&body)
.context("failed to decode token response")
.map_err(Error::TokenExchangeFailed)
}
fn form_urlencode_component(value: &str) -> String {
url::form_urlencoded::byte_serialize(value.as_bytes()).collect()
}
fn format_token_endpoint_error_body(body: &[u8]) -> String {
const LIMIT: usize = 1024;
let mut out = String::with_capacity(body.len().min(LIMIT));
let mut truncated = false;
for ch in String::from_utf8_lossy(body).chars() {
let ch = if ch.is_control() { ' ' } else { ch };
if out.len() + ch.len_utf8() > LIMIT {
truncated = true;
break;
}
out.push(ch);
}
if truncated {
out.push_str("...");
}
out
}

View File

@ -0,0 +1,136 @@
use ::http::uri::PathAndQuery;
use percent_encoding::percent_decode_str;
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use super::Error;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RedirectUri {
pub redirect_uri: String,
pub host: String,
pub port: u16,
pub https: bool,
pub callback_path: PathAndQuery,
}
impl RedirectUri {
pub fn parse(redirect_uri: String) -> Result<Self, Error> {
let http_uri = redirect_uri
.parse::<::http::Uri>()
.map_err(|e| Error::Config(format!("invalid redirectURI: {e}")))?;
let url = url::Url::parse(&redirect_uri)
.map_err(|e| Error::Config(format!("invalid redirectURI: {e}")))?;
if !url.username().is_empty() || url.password().is_some() {
return Err(Error::Config(
"redirectURI must not contain userinfo".into(),
));
}
if url.fragment().is_some() {
return Err(Error::Config(
"redirectURI must not contain a fragment".into(),
));
}
if url.query().is_some() {
return Err(Error::Config(
"redirectURI must not contain query parameters".into(),
));
}
let https = match url.scheme() {
"https" => true,
"http" => false,
other => {
return Err(Error::Config(format!(
"redirectURI must use http or https, got scheme '{other}'"
)));
},
};
let host = url
.host_str()
.ok_or_else(|| Error::Config("redirectURI must include a host".into()))?
.to_ascii_lowercase();
let port = url
.port_or_known_default()
.ok_or_else(|| Error::Config("redirectURI must include an http or https authority".into()))?;
let callback_path = normalize_callback_path(http_uri.path())?;
Ok(Self {
redirect_uri,
host,
port,
https,
callback_path,
})
}
pub fn canonical_uri(&self) -> String {
let scheme = if self.https { "https" } else { "http" };
let host = if self.host.contains(':') {
format!("[{}]", self.host)
} else {
self.host.clone()
};
format!("{scheme}://{host}:{}{}", self.port, self.callback_path)
}
}
impl Serialize for RedirectUri {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
self.redirect_uri.serialize(serializer)
}
}
impl<'de> Deserialize<'de> for RedirectUri {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
#[derive(Deserialize)]
#[serde(untagged)]
enum RedirectUriRepr {
String(String),
Object {
#[serde(rename = "redirectURI")]
redirect_uri: String,
},
}
let redirect_uri = match RedirectUriRepr::deserialize(deserializer)? {
RedirectUriRepr::String(redirect_uri) => redirect_uri,
RedirectUriRepr::Object { redirect_uri } => redirect_uri,
};
Self::parse(redirect_uri).map_err(serde::de::Error::custom)
}
}
fn normalize_callback_path(path: &str) -> Result<PathAndQuery, Error> {
if path.is_empty() || path == "/" {
return Err(Error::Config(
"redirectURI path must be explicit and non-root".into(),
));
}
if !path.starts_with('/') {
return Err(Error::Config("redirectURI path must start with '/'".into()));
}
// Reject dot segments and encoded slash/backslash forms so callback matching stays exact and
// does not depend on downstream path normalization behavior.
for segment in path.split('/').skip(1) {
let decoded = percent_decode_str(segment)
.decode_utf8()
.map_err(|_| Error::Config("redirectURI path contains invalid percent-encoding".into()))?;
if decoded == "." || decoded == ".." {
return Err(Error::Config(
"redirectURI path must not contain dot segments".into(),
));
}
if decoded.contains('/') || decoded.contains('\\') {
return Err(Error::Config(
"redirectURI path must not contain encoded slash or backslash characters".into(),
));
}
}
PathAndQuery::try_from(path).map_err(|e| Error::Config(format!("invalid redirectURI path: {e}")))
}

View File

@ -0,0 +1,309 @@
use std::fmt::Write as _;
use std::time::Duration;
use crate::http::sessionpersistence;
use base64::Engine;
use cookie::Cookie;
use cookie::SameSite;
use rand::RngExt;
use secrecy::ExposeSecret;
use secrecy::SecretString;
use serde::{Serialize, Serializer};
use super::{Error, PolicyId, now_unix};
pub const RESERVED_COOKIE_PREFIX: &str = "agw_oidc_";
// Use a conservative budget below the common ~4 KiB browser per-cookie limit so
// attributes and user-agent differences do not turn borderline values into
// silently dropped session cookies.
const MAX_BROWSER_COOKIE_VALUE_SIZE: usize = 3800;
const ORIGINAL_URI_LIMIT: usize = 2048;
pub(super) fn default_session_ttl() -> Duration {
Duration::from_secs(60 * 60)
}
pub(super) fn default_transaction_ttl() -> Duration {
Duration::from_secs(5 * 60)
}
#[derive(Debug, Clone, serde::Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct TransactionState {
pub policy_id: PolicyId,
pub transaction_id: String,
pub csrf_state: String,
pub nonce: String,
#[serde(serialize_with = "crate::serdes::ser_redact")]
pub pkce_verifier: SecretString,
pub original_uri: String,
pub expires_at_unix: u64,
}
impl Serialize for TransactionState {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
// Cookie payloads must serialize the raw secret values so the encrypted blob can be
// round-tripped. The field-level redact serializers are for external config/debug output.
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
struct SerializableTransactionState<'a> {
policy_id: &'a PolicyId,
transaction_id: &'a str,
csrf_state: &'a str,
nonce: &'a str,
pkce_verifier: &'a str,
original_uri: &'a str,
expires_at_unix: u64,
}
SerializableTransactionState {
policy_id: &self.policy_id,
transaction_id: &self.transaction_id,
csrf_state: &self.csrf_state,
nonce: &self.nonce,
pkce_verifier: self.pkce_verifier.expose_secret(),
original_uri: &self.original_uri,
expires_at_unix: self.expires_at_unix,
}
.serialize(serializer)
}
}
#[derive(Debug, Clone, serde::Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct BrowserSession {
pub policy_id: PolicyId,
#[serde(serialize_with = "crate::serdes::ser_redact")]
pub raw_id_token: SecretString,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub expires_at_unix: Option<u64>,
}
impl Serialize for BrowserSession {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
// Cookie payloads must serialize the raw secret values so the encrypted blob can be
// round-tripped. The field-level redact serializers are for external config/debug output.
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
struct SerializableBrowserSession<'a> {
policy_id: &'a PolicyId,
raw_id_token: &'a str,
expires_at_unix: Option<u64>,
}
SerializableBrowserSession {
policy_id: &self.policy_id,
raw_id_token: self.raw_id_token.expose_secret(),
expires_at_unix: self.expires_at_unix,
}
.serialize(serializer)
}
}
impl BrowserSession {
pub fn is_expired(&self) -> bool {
self
.expires_at_unix
.is_some_and(|expires_at| expires_at <= now_unix())
}
}
#[derive(Debug, Clone, serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub struct SessionConfig {
pub cookie_name: String,
pub transaction_cookie_prefix: String,
pub same_site: SameSiteMode,
pub secure: CookieSecureMode,
#[serde(with = "crate::serdes::serde_dur")]
pub ttl: Duration,
#[serde(with = "crate::serdes::serde_dur")]
pub transaction_ttl: Duration,
pub encoder: sessionpersistence::Encoder,
}
impl SessionConfig {
pub fn decode_transaction(&self, cookie: &str) -> Result<TransactionState, Error> {
let decoded = self
.encoder
.decrypt(cookie)
.map_err(|_| Error::InvalidTransaction)?;
let state: TransactionState =
serde_json::from_slice(&decoded).map_err(|_| Error::InvalidTransaction)?;
if state.expires_at_unix <= now_unix() {
return Err(Error::InvalidTransaction);
}
Ok(state)
}
pub fn encode_transaction(&self, state: &TransactionState) -> Result<String, Error> {
let json = serde_json::to_string(state).map_err(anyhow::Error::from)?;
self
.encoder
.encrypt(&json)
.map_err(|_| Error::InvalidTransaction)
}
pub fn decode_browser_session(&self, cookie: &str) -> Result<BrowserSession, Error> {
let decoded = self
.encoder
.decrypt(cookie)
.map_err(|_| Error::InvalidSession)?;
let session: BrowserSession =
serde_json::from_slice(&decoded).map_err(|_| Error::InvalidSession)?;
if session.is_expired() {
return Err(Error::InvalidSession);
}
Ok(session)
}
pub fn encode_browser_session(&self, session: &BrowserSession) -> Result<String, Error> {
let json = serde_json::to_string(session).map_err(anyhow::Error::from)?;
let encoded = self
.encoder
.encrypt(&json)
.map_err(|_| Error::InvalidSession)?;
if encoded.len() > MAX_BROWSER_COOKIE_VALUE_SIZE {
return Err(Error::SessionCookieTooLarge);
}
Ok(encoded)
}
pub fn set_cookie(&self, name: &str, value: &str, is_https: bool, ttl: Duration) -> String {
cookie_header(
name,
value,
self.same_site,
self.secure,
is_https,
Some(ttl),
false,
)
}
pub fn clear_cookie(&self, name: &str, is_https: bool) -> String {
cookie_header(
name,
"",
self.same_site,
self.secure,
is_https,
Some(Duration::ZERO),
true,
)
}
pub fn transaction_cookie_name(&self, transaction_id: &str) -> String {
format!("{}.{}", self.transaction_cookie_prefix, transaction_id)
}
}
#[derive(Debug, Clone, Copy, Default, serde::Serialize, serde::Deserialize, PartialEq, Eq)]
#[serde(rename_all = "camelCase")]
pub enum SameSiteMode {
#[default]
Lax,
Strict,
None,
}
#[derive(Debug, Clone, Copy, Default, serde::Serialize, serde::Deserialize, PartialEq, Eq)]
#[serde(rename_all = "camelCase")]
pub enum CookieSecureMode {
#[default]
Auto,
Always,
Never,
}
pub(super) fn derive_cookie_names(policy_id: &PolicyId) -> (String, String) {
let digest = aws_lc_rs::digest::digest(&aws_lc_rs::digest::SHA256, policy_id.as_str().as_bytes());
let mut hex = String::with_capacity(32);
for byte in digest.as_ref().iter().take(8) {
let _ = write!(&mut hex, "{byte:02x}");
}
(
format!("{RESERVED_COOKIE_PREFIX}s_{hex}"),
format!("{RESERVED_COOKIE_PREFIX}t_{hex}"),
)
}
pub(super) fn generate_nonce() -> String {
random_token(16)
}
pub(super) fn generate_state() -> String {
random_token(16)
}
pub(super) fn generate_transaction_id() -> String {
random_token(16)
}
pub(super) fn generate_pkce_verifier() -> String {
random_token(32)
}
/// Capture only local post-login redirect targets so callback can safely reflect the stored value.
pub(super) fn normalize_original_uri(path_and_query: Option<&http::uri::PathAndQuery>) -> String {
let original = path_and_query
.map(http::uri::PathAndQuery::as_str)
.unwrap_or("/");
if original.len() > ORIGINAL_URI_LIMIT || !is_safe_local_redirect_target(original) {
return "/".into();
}
original.to_string()
}
fn is_safe_local_redirect_target(target: &str) -> bool {
if !target.starts_with('/') || target.starts_with("//") || target.contains('\\') {
return false;
}
let decoded = percent_encoding::percent_decode_str(target).decode_utf8_lossy();
decoded.starts_with('/') && !decoded.starts_with("//") && !decoded.contains('\\')
}
fn random_token(bytes: usize) -> String {
let mut random = vec![0; bytes];
rand::rng().fill(random.as_mut_slice());
base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(random)
}
fn cookie_header(
name: &str,
value: &str,
same_site: SameSiteMode,
secure_mode: CookieSecureMode,
is_https: bool,
max_age: Option<Duration>,
expire_now: bool,
) -> String {
let secure = match secure_mode {
CookieSecureMode::Always => true,
CookieSecureMode::Never => false,
CookieSecureMode::Auto => is_https,
};
let mut cookie = Cookie::build((name, value))
.path("/")
.http_only(true)
.same_site(match same_site {
SameSiteMode::Lax => SameSite::Lax,
SameSiteMode::Strict => SameSite::Strict,
SameSiteMode::None => SameSite::None,
})
.secure(secure);
if let Some(max_age) = max_age {
let secs = if expire_now { 0 } else { max_age.as_secs() };
let secs = i64::try_from(secs).unwrap_or(i64::MAX);
cookie = cookie.max_age(cookie::time::Duration::seconds(secs));
}
cookie.build().to_string()
}

File diff suppressed because it is too large Load Diff

View File

@ -8,11 +8,9 @@ use agent_core::strng;
use crate::http::Request;
use crate::types::agent;
use crate::types::agent::{
BackendReference, HeaderMatch, HeaderValueMatch, Listener, ListenerProtocol, PathMatch,
QueryValueMatch, Route, RouteBackendReference, RouteMatch, RouteName,
BackendReference, HeaderMatch, HeaderValueMatch, Listener, PathMatch, QueryValueMatch, Route,
RouteBackendReference, RouteMatch, RouteName,
};
use crate::types::discovery::gatewayaddress::Destination;
use crate::types::discovery::{NetworkAddress, WaypointIdentity};
use crate::*;
#[cfg(any(test, feature = "internal_benches"))]
@ -101,8 +99,6 @@ fn matches_request(m: &RouteMatch, request: &Request) -> bool {
pub fn select_best_route(
stores: Stores,
network: Strng,
self_addr: Option<&WaypointIdentity>,
dst: SocketAddr,
listener: &Listener,
request: &Request,
@ -125,106 +121,71 @@ pub fn select_best_route(
// criteria.
let host = http::get_host(request).ok()?;
let (default_response, host) = if matches!(listener.protocol, ListenerProtocol::HBONE) {
let Some(self_id) = self_addr else {
warn!("waypoint requires self address");
return None;
};
// We are going to get a VIP request. Look up the Service
let svc = stores
.read_discovery()
.services
.get_by_vip(&NetworkAddress {
network,
address: dst.ip(),
})?;
let wp = svc.waypoint.as_ref()?;
// Make sure the service is actually bound to us
let is_ours = match &wp.destination {
Destination::Address(addr) => {
let stores_ref = stores.clone();
self_id.matches_address(addr, |ns, hostname| {
let discovery = stores_ref.read_discovery();
let self_svc = discovery.services.get_by_namespaced_host(
&crate::types::discovery::NamespacedHostname {
namespace: ns.clone(),
hostname: hostname.clone(),
},
)?;
Some(self_svc.vips.clone())
})
},
Destination::Hostname(n) => self_id.matches_hostname(n),
};
if !is_ours {
warn!(
"service {} is meant for waypoint {:?}, but we are {}.{}",
svc.hostname, wp.destination, self_id.gateway, self_id.namespace
);
return None;
}
// When routes are attached to a Service via parentRef, they take priority
// over listener-attached routes. If service routes exist but none match,
// the request is rejected (per GAMMA spec).
let svc_nh = svc.namespaced_hostname();
let (has_svc_routes, svc_route_match) = {
let binds = stores.read_binds();
match binds.get_service_routes(&svc_nh) {
Some(svc_routes) => {
let mut result = None;
for hnm in agent::HostnameMatch::all_matches(&svc.hostname) {
result = svc_routes
.get_hostname(&hnm)
.find(|(_, m)| matches_request(m, request))
.map(|(route, matcher)| (route, matcher.path.clone()));
if result.is_some() {
break;
let (default_response, host) =
if let Some(wps) = request.extensions().get::<crate::proxy::WaypointService>() {
// When routes are attached to a Service via parentRef, they take priority
// over listener-attached routes. If service routes exist but none match,
// the request is rejected (per GAMMA spec).
let svc = wps.as_ref();
let svc_nh = svc.namespaced_hostname();
let (has_svc_routes, svc_route_match) = {
let binds = stores.read_binds();
match binds.get_service_routes(&svc_nh) {
Some(svc_routes) => {
let mut result = None;
for hnm in agent::HostnameMatch::all_matches(&svc.hostname) {
result = svc_routes
.get_hostname(&hnm)
.find(|(_, m)| matches_request(m, request))
.map(|(route, matcher)| (route, matcher.path.clone()));
if result.is_some() {
break;
}
}
}
(true, result)
},
None => (false, None),
(true, result)
},
None => (false, None),
}
};
if let Some(result) = svc_route_match {
return Some(result);
}
if has_svc_routes {
// GAMMA: service routes exist but none matched -> reject
return None;
}
};
if let Some(result) = svc_route_match {
return Some(result);
}
if has_svc_routes {
// GAMMA: service routes exist but none matched -> reject
return None;
}
// No service-keyed routes: fall through to hostname matching with default route
let default_route = Route {
key: strng::literal!("_waypoint-default"),
service_key: None,
name: RouteName {
name: strng::literal!("_waypoint-default"),
namespace: svc.namespace.clone(),
rule_name: None,
kind: None,
},
hostnames: vec![],
matches: vec![],
inline_policies: vec![],
backends: vec![RouteBackendReference {
weight: 1,
backend: BackendReference::Service {
name: svc.namespaced_hostname(),
port: dst.port(), // TODO: get from req
// No service-keyed routes: fall through to hostname matching with default route
let default_route = Route {
key: strng::literal!("_waypoint-default"),
service_key: Some(svc.namespaced_hostname()),
name: RouteName {
name: strng::literal!("_waypoint-default"),
namespace: svc.namespace.clone(),
rule_name: None,
kind: None,
},
inline_policies: Vec::new(),
}],
hostnames: vec![],
matches: vec![],
inline_policies: vec![],
backends: vec![RouteBackendReference {
weight: 1,
backend: BackendReference::Service {
name: svc.namespaced_hostname(),
port: dst.port(), // TODO: get from req
},
inline_policies: Vec::new(),
}],
};
let def = Some((
Arc::new(default_route),
PathMatch::PathPrefix(strng::new("/")),
));
(def, Cow::Owned(svc.hostname.to_string()))
} else {
(None, Cow::Borrowed(host))
};
let def = Some((
Arc::new(default_route),
PathMatch::PathPrefix(strng::new("/")),
));
(def, Cow::Owned(svc.hostname.to_string()))
} else {
(None, Cow::Borrowed(host))
};
for hnm in agent::HostnameMatch::all_matches(&host) {
let mut candidates = listener.routes.get_hostname(&hnm);
let best_match = candidates.find(|(_, m)| matches_request(m, request));

View File

@ -15,26 +15,17 @@ use crate::types::agent::{
QueryValueMatch, Route, RouteMatch, RouteSet,
};
use crate::types::discovery::{
GatewayAddress, NamespacedHostname, NetworkAddress, Service, WaypointIdentity,
gatewayaddress::Destination,
GatewayAddress, NamespacedHostname, NetworkAddress, Service, gatewayaddress::Destination,
};
use crate::*;
fn run_test(req: &Request, routes: &[(&str, Vec<&str>, Vec<RouteMatch>)]) -> Option<String> {
let stores = Stores::with_ipv6_enabled(true);
let network = strng::literal!("network");
let dummy_dest = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), 1000);
let listener = setup_listener(routes);
let result = super::select_best_route(
stores.clone(),
network.clone(),
None,
dummy_dest,
&listener,
req,
);
let result = super::select_best_route(stores.clone(), dummy_dest, &listener, req);
result.map(|(r, _)| r.key.to_string())
}
@ -67,6 +58,15 @@ fn setup_listener(routes: &[(&str, Vec<&str>, Vec<RouteMatch>)]) -> Arc<Listener
})
}
fn attach_waypoint_service(req: &mut Request, stores: &Stores, service_key: &NamespacedHostname) {
let svc = stores
.read_discovery()
.services
.get_by_namespaced_host(service_key)
.expect("test service must exist in discovery store");
req.extensions_mut().insert(proxy::WaypointService(svc));
}
#[test]
fn test_hostname_matching() {
let basic_match = vec![RouteMatch {
@ -892,26 +892,16 @@ async fn test_waypoint_hostname_match() {
}),
);
let stores = stores_with_services(vec![svc]);
let self_id = WaypointIdentity {
gateway: strng::new("my-waypoint"),
namespace: strng::new("istio-system"),
};
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
let req = request(
let mut req = request(
"http://my-app.default.svc.cluster.local/",
http::Method::GET,
&[],
);
let listener = hbone_listener();
attach_waypoint_service(&mut req, &stores, &svc_nh());
let result = super::select_best_route(
stores,
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
let result = super::select_best_route(stores, dst, &listener, &req);
assert!(result.is_some(), "should return default waypoint route");
let (route, _) = result.unwrap();
assert_eq!(route.key.as_str(), "_waypoint-default");
@ -935,10 +925,6 @@ async fn test_waypoint_hostname_mismatch() {
}),
);
let stores = stores_with_services(vec![svc]);
let self_id = WaypointIdentity {
gateway: strng::new("my-waypoint"),
namespace: strng::new("istio-system"),
};
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
let req = request(
"http://my-app.default.svc.cluster.local/",
@ -947,14 +933,7 @@ async fn test_waypoint_hostname_mismatch() {
);
let listener = hbone_listener();
let result = super::select_best_route(
stores,
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
let result = super::select_best_route(stores, dst, &listener, &req);
assert!(
result.is_none(),
"should reject service bound to a different waypoint"
@ -979,26 +958,16 @@ async fn test_waypoint_hostname_fqdn_match() {
}),
);
let stores = stores_with_services(vec![svc]);
let self_id = WaypointIdentity {
gateway: strng::new("my-waypoint"),
namespace: strng::new("istio-system"),
};
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
let req = request(
let mut req = request(
"http://my-app.default.svc.cluster.local/",
http::Method::GET,
&[],
);
let listener = hbone_listener();
attach_waypoint_service(&mut req, &stores, &svc_nh());
let result = super::select_best_route(
stores,
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
let result = super::select_best_route(stores, dst, &listener, &req);
assert!(result.is_some(), "should match waypoint with FQDN hostname");
}
@ -1030,26 +999,16 @@ async fn test_waypoint_address_match() {
None,
);
let stores = stores_with_services(vec![svc, waypoint_svc]);
let self_id = WaypointIdentity {
gateway: strng::new("my-waypoint"),
namespace: strng::new("istio-system"),
};
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
let req = request(
let mut req = request(
"http://my-app.default.svc.cluster.local/",
http::Method::GET,
&[],
);
let listener = hbone_listener();
attach_waypoint_service(&mut req, &stores, &svc_nh());
let result = super::select_best_route(
stores,
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
let result = super::select_best_route(stores, dst, &listener, &req);
assert!(result.is_some(), "should match waypoint by address VIP");
let (route, _) = result.unwrap();
assert_eq!(route.key.as_str(), "_waypoint-default");
@ -1082,10 +1041,6 @@ async fn test_waypoint_address_mismatch() {
None,
);
let stores = stores_with_services(vec![svc, waypoint_svc]);
let self_id = WaypointIdentity {
gateway: strng::new("my-waypoint"),
namespace: strng::new("istio-system"),
};
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
let req = request(
"http://my-app.default.svc.cluster.local/",
@ -1094,14 +1049,7 @@ async fn test_waypoint_address_mismatch() {
);
let listener = hbone_listener();
let result = super::select_best_route(
stores,
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
let result = super::select_best_route(stores, dst, &listener, &req);
assert!(
result.is_none(),
"should reject service bound to a different waypoint address"
@ -1120,10 +1068,6 @@ async fn test_waypoint_no_waypoint_on_service() {
None, // no waypoint
);
let stores = stores_with_services(vec![svc]);
let self_id = WaypointIdentity {
gateway: strng::new("my-waypoint"),
namespace: strng::new("istio-system"),
};
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
let req = request(
"http://my-app.default.svc.cluster.local/",
@ -1132,14 +1076,7 @@ async fn test_waypoint_no_waypoint_on_service() {
);
let listener = hbone_listener();
let result = super::select_best_route(
stores,
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
let result = super::select_best_route(stores, dst, &listener, &req);
assert!(
result.is_none(),
"should return None for service without waypoint"
@ -1172,14 +1109,7 @@ async fn test_waypoint_no_self_addr() {
);
let listener = hbone_listener();
let result = super::select_best_route(
stores,
strng::literal!("network"),
None, // no self_addr
dst,
&listener,
&req,
);
let result = super::select_best_route(stores, dst, &listener, &req);
assert!(
result.is_none(),
"should return None when self_addr is not configured"
@ -1190,22 +1120,11 @@ async fn test_waypoint_no_self_addr() {
async fn test_waypoint_unknown_vip() {
// Request to a VIP that doesn't match any known service
let stores = stores_with_services(vec![]);
let self_id = WaypointIdentity {
gateway: strng::new("my-waypoint"),
namespace: strng::new("istio-system"),
};
let dst = SocketAddr::new("10.0.0.200".parse().unwrap(), 80);
let req = request("http://unknown.svc.cluster.local/", http::Method::GET, &[]);
let listener = hbone_listener();
let result = super::select_best_route(
stores,
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
let result = super::select_best_route(stores, dst, &listener, &req);
assert!(result.is_none(), "should return None for unknown VIP");
}
@ -1261,13 +1180,6 @@ fn waypoint_svc() -> Service {
)
}
fn waypoint_self_id() -> WaypointIdentity {
WaypointIdentity {
gateway: strng::new("my-waypoint"),
namespace: strng::new("istio-system"),
}
}
#[tokio::test]
async fn test_service_route_path_match() {
let stores = stores_with_service_routes(
@ -1295,40 +1207,27 @@ async fn test_service_route_path_match() {
),
],
);
let self_id = waypoint_self_id();
let listener = hbone_listener();
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
// /api/v1 matches the prefix route
let req = request(
let mut req = request(
"http://my-app.default.svc.cluster.local/api/v1",
http::Method::GET,
&[],
);
let result = super::select_best_route(
stores.clone(),
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
attach_waypoint_service(&mut req, &stores, &svc_nh());
let result = super::select_best_route(stores.clone(), dst, &listener, &req);
assert_eq!(result.unwrap().0.key.as_str(), "api-route");
// /healthz matches the exact route (higher priority than prefix)
let req = request(
let mut req = request(
"http://my-app.default.svc.cluster.local/healthz",
http::Method::GET,
&[],
);
let result = super::select_best_route(
stores.clone(),
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
attach_waypoint_service(&mut req, &stores, &svc_nh());
let result = super::select_best_route(stores.clone(), dst, &listener, &req);
assert_eq!(result.unwrap().0.key.as_str(), "health-route");
}
@ -1363,23 +1262,16 @@ async fn test_service_route_method_match() {
),
],
);
let self_id = waypoint_self_id();
let listener = hbone_listener();
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
let req = request(
let mut req = request(
"http://my-app.default.svc.cluster.local/",
http::Method::POST,
&[],
);
let result = super::select_best_route(
stores.clone(),
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
attach_waypoint_service(&mut req, &stores, &svc_nh());
let result = super::select_best_route(stores.clone(), dst, &listener, &req);
assert_eq!(result.unwrap().0.key.as_str(), "post-route");
}
@ -1401,40 +1293,27 @@ async fn test_service_route_header_match() {
}],
)],
);
let self_id = waypoint_self_id();
let listener = hbone_listener();
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
// With matching header -> matches
let req = request(
let mut req = request(
"http://my-app.default.svc.cluster.local/",
http::Method::GET,
&[("x-custom", "special")],
);
let result = super::select_best_route(
stores.clone(),
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
attach_waypoint_service(&mut req, &stores, &svc_nh());
let result = super::select_best_route(stores.clone(), dst, &listener, &req);
assert_eq!(result.unwrap().0.key.as_str(), "header-route");
// Without matching header -> GAMMA reject (service routes exist, none match)
let req = request(
let mut req = request(
"http://my-app.default.svc.cluster.local/",
http::Method::GET,
&[],
);
let result = super::select_best_route(
stores.clone(),
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
attach_waypoint_service(&mut req, &stores, &svc_nh());
let result = super::select_best_route(stores.clone(), dst, &listener, &req);
assert!(
result.is_none(),
"should reject when service routes exist but none match"
@ -1457,23 +1336,16 @@ async fn test_service_route_rejects_unmatched() {
}],
)],
);
let self_id = waypoint_self_id();
let listener = hbone_listener();
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
let req = request(
let mut req = request(
"http://my-app.default.svc.cluster.local/other",
http::Method::GET,
&[],
);
let result = super::select_best_route(
stores.clone(),
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
attach_waypoint_service(&mut req, &stores, &svc_nh());
let result = super::select_best_route(stores.clone(), dst, &listener, &req);
assert!(
result.is_none(),
"GAMMA: should reject when service routes exist but none match"
@ -1484,23 +1356,16 @@ async fn test_service_route_rejects_unmatched() {
async fn test_no_service_routes_falls_through_to_default() {
// No service-keyed routes -> default passthrough route
let stores = stores_with_services(vec![waypoint_svc()]);
let self_id = waypoint_self_id();
let listener = hbone_listener();
let dst = SocketAddr::new("10.0.0.100".parse().unwrap(), 80);
let req = request(
let mut req = request(
"http://my-app.default.svc.cluster.local/anything",
http::Method::GET,
&[],
);
let result = super::select_best_route(
stores,
strng::literal!("network"),
Some(&self_id),
dst,
&listener,
&req,
);
attach_waypoint_service(&mut req, &stores, &svc_nh());
let result = super::select_best_route(stores, dst, &listener, &req);
assert!(
result.is_some(),
"should fall through to default route when no service routes"
@ -1549,15 +1414,12 @@ fn bench(b: Bencher, (host, route): (u64, u64)) {
),
});
let stores = Stores::with_ipv6_enabled(true);
let network = strng::literal!("network");
let dummy_dest = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), 1000);
let req = request("http://example.com", http::Method::GET, &[]);
b.bench_local(|| {
divan::black_box(super::select_best_route(
stores.clone(),
network.clone(),
None,
dummy_dest,
&listener,
divan::black_box(&req),

View File

@ -187,6 +187,9 @@ mod aes {
pub fn decrypt(&self, encoded: &str) -> Result<Vec<u8>, Error> {
// Base64 decode
let data = STANDARD.decode(encoded).map_err(|_| Error::InvalidFormat)?;
if data.len() < 12 {
return Err(Error::InvalidFormat);
}
// Extract nonce and ciphertext
let (nonce_bytes, ciphertext) = data.split_at(12);
@ -212,4 +215,18 @@ mod aes {
#[error("invalid format")]
InvalidFormat,
}
#[cfg(test)]
mod tests {
use base64::Engine;
use super::{Encoder, Error};
#[test]
fn short_ciphertexts_fail_cleanly() {
let encoder = Encoder::new(&[0u8; 32]).expect("encoder");
let short = base64::engine::general_purpose::STANDARD.encode([0u8; 11]);
assert!(matches!(encoder.decrypt(&short), Err(Error::InvalidFormat)));
}
}
}

View File

@ -52,6 +52,7 @@ use telemetry::{metrics, trc};
use crate::control::{AuthSource, RootCert};
use crate::telemetry::trc::Protocol;
use crate::types::agent::{ListenerTarget, PolicyTargetRef};
use crate::types::local;
#[derive(serde::Deserialize, Clone, Debug)]
#[serde(rename_all = "camelCase")]
@ -470,6 +471,8 @@ pub struct Config {
pub proxy_metadata: ProxyMetadata,
pub threading_mode: ThreadingMode,
pub session_encoder: http::sessionpersistence::Encoder,
/// Runtime cookie/session crypto for browser OIDC flows.
pub oidc_cookie_encoder: Option<http::sessionpersistence::Encoder>,
/// Handle for tasks/spans emitted on the admin runtime.
#[serde(skip)]
pub admin_runtime_handle: Option<tokio::runtime::Handle>,
@ -492,6 +495,15 @@ impl Config {
listener_name: None,
}
}
pub fn as_policy_context(
&self,
policy_key: impl std::fmt::Display,
) -> Option<local::AttachedPolicyContext> {
Some(local::AttachedPolicyContext {
oidc_policy_id: crate::http::oidc::PolicyId::policy(&policy_key),
oidc_cookie_encoder: self.oidc_cookie_encoder.as_ref(),
})
}
}
#[derive(serde::Serialize, Copy, PartialOrd, PartialEq, Eq, Clone, Debug, Default)]

View File

@ -556,7 +556,10 @@ pub mod from_completions {
if let Some(caching) = prompt_caching {
if caching.cache_messages && supports_caching {
helpers::insert_cache_point_in_last_user_message(&mut bedrock_request.messages);
helpers::insert_message_cache_point(
&mut bedrock_request.messages,
caching.cache_message_offset,
);
}
if caching.cache_tools
&& supports_caching
@ -833,6 +836,8 @@ pub mod from_completions {
cache_creation_input_tokens: usage.cache_write_input_tokens.map(|i| i as u64),
prompt_tokens_details: usage.cache_read_input_tokens.map(|i| UsagePromptDetails {
cached_tokens: Some(i as u64),
audio_tokens: None,
rest: Default::default(),
}),
// TODO: can we get reasoning tokens?
completion_tokens_details: None,
@ -1373,7 +1378,10 @@ pub mod from_messages {
output_tokens: 0,
cache_creation_input_tokens: None,
cache_read_input_tokens: None,
service_tier: None,
},
input_audio_tokens: None,
output_audio_tokens: None,
},
};
let (event_name, event_data) = event.into_sse_tuple();
@ -2019,7 +2027,7 @@ pub mod from_responses {
// Apply user message and tool caching
if let Some(caching) = prompt_caching {
if caching.cache_messages && supports_caching {
insert_cache_point_in_last_user_message(&mut bedrock_request.messages);
insert_message_cache_point(&mut bedrock_request.messages, caching.cache_message_offset);
}
if caching.cache_tools
&& supports_caching
@ -2165,8 +2173,12 @@ pub mod from_responses {
let mut pending_usage: Option<bedrock::TokenUsage> = None;
let mut seen_blocks: HashSet<i32> = HashSet::new();
// Track tool calls for streaming: (index -> (item_id, name, json_buffer))
let mut tool_calls: HashMap<i32, (String, String, String)> = HashMap::new();
// Track tool calls for streaming: (content_block_index -> (item_id, name, json_buffer, output_index))
// output_index is the stable position of this tool call in the response output array.
let mut tool_calls: HashMap<i32, (String, String, String, u32)> = HashMap::new();
// Message item is always output_index 0; tool call items get sequential indices from 1.
let mut next_output_index: u32 = 1;
// Track sequence numbers and item IDs
let mut sequence_number: u64 = 0;
@ -2226,6 +2238,7 @@ pub mod from_responses {
content: Vec::new(),
id: message_item_id.clone(),
role: AssistantRole::Assistant,
phase: None,
status: OutputStatus::InProgress,
}),
});
@ -2239,19 +2252,27 @@ pub mod from_responses {
match start.start {
Some(bedrock::ContentBlockStart::ToolUse(tu)) => {
let tool_call_item_id = format!("call_{:016x}", rand::rng().random::<u64>());
let output_index = next_output_index;
next_output_index += 1;
tool_calls.insert(
start.content_block_index,
(tool_call_item_id.clone(), tu.name.clone(), String::new()),
(
tool_call_item_id.clone(),
tu.name.clone(),
String::new(),
output_index,
),
);
sequence_number += 1;
let item_added_event =
ResponseStreamEvent::ResponseOutputItemAdded(ResponseOutputItemAddedEvent {
sequence_number,
output_index: start.content_block_index as u32,
output_index,
item: OutputItem::FunctionCall(FunctionToolCall {
arguments: String::new(),
call_id: tool_call_item_id.clone(),
namespace: None,
name: tu.name,
id: Some(tool_call_item_id),
status: Some(OutputStatus::InProgress),
@ -2266,7 +2287,7 @@ pub mod from_responses {
ResponseStreamEvent::ResponseContentPartAdded(ResponseContentPartAddedEvent {
sequence_number,
item_id: message_item_id.clone(),
output_index: start.content_block_index as u32,
output_index: 0,
content_index: 0,
part: make_output_part(String::new()),
});
@ -2293,7 +2314,7 @@ pub mod from_responses {
ResponseStreamEvent::ResponseOutputTextDelta(ResponseTextDeltaEvent {
sequence_number,
item_id: message_item_id.clone(),
output_index: delta.content_block_index as u32,
output_index: 0,
content_index: 0,
delta: text,
logprobs: None,
@ -2307,7 +2328,7 @@ pub mod from_responses {
ResponseStreamEvent::ResponseOutputTextDelta(ResponseTextDeltaEvent {
sequence_number,
item_id: message_item_id.clone(),
output_index: delta.content_block_index as u32,
output_index: 0,
content_index: 0,
delta: t,
logprobs: None,
@ -2320,7 +2341,7 @@ pub mod from_responses {
ResponseStreamEvent::ResponseOutputTextDelta(ResponseTextDeltaEvent {
sequence_number,
item_id: message_item_id.clone(),
output_index: delta.content_block_index as u32,
output_index: 0,
content_index: 0,
delta: "[REDACTED]".to_string(),
logprobs: None,
@ -2330,7 +2351,7 @@ pub mod from_responses {
_ => {},
},
bedrock::ContentBlockDelta::ToolUse(tu) => {
if let Some((item_id, _name, buffer)) =
if let Some((item_id, _name, buffer, output_index)) =
tool_calls.get_mut(&delta.content_block_index)
{
buffer.push_str(&tu.input);
@ -2340,7 +2361,7 @@ pub mod from_responses {
ResponseFunctionCallArgumentsDeltaEvent {
sequence_number,
item_id: item_id.clone(),
output_index: delta.content_block_index as u32,
output_index: *output_index,
delta: tu.input,
},
);
@ -2354,15 +2375,18 @@ pub mod from_responses {
},
bedrock::ConverseStreamOutput::ContentBlockStop(stop) => {
let mut events: Vec<(&'static str, ResponseStreamEvent)> = Vec::new();
let was_tracked = seen_blocks.remove(&stop.content_block_index);
if let Some((item_id, name, buffer)) = tool_calls.remove(&stop.content_block_index) {
if let Some((item_id, name, buffer, output_index)) =
tool_calls.remove(&stop.content_block_index)
{
sequence_number += 1;
let args_done_event = ResponseStreamEvent::ResponseFunctionCallArgumentsDone(
ResponseFunctionCallArgumentsDoneEvent {
name: Some(name.clone()),
sequence_number,
item_id: item_id.clone(),
output_index: stop.content_block_index as u32,
output_index,
arguments: buffer.clone(),
},
);
@ -2372,23 +2396,24 @@ pub mod from_responses {
let item_done_event =
ResponseStreamEvent::ResponseOutputItemDone(ResponseOutputItemDoneEvent {
sequence_number,
output_index: stop.content_block_index as u32,
output_index,
item: OutputItem::FunctionCall(FunctionToolCall {
arguments: buffer,
call_id: item_id.clone(),
namespace: None,
name,
id: Some(item_id),
status: Some(OutputStatus::Completed),
}),
});
events.push(("event", item_done_event));
} else if seen_blocks.remove(&stop.content_block_index) {
} else if was_tracked {
sequence_number += 1;
let part_done_event =
ResponseStreamEvent::ResponseContentPartDone(ResponseContentPartDoneEvent {
sequence_number,
item_id: message_item_id.clone(),
output_index: stop.content_block_index as u32,
output_index: 0,
content_index: 0,
part: make_output_part(String::new()),
});
@ -2425,6 +2450,7 @@ pub mod from_responses {
content: Vec::new(),
id: message_item_id.clone(),
role: AssistantRole::Assistant,
phase: None,
status: OutputStatus::Completed,
}),
});
@ -2561,7 +2587,7 @@ mod helpers {
(word_count * 13) / 10
}
pub fn insert_cache_point_in_last_user_message(messages: &mut [bedrock::Message]) {
pub fn insert_message_cache_point(messages: &mut [bedrock::Message], offset: usize) {
// Strategy: Cache everything BEFORE the last message (not including it)
// This caches the conversation history but not the current turn's input
//
@ -2572,6 +2598,10 @@ mod helpers {
// This way:
// - Conversation history: cached (cheap reads on subsequent turns)
// - Current input: full price (it's new each turn anyway)
//
// The `offset` parameter shifts the cache point further back:
// offset 0 → second-to-last message (default)
// offset N → N additional messages back from default, clamped to bounds
let len = messages.len();
@ -2580,16 +2610,16 @@ mod helpers {
return;
}
// Insert cache point in the second-to-last message
// This caches all history BEFORE the current turn
let second_to_last_idx = len - 2;
messages[second_to_last_idx]
// Clamp so the index never goes below 0
let target_idx = (len - 2).saturating_sub(offset);
messages[target_idx]
.content
.push(bedrock::ContentBlock::CachePoint(create_cache_point()));
tracing::debug!(
"Inserted cachePoint before last message (in message at index {})",
second_to_last_idx
"Inserted cachePoint in message at index {} (offset={})",
target_idx,
offset
);
}
@ -2796,6 +2826,8 @@ impl ConverseResponseAdapter {
.cache_read_input_tokens
.map(|i| UsagePromptDetails {
cached_tokens: Some(i as u64),
audio_tokens: None,
rest: Default::default(),
}),
cache_creation_input_tokens: token_usage.cache_write_input_tokens.map(|i| i as u64),
})
@ -2858,6 +2890,7 @@ impl ConverseResponseAdapter {
responsest::FunctionToolCall {
arguments: arguments_str,
call_id: tool_use.tool_use_id.clone(),
namespace: None,
name: tool_use.name.clone(),
id: Some(tool_use.tool_use_id.clone()),
status: Some(responsest::OutputStatus::Completed),
@ -2876,6 +2909,7 @@ impl ConverseResponseAdapter {
outputs.push(responsest::OutputItem::Message(responsest::OutputMessage {
id: format!("msg_{:016x}", rand::rng().random::<u64>()),
role: responsest::AssistantRole::Assistant,
phase: None,
content: text_parts,
status: responsest::OutputStatus::Completed,
}));
@ -2997,12 +3031,14 @@ impl ConverseResponseAdapter {
output_tokens: u.output_tokens,
cache_creation_input_tokens: u.cache_write_input_tokens,
cache_read_input_tokens: u.cache_read_input_tokens,
service_tier: None,
})
.unwrap_or(messagest::Usage {
input_tokens: 0,
output_tokens: 0,
cache_creation_input_tokens: None,
cache_read_input_tokens: None,
service_tier: None,
});
Ok(messagest::MessagesResponse {
@ -3014,6 +3050,8 @@ impl ConverseResponseAdapter {
stop_reason: Some(from_messages::translate_stop_reason(self.stop_reason)),
stop_sequence: None,
usage,
input_audio_tokens: None,
output_audio_tokens: None,
})
}
}

View File

@ -1225,3 +1225,77 @@ fn test_embeddings_error_translation() {
assert_eq!(error_resp["error"]["type"], "invalid_request_error");
assert_eq!(error_resp["error"]["message"], "Model not found");
}
fn make_message(role: types::bedrock::Role, text: &str) -> types::bedrock::Message {
types::bedrock::Message {
role,
content: vec![types::bedrock::ContentBlock::Text(text.to_string())],
}
}
fn has_cache_point(msg: &types::bedrock::Message) -> bool {
msg
.content
.iter()
.any(|b| matches!(b, types::bedrock::ContentBlock::CachePoint(_)))
}
#[test]
fn test_insert_cache_point_default_offset() {
let mut msgs = vec![
make_message(types::bedrock::Role::User, "Hello"),
make_message(types::bedrock::Role::Assistant, "Hi"),
make_message(types::bedrock::Role::User, "How are you?"),
];
helpers::insert_message_cache_point(&mut msgs, 0);
assert!(has_cache_point(&msgs[1]));
assert!(!has_cache_point(&msgs[0]));
assert!(!has_cache_point(&msgs[2]));
}
#[test]
fn test_insert_cache_point_offset_shifts_back() {
let mut msgs = vec![
make_message(types::bedrock::Role::User, "a"),
make_message(types::bedrock::Role::Assistant, "b"),
make_message(types::bedrock::Role::User, "c"),
make_message(types::bedrock::Role::Assistant, "d"),
make_message(types::bedrock::Role::User, "e"),
];
helpers::insert_message_cache_point(&mut msgs, 2);
// default position is index 3 (len-2), offset 2 → index 1
assert!(has_cache_point(&msgs[1]));
for (i, msg) in msgs.iter().enumerate() {
if i != 1 {
assert!(!has_cache_point(msg));
}
}
}
#[test]
fn test_insert_cache_point_offset_clamps_to_zero() {
let mut msgs = vec![
make_message(types::bedrock::Role::User, "a"),
make_message(types::bedrock::Role::Assistant, "b"),
make_message(types::bedrock::Role::User, "c"),
];
// offset 100 should clamp to index 0
helpers::insert_message_cache_point(&mut msgs, 100);
assert!(has_cache_point(&msgs[0]));
assert!(!has_cache_point(&msgs[1]));
assert!(!has_cache_point(&msgs[2]));
}
#[test]
fn test_insert_cache_point_single_message_noop() {
let mut msgs = vec![make_message(types::bedrock::Role::User, "only")];
helpers::insert_message_cache_point(&mut msgs, 0);
assert!(!has_cache_point(&msgs[0]));
}
#[test]
fn test_insert_cache_point_empty_messages_noop() {
let mut msgs: Vec<types::bedrock::Message> = vec![];
helpers::insert_message_cache_point(&mut msgs, 0);
assert!(msgs.is_empty());
}

View File

@ -160,6 +160,7 @@ pub mod from_messages {
choices,
model,
usage,
service_tier,
..
} = resp;
// Anthropic only supports one choice
@ -221,7 +222,20 @@ pub mod from_messages {
.unwrap_or(0),
cache_creation_input_tokens: None,
cache_read_input_tokens: None,
service_tier,
},
input_audio_tokens: usage.as_ref().and_then(|u| {
u.prompt_tokens_details
.as_ref()
.and_then(|d| d.audio_tokens)
.map(|t| t as usize)
}),
output_audio_tokens: usage.as_ref().and_then(|u| {
u.completion_tokens_details
.as_ref()
.and_then(|d| d.audio_tokens)
.map(|t| t as usize)
}),
content,
})
}
@ -464,7 +478,10 @@ pub mod from_messages {
output_tokens: 0,
cache_creation_input_tokens: None,
cache_read_input_tokens: None,
service_tier: None,
},
input_audio_tokens: None,
output_audio_tokens: None,
},
},
);
@ -819,6 +836,13 @@ pub mod from_messages {
})
.collect_vec();
// "Function tools with reasoning_effort are not supported for gpt-5.4
let reasoning_effort = if !tools.is_empty() && model.starts_with("gpt-5.4") {
None
} else {
reasoning_effort
};
let mut parallel_tool_calls = None;
let tool_choice = tool_choice.map(|choice| match choice {
messages::ToolChoice::Auto {
@ -939,12 +963,23 @@ pub fn passthrough_stream(
}
if !seen_provider {
seen_provider = true;
log.non_atomic_mutate(|r| r.response.provider_model = Some(strng::new(&f.model)));
log.non_atomic_mutate(|r| {
r.response.provider_model = Some(strng::new(&f.model));
r.response.service_tier = f.service_tier.as_deref().map(Into::into);
});
}
if let Some(u) = f.usage {
log.non_atomic_mutate(|r| {
r.response.input_tokens = Some(u.prompt_tokens as u64);
r.response.input_audio_tokens = u
.prompt_tokens_details
.as_ref()
.and_then(|d| d.audio_tokens);
r.response.output_tokens = Some(u.completion_tokens as u64);
r.response.output_audio_tokens = u
.completion_tokens_details
.as_ref()
.and_then(|d| d.audio_tokens);
r.response.total_tokens = Some(u.total_tokens as u64);
r.response.cached_input_tokens = u
.prompt_tokens_details

View File

@ -474,6 +474,8 @@ pub mod from_completions {
.cache_read_input_tokens
.map(|i| UsagePromptDetails {
cached_tokens: Some(i as u64),
audio_tokens: None,
rest: Default::default(),
}),
cache_creation_input_tokens: resp.usage.cache_creation_input_tokens.map(|i| i as u64),
@ -488,7 +490,7 @@ pub mod from_completions {
model: resp.model,
choices,
usage: Some(usage),
service_tier: None,
service_tier: resp.usage.service_tier,
system_fingerprint: None,
}
}
@ -514,6 +516,7 @@ pub mod from_completions {
pub fn translate_stream(b: Body, buffer_limit: usize, log: AmendOnDrop) -> Body {
let mut message_id = None;
let mut model = String::new();
let mut service_tier = None;
let created = chrono::Utc::now().timestamp() as u32;
// let mut finish_reason = None;
let mut saw_token = false;
@ -532,7 +535,7 @@ pub mod from_completions {
model: model.clone(),
object: "chat.completion.chunk".to_string(),
system_fingerprint: None,
service_tier: None,
service_tier: service_tier.clone(),
created,
choices,
usage,
@ -546,6 +549,7 @@ pub mod from_completions {
messages::MessagesStreamEvent::MessageStart { message } => {
message_id = Some(message.id);
model = message.model.clone();
service_tier = message.usage.service_tier.clone();
log.non_atomic_mutate(|r| {
r.response.output_tokens = Some(message.usage.output_tokens as u64);
r.response.input_tokens = Some(message.usage.input_tokens as u64);
@ -553,6 +557,7 @@ pub mod from_completions {
message.usage.cache_read_input_tokens.map(|i| i as u64);
r.response.cache_creation_input_tokens =
message.usage.cache_creation_input_tokens.map(|i| i as u64);
r.response.service_tier = message.usage.service_tier.as_deref().map(Into::into);
r.response.provider_model = Some(strng::new(&message.model))
});
// no need to respond with anything yet
@ -676,6 +681,8 @@ pub mod from_completions {
cache_read_input_tokens: usage.cache_read_input_tokens.map(|i| i as u64),
prompt_tokens_details: usage.cache_read_input_tokens.map(|i| UsagePromptDetails {
cached_tokens: Some(i as u64),
audio_tokens: None,
rest: Default::default(),
}),
cache_creation_input_tokens: usage.cache_creation_input_tokens.map(|i| i as u64),
@ -723,6 +730,7 @@ pub fn passthrough_stream(b: Body, buffer_limit: usize, log: AmendOnDrop) -> Bod
r.response.cached_input_tokens = message.usage.cache_read_input_tokens.map(|i| i as u64);
r.response.cache_creation_input_tokens =
message.usage.cache_creation_input_tokens.map(|i| i as u64);
r.response.service_tier = message.usage.service_tier.as_deref().map(Into::into);
r.response.provider_model = Some(strng::new(&message.model))
});
},

View File

@ -20,6 +20,11 @@ pub fn passthrough_stream(b: Body, buffer_limit: usize, log: AmendOnDrop) -> Bod
types::responses::typed::ResponseStreamEvent::ResponseCreated(created) => {
log.non_atomic_mutate(|r| {
r.response.provider_model = Some(strng::new(&created.response.model));
r.response.service_tier = created
.response
.service_tier
.as_ref()
.and_then(types::serialize_str);
if let Some(usage) = &created.response.usage {
r.response.input_tokens = Some(usage.input_tokens as u64);
r.response.output_tokens = Some(usage.output_tokens as u64);
@ -42,6 +47,11 @@ pub fn passthrough_stream(b: Body, buffer_limit: usize, log: AmendOnDrop) -> Bod
types::responses::typed::ResponseStreamEvent::ResponseCompleted(completed) => {
log.non_atomic_mutate(|r| {
r.response.provider_model = Some(strng::new(&completed.response.model));
r.response.service_tier = completed
.response
.service_tier
.as_ref()
.and_then(types::serialize_str);
if let Some(usage) = &completed.response.usage {
r.response.input_tokens = Some(usage.input_tokens as u64);
r.response.output_tokens = Some(usage.output_tokens as u64);

View File

@ -225,6 +225,12 @@ impl LLMInfo {
pub struct LLMResponse {
#[serde(skip_serializing_if = "Option::is_none")]
pub input_tokens: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub input_image_tokens: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub input_text_tokens: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub input_audio_tokens: Option<u64>,
/// count_tokens contains the number of tokens in the request, when using the token counting endpoint
/// These are not counted as 'input tokens' since they do not consume input tokens.
#[serde(skip_serializing_if = "Option::is_none")]
@ -232,6 +238,12 @@ pub struct LLMResponse {
#[serde(skip_serializing_if = "Option::is_none")]
pub output_tokens: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub output_image_tokens: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub output_text_tokens: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub output_audio_tokens: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub total_tokens: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub reasoning_tokens: Option<u64>,
@ -239,7 +251,8 @@ pub struct LLMResponse {
pub cache_creation_input_tokens: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub cached_input_tokens: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub service_tier: Option<Strng>,
#[serde(skip_serializing_if = "Option::is_none")]
pub provider_model: Option<Strng>,
#[serde(skip_serializing_if = "Option::is_none")]

View File

@ -187,6 +187,9 @@ pub struct PromptCachingConfig {
#[serde(rename = "minTokens")]
pub min_tokens: Option<usize>,
#[serde(rename = "cacheMessageOffset")]
pub cache_message_offset: usize,
}
impl Default for PromptCachingConfig {
@ -196,6 +199,7 @@ impl Default for PromptCachingConfig {
cache_messages: true,
cache_tools: false,
min_tokens: Some(1024),
cache_message_offset: 0,
}
}
}

View File

@ -161,6 +161,7 @@ fn test_prompt_caching_policy_deserialization() {
assert!(caching.cache_messages);
assert!(!caching.cache_tools);
assert_eq!(caching.min_tokens, Some(1024));
assert_eq!(caching.cache_message_offset, 0);
}
#[test]
@ -179,6 +180,7 @@ fn test_prompt_caching_policy_defaults() {
assert!(caching.cache_messages); // Default: true
assert!(!caching.cache_tools); // Default: false
assert_eq!(caching.min_tokens, Some(1024)); // Default: 1024
assert_eq!(caching.cache_message_offset, 0); // Default: 0
}
#[test]
@ -212,6 +214,24 @@ fn test_prompt_caching_explicit_disable() {
assert!(policy.prompt_caching.is_none());
}
#[test]
fn test_prompt_caching_with_offset() {
use serde_json::json;
let json = json!({
"promptCaching": {
"cacheMessages": true,
"cacheMessageOffset": 4
}
});
let policy: Policy = serde_json::from_value(json).unwrap();
let caching = policy.prompt_caching.unwrap();
assert!(caching.cache_messages);
assert_eq!(caching.cache_message_offset, 4);
}
#[test]
fn test_resolve_route() {
let mut routes = IndexMap::new();

View File

@ -382,7 +382,8 @@ mod response {
BEDROCK_TO_RESPONSES,
];
const BEDROCK_RESPONSES: &[(&str, &[&str])] = &[("basic", ALL_BEDROCK), ("tool", ALL_BEDROCK)];
const BEDROCK_STREAM_RESPONSES: &[(&str, &[&str])] = &[("basic", ALL_BEDROCK)];
const BEDROCK_STREAM_RESPONSES: &[(&str, &[&str])] =
&[("basic", ALL_BEDROCK), ("tool", ALL_BEDROCK)];
const ALL_ANTHROPIC: &[&str] = &[
MESSAGES_TO_MESSAGES,
@ -406,7 +407,10 @@ mod response {
];
const COMPLETIONS_RESPONSES: &[(&str, &[&str])] = &[
("basic", ALL_COMPLETIONS),
("audio", ALL_COMPLETIONS),
("openrouter_reasoning", ALL_COMPLETIONS),
("gemini_zero_completion_tokens", ALL_COMPLETIONS),
("gemini_with_completion_tokens", ALL_COMPLETIONS),
];
const COMPLETIONS_STREAM_RESPONSES: &[(&str, &[&str])] = &[("stream", ALL_COMPLETIONS)];
@ -420,11 +424,13 @@ mod response {
const ALL_RESPONSES: &[&str] = &[RESPONSES_TO_RESPONSES, RESPONSES_TO_DETECT];
const RESPONSES_RESPONSES: &[(&str, &[&str])] = &[("basic", ALL_RESPONSES)];
const RESPONSES_STREAM_RESPONSES: &[(&str, &[&str])] = &[("stream", ALL_RESPONSES)];
const RESPONSES_STREAM_RESPONSES: &[(&str, &[&str])] =
&[("stream", ALL_RESPONSES), ("stream-image", ALL_RESPONSES)];
const DETECT_RESPONSES: &[(&str, &[&str])] = &[
("non-json", &[COMPLETIONS_TO_DETECT]),
("broken-sse", &[COMPLETIONS_TO_DETECT]),
("stream-image-generation", &[COMPLETIONS_TO_DETECT]),
];
#[tokio::test]
@ -869,23 +875,27 @@ fn test_prompt_enrichment() {
"requests/policies/openai_with_messages.json",
apply_test_prompts,
);
test_request::<types::responses::Request>(
OPENAI,
"requests/policies/openai_with_text_input.json",
apply_test_prompts,
);
test_request::<types::responses::Request>(
OPENAI,
"requests/responses/assistant-history.json",
apply_test_prompts,
);
}
#[test]
fn test_get_messages() {
use crate::llm::types::RequestType;
let input_path = fixture_path("requests/completions/full.json");
let input_str = &fs::read_to_string(&input_path).expect("Failed to read input file");
let input_raw: Value = serde_json::from_str(input_str).expect("Failed to parse input json");
fn extract_messages<R: RequestType + DeserializeOwned>(
input: &str,
path: &Path,
raw: &Value,
provider: &str,
) {
let request: R = serde_json::from_str(input).expect("Failed to parse json");
fn extract_messages<R: RequestType + DeserializeOwned>(fixture: &str, provider: &str) {
let path = fixture_path(fixture);
let input_str = fs::read_to_string(&path).expect("Failed to read input file");
let raw: Value = serde_json::from_str(&input_str).expect("Failed to parse input json");
let request: R = serde_json::from_str(&input_str).expect("Failed to parse json");
let out: Vec<Value> = request
.get_messages()
@ -898,10 +908,9 @@ fn test_get_messages() {
})
.collect();
let (snapshot_path, snapshot_name) =
snapshot_path_and_name("requests/completions/full.json", provider);
let (snapshot_path, snapshot_name) = snapshot_path_and_name(fixture, provider);
insta::with_settings!({
info => raw,
info => &raw,
description => path.to_string_lossy().to_string(),
omit_expression => true,
prepend_module_to_snapshot => false,
@ -912,17 +921,17 @@ fn test_get_messages() {
}
extract_messages::<types::completions::Request>(
input_str,
&input_path,
&input_raw,
"requests/completions/full.json",
"get-messages-completions",
);
extract_messages::<types::messages::Request>(
input_str,
&input_path,
&input_raw,
"requests/completions/full.json",
"get-messages-messages",
);
extract_messages::<types::responses::Request>(
"requests/responses/assistant-history.json",
"get-messages-responses",
);
}
/// Verifies that `process_response` routes a non-success response through
@ -1051,3 +1060,25 @@ fn setup_request_openai_normalizes_trailing_slash_in_path_prefix() {
assert_eq!(req.uri().path(), "/v1/custom/chat/completions");
assert_eq!(req.uri().query(), Some("trace=repro"));
}
#[test]
fn completions_response_missing_message_and_usage_fields() {
// Gemini's OpenAI-compat endpoint can omit `message` from choices and
// `completion_tokens` from usage. Verify deserialization succeeds with defaults.
let json = r#"{
"id": "1",
"object": "chat.completion",
"created": 0,
"model": "google/gemini-2.5-flash",
"choices": [{"index": 0, "finish_reason": "length"}],
"usage": {"prompt_tokens": 5, "total_tokens": 12}
}"#;
let resp: types::completions::Response = serde_json::from_str(json).unwrap();
assert_eq!(resp.choices.len(), 1);
assert_eq!(resp.choices[0].message.content, None);
assert_eq!(resp.choices[0].message.role, None);
let usage = resp.usage.unwrap();
assert_eq!(usage.prompt_tokens, 5);
assert_eq!(usage.completion_tokens, 0);
assert_eq!(usage.total_tokens, 12);
}

View File

@ -8,9 +8,15 @@ info:
type: adaptive
output_config:
effort: high
output_format: markdown
messages:
- role: user
content: Give one concise insight about distributed consensus.
content:
- type: text
text: Give one concise insight about distributed consensus.
cache_control:
type: ephemeral
scope: turn
---
{
"modelId": "claude-opus-4-6",
@ -20,6 +26,11 @@ info:
"content": [
{
"text": "Give one concise insight about distributed consensus."
},
{
"cachePoint": {
"type": "default"
}
}
]
}

View File

@ -8,9 +8,15 @@ info:
type: adaptive
output_config:
effort: high
output_format: markdown
messages:
- role: user
content: Give one concise insight about distributed consensus.
content:
- type: text
text: Give one concise insight about distributed consensus.
cache_control:
type: ephemeral
scope: turn
---
{
"messages": [

View File

@ -7,10 +7,20 @@
"output_config": {
"effort": "high"
},
"output_format": "markdown",
"messages": [
{
"role": "user",
"content": "Give one concise insight about distributed consensus."
"content": [
{
"type": "text",
"text": "Give one concise insight about distributed consensus.",
"cache_control": {
"type": "ephemeral",
"scope": "turn"
}
}
]
}
]
}

View File

@ -8,23 +8,34 @@ info:
type: adaptive
output_config:
effort: high
output_format: markdown
messages:
- role: user
content: Give one concise insight about distributed consensus.
content:
- type: text
text: Give one concise insight about distributed consensus.
cache_control:
type: ephemeral
scope: turn
---
{
"messages": [
{
"role": "user",
"content": "Give one concise insight about distributed consensus."
"content": [
{
"type": "text",
"text": "Give one concise insight about distributed consensus.",
"cache_control": {
"type": "ephemeral"
}
}
]
}
],
"anthropic_version": "vertex-2023-10-16",
"max_tokens": 1024,
"thinking": {
"type": "adaptive"
},
"output_config": {
"effort": "high"
}
}

View File

@ -1,5 +1,6 @@
---
source: crates/agentgateway/src/llm/tests.rs
assertion_line: 136
description: src/llm/tests/requests/policies/openai_with_inputs.json
info:
model: gpt-4o
@ -40,13 +41,13 @@ info:
},
{
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "Hello, world"
}
],
"role": "user"
]
},
{
"type": "message",

View File

@ -0,0 +1,5 @@
{
"model": "gpt-4o",
"max_tokens": 1024,
"input": "Hello, world"
}

View File

@ -0,0 +1,74 @@
---
source: crates/agentgateway/src/llm/tests.rs
description: src/llm/tests/requests/policies/openai_with_text_input.json
info:
model: gpt-4o
max_tokens: 1024
input: "Hello, world"
---
{
"input": [
{
"type": "message",
"content": [
{
"type": "input_text",
"text": "prepend system prompt"
}
],
"role": "system"
},
{
"type": "message",
"content": [
{
"type": "input_text",
"text": "prepend user message"
}
],
"role": "user"
},
{
"type": "message",
"role": "assistant",
"content": "prepend assistant message"
},
{
"type": "message",
"content": [
{
"type": "input_text",
"text": "Hello, world"
}
],
"role": "user"
},
{
"type": "message",
"content": [
{
"type": "input_text",
"text": "append user message"
}
],
"role": "user"
},
{
"type": "message",
"content": [
{
"type": "input_text",
"text": "append system prompt"
}
],
"role": "system"
},
{
"type": "message",
"role": "assistant",
"content": "append assistant prompt"
}
],
"model": "gpt-4o",
"max_tokens": 1024
}

View File

@ -0,0 +1,30 @@
---
source: crates/agentgateway/src/llm/tests.rs
description: src/llm/tests/requests/responses/assistant-history.json
info:
model: gpt-4o
input:
- type: message
role: assistant
phase: final_answer
content:
- type: output_text
text: Here is the answer.
- type: refusal
text: I cannot help with that part.
- type: message
role: user
content:
- type: input_text
text: Follow up question.
---
[
{
"role": "assistant",
"content": "Here is the answer."
},
{
"role": "user",
"content": "Follow up question."
}
]

Some files were not shown because too many files have changed in this diff Show More