Compare commits
8 Commits
main
...
cursor/ope
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b38d44a5d4 | ||
|
|
de3a8a6d77 | ||
|
|
038aa5bc3b | ||
|
|
e47c316e47 | ||
|
|
8581cc5e24 | ||
|
|
53bba774c0 | ||
|
|
84f6c8f00f | ||
|
|
d71b7e0a7d |
@ -7,6 +7,7 @@ VITE_GOOGLE_GENERATIVE_AI_API_KEY=
|
||||
VITE_ALIBABA_API_KEY=
|
||||
VITE_OPENAI_API_KEY=
|
||||
VITE_ANTHROPIC_API_KEY=
|
||||
VITE_OPENCODE_API_KEY=
|
||||
VITE_XAI_API_KEY=
|
||||
|
||||
# Developer-only: write one JSON file per AI run to ./ai-logs/.
|
||||
|
||||
@ -27,7 +27,7 @@ Use Bun for all package and script operations. Never use npm or manually edit `b
|
||||
- Keep editor mutations immutable; undo/redo snapshots depend on it.
|
||||
- This public repository owns only the editor. The production marketing site is maintained privately and deployed separately; never add its source, styles, assets, or production-only configuration here.
|
||||
- Never commit secrets, private screenshots, unlicensed assets, PSD/PSB files, or generated `dist/`.
|
||||
- AI provider keys are entered in the browser (stored in `localStorage`) with optional `.env.local` `VITE_*` fallback for local dev; only Moonshot uses the local CORS proxy. Never commit `.env.local` or deploy a keyed build.
|
||||
- AI provider keys are entered in the browser (stored in `localStorage`) with optional `.env.local` `VITE_*` fallback for local dev; Moonshot and OpenCode use a same-origin CORS proxy. Never commit `.env.local` or deploy a keyed build.
|
||||
- Use shadcn/Base UI and Hugeicons for editor chrome. Do not add a parallel UI or token system.
|
||||
- Update generated `src/components/ui/` files only with the shadcn CLI.
|
||||
- Use Conventional Commit messages and keep commits independently coherent.
|
||||
|
||||
11
README.md
11
README.md
@ -23,7 +23,7 @@ Projects live entirely in your browser — no Shotluma account or project backen
|
||||
|
||||
- **AI that designs, not renders** — the agent works through editor tool calls, so its output is a normal project: every element it creates is selectable and editable, and you can revise a single screen with the magic-cursor action instead of regenerating everything.
|
||||
- **Use your ChatGPT plan** — connect the Codex app or CLI to the hosted editor without giving Shotluma an API key.
|
||||
- **Bring your own model** — also works with Moonshot, Google, Qwen, OpenAI, Anthropic, xAI, and OpenRouter; pick provider and model per run.
|
||||
- **Bring your own model** — also works with Moonshot, Google, Qwen, OpenAI, Anthropic, xAI, OpenRouter, and OpenCode Zen/Go; pick provider and model per run.
|
||||
- **Full screenshot sets** — design a multi-screen story on portrait artboards, not one image at a time.
|
||||
- **Direct manipulation** — typography, gradients, shapes, images, mockups, alignment, stacking, and spacing are all hand-editable, with or without AI.
|
||||
- **Perspective device mockups** — drop a raw screenshot into a device frame with correct 3D geometry.
|
||||
@ -84,6 +84,7 @@ VITE_ALIBABA_API_KEY=
|
||||
VITE_OPENAI_API_KEY=
|
||||
VITE_ANTHROPIC_API_KEY=
|
||||
VITE_XAI_API_KEY=
|
||||
VITE_OPENCODE_API_KEY=
|
||||
```
|
||||
|
||||
Restart the dev server after changing `.env.local` keys. Pick the provider and model in the generation dialog — models with configurable reasoning also expose their effort levels there. If a selected provider has no key, the dialog offers **Enter API key** and disables generation until one is saved.
|
||||
@ -91,7 +92,7 @@ Restart the dev server after changing `.env.local` keys. Pick the provider and m
|
||||
How it works, and what to know:
|
||||
|
||||
- Codex runs go from the hosted browser to the paired loopback connector and then to Codex App Server. The connector exposes only the account, model, thread, turn, and Shotluma dynamic-tool RPC surface; Codex gets a read-only empty workspace with network access disabled.
|
||||
- Google, Qwen, OpenAI, Anthropic, xAI, and OpenRouter are called directly from the browser via the AI SDK. Moonshot works only on localhost through the local `/api/moonshot` CORS proxy.
|
||||
- Google, Qwen, OpenAI, Anthropic, xAI, and OpenRouter are called directly from the browser via the AI SDK. Moonshot works only on localhost through the local `/api/moonshot` CORS proxy. OpenCode Zen and Go use a same-origin `/api/opencode` proxy (Vite locally, Cloudflare Worker when hosted) because those APIs do not answer browser CORS preflight. Models without native vision send screenshots and previews through a second OpenCode vision model first.
|
||||
- When you start a run, your description and selected screenshots are sent to the selected provider or through Codex. Normal editing, persistence, and export never call any AI service. Provider limits or charges may apply.
|
||||
- Browser-stored keys are visible to same-origin JavaScript by design. Optional `VITE_*` keys are exposed only by the local dev server; production builds replace every provider env key with an empty value. Never commit `.env.local` (see [Self-hosting](#self-hosting)).
|
||||
- Prompts can be written in any language; generated canvas copy and summaries are in English.
|
||||
@ -109,7 +110,7 @@ For debugging, set `SHOTLUMA_AI_LOGGING=true` in `.env.local` to write one JSON
|
||||
| `bun run test` | Run the Vitest suite once |
|
||||
| `bun run test:coverage` | Tests with coverage thresholds |
|
||||
| `bun run build` | Type-check and build the production bundle |
|
||||
| `bun run preview` | Serve the production bundle locally, including the Moonshot proxy |
|
||||
| `bun run preview` | Serve the production bundle locally, including the Moonshot and OpenCode proxies |
|
||||
| `bun run audit` | Check dependencies for known vulnerabilities |
|
||||
| `bun run check` | All required local and CI quality gates |
|
||||
|
||||
@ -117,13 +118,13 @@ Changes to rendering, export, or AI behavior also need manual verification in th
|
||||
|
||||
## Self-hosting
|
||||
|
||||
`bun run build` produces a static app in `dist/`, including the downloadable Codex connector at `codex/shotluma-codex-bridge.mjs`. Provider env values are stripped from the bundle, so it can be hosted as a static site while users connect Codex or supply their own direct-provider keys in the browser.
|
||||
`bun run build` produces a static app in `dist/`, including the downloadable Codex connector at `codex/shotluma-codex-bridge.mjs`. Provider env values are stripped from the bundle, so it can be hosted as a static site while users connect Codex or supply their own direct-provider keys in the browser. OpenCode on a hosted origin also needs the `/api/opencode` Worker in `scripts/shotluma-worker.ts` (the official `app.shotluma.com` deploy includes it).
|
||||
|
||||
The official editor is deployed at `https://app.shotluma.com`. The marketing
|
||||
site at `https://shotluma.com` is maintained and deployed separately; its source
|
||||
and production configuration are not part of this repository.
|
||||
|
||||
Codex connection prompts pair against the deployment's own origin. Google, Qwen, OpenAI, Anthropic, xAI, and OpenRouter work with browser-entered keys. Moonshot
|
||||
Codex connection prompts pair against the deployment's own origin. Google, Qwen, OpenAI, Anthropic, xAI, OpenRouter, and OpenCode work with browser-entered keys. Moonshot
|
||||
remains local-only; offering it on a hosted deployment requires an authenticated
|
||||
proxy. A hosted workflow with shared credentials likewise needs a backend or
|
||||
short-lived credential exchange.
|
||||
|
||||
@ -32,8 +32,8 @@ Maintainers will acknowledge reports and coordinate a fix and disclosure on a be
|
||||
- `.env.local` is ignored by Git and must remain untracked.
|
||||
- Production builds replace provider env keys with empty values. Do not remove this build boundary or publish a manually altered keyed bundle.
|
||||
- Uploaded images and projects stay in IndexedDB during normal editing.
|
||||
- Screenshots selected for an AI run are sent to the selected provider. Google, Qwen, OpenAI, Anthropic, and xAI are contacted directly; Moonshot works only on localhost through the same-origin local CORS proxy. Opt-in overlay generation contacts OpenAI separately.
|
||||
- Screenshots selected for an AI run are sent to the selected provider. Google, Qwen, OpenAI, Anthropic, and xAI are contacted directly; Moonshot works only on localhost through the same-origin local CORS proxy. OpenCode Zen and Go use the same-origin `/api/opencode` proxy. Opt-in overlay generation contacts OpenAI separately.
|
||||
- Users are responsible for protecting provider keys and usage quotas.
|
||||
- A hosted deployment can use browser-entered keys with the direct providers. A deployment that supplies shared credentials or offers Moonshot requires a separate authenticated backend design.
|
||||
- A hosted deployment can use browser-entered keys with the direct providers and OpenCode. A deployment that supplies shared credentials or offers Moonshot requires a separate authenticated backend design.
|
||||
|
||||
If a key is accidentally exposed, revoke it at the provider immediately, remove it from the current files, and treat it as compromised even if the Git commit is later rewritten.
|
||||
|
||||
@ -17,10 +17,11 @@ Optional AI path
|
||||
Browser → local Shotluma bridge → Codex App Server → ChatGPT plan
|
||||
Browser → AI SDK provider → Google / Qwen / OpenAI / Anthropic / xAI / OpenRouter
|
||||
Browser → /api/moonshot/* → Vite proxy → Moonshot API
|
||||
Browser → /api/opencode/* → Vite or Cloudflare Worker proxy → OpenCode Zen/Go
|
||||
Browser → /api/ai-run-logs → Vite → ./ai-logs/*.json (developer opt-in)
|
||||
```
|
||||
|
||||
The core editor has no backend requirement. Hosted users can connect a Codex app or CLI already signed in with ChatGPT on the same computer; `app.shotluma.com` remains the application host. The browser talks to a paired loopback bridge, and the bridge talks to Codex App Server over stdio. The alternative AI path uses bring-your-own keys entered in the browser (`localStorage`), with optional `.env.local` `VITE_*` fallback served only during local development. Production builds replace all provider env values with empty strings. Google, Qwen, OpenAI, Anthropic, xAI, and OpenRouter are called directly; Moonshot is enabled only on localhost and uses a same-origin proxy because its API does not support the required browser CORS flow.
|
||||
The core editor has no backend requirement. Hosted users can connect a Codex app or CLI already signed in with ChatGPT on the same computer; `app.shotluma.com` remains the application host. The browser talks to a paired loopback bridge, and the bridge talks to Codex App Server over stdio. The alternative AI path uses bring-your-own keys entered in the browser (`localStorage`), with optional `.env.local` `VITE_*` fallback served only during local development. Production builds replace all provider env values with empty strings. Google, Qwen, OpenAI, Anthropic, xAI, and OpenRouter are called directly; Moonshot is enabled only on localhost and uses a same-origin proxy because its API does not support the required browser CORS flow. OpenCode Zen and Go share one API key and use the same-origin `/api/opencode` proxy (Vite in local dev, Cloudflare Worker when hosted) for the same CORS reason.
|
||||
|
||||
This public repository contains and deploys only the editor at
|
||||
`app.shotluma.com`. The marketing site at `shotluma.com` has a separate private
|
||||
@ -139,6 +140,11 @@ The AI feature is split into explicit layers:
|
||||
| `src/html-slide/device-attributes.ts` | `<shotluma-device>` placeholder parsing |
|
||||
| `src/ai/prompt-caching.ts` | OpenAI, Anthropic, and Qwen prompt-cache routing and breakpoints |
|
||||
| `src/ai/chat-tool-images.ts` | Image relay for providers whose chat tool results support text only |
|
||||
| `src/ai/describe-images.ts` | Replace image parts with text descriptions for models without vision |
|
||||
| `src/ai/opencode-models.ts` | Runtime OpenCode Zen/Go catalog, vision flags, and fallback picker |
|
||||
| `src/ai/opencode-client.ts` | OpenAI-compat chat client against the same-origin OpenCode proxy |
|
||||
| `scripts/opencode-proxy.ts` | URL rewrite and header allowlist for Zen/Go |
|
||||
| `scripts/shotluma-worker.ts` | Hosted Cloudflare Worker that proxies `/api/opencode/*` |
|
||||
| `src/ai/tools.ts` | Tool composition and generate/edit tool boundary |
|
||||
| `src/ai/*-tools.ts`, `src/ai/*-tool.ts` | Focused slide, media, text, update, inspection, and overlay-asset tools |
|
||||
| `src/ai/chroma-key.ts` | Pure soft chroma-key pixel math for overlay cutouts |
|
||||
@ -168,13 +174,15 @@ Rich text is built from structured highlight input. The model never writes raw H
|
||||
|
||||
The generate dialog covers input only. Once a run starts it closes and `src/components/AiRunBand.tsx` takes over as a band floating over the canvas: assistant prose and reasoning at reading size (accumulated per sentence by `src/ai/run-narration.ts`), a screen rail fed by the model's `declare_plan` call and reconciled against reality by `src/ai/run-plan.ts`, and an action count in place of a tool log. The band is also where a finished or failed run reports its result, so no run hands control back to the centered dialog.
|
||||
|
||||
`src/ai/provider-catalog.ts` owns the selectable providers, models, transport metadata, and model-specific reasoning-effort choices. `src/ai/provider-config.ts` resolves keys from browser `localStorage` (API keys dialog) with optional `.env.local` `VITE_*` fallback, and `src/ai/runner.ts` routes either to the Codex bridge or a lazily loaded native AI SDK provider. The runner passes portable efforts through the AI SDK's standardized `reasoning` option so each native provider can map it to its own API; OpenAI GPT-5.6 and Moonshot/Kimi K3 `max` go through OpenAI-compat `providerOptions` because the shared option has no `max`. The generate modal presents one model picker grouped by provider, with reasoning effort as secondary chips when the model supports it, a Codex connection tutorial, and an API keys dialog for key-based providers. Requests go directly from the local browser to Google, Alibaba/Qwen, OpenAI, Anthropic, xAI, or OpenRouter. Moonshot uses the OpenAI chat provider through the only Vite proxy route.
|
||||
`src/ai/provider-catalog.ts` owns the selectable providers, models, transport metadata, and model-specific reasoning-effort choices. `src/ai/provider-config.ts` resolves keys from browser `localStorage` (API keys dialog) with optional `.env.local` `VITE_*` fallback, and `src/ai/runner.ts` routes either to the Codex bridge or a lazily loaded native AI SDK provider. The runner passes portable efforts through the AI SDK's standardized `reasoning` option so each native provider can map it to its own API; OpenAI GPT-5.6 and Moonshot/Kimi K3 `max` go through OpenAI-compat `providerOptions` because the shared option has no `max`. The generate modal presents one model picker grouped by provider, with reasoning effort as secondary chips when the model supports it, a Codex connection tutorial, and an API keys dialog for key-based providers. Requests go directly from the local browser to Google, Alibaba/Qwen, OpenAI, Anthropic, xAI, or OpenRouter. Moonshot uses the OpenAI chat provider through the local Vite proxy. OpenCode Zen and Go share one API key, call chat-completions through `/api/opencode`, and describe images with a second vision model when the selected model has no native vision.
|
||||
|
||||
Codex pairing is deliberately not OAuth inside Shotluma. The setup prompt downloads the connector asset emitted with the hosted app, asks Codex to inspect it, and starts it as a detached user process. The browser stores a random pairing token and the exact app origin. The connector binds only `127.0.0.1:47447`, stores its matching configuration with user-only permissions, requires both the exact `Origin` header and bearer pairing token, and never reads Codex authentication files. It starts `codex app-server` over stdio and allowlists only account/rate-limit/model reads plus ephemeral thread and turn methods. Thread and turn requests are overwritten to use an empty read-only workspace, no approvals, and no network. The model can change the canvas only through the same `createEditorTools` dynamic tools used by direct providers.
|
||||
|
||||
The bridge uses Codex App Server's experimental dynamic-tool API. Keep bridge and web-client versions compatible, fail closed on unknown RPC methods, and retain the API-key provider path as a fallback while this integration is experimental.
|
||||
|
||||
OpenRouter is the one provider with a runtime model catalog: `src/ai/openrouter-models.ts` fetches the public OpenRouter `/models` endpoint (no key required), keeps only models that accept image input and support tool calling, caches the result in `localStorage` for an hour, and registers it with the static catalog so selections resolve everywhere. The curated OpenRouter shortlist in `provider-catalog.ts` doubles as the offline fallback, and the model picker adds a searchable browser over the fetched catalog. Requests use the OpenAI chat provider against `https://openrouter.ai/api/v1`.
|
||||
OpenRouter is one runtime model catalog: `src/ai/openrouter-models.ts` fetches the public OpenRouter `/models` endpoint (no key required), keeps only models that accept image input and support tool calling, caches the result in `localStorage` for an hour, and registers it with the static catalog so selections resolve everywhere. The curated OpenRouter shortlist in `provider-catalog.ts` doubles as the offline fallback, and the model picker adds a searchable browser over the fetched catalog. Requests use the OpenAI chat provider against `https://openrouter.ai/api/v1`.
|
||||
|
||||
OpenCode Zen and Go are the other runtime catalogs: `src/ai/opencode-models.ts` fetches each gateway's `/v1/models` list through the same-origin proxy, keeps every tool-capable model including those without vision, and caches for an hour. Text-only models stay selectable; `src/ai/describe-images.ts` sends screenshots and slide previews to a vision model from the same gateway (defaulting to a cheap one such as MiMo V2.5 Free on Zen or MiMo V2.5 on Go) and replaces the image parts with the description before the coding model runs.
|
||||
|
||||
Keys are intentionally browser-visible and stored unencrypted. Never commit `.env.local` or weaken the production-build stripping of provider env values. Use dedicated keys with restrictive quotas. A hosted deployment that must hide or share credentials needs an authenticated backend.
|
||||
|
||||
|
||||
@ -13,11 +13,24 @@ Generate mode does not spend a model turn re-reading canvas state already presen
|
||||
|
||||
Preview images are rendered at the editor's native 330 px width rather than upscaled, use OpenAI's low-detail vision path because exact geometry arrives separately, and are capped at three attempts per generated screen (four for edit mode's before/after checks). Mutations return only warnings involving the changed element; the preview is the one deduplicated slide-wide validation point. Measurement boxes remain rounded to one decimal (`src/ai/measure.ts`).
|
||||
|
||||
OpenAI Responses, Anthropic, Google, xAI, and Codex accept image-bearing tool results directly. OpenAI-compatible Chat Completions and Alibaba do not: before Moonshot, OpenRouter, or Qwen steps, `src/ai/chat-tool-images.ts` moves image files out of the tool-result JSON and into a following multimodal user message. Never allow base64 image data to be serialized as model-visible text.
|
||||
OpenAI Responses, Anthropic, Google, xAI, and Codex accept image-bearing tool results directly. OpenAI-compatible Chat Completions and Alibaba do not: before Moonshot, OpenRouter, Qwen, or chat-completions OpenCode steps, `src/ai/chat-tool-images.ts` moves image files out of the tool-result JSON and into a following multimodal user message. Never allow base64 image data to be serialized as model-visible text.
|
||||
|
||||
OpenCode models without native vision then go through `src/ai/describe-images.ts`: each remaining image part is sent to a vision-capable model on the same Zen or Go gateway, and the coding model receives the text description instead. Do not send raw images to a text-only OpenCode model.
|
||||
|
||||
## Security and provider boundary
|
||||
|
||||
The browser uses the AI SDK's native Google, Alibaba/Qwen, OpenAI, Anthropic, and xAI providers directly. OpenRouter is called directly through the OpenAI chat provider against `https://openrouter.ai/api/v1`. Moonshot uses the OpenAI chat provider through the local `/api/moonshot` CORS proxy.
|
||||
The browser uses the AI SDK's native Google, Alibaba/Qwen, OpenAI, Anthropic, and xAI providers directly. OpenRouter is called directly through the OpenAI chat provider against `https://openrouter.ai/api/v1`. Moonshot uses the OpenAI chat provider through the local `/api/moonshot` CORS proxy. OpenCode Zen and Go go through `/api/opencode/{zen|go}/v1` because `opencode.ai` does not answer browser CORS preflight.
|
||||
|
||||
OpenCode is a multi-dialect gateway, not an OpenAI-compatible one. Each model answers on exactly one endpoint, and calling any other returns a 400 or an `Endpoint is unavailable` 5xx rather than routing itself. `src/ai/opencode-dialects.ts` owns the mapping and `src/ai/opencode-client.ts` builds the matching provider:
|
||||
|
||||
| Dialect | Endpoint | Provider | Families |
|
||||
| --- | --- | --- | --- |
|
||||
| `responses` | `/v1/responses` | `@ai-sdk/openai` Responses | `gpt-*`, `grok-*`, `muse-spark-*` |
|
||||
| `messages` | `/v1/messages` | `@ai-sdk/anthropic` | `claude-*`, `qwen*`, `minimax-*` on Go only |
|
||||
| `google` | `/v1/models/<id>:…` | `@ai-sdk/google` | `gemini-*` |
|
||||
| `chat` | `/v1/chat/completions` | `@ai-sdk/openai` chat | everything else, including `minimax-*` on Zen |
|
||||
|
||||
Match `src/ai/opencode-dialects.ts` against the endpoint tables at <https://opencode.ai/docs/zen> and <https://opencode.ai/docs/go> whenever OpenCode adds a family; prefixes are matched rather than exact ids because `/v1/models` only reports ids and keeps growing. Chat completions is the default for an unrecognised id because every open-weight family lands there. The proxy header allowlist in `scripts/opencode-proxy.ts` must keep carrying `x-api-key`, `anthropic-version`, `anthropic-beta`, and `x-goog-api-key` alongside `authorization`, or the non-OpenAI dialects lose their credentials.
|
||||
|
||||
Codex is a separate no-API-key transport for hosted users. The browser pairs with `scripts/shotluma-codex-bridge.ts` on `127.0.0.1`; that process launches `codex app-server` over stdio and uses Codex's existing ChatGPT authentication. The user stays on `app.shotluma.com` and never needs a repository checkout or local Vite server.
|
||||
|
||||
@ -29,7 +42,7 @@ Codex is a separate no-API-key transport for hosted users. The browser pairs wit
|
||||
- `scripts/codex-bridge-asset-plugin.ts` must emit the reviewed connector source at `/codex/shotluma-codex-bridge.mjs` in both development and production builds. The copied setup prompt must use the current browser origin so normal hosted and self-hosted deployments pair only with themselves.
|
||||
- Open the ChatGPT desktop app with the documented `codex://threads/new?prompt=` deep link and URI-encode the complete setup prompt. Deep links prefill but do not submit the composer, so the tutorial must tell the user to press Send and retain the manual copy fallback.
|
||||
- Keys are intentionally visible to same-origin browser JavaScript. Never commit `.env.local`, reuse a shared production credential, or weaken the production build boundary. Recommend dedicated keys with restrictive quotas.
|
||||
- Do not add a proxy for providers whose browser API supports the required CORS flow. Moonshot is available only on localhost through `/api/moonshot`; a hosted deployment that offers Moonshot or must hide keys needs a separate authenticated backend design.
|
||||
- Do not add a proxy for providers whose browser API supports the required CORS flow. Moonshot is available only on localhost through `/api/moonshot`; a hosted deployment that offers Moonshot or must hide keys needs a separate authenticated backend design. OpenCode is the exception that is proxied in production as well: `scripts/shotluma-worker.ts` forwards `/api/opencode/*` to `opencode.ai`, copying only the allowlisted auth headers upstream without storing or logging the caller's key.
|
||||
- Never return secrets or raw data URLs in model-visible state.
|
||||
- Keep uploads browser-local except for screenshots and app logos explicitly included in an AI run.
|
||||
- In generate mode, collect app name and app logo separately from the app description and screenshots. Pass the name and logo asset id through the user message and attach the logo image. Instruct the model to place the logo with `add_image` in element runs and with an `<img src="asset:…">` tag in HTML screen runs — never as a device screenshot.
|
||||
@ -108,4 +121,6 @@ Reasoning **visibility** is a separate switch from reasoning **effort**, and pro
|
||||
|
||||
This matters more than a missing nicety because `prompt.ts` tells the model not to narrate its work turn by turn. Reasoning is therefore the only live prose during a run, and for a model that streams none, the tool-activity line in the run band is the only remaining signal — it is load-bearing, not decoration.
|
||||
|
||||
OpenRouter is the only provider with a runtime model catalog (`src/ai/openrouter-models.ts`): the public `/models` endpoint is fetched without a key, filtered to models that accept image input and support tool calling — both are hard requirements for a Shotluma run — and cached in `localStorage` for an hour. Loaded models are registered with `setDynamicOpenRouterModels` so catalog lookups resolve them; unknown OpenRouter ids synthesize a minimal option instead of throwing, while every other provider keeps failing fast on unknown models. Keep the curated OpenRouter shortlist in `provider-catalog.ts` working as the fetch fallback, keep the vision+tools filter intact, and offer reasoning effort only when the fetched model advertises the `reasoning` parameter.
|
||||
OpenRouter is a runtime model catalog (`src/ai/openrouter-models.ts`): the public `/models` endpoint is fetched without a key, filtered to models that accept image input and support tool calling — both are hard requirements for a Shotluma run — and cached in `localStorage` for an hour. Loaded models are registered with `setDynamicProviderModels` so catalog lookups resolve them; unknown OpenRouter ids synthesize a minimal option instead of throwing, while every other static provider keeps failing fast on unknown models. Keep the curated OpenRouter shortlist in `provider-catalog.ts` working as the fetch fallback, keep the vision+tools filter intact, and offer reasoning effort only when the fetched model advertises the `reasoning` parameter.
|
||||
|
||||
OpenCode Zen and Go are also runtime catalogs (`src/ai/opencode-models.ts`). Fetch `/v1/models` through the same-origin proxy, keep text-only tool models, and mark them `supportsVision: false`. The generate modal exposes a vision-fallback picker for those models. Unknown OpenCode ids synthesize like OpenRouter.
|
||||
|
||||
@ -17,6 +17,7 @@ describe('client provider environment', () => {
|
||||
expect(values.VITE_MOONSHOT_API_KEY).toBe('moonshot-public')
|
||||
expect(values.VITE_OPENAI_API_KEY).toBe('openai-public')
|
||||
expect(values.VITE_XAI_API_KEY).toBe('')
|
||||
expect(values.VITE_OPENCODE_API_KEY).toBe('')
|
||||
expect(values).not.toHaveProperty('MOONSHOT_API_KEY')
|
||||
})
|
||||
|
||||
@ -28,6 +29,7 @@ describe('client provider environment', () => {
|
||||
VITE_OPENAI_API_KEY: '',
|
||||
VITE_ANTHROPIC_API_KEY: '',
|
||||
VITE_XAI_API_KEY: '',
|
||||
VITE_OPENCODE_API_KEY: '',
|
||||
})
|
||||
expect(createClientProviderDefinitions(environment, 'build')).toMatchObject({
|
||||
'import.meta.env.VITE_MOONSHOT_API_KEY': '""',
|
||||
|
||||
@ -5,6 +5,7 @@ export const AI_PROVIDER_ENV_VARIABLES = [
|
||||
'VITE_OPENAI_API_KEY',
|
||||
'VITE_ANTHROPIC_API_KEY',
|
||||
'VITE_XAI_API_KEY',
|
||||
'VITE_OPENCODE_API_KEY',
|
||||
] as const
|
||||
|
||||
export type AiProviderEnvironmentVariable = typeof AI_PROVIDER_ENV_VARIABLES[number]
|
||||
|
||||
138
scripts/opencode-proxy.test.ts
Normal file
138
scripts/opencode-proxy.test.ts
Normal file
@ -0,0 +1,138 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import {
|
||||
copyOpencodeProxyHeaders,
|
||||
isOpencodeProxyMethod,
|
||||
proxyOpencodeRequest,
|
||||
rewriteOpencodeProxyUrl,
|
||||
} from './opencode-proxy'
|
||||
|
||||
describe('OpenCode CORS proxy rewrite', () => {
|
||||
it('maps Zen and Go prefixes onto the public OpenCode origins', () => {
|
||||
expect(rewriteOpencodeProxyUrl(
|
||||
new URL('https://app.shotluma.com/api/opencode/zen/v1/chat/completions'),
|
||||
)?.toString()).toBe('https://opencode.ai/zen/v1/chat/completions')
|
||||
expect(rewriteOpencodeProxyUrl(
|
||||
new URL('https://app.shotluma.com/api/opencode/go/v1/models?foo=1'),
|
||||
)?.toString()).toBe('https://opencode.ai/zen/go/v1/models?foo=1')
|
||||
expect(rewriteOpencodeProxyUrl(
|
||||
new URL('https://app.shotluma.com/api/opencode/zen'),
|
||||
)?.toString()).toBe('https://opencode.ai/zen/')
|
||||
expect(rewriteOpencodeProxyUrl(
|
||||
new URL('https://app.shotluma.com/api/moonshot/v1/chat/completions'),
|
||||
)).toBeNull()
|
||||
expect(rewriteOpencodeProxyUrl(
|
||||
new URL('https://app.shotluma.com/api/opencode/go'),
|
||||
)?.toString()).toBe('https://opencode.ai/zen/go/')
|
||||
})
|
||||
|
||||
it('maps every dialect path each gateway serves', () => {
|
||||
expect(rewriteOpencodeProxyUrl(
|
||||
new URL('https://app.shotluma.com/api/opencode/go/v1/responses'),
|
||||
)?.toString()).toBe('https://opencode.ai/zen/go/v1/responses')
|
||||
expect(rewriteOpencodeProxyUrl(
|
||||
new URL('https://app.shotluma.com/api/opencode/zen/v1/messages'),
|
||||
)?.toString()).toBe('https://opencode.ai/zen/v1/messages')
|
||||
expect(rewriteOpencodeProxyUrl(
|
||||
new URL('https://app.shotluma.com/api/opencode/zen/v1/models/gemini-3.7-flash:streamGenerateContent'),
|
||||
)?.toString()).toBe('https://opencode.ai/zen/v1/models/gemini-3.7-flash:streamGenerateContent')
|
||||
})
|
||||
|
||||
it('does not treat a zen path as go', () => {
|
||||
expect(rewriteOpencodeProxyUrl(
|
||||
new URL('https://app.shotluma.com/api/opencode/zen/v1/models'),
|
||||
)?.toString()).toBe('https://opencode.ai/zen/v1/models')
|
||||
})
|
||||
})
|
||||
|
||||
describe('OpenCode CORS proxy headers and methods', () => {
|
||||
it('forwards only the allowlisted request headers', () => {
|
||||
const headers = copyOpencodeProxyHeaders(new Headers({
|
||||
authorization: 'Bearer test-key',
|
||||
'content-type': 'application/json',
|
||||
accept: 'text/event-stream',
|
||||
cookie: 'session=secret',
|
||||
host: 'app.shotluma.com',
|
||||
}))
|
||||
expect(Object.fromEntries(headers.entries())).toEqual({
|
||||
authorization: 'Bearer test-key',
|
||||
'content-type': 'application/json',
|
||||
accept: 'text/event-stream',
|
||||
})
|
||||
})
|
||||
|
||||
it('forwards the Anthropic and Google credentials the other dialects use', () => {
|
||||
const headers = copyOpencodeProxyHeaders(new Headers({
|
||||
'x-api-key': 'sk-test',
|
||||
'anthropic-version': '2023-06-01',
|
||||
'anthropic-beta': 'fine-grained-tool-streaming-2025-05-14',
|
||||
'x-goog-api-key': 'goog-test',
|
||||
origin: 'https://evil.example',
|
||||
}))
|
||||
expect(Object.fromEntries(headers.entries())).toEqual({
|
||||
'x-api-key': 'sk-test',
|
||||
'anthropic-version': '2023-06-01',
|
||||
'anthropic-beta': 'fine-grained-tool-streaming-2025-05-14',
|
||||
'x-goog-api-key': 'goog-test',
|
||||
})
|
||||
})
|
||||
|
||||
it('allows GET, POST, and OPTIONS only', () => {
|
||||
expect(isOpencodeProxyMethod('get')).toBe(true)
|
||||
expect(isOpencodeProxyMethod('POST')).toBe(true)
|
||||
expect(isOpencodeProxyMethod('OPTIONS')).toBe(true)
|
||||
expect(isOpencodeProxyMethod('PUT')).toBe(false)
|
||||
})
|
||||
|
||||
it('answers OPTIONS without contacting OpenCode', async () => {
|
||||
const fetchMock = vi.fn()
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
const response = await proxyOpencodeRequest(new Request(
|
||||
'https://app.shotluma.com/api/opencode/zen/v1/models',
|
||||
{ method: 'OPTIONS' },
|
||||
))
|
||||
expect(response.status).toBe(204)
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
vi.unstubAllGlobals()
|
||||
})
|
||||
|
||||
it('rejects unsupported methods and unknown paths', async () => {
|
||||
expect((await proxyOpencodeRequest(new Request(
|
||||
'https://app.shotluma.com/api/opencode/zen/v1/models',
|
||||
{ method: 'DELETE' },
|
||||
))).status).toBe(405)
|
||||
expect((await proxyOpencodeRequest(new Request(
|
||||
'https://app.shotluma.com/api/moonshot/v1/models',
|
||||
{ method: 'GET' },
|
||||
))).status).toBe(404)
|
||||
})
|
||||
|
||||
it('forwards GET and POST to the rewritten OpenCode origin', async () => {
|
||||
const fetchMock = vi.fn().mockResolvedValue(new Response('ok', { status: 200 }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
|
||||
const getResponse = await proxyOpencodeRequest(new Request(
|
||||
'https://app.shotluma.com/api/opencode/go/v1/models',
|
||||
{ method: 'GET', headers: { authorization: 'Bearer test-key' } },
|
||||
))
|
||||
expect(getResponse.status).toBe(200)
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1)
|
||||
const [getUrl, getInit] = fetchMock.mock.calls[0] as [URL, RequestInit]
|
||||
expect(getUrl.toString()).toBe('https://opencode.ai/zen/go/v1/models')
|
||||
expect(getInit.method).toBe('GET')
|
||||
expect(getInit.body).toBeUndefined()
|
||||
|
||||
await proxyOpencodeRequest(new Request(
|
||||
'https://app.shotluma.com/api/opencode/zen/v1/chat/completions',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: '{"model":"kimi-k3"}',
|
||||
},
|
||||
))
|
||||
const [, postInit] = fetchMock.mock.calls[1] as [URL, RequestInit & { duplex?: string }]
|
||||
expect(postInit.method).toBe('POST')
|
||||
expect(postInit.body).toBeDefined()
|
||||
expect(postInit.duplex).toBe('half')
|
||||
vi.unstubAllGlobals()
|
||||
})
|
||||
})
|
||||
89
scripts/opencode-proxy.ts
Normal file
89
scripts/opencode-proxy.ts
Normal file
@ -0,0 +1,89 @@
|
||||
export const OPENCODE_ZEN_UPSTREAM = 'https://opencode.ai/zen'
|
||||
export const OPENCODE_GO_UPSTREAM = 'https://opencode.ai/zen/go'
|
||||
|
||||
const GO_PREFIX = '/api/opencode/go'
|
||||
const ZEN_PREFIX = '/api/opencode/zen'
|
||||
|
||||
/**
|
||||
* OpenCode serves each model on one of four dialects, so the proxy has to carry
|
||||
* the auth and versioning headers of all of them: `authorization` for the
|
||||
* OpenAI Responses and Chat Completions endpoints, `x-api-key` plus the
|
||||
* `anthropic-*` headers for `/v1/messages`, and `x-goog-api-key` for the Gemini
|
||||
* endpoint. Anything outside this list (cookies, origin, client hints) is
|
||||
* dropped so the proxy stays a pure credential passthrough.
|
||||
*/
|
||||
export const OPENCODE_FORWARDED_HEADERS = [
|
||||
'authorization',
|
||||
'content-type',
|
||||
'accept',
|
||||
'x-api-key',
|
||||
'anthropic-version',
|
||||
'anthropic-beta',
|
||||
'x-goog-api-key',
|
||||
] as const
|
||||
|
||||
const joinUpstream = (origin: string, suffix: string): URL => {
|
||||
const base = `${origin.replace(/\/$/u, '')}/`
|
||||
const relative = suffix.replace(/^\//u, '')
|
||||
return new URL(relative, base)
|
||||
}
|
||||
|
||||
/**
|
||||
* Map a same-origin OpenCode CORS-proxy path onto the Zen or Go upstream.
|
||||
* `/api/opencode/go` is matched before `/api/opencode/zen` so the prefixes
|
||||
* cannot overlap.
|
||||
*/
|
||||
export const rewriteOpencodeProxyUrl = (requestUrl: URL): URL | null => {
|
||||
const path = requestUrl.pathname
|
||||
const search = requestUrl.search
|
||||
if (path === GO_PREFIX || path.startsWith(`${GO_PREFIX}/`)) {
|
||||
const upstream = joinUpstream(OPENCODE_GO_UPSTREAM, path.slice(GO_PREFIX.length))
|
||||
upstream.search = search
|
||||
return upstream
|
||||
}
|
||||
if (path === ZEN_PREFIX || path.startsWith(`${ZEN_PREFIX}/`)) {
|
||||
const upstream = joinUpstream(OPENCODE_ZEN_UPSTREAM, path.slice(ZEN_PREFIX.length))
|
||||
upstream.search = search
|
||||
return upstream
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
export const copyOpencodeProxyHeaders = (source: Headers): Headers => {
|
||||
const headers = new Headers()
|
||||
for (const name of OPENCODE_FORWARDED_HEADERS) {
|
||||
const value = source.get(name)
|
||||
if (value) headers.set(name, value)
|
||||
}
|
||||
return headers
|
||||
}
|
||||
|
||||
export const isOpencodeProxyMethod = (method: string): boolean => {
|
||||
const normalized = method.toUpperCase()
|
||||
return normalized === 'GET' || normalized === 'POST' || normalized === 'OPTIONS'
|
||||
}
|
||||
|
||||
export const proxyOpencodeRequest = async (request: Request): Promise<Response> => {
|
||||
if (request.method.toUpperCase() === 'OPTIONS') {
|
||||
return new Response(null, { status: 204 })
|
||||
}
|
||||
if (!isOpencodeProxyMethod(request.method)) {
|
||||
return new Response('Method not allowed', { status: 405 })
|
||||
}
|
||||
const upstream = rewriteOpencodeProxyUrl(new URL(request.url))
|
||||
if (!upstream) {
|
||||
return new Response('Not found', { status: 404 })
|
||||
}
|
||||
const method = request.method.toUpperCase()
|
||||
// Node's fetch types require `duplex` when forwarding a stream body.
|
||||
const init: RequestInit & { duplex?: 'half' } = {
|
||||
method,
|
||||
headers: copyOpencodeProxyHeaders(request.headers),
|
||||
redirect: 'manual',
|
||||
}
|
||||
if (method !== 'GET' && method !== 'HEAD') {
|
||||
init.body = request.body
|
||||
init.duplex = 'half'
|
||||
}
|
||||
return fetch(upstream, init)
|
||||
}
|
||||
23
scripts/shotluma-worker.ts
Normal file
23
scripts/shotluma-worker.ts
Normal file
@ -0,0 +1,23 @@
|
||||
import { proxyOpencodeRequest, rewriteOpencodeProxyUrl } from './opencode-proxy'
|
||||
|
||||
type AssetFetcher = {
|
||||
fetch: (request: Request) => Promise<Response>
|
||||
}
|
||||
|
||||
export type ShotlumaWorkerEnv = {
|
||||
ASSETS: AssetFetcher
|
||||
}
|
||||
|
||||
/**
|
||||
* Hosted same-origin CORS proxy for OpenCode Zen/Go. Browser keys stay on the
|
||||
* request; the worker only rewrites the URL so app.shotluma.com can call an
|
||||
* API that does not answer OPTIONS preflight.
|
||||
*/
|
||||
export default {
|
||||
async fetch(request: Request, env: ShotlumaWorkerEnv): Promise<Response> {
|
||||
if (rewriteOpencodeProxyUrl(new URL(request.url))) {
|
||||
return proxyOpencodeRequest(request)
|
||||
}
|
||||
return env.ASSETS.fetch(request)
|
||||
},
|
||||
}
|
||||
216
src/ai/describe-images.test.ts
Normal file
216
src/ai/describe-images.test.ts
Normal file
@ -0,0 +1,216 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import {
|
||||
collectMessageImages,
|
||||
describeMessageImages,
|
||||
imagePartKey,
|
||||
replaceMessageImages,
|
||||
toVisionFilePart,
|
||||
} from './describe-images'
|
||||
import type { FilePart, ModelMessage, UserModelMessage } from 'ai'
|
||||
|
||||
const filePart = (data: string, mediaType = 'image/png'): FilePart => ({
|
||||
type: 'file',
|
||||
mediaType,
|
||||
data,
|
||||
})
|
||||
|
||||
const userImages = (...parts: FilePart[]): UserModelMessage => ({
|
||||
role: 'user',
|
||||
content: [{ type: 'text', text: 'Screenshots:' }, ...parts],
|
||||
})
|
||||
|
||||
describe('describeMessageImages', () => {
|
||||
it('leaves messages without images unchanged', async () => {
|
||||
const messages: ModelMessage[] = [{ role: 'user', content: 'hello' }]
|
||||
expect(await describeMessageImages(messages, async () => 'nope')).toBe(messages)
|
||||
expect(collectMessageImages(messages)).toEqual([])
|
||||
})
|
||||
|
||||
it('replaces image files with text descriptions and skips tool payloads', async () => {
|
||||
const shot = filePart('data:image/png;base64,abc')
|
||||
const messages: ModelMessage[] = [
|
||||
userImages(shot),
|
||||
{
|
||||
role: 'tool',
|
||||
content: [{
|
||||
type: 'tool-result',
|
||||
toolCallId: 'call-1',
|
||||
toolName: 'render_slide_preview',
|
||||
output: {
|
||||
type: 'content',
|
||||
value: [{
|
||||
type: 'file',
|
||||
mediaType: 'image/jpeg',
|
||||
data: { type: 'data', data: 'preview-bytes' },
|
||||
}],
|
||||
},
|
||||
}],
|
||||
},
|
||||
]
|
||||
|
||||
const described = await describeMessageImages(messages, async (part) => {
|
||||
return part.mediaType === 'image/jpeg' ? 'preview layout' : 'app screenshot'
|
||||
})
|
||||
|
||||
expect(collectMessageImages(messages)).toHaveLength(1)
|
||||
expect(described[0]).toMatchObject({
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: 'Screenshots:' },
|
||||
{ type: 'text', text: 'Image description:\napp screenshot' },
|
||||
],
|
||||
})
|
||||
expect(described[1]).toBe(messages[1])
|
||||
})
|
||||
|
||||
it('describes identical payloads once and reports failures as text', async () => {
|
||||
const first = filePart('same-bytes')
|
||||
const second = filePart('same-bytes')
|
||||
const broken = filePart('broken')
|
||||
let calls = 0
|
||||
const described = await describeMessageImages(
|
||||
[userImages(first, second, broken)],
|
||||
async (part) => {
|
||||
calls += 1
|
||||
if (imagePartKey(part).includes('broken')) throw new Error('vision timeout')
|
||||
return ' shared view '
|
||||
},
|
||||
)
|
||||
|
||||
expect(calls).toBe(2)
|
||||
const content = (described[0] as UserModelMessage).content
|
||||
if (!Array.isArray(content)) throw new Error('expected parts')
|
||||
expect(content.map((part) => 'text' in part ? part.text : '')).toEqual([
|
||||
'Screenshots:',
|
||||
'Image description:\nshared view',
|
||||
'Image description:\nshared view',
|
||||
'[Image could not be described: vision timeout]',
|
||||
])
|
||||
})
|
||||
|
||||
it('reuses a shared cache across calls instead of re-describing per step', async () => {
|
||||
const shot = filePart('step-bytes')
|
||||
const cache = new Map<string, string>()
|
||||
let calls = 0
|
||||
const describeImage = async () => {
|
||||
calls += 1
|
||||
return 'described once'
|
||||
}
|
||||
|
||||
const first = await describeMessageImages([userImages(shot)], describeImage, cache)
|
||||
const second = await describeMessageImages([userImages(shot)], describeImage, cache)
|
||||
|
||||
expect(calls).toBe(1)
|
||||
for (const described of [first, second]) {
|
||||
const content = (described[0] as UserModelMessage).content
|
||||
if (!Array.isArray(content)) throw new Error('expected parts')
|
||||
expect(content[1]).toEqual({
|
||||
type: 'text',
|
||||
text: 'Image description:\ndescribed once',
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
it('keys distinct binary payloads separately, including ArrayBuffers', async () => {
|
||||
const bytesPart = (bytes: number[]): FilePart => ({
|
||||
type: 'file',
|
||||
mediaType: 'image/png',
|
||||
data: new Uint8Array(bytes),
|
||||
})
|
||||
// Same length and first byte must not collide.
|
||||
expect(imagePartKey(bytesPart([9, 8, 7]))).not.toBe(imagePartKey(bytesPart([9, 0, 0])))
|
||||
const bufferPart = (bytes: number[]): FilePart => ({
|
||||
type: 'file',
|
||||
mediaType: 'image/png',
|
||||
data: new Uint8Array(bytes).buffer,
|
||||
})
|
||||
expect(imagePartKey(bufferPart([1, 2, 3]))).not.toBe(imagePartKey(bufferPart([1, 2, 4])))
|
||||
// An ArrayBuffer keys by content, matching its Uint8Array view.
|
||||
expect(imagePartKey(bufferPart([1, 2, 3]))).toBe(imagePartKey(bytesPart([1, 2, 3])))
|
||||
|
||||
const descriptions: string[] = []
|
||||
const described = await describeMessageImages(
|
||||
[userImages(bytesPart([9, 8, 7]), bytesPart([9, 0, 0]))],
|
||||
async () => {
|
||||
descriptions.push(`payload ${descriptions.length + 1}`)
|
||||
return descriptions[descriptions.length - 1] ?? ''
|
||||
},
|
||||
)
|
||||
expect(descriptions).toHaveLength(2)
|
||||
expect(described[0]).toMatchObject({
|
||||
content: [
|
||||
{ type: 'text', text: 'Screenshots:' },
|
||||
{ type: 'text', text: 'Image description:\npayload 1' },
|
||||
{ type: 'text', text: 'Image description:\npayload 2' },
|
||||
],
|
||||
})
|
||||
})
|
||||
|
||||
it('converts legacy image parts for the vision request', () => {
|
||||
const converted = toVisionFilePart({
|
||||
type: 'image',
|
||||
image: 'abc',
|
||||
mediaType: 'image/jpeg',
|
||||
})
|
||||
expect(converted).toEqual({
|
||||
type: 'file',
|
||||
mediaType: 'image/jpeg',
|
||||
data: { type: 'data', data: 'abc' },
|
||||
})
|
||||
expect(replaceMessageImages([], new Map())).toEqual([])
|
||||
})
|
||||
|
||||
it('covers image-part keys, empty descriptions, and tagged data wrappers', async () => {
|
||||
const url = new URL('https://example.test/shot.png')
|
||||
expect(toVisionFilePart({ type: 'file', mediaType: 'image/png', data: 'kept' })).toEqual({
|
||||
type: 'file',
|
||||
mediaType: 'image/png',
|
||||
data: 'kept',
|
||||
})
|
||||
expect(toVisionFilePart({
|
||||
type: 'image',
|
||||
image: url,
|
||||
})).toEqual({
|
||||
type: 'file',
|
||||
mediaType: 'image/png',
|
||||
data: { type: 'url', url },
|
||||
})
|
||||
expect(imagePartKey({
|
||||
type: 'file',
|
||||
mediaType: 'image/webp',
|
||||
data: { type: 'data', data: 'nested' },
|
||||
})).toContain('nested')
|
||||
expect(imagePartKey({
|
||||
type: 'file',
|
||||
mediaType: 'image/png',
|
||||
data: { type: 'url', url },
|
||||
})).toContain('example.test')
|
||||
expect(imagePartKey({
|
||||
type: 'file',
|
||||
mediaType: 'image/png',
|
||||
data: new Uint8Array([9, 8, 7]),
|
||||
})).toContain('bytes:3:')
|
||||
expect(imagePartKey({
|
||||
type: 'image',
|
||||
image: 'abc',
|
||||
mediaType: 'image/jpeg',
|
||||
})).toContain('abc')
|
||||
|
||||
const described = await describeMessageImages(
|
||||
[{
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'image', image: 'abc', mediaType: 'image/jpeg' },
|
||||
{ type: 'file', mediaType: 'application/pdf', data: 'not-an-image' },
|
||||
],
|
||||
}],
|
||||
async () => ' ',
|
||||
)
|
||||
expect(described[0]).toMatchObject({
|
||||
content: [
|
||||
{ type: 'text', text: '[Image description was empty]' },
|
||||
{ type: 'file', mediaType: 'application/pdf', data: 'not-an-image' },
|
||||
],
|
||||
})
|
||||
})
|
||||
})
|
||||
167
src/ai/describe-images.ts
Normal file
167
src/ai/describe-images.ts
Normal file
@ -0,0 +1,167 @@
|
||||
import type { FilePart, ModelMessage, TextPart } from 'ai'
|
||||
|
||||
type LegacyImagePart = {
|
||||
type: 'image'
|
||||
image: unknown
|
||||
mediaType?: string
|
||||
}
|
||||
|
||||
type DescribeablePart = FilePart | LegacyImagePart
|
||||
|
||||
const isRecord = (value: unknown): value is Record<string, unknown> =>
|
||||
typeof value === 'object' && value !== null
|
||||
|
||||
const isFilePart = (part: unknown): part is FilePart =>
|
||||
isRecord(part) && part['type'] === 'file' && typeof part['mediaType'] === 'string'
|
||||
|
||||
const isLegacyImagePart = (part: unknown): part is LegacyImagePart =>
|
||||
isRecord(part) && part['type'] === 'image'
|
||||
|
||||
const isImageMediaType = (mediaType: string | undefined): boolean =>
|
||||
typeof mediaType === 'string' && mediaType.toLowerCase().startsWith('image/')
|
||||
|
||||
export const OPENCODE_VISION_PROMPT = `Describe this image for a designer who cannot see it.
|
||||
Focus on layout, visual hierarchy, colors, typography, exact visible copy, UI elements, icons, device frames, and spacing.
|
||||
Be concrete and complete. Do not suggest improvements.`
|
||||
|
||||
// Two FNV-1a variants over the full payload keep same-length images from
|
||||
// sharing a cache key without pulling in a hashing dependency.
|
||||
const hashBytes = (bytes: Uint8Array): string => {
|
||||
let a = 0x811c9dc5
|
||||
let b = 0x811c9dc5
|
||||
for (const byte of bytes) {
|
||||
a = Math.imul(a ^ byte, 0x01000193)
|
||||
b = Math.imul(b ^ byte, 0x85ebca77)
|
||||
}
|
||||
return `${(a >>> 0).toString(16)}-${(b >>> 0).toString(16)}`
|
||||
}
|
||||
|
||||
const serializeImageData = (data: unknown): string => {
|
||||
if (typeof data === 'string') return data
|
||||
if (data instanceof URL) return data.toString()
|
||||
if (isRecord(data) && data['type'] === 'data') return serializeImageData(data['data'])
|
||||
if (isRecord(data) && data['type'] === 'url') return serializeImageData(data['url'])
|
||||
if (data instanceof Uint8Array) return `bytes:${data.byteLength}:${hashBytes(data)}`
|
||||
if (data instanceof ArrayBuffer) return serializeImageData(new Uint8Array(data))
|
||||
return JSON.stringify(data)
|
||||
}
|
||||
|
||||
export const imagePartKey = (part: DescribeablePart): string => {
|
||||
if (isFilePart(part)) {
|
||||
return `${part.mediaType}:${serializeImageData(part.data)}`
|
||||
}
|
||||
return `image:${part.mediaType ?? 'image'}:${serializeImageData(part.image)}`
|
||||
}
|
||||
|
||||
const isDescribeablePart = (part: unknown): part is DescribeablePart => {
|
||||
if (isLegacyImagePart(part)) return true
|
||||
return isFilePart(part) && isImageMediaType(part.mediaType)
|
||||
}
|
||||
|
||||
const collectFromContent = (content: unknown, images: DescribeablePart[]): void => {
|
||||
if (!Array.isArray(content)) return
|
||||
for (const part of content as unknown[]) {
|
||||
if (isDescribeablePart(part)) images.push(part)
|
||||
}
|
||||
}
|
||||
|
||||
export const collectMessageImages = (
|
||||
messages: readonly ModelMessage[],
|
||||
): readonly DescribeablePart[] => {
|
||||
const images: DescribeablePart[] = []
|
||||
for (const message of messages) {
|
||||
if (message.role === 'tool') continue
|
||||
collectFromContent(message.content, images)
|
||||
}
|
||||
return images
|
||||
}
|
||||
|
||||
const toDescriptionPart = (description: string): TextPart => ({
|
||||
type: 'text',
|
||||
text: description.startsWith('[Image')
|
||||
? description
|
||||
: `Image description:\n${description}`,
|
||||
})
|
||||
|
||||
const replaceContentImages = (
|
||||
content: unknown,
|
||||
descriptions: ReadonlyMap<string, string>,
|
||||
): unknown => {
|
||||
if (!Array.isArray(content)) return content
|
||||
const parts = content as unknown[]
|
||||
const next = parts.map((part) => {
|
||||
if (!isDescribeablePart(part)) return part
|
||||
const description = descriptions.get(imagePartKey(part))
|
||||
if (description === undefined) return part
|
||||
return toDescriptionPart(description)
|
||||
})
|
||||
const changed = next.some((part, index) => part !== parts[index])
|
||||
return changed ? next : content
|
||||
}
|
||||
|
||||
export const replaceMessageImages = (
|
||||
messages: ModelMessage[],
|
||||
descriptions: ReadonlyMap<string, string>,
|
||||
): ModelMessage[] => {
|
||||
if (descriptions.size === 0) return messages
|
||||
const next = messages.map((message) => {
|
||||
if (message.role === 'tool') return message
|
||||
const content = replaceContentImages(message.content, descriptions)
|
||||
if (content === message.content) return message
|
||||
return { ...message, content } as ModelMessage
|
||||
})
|
||||
const changed = next.some((message, index) => message !== messages[index])
|
||||
return changed ? next : messages
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace image file parts with text descriptions so a text-only model can
|
||||
* still use screenshots and slide previews. Descriptions are cached by image
|
||||
* payload so repeated previews of the same bytes are described once. Pass a
|
||||
* shared `cache` to reuse descriptions across calls — each tool-loop step
|
||||
* re-prepares the full message history, and describe calls cost tens of
|
||||
* seconds each, so re-describing per step would stall every step.
|
||||
*/
|
||||
export const describeMessageImages = async (
|
||||
messages: ModelMessage[],
|
||||
describeImage: (part: FilePart) => Promise<string>,
|
||||
cache?: Map<string, string>,
|
||||
): Promise<ModelMessage[]> => {
|
||||
const images = collectMessageImages(messages)
|
||||
if (images.length === 0) return messages
|
||||
|
||||
const descriptions = cache ?? new Map<string, string>()
|
||||
for (const image of images) {
|
||||
const key = imagePartKey(image)
|
||||
if (descriptions.has(key)) continue
|
||||
try {
|
||||
const description = (await describeImage(toVisionFilePart(image))).trim()
|
||||
descriptions.set(
|
||||
key,
|
||||
description.length > 0 ? description : '[Image description was empty]',
|
||||
)
|
||||
} catch (error) {
|
||||
const detail = error instanceof Error ? error.message : String(error)
|
||||
descriptions.set(key, `[Image could not be described: ${detail}]`)
|
||||
}
|
||||
}
|
||||
|
||||
return replaceMessageImages(messages, descriptions)
|
||||
}
|
||||
|
||||
export const toVisionFilePart = (part: DescribeablePart): FilePart => {
|
||||
if (isFilePart(part)) return part
|
||||
const mediaType = part.mediaType ?? 'image/png'
|
||||
const image = part.image
|
||||
if (image instanceof URL) {
|
||||
return { type: 'file', mediaType, data: { type: 'url', url: image } }
|
||||
}
|
||||
if (
|
||||
typeof image === 'string'
|
||||
|| image instanceof Uint8Array
|
||||
|| image instanceof ArrayBuffer
|
||||
) {
|
||||
return { type: 'file', mediaType, data: { type: 'data', data: image } }
|
||||
}
|
||||
return { type: 'file', mediaType, data: { type: 'data', data: serializeImageData(image) } }
|
||||
}
|
||||
17
src/ai/opencode-client.test.ts
Normal file
17
src/ai/opencode-client.test.ts
Normal file
@ -0,0 +1,17 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { opencodeBaseUrl } from './opencode-client'
|
||||
import { opencodeRequestModelId } from './opencode-models'
|
||||
|
||||
describe('OpenCode client routing', () => {
|
||||
it('builds same-origin proxy base URLs for Zen and Go', () => {
|
||||
expect(opencodeBaseUrl('opencode-zen', 'https://app.shotluma.com'))
|
||||
.toBe('https://app.shotluma.com/api/opencode/zen/v1')
|
||||
expect(opencodeBaseUrl('opencode-go', 'http://127.0.0.1:4173'))
|
||||
.toBe('http://127.0.0.1:4173/api/opencode/go/v1')
|
||||
})
|
||||
|
||||
it('sends the upstream model id rather than the catalog prefix', () => {
|
||||
expect(opencodeRequestModelId('opencode-go/glm-5.3', 'glm-5.3')).toBe('glm-5.3')
|
||||
expect(opencodeRequestModelId('opencode-zen/kimi-k3')).toBe('kimi-k3')
|
||||
})
|
||||
})
|
||||
71
src/ai/opencode-client.ts
Normal file
71
src/ai/opencode-client.ts
Normal file
@ -0,0 +1,71 @@
|
||||
import { generateText, type FilePart } from 'ai'
|
||||
import { OPENCODE_VISION_PROMPT, toVisionFilePart } from './describe-images'
|
||||
import { opencodeModelDialect } from './opencode-dialects'
|
||||
import type { OpencodeProviderId } from './opencode-models'
|
||||
|
||||
export const opencodeBaseUrl = (
|
||||
providerId: OpencodeProviderId,
|
||||
origin: string,
|
||||
): string =>
|
||||
`${origin}/api/opencode/${providerId === 'opencode-go' ? 'go' : 'zen'}/v1`
|
||||
|
||||
/**
|
||||
* Build a language model for one OpenCode model on the dialect that gateway
|
||||
* serves it on. All four dialects share the same proxied base URL; only the
|
||||
* provider package and the path it appends differ.
|
||||
*/
|
||||
export const createOpencodeModel = async (options: {
|
||||
providerId: OpencodeProviderId
|
||||
modelId: string
|
||||
apiKey: string
|
||||
origin?: string
|
||||
}) => {
|
||||
const { providerId, modelId, apiKey } = options
|
||||
const baseURL = opencodeBaseUrl(providerId, options.origin ?? window.location.origin)
|
||||
switch (opencodeModelDialect(providerId, modelId)) {
|
||||
case 'responses': {
|
||||
const { createOpenAI } = await import('@ai-sdk/openai')
|
||||
return createOpenAI({ apiKey, baseURL }).responses(modelId)
|
||||
}
|
||||
case 'messages': {
|
||||
const { createAnthropic } = await import('@ai-sdk/anthropic')
|
||||
return createAnthropic({ apiKey, baseURL })(modelId)
|
||||
}
|
||||
case 'google': {
|
||||
const { createGoogle } = await import('@ai-sdk/google')
|
||||
return createGoogle({ apiKey, baseURL })(modelId)
|
||||
}
|
||||
case 'chat': {
|
||||
const { createOpenAI } = await import('@ai-sdk/openai')
|
||||
return createOpenAI({ apiKey, baseURL }).chat(modelId)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export const createOpencodeImageDescriber = (options: {
|
||||
providerId: OpencodeProviderId
|
||||
visionModelId: string
|
||||
apiKey: string
|
||||
signal?: AbortSignal
|
||||
}): ((part: FilePart) => Promise<string>) => {
|
||||
let modelPromise: ReturnType<typeof createOpencodeModel> | null = null
|
||||
return async (part) => {
|
||||
modelPromise ??= createOpencodeModel({
|
||||
providerId: options.providerId,
|
||||
modelId: options.visionModelId,
|
||||
apiKey: options.apiKey,
|
||||
})
|
||||
const { text } = await generateText({
|
||||
model: await modelPromise,
|
||||
messages: [{
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: OPENCODE_VISION_PROMPT },
|
||||
toVisionFilePart(part),
|
||||
],
|
||||
}],
|
||||
...(options.signal ? { abortSignal: options.signal } : {}),
|
||||
})
|
||||
return text
|
||||
}
|
||||
}
|
||||
61
src/ai/opencode-dialects.test.ts
Normal file
61
src/ai/opencode-dialects.test.ts
Normal file
@ -0,0 +1,61 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import {
|
||||
opencodeDialectNeedsChatToolImageRelay,
|
||||
opencodeModelDialect,
|
||||
opencodeSelectionDialect,
|
||||
} from './opencode-dialects'
|
||||
|
||||
describe('OpenCode dialect routing', () => {
|
||||
it('routes GPT, Grok, and Muse Spark to the Responses endpoint', () => {
|
||||
expect(opencodeModelDialect('opencode-zen', 'gpt-5.6-terra')).toBe('responses')
|
||||
expect(opencodeModelDialect('opencode-go', 'gpt-5.6-luna')).toBe('responses')
|
||||
expect(opencodeModelDialect('opencode-go', 'grok-4.5')).toBe('responses')
|
||||
expect(opencodeModelDialect('opencode-zen', 'grok-4.6')).toBe('responses')
|
||||
expect(opencodeModelDialect('opencode-go', 'muse-spark-1.2-contributor'))
|
||||
.toBe('responses')
|
||||
})
|
||||
|
||||
it('routes Claude and Qwen to the Anthropic messages endpoint', () => {
|
||||
expect(opencodeModelDialect('opencode-zen', 'claude-sonnet-5')).toBe('messages')
|
||||
expect(opencodeModelDialect('opencode-zen', 'qwen3.7-max')).toBe('messages')
|
||||
expect(opencodeModelDialect('opencode-go', 'qwen3.8-max')).toBe('messages')
|
||||
})
|
||||
|
||||
it('routes Gemini to the Google endpoint', () => {
|
||||
expect(opencodeModelDialect('opencode-zen', 'gemini-3.7-flash')).toBe('google')
|
||||
})
|
||||
|
||||
it('keeps the open-weight families and unknown ids on chat completions', () => {
|
||||
expect(opencodeModelDialect('opencode-zen', 'kimi-k3')).toBe('chat')
|
||||
expect(opencodeModelDialect('opencode-go', 'mimo-v2.5')).toBe('chat')
|
||||
expect(opencodeModelDialect('opencode-go', 'glm-5.3')).toBe('chat')
|
||||
expect(opencodeModelDialect('opencode-zen', 'deepseek-v4-pro')).toBe('chat')
|
||||
expect(opencodeModelDialect('opencode-zen', 'hy3-free')).toBe('chat')
|
||||
expect(opencodeModelDialect('opencode-go', 'some-future-open-model')).toBe('chat')
|
||||
})
|
||||
|
||||
it('splits MiniMax by gateway because Zen and Go disagree', () => {
|
||||
expect(opencodeModelDialect('opencode-zen', 'minimax-m2.5')).toBe('chat')
|
||||
expect(opencodeModelDialect('opencode-go', 'minimax-m3')).toBe('messages')
|
||||
})
|
||||
|
||||
it('resolves a selection and ignores non-OpenCode providers', () => {
|
||||
expect(opencodeSelectionDialect({
|
||||
provider: 'opencode-go',
|
||||
model: 'opencode-go/gpt-5.6-luna',
|
||||
})).toBe('responses')
|
||||
expect(opencodeSelectionDialect({
|
||||
provider: 'opencode-zen',
|
||||
model: 'opencode-zen/kimi-k3',
|
||||
})).toBe('chat')
|
||||
expect(opencodeSelectionDialect({ provider: 'openai', model: 'gpt-5.6-terra' }))
|
||||
.toBeNull()
|
||||
})
|
||||
|
||||
it('relays tool images only on the chat-completions dialect', () => {
|
||||
expect(opencodeDialectNeedsChatToolImageRelay('chat')).toBe(true)
|
||||
expect(opencodeDialectNeedsChatToolImageRelay('responses')).toBe(false)
|
||||
expect(opencodeDialectNeedsChatToolImageRelay('messages')).toBe(false)
|
||||
expect(opencodeDialectNeedsChatToolImageRelay('google')).toBe(false)
|
||||
})
|
||||
})
|
||||
74
src/ai/opencode-dialects.ts
Normal file
74
src/ai/opencode-dialects.ts
Normal file
@ -0,0 +1,74 @@
|
||||
import {
|
||||
isOpencodeProviderId,
|
||||
opencodeRequestModelId,
|
||||
type OpencodeProviderId,
|
||||
} from './opencode-models'
|
||||
import { getAiModel, type AiModelSelection } from './provider-catalog'
|
||||
|
||||
/**
|
||||
* OpenCode Zen and Go are multi-dialect gateways: every model answers on exactly
|
||||
* one endpoint, and the wrong one fails with a 400 or an "Endpoint is
|
||||
* unavailable" 5xx instead of falling back. The table mirrors the endpoint
|
||||
* columns published at https://opencode.ai/docs/zen and https://opencode.ai/docs/go.
|
||||
*/
|
||||
export type OpencodeDialect = 'responses' | 'messages' | 'google' | 'chat'
|
||||
|
||||
const PREFIX_DIALECTS: readonly (readonly [string, OpencodeDialect])[] = [
|
||||
['gpt-', 'responses'],
|
||||
['grok-', 'responses'],
|
||||
['muse-spark-', 'responses'],
|
||||
['claude-', 'messages'],
|
||||
['qwen', 'messages'],
|
||||
['gemini-', 'google'],
|
||||
]
|
||||
|
||||
/** MiniMax is the one family the two gateways route differently. */
|
||||
const GATEWAY_PREFIX_DIALECTS: Record<
|
||||
OpencodeProviderId,
|
||||
readonly (readonly [string, OpencodeDialect])[]
|
||||
> = {
|
||||
'opencode-zen': [['minimax-', 'chat']],
|
||||
'opencode-go': [['minimax-', 'messages']],
|
||||
}
|
||||
|
||||
const matchPrefix = (
|
||||
modelId: string,
|
||||
table: readonly (readonly [string, OpencodeDialect])[],
|
||||
): OpencodeDialect | null => {
|
||||
for (const [prefix, dialect] of table) {
|
||||
if (modelId.startsWith(prefix)) return dialect
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the endpoint dialect for an upstream OpenCode model id. Prefixes are
|
||||
* used rather than a fixed id list because `/v1/models` keeps gaining entries;
|
||||
* the open-weight families (DeepSeek, GLM, Kimi, MiMo, Hy3, Laguna, Nemotron,
|
||||
* Big Pickle, and the free tier) all answer on chat completions, which makes it
|
||||
* the correct default for an unrecognised id.
|
||||
*/
|
||||
export const opencodeModelDialect = (
|
||||
providerId: OpencodeProviderId,
|
||||
modelId: string,
|
||||
): OpencodeDialect =>
|
||||
matchPrefix(modelId, GATEWAY_PREFIX_DIALECTS[providerId])
|
||||
?? matchPrefix(modelId, PREFIX_DIALECTS)
|
||||
?? 'chat'
|
||||
|
||||
/** The dialect a selection resolves to, or `null` for a non-OpenCode provider. */
|
||||
export const opencodeSelectionDialect = (
|
||||
selection: AiModelSelection,
|
||||
): OpencodeDialect | null => {
|
||||
if (!isOpencodeProviderId(selection.provider)) return null
|
||||
const model = getAiModel(selection)
|
||||
return opencodeModelDialect(
|
||||
selection.provider,
|
||||
opencodeRequestModelId(model.id, model.providerModelId),
|
||||
)
|
||||
}
|
||||
|
||||
/** Only the chat-completions dialect rejects images inside tool results. */
|
||||
export const opencodeDialectNeedsChatToolImageRelay = (
|
||||
dialect: OpencodeDialect,
|
||||
): boolean => dialect === 'chat'
|
||||
171
src/ai/opencode-models.test.ts
Normal file
171
src/ai/opencode-models.test.ts
Normal file
@ -0,0 +1,171 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import {
|
||||
formatOpencodeModelLabel,
|
||||
loadAllOpencodeModels,
|
||||
loadOpencodeModels,
|
||||
opencodeCatalogId,
|
||||
opencodeModelsUrl,
|
||||
opencodeModelSupportsVision,
|
||||
parseOpencodeModels,
|
||||
pickOpencodeVisionModel,
|
||||
readCachedOpencodeModels,
|
||||
toOpencodeModelOption,
|
||||
} from './opencode-models'
|
||||
import {
|
||||
getAiProvider,
|
||||
getDynamicProviderModels,
|
||||
setDynamicProviderModels,
|
||||
} from './provider-catalog'
|
||||
|
||||
const createMemoryStorage = (entries: Record<string, string> = {}) => {
|
||||
const store = new Map(Object.entries(entries))
|
||||
return {
|
||||
getItem: (key: string) => store.get(key) ?? null,
|
||||
setItem: (key: string, value: string) => {
|
||||
store.set(key, value)
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
setDynamicProviderModels('opencode-zen', [])
|
||||
setDynamicProviderModels('opencode-go', [])
|
||||
vi.unstubAllGlobals()
|
||||
})
|
||||
|
||||
describe('OpenCode model catalog', () => {
|
||||
it('formats labels and marks known text-only models', () => {
|
||||
expect(formatOpencodeModelLabel('gpt-5.6-luna')).toBe('GPT 5.6 Luna')
|
||||
expect(formatOpencodeModelLabel('kimi-k3')).toBe('Kimi K3')
|
||||
expect(formatOpencodeModelLabel('mimo-v2.5-pro')).toBe('MiMo V2.5 Pro')
|
||||
expect(formatOpencodeModelLabel('big-pickle')).toBe('Big Pickle')
|
||||
expect(opencodeModelSupportsVision('kimi-k3')).toBe(true)
|
||||
expect(opencodeModelSupportsVision('glm-5.3')).toBe(false)
|
||||
expect(opencodeCatalogId('opencode-go', 'glm-5.3')).toBe('opencode-go/glm-5.3')
|
||||
})
|
||||
|
||||
it('parses the live /models list and keeps text-only tool models', () => {
|
||||
const models = parseOpencodeModels('opencode-go', {
|
||||
data: [
|
||||
{ id: 'glm-5.3' },
|
||||
{ id: 'kimi-k3' },
|
||||
{ id: 'kimi-k3' },
|
||||
{ id: '' },
|
||||
'nope',
|
||||
],
|
||||
})
|
||||
expect(models.map((model) => model.id)).toEqual([
|
||||
'opencode-go/glm-5.3',
|
||||
'opencode-go/kimi-k3',
|
||||
])
|
||||
expect(toOpencodeModelOption('opencode-go', 'glm-5.3')).toMatchObject({
|
||||
providerModelId: 'glm-5.3',
|
||||
supportsVision: false,
|
||||
reasoningEfforts: ['low', 'medium', 'high'],
|
||||
})
|
||||
expect(toOpencodeModelOption('opencode-zen', 'kimi-k3').supportsVision).toBeUndefined()
|
||||
expect(parseOpencodeModels('opencode-zen', null)).toEqual([])
|
||||
})
|
||||
|
||||
it('picks a cheap vision fallback and honors an explicit preference', () => {
|
||||
const models = [
|
||||
toOpencodeModelOption('opencode-zen', 'glm-5.2'),
|
||||
toOpencodeModelOption('opencode-zen', 'gpt-5-nano'),
|
||||
toOpencodeModelOption('opencode-zen', 'claude-sonnet-5'),
|
||||
]
|
||||
expect(pickOpencodeVisionModel('opencode-zen', models)?.id).toBe(
|
||||
'opencode-zen/gpt-5-nano',
|
||||
)
|
||||
expect(pickOpencodeVisionModel(
|
||||
'opencode-zen',
|
||||
models,
|
||||
'opencode-zen/claude-sonnet-5',
|
||||
)?.id).toBe('opencode-zen/claude-sonnet-5')
|
||||
expect(pickOpencodeVisionModel(
|
||||
'opencode-go',
|
||||
[toOpencodeModelOption('opencode-go', 'glm-5.3')],
|
||||
)).toBeUndefined()
|
||||
expect(pickOpencodeVisionModel(
|
||||
'opencode-go',
|
||||
[toOpencodeModelOption('opencode-go', 'kimi-k3')],
|
||||
)?.id).toBe('opencode-go/kimi-k3')
|
||||
expect(formatOpencodeModelLabel('unknown-xyz')).toBe('Unknown Xyz')
|
||||
expect(parseOpencodeModels('opencode-zen', { data: 'nope' })).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('loadOpencodeModels', () => {
|
||||
it('serves a fresh cache without fetching', async () => {
|
||||
const cachedModel = toOpencodeModelOption('opencode-zen', 'kimi-k3')
|
||||
const storage = createMemoryStorage({
|
||||
'shotluma-opencode-zen-models': JSON.stringify({
|
||||
fetchedAt: 500,
|
||||
models: [cachedModel],
|
||||
}),
|
||||
})
|
||||
const fetchMock = vi.fn()
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
|
||||
const result = await loadOpencodeModels('opencode-zen', { now: 600, storage })
|
||||
expect(result).toEqual({ models: [cachedModel], source: 'cache' })
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
expect(getDynamicProviderModels('opencode-zen')).toEqual([cachedModel])
|
||||
expect(readCachedOpencodeModels('opencode-zen', 600, storage)).toEqual([cachedModel])
|
||||
})
|
||||
|
||||
it('fetches, caches, and registers the remote catalog', async () => {
|
||||
const storage = createMemoryStorage()
|
||||
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ data: [{ id: 'glm-5.2' }, { id: 'kimi-k3' }] }),
|
||||
}))
|
||||
|
||||
const result = await loadOpencodeModels('opencode-zen', { now: 42, storage })
|
||||
expect(result.source).toBe('remote')
|
||||
expect(result.models.map((model) => model.id)).toEqual([
|
||||
'opencode-zen/glm-5.2',
|
||||
'opencode-zen/kimi-k3',
|
||||
])
|
||||
expect(getDynamicProviderModels('opencode-zen')).toEqual(result.models)
|
||||
})
|
||||
|
||||
it('falls back to the curated shortlist when the fetch fails', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({ ok: false, status: 503 }))
|
||||
|
||||
const result = await loadOpencodeModels('opencode-go', { now: 42, storage: null })
|
||||
expect(result.source).toBe('fallback')
|
||||
expect(result.models).toBe(getAiProvider('opencode-go').models)
|
||||
})
|
||||
|
||||
it('ignores stale or invalid cache and coalesces remote loads', async () => {
|
||||
const storage = createMemoryStorage({
|
||||
'shotluma-opencode-zen-models': '{not-json',
|
||||
'shotluma-opencode-go-models': JSON.stringify({
|
||||
fetchedAt: 1,
|
||||
models: [toOpencodeModelOption('opencode-go', 'kimi-k3')],
|
||||
}),
|
||||
})
|
||||
expect(readCachedOpencodeModels('opencode-zen', 10, storage)).toBeNull()
|
||||
expect(readCachedOpencodeModels('opencode-go', 60 * 60 * 1000 + 2, storage)).toBeNull()
|
||||
expect(opencodeModelsUrl('opencode-zen', 'https://app.shotluma.com'))
|
||||
.toBe('https://app.shotluma.com/api/opencode/zen/v1/models')
|
||||
expect(opencodeModelsUrl('opencode-go', 'null'))
|
||||
.toBe('https://opencode.ai/zen/go/v1/models')
|
||||
|
||||
const fetchMock = vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ data: [{ id: 'kimi-k3' }] }),
|
||||
})
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
const [first, second, all] = await Promise.all([
|
||||
loadOpencodeModels('opencode-zen', { now: 42, storage: null }),
|
||||
loadOpencodeModels('opencode-zen', { now: 42, storage: null }),
|
||||
loadAllOpencodeModels({ now: 42, storage: null }),
|
||||
])
|
||||
expect(first.source).toBe('remote')
|
||||
expect(second.source).toBe('remote')
|
||||
expect(all).toHaveLength(2)
|
||||
// One request per gateway, not one per caller.
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
})
|
||||
361
src/ai/opencode-models.ts
Normal file
361
src/ai/opencode-models.ts
Normal file
@ -0,0 +1,361 @@
|
||||
import {
|
||||
OPENROUTER_REASONING_EFFORTS,
|
||||
getAiProvider,
|
||||
setDynamicProviderModels,
|
||||
type AiModelOption,
|
||||
type AiProviderId,
|
||||
} from './provider-catalog'
|
||||
|
||||
export type OpencodeGateway = 'zen' | 'go'
|
||||
|
||||
export const OPENCODE_PROVIDERS = [
|
||||
'opencode-zen',
|
||||
'opencode-go',
|
||||
] as const satisfies readonly AiProviderId[]
|
||||
|
||||
export type OpencodeProviderId = (typeof OPENCODE_PROVIDERS)[number]
|
||||
|
||||
export const isOpencodeProviderId = (
|
||||
value: unknown,
|
||||
): value is OpencodeProviderId =>
|
||||
value === 'opencode-zen' || value === 'opencode-go'
|
||||
|
||||
export const OPENCODE_MODELS_TTL_MS = 60 * 60 * 1000
|
||||
|
||||
// Bounds the catalog request so a stalled gateway falls back to the curated
|
||||
// shortlist instead of leaving the model browser loading forever.
|
||||
export const OPENCODE_MODELS_REQUEST_TIMEOUT_MS = 10_000
|
||||
|
||||
const STORAGE_KEYS: Record<OpencodeProviderId, string> = {
|
||||
'opencode-zen': 'shotluma-opencode-zen-models',
|
||||
'opencode-go': 'shotluma-opencode-go-models',
|
||||
}
|
||||
|
||||
const GATEWAY_BY_PROVIDER: Record<OpencodeProviderId, OpencodeGateway> = {
|
||||
'opencode-zen': 'zen',
|
||||
'opencode-go': 'go',
|
||||
}
|
||||
|
||||
/**
|
||||
* Models.dev reports these OpenCode ids as text-only; the runner describes
|
||||
* images for them instead of attaching them. Unknown live ids default to
|
||||
* vision-capable so screenshots reach new models untouched — a wrong default
|
||||
* fails loudly at the provider instead of silently flattening images to text.
|
||||
*/
|
||||
const OPENCODE_TEXT_ONLY_MODELS = new Set([
|
||||
'big-pickle',
|
||||
'deepseek-v4-flash',
|
||||
'deepseek-v4-flash-free',
|
||||
'deepseek-v4-pro',
|
||||
'glm-5',
|
||||
'glm-5.1',
|
||||
'glm-5.2',
|
||||
'glm-5.3',
|
||||
'gpt-5.3-codex-spark',
|
||||
'hy3',
|
||||
'hy3-free',
|
||||
'hy3-preview',
|
||||
'laguna-s-2.1-free',
|
||||
'minimax-m2.5',
|
||||
'minimax-m2.7',
|
||||
'nemotron-3-ultra-free',
|
||||
'nemotron-3.5-lightning-free',
|
||||
'qwen3.7-max',
|
||||
])
|
||||
|
||||
const OPENCODE_LABEL_OVERRIDES: Record<string, string> = {
|
||||
'big-pickle': 'Big Pickle',
|
||||
hy3: 'Hy3',
|
||||
'hy3-free': 'Hy3 Free',
|
||||
'hy3-preview': 'Hy3 Preview',
|
||||
'mimo-v2.5': 'MiMo V2.5',
|
||||
'mimo-v2.5-free': 'MiMo V2.5 Free',
|
||||
'mimo-v2.5-pro': 'MiMo V2.5 Pro',
|
||||
'mimo-v2-omni': 'MiMo V2 Omni',
|
||||
'mimo-v2-pro': 'MiMo V2 Pro',
|
||||
}
|
||||
|
||||
const DEFAULT_VISION_MODEL_IDS: Record<OpencodeProviderId, readonly string[]> = {
|
||||
'opencode-zen': [
|
||||
'opencode-zen/mimo-v2.5-free',
|
||||
'opencode-zen/gpt-5-nano',
|
||||
'opencode-zen/gemini-3.5-flash-lite',
|
||||
'opencode-zen/gpt-5.4-nano',
|
||||
],
|
||||
'opencode-go': [
|
||||
'opencode-go/mimo-v2.5',
|
||||
'opencode-go/gpt-5.6-luna',
|
||||
'opencode-go/minimax-m3',
|
||||
],
|
||||
}
|
||||
|
||||
const TOKEN_LABELS: Record<string, string> = {
|
||||
claude: 'Claude',
|
||||
deepseek: 'DeepSeek',
|
||||
gemini: 'Gemini',
|
||||
glm: 'GLM',
|
||||
gpt: 'GPT',
|
||||
grok: 'Grok',
|
||||
kimi: 'Kimi',
|
||||
laguna: 'Laguna',
|
||||
minimax: 'MiniMax',
|
||||
mimo: 'MiMo',
|
||||
muse: 'Muse',
|
||||
nemotron: 'Nemotron',
|
||||
qwen: 'Qwen',
|
||||
}
|
||||
|
||||
export const formatOpencodeModelLabel = (modelId: string): string => {
|
||||
const override = OPENCODE_LABEL_OVERRIDES[modelId]
|
||||
if (override) return override
|
||||
return modelId.split('-').map((token) => {
|
||||
const mapped = TOKEN_LABELS[token]
|
||||
if (mapped) return mapped
|
||||
if (/[0-9]/.test(token)) return token.toUpperCase()
|
||||
if (token.length === 0) return token
|
||||
return `${token[0]?.toUpperCase() ?? ''}${token.slice(1)}`
|
||||
}).join(' ')
|
||||
}
|
||||
|
||||
export const opencodeModelSupportsVision = (modelId: string): boolean =>
|
||||
!OPENCODE_TEXT_ONLY_MODELS.has(modelId)
|
||||
|
||||
export const opencodeCatalogId = (
|
||||
providerId: OpencodeProviderId,
|
||||
modelId: string,
|
||||
): string => `${providerId}/${modelId}`
|
||||
|
||||
export const opencodeProviderModelId = (catalogId: string): string =>
|
||||
catalogId.replace(/^opencode-(?:zen|go)\//, '')
|
||||
|
||||
/** The upstream id to send for a catalog entry, preferring the stored mapping. */
|
||||
export const opencodeRequestModelId = (
|
||||
catalogId: string,
|
||||
providerModelId?: string,
|
||||
): string => providerModelId ?? opencodeProviderModelId(catalogId)
|
||||
|
||||
const gatewayLabel = (providerId: OpencodeProviderId): string =>
|
||||
providerId === 'opencode-go' ? 'OpenCode Go' : 'OpenCode Zen'
|
||||
|
||||
const describeOpencodeModel = (
|
||||
providerId: OpencodeProviderId,
|
||||
supportsVision: boolean,
|
||||
): string => {
|
||||
const gateway = gatewayLabel(providerId)
|
||||
if (supportsVision) return `${gateway} · vision and tools`
|
||||
return `${gateway} · no native vision · a vision model describes images first`
|
||||
}
|
||||
|
||||
export const toOpencodeModelOption = (
|
||||
providerId: OpencodeProviderId,
|
||||
modelId: string,
|
||||
): AiModelOption => {
|
||||
const supportsVision = opencodeModelSupportsVision(modelId)
|
||||
return {
|
||||
id: opencodeCatalogId(providerId, modelId),
|
||||
providerModelId: modelId,
|
||||
label: formatOpencodeModelLabel(modelId),
|
||||
description: describeOpencodeModel(providerId, supportsVision),
|
||||
reasoningEfforts: OPENROUTER_REASONING_EFFORTS,
|
||||
...(supportsVision ? {} : { supportsVision: false }),
|
||||
}
|
||||
}
|
||||
|
||||
const parseModelId = (value: unknown): string | null => {
|
||||
if (!value || typeof value !== 'object') return null
|
||||
const id = (value as Record<string, unknown>)['id']
|
||||
return typeof id === 'string' && id.length > 0 ? id : null
|
||||
}
|
||||
|
||||
/** Parse an OpenAI-compatible `/models` list into OpenCode catalog options. */
|
||||
export const parseOpencodeModels = (
|
||||
providerId: OpencodeProviderId,
|
||||
payload: unknown,
|
||||
): AiModelOption[] => {
|
||||
if (!payload || typeof payload !== 'object') return []
|
||||
const data = (payload as Record<string, unknown>)['data']
|
||||
if (!Array.isArray(data)) return []
|
||||
const seen = new Set<string>()
|
||||
const models: AiModelOption[] = []
|
||||
for (const entry of data) {
|
||||
const modelId = parseModelId(entry)
|
||||
if (!modelId || seen.has(modelId)) continue
|
||||
seen.add(modelId)
|
||||
models.push(toOpencodeModelOption(providerId, modelId))
|
||||
}
|
||||
return models.sort((left, right) => left.label.localeCompare(right.label))
|
||||
}
|
||||
|
||||
export const listOpencodeVisionModels = (
|
||||
models: readonly AiModelOption[],
|
||||
): readonly AiModelOption[] =>
|
||||
models.filter((model) => model.supportsVision !== false)
|
||||
|
||||
export const pickOpencodeVisionModel = (
|
||||
providerId: OpencodeProviderId,
|
||||
models: readonly AiModelOption[],
|
||||
preferredId?: string,
|
||||
): AiModelOption | undefined => {
|
||||
const visionModels = listOpencodeVisionModels(models)
|
||||
if (visionModels.length === 0) return undefined
|
||||
if (preferredId) {
|
||||
const preferred = visionModels.find((model) => model.id === preferredId)
|
||||
if (preferred) return preferred
|
||||
}
|
||||
for (const candidateId of DEFAULT_VISION_MODEL_IDS[providerId]) {
|
||||
const match = visionModels.find((model) => model.id === candidateId)
|
||||
if (match) return match
|
||||
}
|
||||
return visionModels[0]
|
||||
}
|
||||
|
||||
type CachedOpencodeModels = {
|
||||
fetchedAt: number
|
||||
models: AiModelOption[]
|
||||
}
|
||||
|
||||
const parseCachedModels = (raw: string | null): CachedOpencodeModels | null => {
|
||||
if (!raw) return null
|
||||
let parsed: unknown
|
||||
try {
|
||||
parsed = JSON.parse(raw)
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
if (!parsed || typeof parsed !== 'object') return null
|
||||
const record = parsed as Record<string, unknown>
|
||||
if (typeof record['fetchedAt'] !== 'number' || !Array.isArray(record['models'])) {
|
||||
return null
|
||||
}
|
||||
const models = record['models'].filter((entry): entry is AiModelOption => {
|
||||
if (!entry || typeof entry !== 'object') return false
|
||||
const model = entry as Record<string, unknown>
|
||||
return typeof model['id'] === 'string'
|
||||
&& typeof model['label'] === 'string'
|
||||
&& typeof model['description'] === 'string'
|
||||
})
|
||||
return { fetchedAt: record['fetchedAt'], models }
|
||||
}
|
||||
|
||||
type ModelStorage = Pick<Storage, 'getItem' | 'setItem'>
|
||||
|
||||
const getBrowserStorage = (): ModelStorage | null => {
|
||||
try {
|
||||
if (typeof localStorage === 'undefined') return null
|
||||
return localStorage
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
export const opencodeModelsUrl = (
|
||||
providerId: OpencodeProviderId,
|
||||
origin: string = typeof window === 'undefined' ? '' : window.location.origin,
|
||||
): string => {
|
||||
const gateway = GATEWAY_BY_PROVIDER[providerId]
|
||||
if (origin && origin !== 'null') {
|
||||
return `${origin}/api/opencode/${gateway}/v1/models`
|
||||
}
|
||||
return gateway === 'go'
|
||||
? 'https://opencode.ai/zen/go/v1/models'
|
||||
: 'https://opencode.ai/zen/v1/models'
|
||||
}
|
||||
|
||||
export const readCachedOpencodeModels = (
|
||||
providerId: OpencodeProviderId,
|
||||
now: number,
|
||||
storage: ModelStorage | null = getBrowserStorage(),
|
||||
): AiModelOption[] | null => {
|
||||
if (!storage) return null
|
||||
let cached: CachedOpencodeModels | null
|
||||
try {
|
||||
cached = parseCachedModels(storage.getItem(STORAGE_KEYS[providerId]))
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
if (!cached || cached.models.length === 0) return null
|
||||
if (now - cached.fetchedAt > OPENCODE_MODELS_TTL_MS) return null
|
||||
return cached.models
|
||||
}
|
||||
|
||||
const writeCachedOpencodeModels = (
|
||||
providerId: OpencodeProviderId,
|
||||
models: AiModelOption[],
|
||||
now: number,
|
||||
storage: ModelStorage | null,
|
||||
): void => {
|
||||
if (!storage) return
|
||||
try {
|
||||
storage.setItem(
|
||||
STORAGE_KEYS[providerId],
|
||||
JSON.stringify({ fetchedAt: now, models } satisfies CachedOpencodeModels),
|
||||
)
|
||||
} catch {
|
||||
// A full or blocked storage only loses the cache, not the feature.
|
||||
}
|
||||
}
|
||||
|
||||
export type OpencodeModelsResult = {
|
||||
models: readonly AiModelOption[]
|
||||
source: 'remote' | 'cache' | 'fallback'
|
||||
}
|
||||
|
||||
const pendingFetches = new Map<OpencodeProviderId, Promise<OpencodeModelsResult>>()
|
||||
|
||||
const fetchRemoteModels = async (
|
||||
providerId: OpencodeProviderId,
|
||||
now: number,
|
||||
storage: ModelStorage | null,
|
||||
): Promise<OpencodeModelsResult> => {
|
||||
const response = await fetch(opencodeModelsUrl(providerId), {
|
||||
signal: AbortSignal.timeout(OPENCODE_MODELS_REQUEST_TIMEOUT_MS),
|
||||
})
|
||||
if (!response.ok) {
|
||||
throw new Error(`OpenCode models request failed: ${response.status}`)
|
||||
}
|
||||
const models = parseOpencodeModels(providerId, await response.json() as unknown)
|
||||
if (models.length === 0) throw new Error('OpenCode models response was empty')
|
||||
writeCachedOpencodeModels(providerId, models, now, storage)
|
||||
setDynamicProviderModels(providerId, models)
|
||||
return { models, source: 'remote' }
|
||||
}
|
||||
|
||||
/**
|
||||
* Load an OpenCode gateway catalog: fresh cache, then the proxied `/models`
|
||||
* list, then the curated shortlist. Successful loads register the models so
|
||||
* selections resolve everywhere. Never rejects.
|
||||
*/
|
||||
export const loadOpencodeModels = (
|
||||
providerId: OpencodeProviderId,
|
||||
options?: {
|
||||
now?: number
|
||||
storage?: ModelStorage | null
|
||||
},
|
||||
): Promise<OpencodeModelsResult> => {
|
||||
const now = options?.now ?? Date.now()
|
||||
const storage = options?.storage === undefined ? getBrowserStorage() : options.storage
|
||||
|
||||
const cached = readCachedOpencodeModels(providerId, now, storage)
|
||||
if (cached) {
|
||||
setDynamicProviderModels(providerId, cached)
|
||||
return Promise.resolve({ models: cached, source: 'cache' })
|
||||
}
|
||||
|
||||
const pending = pendingFetches.get(providerId)
|
||||
?? fetchRemoteModels(providerId, now, storage)
|
||||
.catch((): OpencodeModelsResult => ({
|
||||
models: getAiProvider(providerId).models,
|
||||
source: 'fallback',
|
||||
}))
|
||||
.finally(() => {
|
||||
pendingFetches.delete(providerId)
|
||||
})
|
||||
pendingFetches.set(providerId, pending)
|
||||
return pending
|
||||
}
|
||||
|
||||
export const loadAllOpencodeModels = (options?: {
|
||||
now?: number
|
||||
storage?: ModelStorage | null
|
||||
}): Promise<readonly OpencodeModelsResult[]> =>
|
||||
Promise.all(OPENCODE_PROVIDERS.map((providerId) => loadOpencodeModels(providerId, options)))
|
||||
@ -1,3 +1,4 @@
|
||||
import { opencodeSelectionDialect } from './opencode-dialects'
|
||||
import {
|
||||
getAiStreamReasoningOptions,
|
||||
type AiModelSelection,
|
||||
@ -34,12 +35,19 @@ export type AiStreamRequestOptions = {
|
||||
* still thinks and still bills those tokens while withholding every thought
|
||||
* summary — the run band then has no live prose to show for the whole run.
|
||||
*
|
||||
* OpenCode's Gemini models run on the same Google provider through the gateway,
|
||||
* so they need the same option to stream thoughts.
|
||||
*
|
||||
* Keep this object limited to `includeThoughts`. Adding `thinkingLevel` or
|
||||
* `thinkingBudget` here would take full precedence over the portable `reasoning`
|
||||
* option and silently discard the user's effort choice.
|
||||
*/
|
||||
const GOOGLE_THOUGHT_OPTIONS = { thinkingConfig: { includeThoughts: true } } as const
|
||||
|
||||
const usesGoogleGeneration = (selection: AiModelSelection): boolean =>
|
||||
selection.provider === 'google'
|
||||
|| opencodeSelectionDialect(selection) === 'google'
|
||||
|
||||
/**
|
||||
* Combine the reasoning-effort options with OpenAI's request-wide cache
|
||||
* routing. Implicit mode intentionally stays enabled: its moving breakpoint
|
||||
@ -57,7 +65,7 @@ export const buildStreamRequestOptions = (
|
||||
promptCacheKey: string,
|
||||
): AiStreamRequestOptions => {
|
||||
const reasoningOptions = getAiStreamReasoningOptions(selection)
|
||||
if (selection.provider === 'google') {
|
||||
if (usesGoogleGeneration(selection)) {
|
||||
return {
|
||||
...(reasoningOptions && 'reasoning' in reasoningOptions
|
||||
? { reasoning: reasoningOptions.reasoning }
|
||||
|
||||
@ -11,8 +11,11 @@ import {
|
||||
getAiStreamReasoningOptions,
|
||||
getDefaultAiModel,
|
||||
getDynamicOpenRouterModels,
|
||||
getAiProviderKeyFields,
|
||||
isAiProviderId,
|
||||
modelSupportsVision,
|
||||
setDynamicOpenRouterModels,
|
||||
setDynamicProviderModels,
|
||||
toAiSdkReasoningEffort,
|
||||
} from './provider-catalog'
|
||||
|
||||
@ -27,6 +30,8 @@ describe('AI provider catalog', () => {
|
||||
'anthropic',
|
||||
'xai',
|
||||
'openrouter',
|
||||
'opencode-zen',
|
||||
'opencode-go',
|
||||
])
|
||||
expect(AI_PROVIDERS.map((provider) => provider.envVar)).toEqual([
|
||||
'',
|
||||
@ -37,10 +42,12 @@ describe('AI provider catalog', () => {
|
||||
'VITE_ANTHROPIC_API_KEY',
|
||||
'VITE_XAI_API_KEY',
|
||||
'VITE_OPENROUTER_API_KEY',
|
||||
'VITE_OPENCODE_API_KEY',
|
||||
'VITE_OPENCODE_API_KEY',
|
||||
])
|
||||
expect(AI_PROVIDERS.filter((provider) => provider.transport === 'proxy').map(
|
||||
(provider) => provider.id,
|
||||
)).toEqual(['moonshot'])
|
||||
)).toEqual(['moonshot', 'opencode-zen', 'opencode-go'])
|
||||
expect(getAiProvider('codex')).toMatchObject({
|
||||
auth: 'chatgpt',
|
||||
transport: 'bridge',
|
||||
@ -219,4 +226,47 @@ describe('AI provider catalog', () => {
|
||||
providerOptions: { openai: { reasoningEffort: 'max' } },
|
||||
})
|
||||
})
|
||||
|
||||
it('shares one OpenCode key field and synthesizes runtime catalog models', () => {
|
||||
const opencodeField = getAiProviderKeyFields().find((field) => field.id === 'opencode')
|
||||
expect(opencodeField).toMatchObject({
|
||||
label: 'OpenCode',
|
||||
envVar: 'VITE_OPENCODE_API_KEY',
|
||||
providerIds: ['opencode-zen', 'opencode-go'],
|
||||
})
|
||||
expect(modelSupportsVision(getAiModel({
|
||||
provider: 'opencode-go',
|
||||
model: 'opencode-go/glm-5.3',
|
||||
}))).toBe(false)
|
||||
expect(modelSupportsVision(getAiModel({
|
||||
provider: 'opencode-zen',
|
||||
model: 'opencode-zen/kimi-k3',
|
||||
}))).toBe(true)
|
||||
|
||||
const synthesized = getAiModel({
|
||||
provider: 'opencode-go',
|
||||
model: 'opencode-go/new-model',
|
||||
})
|
||||
expect(synthesized).toEqual({
|
||||
id: 'opencode-go/new-model',
|
||||
label: 'opencode-go/new-model',
|
||||
description: 'OpenCode Go model',
|
||||
})
|
||||
const dynamicModel = {
|
||||
id: 'opencode-go/new-model',
|
||||
label: 'New Go Model',
|
||||
description: 'fetched',
|
||||
supportsVision: false,
|
||||
}
|
||||
setDynamicProviderModels('opencode-go', [dynamicModel])
|
||||
expect(getAiModel({
|
||||
provider: 'opencode-go',
|
||||
model: 'opencode-go/new-model',
|
||||
})).toBe(dynamicModel)
|
||||
expect(findAiModelById('opencode-go/new-model')).toEqual({
|
||||
provider: getAiProvider('opencode-go'),
|
||||
model: dynamicModel,
|
||||
})
|
||||
setDynamicProviderModels('opencode-go', [])
|
||||
})
|
||||
})
|
||||
|
||||
@ -7,6 +7,8 @@ export type AiProviderId =
|
||||
| 'anthropic'
|
||||
| 'xai'
|
||||
| 'openrouter'
|
||||
| 'opencode-zen'
|
||||
| 'opencode-go'
|
||||
|
||||
export type AiReasoningEffort =
|
||||
| 'minimal'
|
||||
@ -23,6 +25,7 @@ export type AiModelSelection = {
|
||||
provider: AiProviderId
|
||||
model: string
|
||||
reasoningEffort?: AiReasoningEffort
|
||||
visionModel?: string
|
||||
}
|
||||
|
||||
export type AiModelOption = {
|
||||
@ -31,6 +34,13 @@ export type AiModelOption = {
|
||||
label: string
|
||||
description: string
|
||||
reasoningEfforts?: readonly AiReasoningEffort[]
|
||||
/** Omitted means the model accepts image input natively. */
|
||||
supportsVision?: boolean
|
||||
}
|
||||
|
||||
export type AiProviderKeyGroup = {
|
||||
id: string
|
||||
label: string
|
||||
}
|
||||
|
||||
export type AiProviderOption = {
|
||||
@ -40,8 +50,12 @@ export type AiProviderOption = {
|
||||
auth: 'apiKey' | 'chatgpt'
|
||||
transport: 'bridge' | 'direct' | 'proxy'
|
||||
models: readonly AiModelOption[]
|
||||
keyGroup?: AiProviderKeyGroup
|
||||
}
|
||||
|
||||
export const modelSupportsVision = (model: AiModelOption): boolean =>
|
||||
model.supportsVision !== false
|
||||
|
||||
export const DEFAULT_AI_REASONING_EFFORT = 'high' as const satisfies AiReasoningEffort
|
||||
|
||||
export const AI_REASONING_EFFORT_LABELS: Record<AiReasoningEffort, string> = {
|
||||
@ -320,6 +334,86 @@ export const AI_PROVIDERS: readonly AiProviderOption[] = [
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'opencode-zen',
|
||||
label: 'OpenCode Zen',
|
||||
envVar: 'VITE_OPENCODE_API_KEY',
|
||||
auth: 'apiKey',
|
||||
transport: 'proxy',
|
||||
keyGroup: { id: 'opencode', label: 'OpenCode' },
|
||||
// Curated recommendations; the full tools catalog is fetched at runtime
|
||||
// (src/ai/opencode-models.ts) and this list doubles as the offline fallback.
|
||||
models: [
|
||||
{
|
||||
id: 'opencode-zen/gpt-5.6-terra',
|
||||
providerModelId: 'gpt-5.6-terra',
|
||||
label: 'GPT 5.6 Terra',
|
||||
description: 'Recommended · vision and tools',
|
||||
reasoningEfforts: OPENROUTER_REASONING_EFFORTS,
|
||||
},
|
||||
{
|
||||
id: 'opencode-zen/claude-sonnet-5',
|
||||
providerModelId: 'claude-sonnet-5',
|
||||
label: 'Claude Sonnet 5',
|
||||
description: 'Balanced quality and cost',
|
||||
reasoningEfforts: OPENROUTER_REASONING_EFFORTS,
|
||||
},
|
||||
{
|
||||
id: 'opencode-zen/kimi-k3',
|
||||
providerModelId: 'kimi-k3',
|
||||
label: 'Kimi K3',
|
||||
description: 'Open coding model with vision',
|
||||
reasoningEfforts: OPENROUTER_REASONING_EFFORTS,
|
||||
},
|
||||
{
|
||||
id: 'opencode-zen/glm-5.2',
|
||||
providerModelId: 'glm-5.2',
|
||||
label: 'GLM 5.2',
|
||||
description: 'No native vision · a vision model describes images first',
|
||||
reasoningEfforts: OPENROUTER_REASONING_EFFORTS,
|
||||
supportsVision: false,
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'opencode-go',
|
||||
label: 'OpenCode Go',
|
||||
envVar: 'VITE_OPENCODE_API_KEY',
|
||||
auth: 'apiKey',
|
||||
transport: 'proxy',
|
||||
keyGroup: { id: 'opencode', label: 'OpenCode' },
|
||||
models: [
|
||||
{
|
||||
id: 'opencode-go/gpt-5.6-luna',
|
||||
providerModelId: 'gpt-5.6-luna',
|
||||
label: 'GPT 5.6 Luna',
|
||||
description: 'Recommended · vision and tools',
|
||||
reasoningEfforts: OPENROUTER_REASONING_EFFORTS,
|
||||
},
|
||||
{
|
||||
id: 'opencode-go/kimi-k3',
|
||||
providerModelId: 'kimi-k3',
|
||||
label: 'Kimi K3',
|
||||
description: 'Open coding model with vision',
|
||||
reasoningEfforts: OPENROUTER_REASONING_EFFORTS,
|
||||
},
|
||||
{
|
||||
id: 'opencode-go/mimo-v2.5',
|
||||
providerModelId: 'mimo-v2.5',
|
||||
label: 'MiMo V2.5',
|
||||
description: 'Fast open model with vision',
|
||||
reasoningEfforts: OPENROUTER_REASONING_EFFORTS,
|
||||
},
|
||||
{
|
||||
id: 'opencode-go/glm-5.3',
|
||||
providerModelId: 'glm-5.3',
|
||||
label: 'GLM 5.3',
|
||||
description: 'No native vision · a vision model describes images first',
|
||||
reasoningEfforts: OPENROUTER_REASONING_EFFORTS,
|
||||
supportsVision: false,
|
||||
},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
export const DEFAULT_AI_SELECTION: AiModelSelection = {
|
||||
@ -331,37 +425,93 @@ export const DEFAULT_AI_SELECTION: AiModelSelection = {
|
||||
export const isAiProviderId = (value: unknown): value is AiProviderId =>
|
||||
typeof value === 'string' && AI_PROVIDERS.some((provider) => provider.id === value)
|
||||
|
||||
export type AiProviderKeyField = {
|
||||
id: string
|
||||
label: string
|
||||
envVar: string
|
||||
providerIds: readonly AiProviderId[]
|
||||
}
|
||||
|
||||
/** One API-key field per credential, sharing Zen/Go under a single OpenCode key. */
|
||||
export const getAiProviderKeyFields = (): readonly AiProviderKeyField[] => {
|
||||
const fields: AiProviderKeyField[] = []
|
||||
const indexById = new Map<string, number>()
|
||||
for (const provider of AI_PROVIDERS) {
|
||||
if (provider.auth !== 'apiKey') continue
|
||||
const id = provider.keyGroup?.id ?? provider.id
|
||||
const existingIndex = indexById.get(id)
|
||||
if (existingIndex !== undefined) {
|
||||
const existing = fields[existingIndex]
|
||||
if (!existing) continue
|
||||
fields[existingIndex] = {
|
||||
...existing,
|
||||
providerIds: [...existing.providerIds, provider.id],
|
||||
}
|
||||
continue
|
||||
}
|
||||
indexById.set(id, fields.length)
|
||||
fields.push({
|
||||
id,
|
||||
label: provider.keyGroup?.label ?? provider.label,
|
||||
envVar: provider.envVar,
|
||||
providerIds: [provider.id],
|
||||
})
|
||||
}
|
||||
return fields
|
||||
}
|
||||
|
||||
/**
|
||||
* OpenRouter models discovered at runtime (src/ai/openrouter-models.ts).
|
||||
* Registered here so catalog lookups resolve dynamic selections; the static
|
||||
* catalog stays the source of truth for every other provider.
|
||||
* Runtime catalogs (OpenRouter, OpenCode Zen/Go) register here so lookups
|
||||
* resolve dynamic selections. The static catalog stays the fallback.
|
||||
*/
|
||||
let dynamicOpenRouterModels: readonly AiModelOption[] = []
|
||||
const DYNAMIC_CATALOG_PROVIDERS = new Set<AiProviderId>([
|
||||
'openrouter',
|
||||
'opencode-zen',
|
||||
'opencode-go',
|
||||
])
|
||||
|
||||
const dynamicModelsByProvider: Partial<Record<AiProviderId, readonly AiModelOption[]>> = {}
|
||||
|
||||
export const setDynamicProviderModels = (
|
||||
providerId: AiProviderId,
|
||||
models: readonly AiModelOption[],
|
||||
): void => {
|
||||
dynamicModelsByProvider[providerId] = models
|
||||
}
|
||||
|
||||
export const getDynamicProviderModels = (
|
||||
providerId: AiProviderId,
|
||||
): readonly AiModelOption[] =>
|
||||
dynamicModelsByProvider[providerId] ?? []
|
||||
|
||||
export const setDynamicOpenRouterModels = (
|
||||
models: readonly AiModelOption[],
|
||||
): void => {
|
||||
dynamicOpenRouterModels = models
|
||||
setDynamicProviderModels('openrouter', models)
|
||||
}
|
||||
|
||||
export const getDynamicOpenRouterModels = (): readonly AiModelOption[] =>
|
||||
dynamicOpenRouterModels
|
||||
getDynamicProviderModels('openrouter')
|
||||
|
||||
const findOpenRouterModel = (modelId: string): AiModelOption | undefined =>
|
||||
getAiProvider('openrouter').models.find((option) => option.id === modelId)
|
||||
?? dynamicOpenRouterModels.find((option) => option.id === modelId)
|
||||
const findDynamicCatalogModel = (
|
||||
providerId: AiProviderId,
|
||||
modelId: string,
|
||||
): AiModelOption | undefined =>
|
||||
getAiProvider(providerId).models.find((option) => option.id === modelId)
|
||||
?? getDynamicProviderModels(providerId).find((option) => option.id === modelId)
|
||||
|
||||
/**
|
||||
* A dynamic selection may outlive the fetched catalog (e.g. the model list is
|
||||
* still loading); synthesize a minimal option so lookups never throw for the
|
||||
* dynamic provider. Reasoning support is unknown, so no effort is offered.
|
||||
*/
|
||||
const synthesizeOpenRouterModel = (modelId: string): AiModelOption => ({
|
||||
const synthesizeDynamicCatalogModel = (
|
||||
providerId: AiProviderId,
|
||||
modelId: string,
|
||||
): AiModelOption => ({
|
||||
id: modelId,
|
||||
label: modelId,
|
||||
description: 'OpenRouter model',
|
||||
description: `${getAiProvider(providerId).label} model`,
|
||||
})
|
||||
|
||||
export const isDynamicCatalogProvider = (providerId: AiProviderId): boolean =>
|
||||
DYNAMIC_CATALOG_PROVIDERS.has(providerId)
|
||||
|
||||
export const getAiProvider = (providerId: AiProviderId): AiProviderOption => {
|
||||
const provider = AI_PROVIDERS.find((option) => option.id === providerId)
|
||||
if (!provider) throw new Error(`Unknown AI provider: ${providerId}`)
|
||||
@ -375,9 +525,9 @@ export const getDefaultAiModel = (providerId: AiProviderId): AiModelOption => {
|
||||
}
|
||||
|
||||
export const getAiModel = (selection: AiModelSelection): AiModelOption => {
|
||||
if (selection.provider === 'openrouter') {
|
||||
return findOpenRouterModel(selection.model)
|
||||
?? synthesizeOpenRouterModel(selection.model)
|
||||
if (isDynamicCatalogProvider(selection.provider)) {
|
||||
return findDynamicCatalogModel(selection.provider, selection.model)
|
||||
?? synthesizeDynamicCatalogModel(selection.provider, selection.model)
|
||||
}
|
||||
const model = getAiProvider(selection.provider).models.find(
|
||||
(option) => option.id === selection.model,
|
||||
@ -395,9 +545,13 @@ export const findAiModelById = (
|
||||
const model = provider.models.find((option) => option.id === modelId)
|
||||
if (model) return { provider, model }
|
||||
}
|
||||
const dynamicModel = dynamicOpenRouterModels.find((option) => option.id === modelId)
|
||||
if (dynamicModel) {
|
||||
return { provider: getAiProvider('openrouter'), model: dynamicModel }
|
||||
for (const providerId of DYNAMIC_CATALOG_PROVIDERS) {
|
||||
const dynamicModel = getDynamicProviderModels(providerId).find(
|
||||
(option) => option.id === modelId,
|
||||
)
|
||||
if (dynamicModel) {
|
||||
return { provider: getAiProvider(providerId), model: dynamicModel }
|
||||
}
|
||||
}
|
||||
throw new Error(`Unknown AI model: ${modelId}`)
|
||||
}
|
||||
|
||||
@ -43,8 +43,9 @@ describe('AI provider configuration', () => {
|
||||
VITE_ANTHROPIC_API_KEY: 'anthropic-key',
|
||||
VITE_XAI_API_KEY: ' xai-key ',
|
||||
VITE_OPENROUTER_API_KEY: ' openrouter-key ',
|
||||
VITE_OPENCODE_API_KEY: ' opencode-key ',
|
||||
})).toEqual({
|
||||
codex: '',
|
||||
...createEmptyAiProviderKeys(),
|
||||
moonshot: 'moonshot-key',
|
||||
google: 'google-key',
|
||||
qwen: 'qwen-key',
|
||||
@ -52,6 +53,8 @@ describe('AI provider configuration', () => {
|
||||
anthropic: 'anthropic-key',
|
||||
xai: 'xai-key',
|
||||
openrouter: 'openrouter-key',
|
||||
'opencode-zen': 'opencode-key',
|
||||
'opencode-go': 'opencode-key',
|
||||
})
|
||||
})
|
||||
|
||||
@ -63,14 +66,8 @@ describe('AI provider configuration', () => {
|
||||
})
|
||||
|
||||
expect(keys).toEqual({
|
||||
codex: '',
|
||||
moonshot: '',
|
||||
google: '',
|
||||
qwen: '',
|
||||
...createEmptyAiProviderKeys(),
|
||||
openai: 'openai-key',
|
||||
anthropic: '',
|
||||
xai: '',
|
||||
openrouter: '',
|
||||
})
|
||||
expect(getAiProviderAvailability(keys)).toEqual({
|
||||
codex: false,
|
||||
@ -81,6 +78,8 @@ describe('AI provider configuration', () => {
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: false,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
})
|
||||
})
|
||||
|
||||
@ -94,6 +93,8 @@ describe('AI provider configuration', () => {
|
||||
anthropic: true,
|
||||
xai: true,
|
||||
openrouter: true,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
})).toEqual({
|
||||
provider: 'openai',
|
||||
model: 'gpt-5.6-terra',
|
||||
@ -108,6 +109,8 @@ describe('AI provider configuration', () => {
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: true,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
})).toEqual({
|
||||
provider: 'openrouter',
|
||||
model: 'anthropic/claude-sonnet-5',
|
||||
@ -122,6 +125,8 @@ describe('AI provider configuration', () => {
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: false,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
})).toEqual({
|
||||
provider: 'codex',
|
||||
model: 'codex/gpt-5.6-terra',
|
||||
@ -136,6 +141,8 @@ describe('AI provider configuration', () => {
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: false,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
})).toEqual({
|
||||
provider: 'moonshot',
|
||||
model: 'kimi-k3',
|
||||
@ -154,14 +161,9 @@ describe('AI provider configuration', () => {
|
||||
}
|
||||
|
||||
expect(mergeAiProviderKeys(environment, stored)).toEqual({
|
||||
codex: '',
|
||||
moonshot: '',
|
||||
...createEmptyAiProviderKeys(),
|
||||
google: 'env-google',
|
||||
qwen: '',
|
||||
openai: 'browser-openai',
|
||||
anthropic: '',
|
||||
xai: '',
|
||||
openrouter: '',
|
||||
})
|
||||
})
|
||||
|
||||
@ -176,6 +178,20 @@ describe('AI provider configuration', () => {
|
||||
...createEmptyAiProviderKeys(),
|
||||
openai: 'sk-test',
|
||||
})
|
||||
expect(parseStoredAiProviderKeys(JSON.stringify({
|
||||
opencode: ' zen-go-key ',
|
||||
}))).toEqual({
|
||||
...createEmptyAiProviderKeys(),
|
||||
'opencode-zen': 'zen-go-key',
|
||||
'opencode-go': 'zen-go-key',
|
||||
})
|
||||
expect(parseStoredAiProviderKeys(JSON.stringify({
|
||||
'opencode-zen': ' zen-only-key ',
|
||||
}))).toEqual({
|
||||
...createEmptyAiProviderKeys(),
|
||||
'opencode-zen': 'zen-only-key',
|
||||
'opencode-go': 'zen-only-key',
|
||||
})
|
||||
|
||||
const storage = createMemoryStorage()
|
||||
expect(writeStoredAiProviderKeys({
|
||||
@ -238,6 +254,8 @@ describe('AI provider configuration', () => {
|
||||
expect(isLocalAiProxyHostname('[::1]')).toBe(true)
|
||||
expect(isLocalAiProxyHostname('app.shotluma.com')).toBe(false)
|
||||
expect(getAiProviderTransportAvailability('app.shotluma.com').moonshot).toBe(false)
|
||||
expect(getAiProviderTransportAvailability('app.shotluma.com')['opencode-zen']).toBe(true)
|
||||
expect(getAiProviderTransportAvailability('app.shotluma.com')['opencode-go']).toBe(true)
|
||||
})
|
||||
|
||||
it('does not mark a stored Moonshot key as usable without the local proxy', () => {
|
||||
|
||||
@ -22,6 +22,7 @@ type AiProviderEnvironment = {
|
||||
VITE_ANTHROPIC_API_KEY?: unknown
|
||||
VITE_XAI_API_KEY?: unknown
|
||||
VITE_OPENROUTER_API_KEY?: unknown
|
||||
VITE_OPENCODE_API_KEY?: unknown
|
||||
}
|
||||
|
||||
export const AI_PROVIDER_KEYS_STORAGE_KEY = 'shotluma-ai-provider-keys'
|
||||
@ -40,20 +41,27 @@ export const createEmptyAiProviderKeys = (): AiProviderKeys => ({
|
||||
anthropic: '',
|
||||
xai: '',
|
||||
openrouter: '',
|
||||
'opencode-zen': '',
|
||||
'opencode-go': '',
|
||||
})
|
||||
|
||||
export const createAiProviderKeys = (
|
||||
environment: AiProviderEnvironment,
|
||||
): AiProviderKeys => ({
|
||||
codex: '',
|
||||
moonshot: readKey(environment.VITE_MOONSHOT_API_KEY),
|
||||
google: readKey(environment.VITE_GOOGLE_GENERATIVE_AI_API_KEY),
|
||||
qwen: readKey(environment.VITE_ALIBABA_API_KEY),
|
||||
openai: readKey(environment.VITE_OPENAI_API_KEY),
|
||||
anthropic: readKey(environment.VITE_ANTHROPIC_API_KEY),
|
||||
xai: readKey(environment.VITE_XAI_API_KEY),
|
||||
openrouter: readKey(environment.VITE_OPENROUTER_API_KEY),
|
||||
})
|
||||
): AiProviderKeys => {
|
||||
const opencode = readKey(environment.VITE_OPENCODE_API_KEY)
|
||||
return {
|
||||
codex: '',
|
||||
moonshot: readKey(environment.VITE_MOONSHOT_API_KEY),
|
||||
google: readKey(environment.VITE_GOOGLE_GENERATIVE_AI_API_KEY),
|
||||
qwen: readKey(environment.VITE_ALIBABA_API_KEY),
|
||||
openai: readKey(environment.VITE_OPENAI_API_KEY),
|
||||
anthropic: readKey(environment.VITE_ANTHROPIC_API_KEY),
|
||||
xai: readKey(environment.VITE_XAI_API_KEY),
|
||||
openrouter: readKey(environment.VITE_OPENROUTER_API_KEY),
|
||||
'opencode-zen': opencode,
|
||||
'opencode-go': opencode,
|
||||
}
|
||||
}
|
||||
|
||||
export const getAiProviderAvailability = (
|
||||
keys: AiProviderKeys,
|
||||
@ -66,6 +74,8 @@ export const getAiProviderAvailability = (
|
||||
anthropic: keys.anthropic.length > 0,
|
||||
xai: keys.xai.length > 0,
|
||||
openrouter: keys.openrouter.length > 0,
|
||||
'opencode-zen': keys['opencode-zen'].length > 0,
|
||||
'opencode-go': keys['opencode-go'].length > 0,
|
||||
})
|
||||
|
||||
const getBrowserHostname = (): string => {
|
||||
@ -95,6 +105,8 @@ export const getAiProviderTransportAvailability = (
|
||||
anthropic: true,
|
||||
xai: true,
|
||||
openrouter: true,
|
||||
'opencode-zen': true,
|
||||
'opencode-go': true,
|
||||
})
|
||||
|
||||
export const mergeAiProviderKeys = (
|
||||
@ -125,6 +137,11 @@ export const parseStoredAiProviderKeys = (raw: string | null): AiProviderKeys =>
|
||||
for (const providerId of PROVIDER_IDS) {
|
||||
keys[providerId] = readKey(record[providerId])
|
||||
}
|
||||
const sharedOpencode = readKey(record['opencode'])
|
||||
|| keys['opencode-zen']
|
||||
|| keys['opencode-go']
|
||||
keys['opencode-zen'] = sharedOpencode
|
||||
keys['opencode-go'] = sharedOpencode
|
||||
return keys
|
||||
}
|
||||
|
||||
@ -232,6 +249,7 @@ export const ENVIRONMENT_AI_PROVIDER_KEYS = createAiProviderKeys({
|
||||
VITE_ANTHROPIC_API_KEY: import.meta.env.VITE_ANTHROPIC_API_KEY,
|
||||
VITE_XAI_API_KEY: import.meta.env.VITE_XAI_API_KEY,
|
||||
VITE_OPENROUTER_API_KEY: import.meta.env.VITE_OPENROUTER_API_KEY,
|
||||
VITE_OPENCODE_API_KEY: import.meta.env.VITE_OPENCODE_API_KEY,
|
||||
})
|
||||
|
||||
export const AI_PROVIDER_AVAILABILITY = getResolvedAiProviderAvailability()
|
||||
|
||||
98
src/ai/runner.test.ts
Normal file
98
src/ai/runner.test.ts
Normal file
@ -0,0 +1,98 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { prepareGenerationMessages, type OpencodeVisionFallback } from './runner'
|
||||
import type { ModelMessage } from 'ai'
|
||||
|
||||
const textOnlyMessages: ModelMessage[] = [
|
||||
{ role: 'user', content: 'Design three slides' },
|
||||
]
|
||||
|
||||
const messagesWithUserImage: ModelMessage[] = [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: 'Screenshot:' },
|
||||
{ type: 'file', mediaType: 'image/png', data: 'data:image/png;base64,abc' },
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
const messagesWithToolPreview: ModelMessage[] = [
|
||||
{ role: 'user', content: 'Design three slides' },
|
||||
{
|
||||
role: 'tool',
|
||||
content: [{
|
||||
type: 'tool-result',
|
||||
toolCallId: 'call-1',
|
||||
toolName: 'render_slide_preview',
|
||||
output: {
|
||||
type: 'content',
|
||||
value: [{
|
||||
type: 'file',
|
||||
mediaType: 'image/png',
|
||||
data: { type: 'data', data: 'preview-bytes' },
|
||||
}],
|
||||
},
|
||||
}],
|
||||
},
|
||||
]
|
||||
|
||||
const prepare = (
|
||||
messages: ModelMessage[],
|
||||
resolveVisionFallback: (() => OpencodeVisionFallback) | null,
|
||||
needsChatImageRelay = false,
|
||||
) => prepareGenerationMessages({
|
||||
messages,
|
||||
movingCacheProvider: null,
|
||||
needsChatImageRelay,
|
||||
resolveVisionFallback,
|
||||
visionAnnouncement: { sent: false },
|
||||
descriptionCache: new Map(),
|
||||
})
|
||||
|
||||
describe('prepareGenerationMessages', () => {
|
||||
it('never resolves the vision fallback for image-free requests', async () => {
|
||||
const resolveVisionFallback = vi.fn((): OpencodeVisionFallback => {
|
||||
throw new Error('no OpenCode vision model is available')
|
||||
})
|
||||
const prepared = await prepare(textOnlyMessages, resolveVisionFallback)
|
||||
expect(prepared).toBe(textOnlyMessages)
|
||||
expect(resolveVisionFallback).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('surfaces the missing-vision-model error once an image appears', async () => {
|
||||
const resolveVisionFallback = (): OpencodeVisionFallback => {
|
||||
throw new Error('no OpenCode vision model is available')
|
||||
}
|
||||
await expect(prepare(messagesWithUserImage, resolveVisionFallback))
|
||||
.rejects.toThrow('no OpenCode vision model is available')
|
||||
})
|
||||
|
||||
it('describes user images through the resolved fallback', async () => {
|
||||
const describeImage = vi.fn().mockResolvedValue('a login screen')
|
||||
const prepared = await prepare(messagesWithUserImage, () => ({
|
||||
describeImage,
|
||||
visionLabel: 'MiMo V2.5',
|
||||
}))
|
||||
expect(describeImage).toHaveBeenCalledTimes(1)
|
||||
expect(prepared[0]).toMatchObject({
|
||||
content: [
|
||||
{ type: 'text', text: 'Screenshot:' },
|
||||
{ type: 'text', text: 'Image description:\na login screen' },
|
||||
],
|
||||
})
|
||||
})
|
||||
|
||||
it('relays tool preview images for non-chat dialects when the fallback is active', async () => {
|
||||
const describeImage = vi.fn().mockResolvedValue('slide preview')
|
||||
const prepared = await prepare(messagesWithToolPreview, () => ({
|
||||
describeImage,
|
||||
visionLabel: 'MiMo V2.5',
|
||||
}), false)
|
||||
expect(describeImage).toHaveBeenCalledTimes(1)
|
||||
const relayedUserMessage = prepared[prepared.length - 1]
|
||||
expect(relayedUserMessage).toMatchObject({
|
||||
role: 'user',
|
||||
content: [{ type: 'text', text: 'Image description:\nslide preview' }],
|
||||
})
|
||||
})
|
||||
})
|
||||
171
src/ai/runner.ts
171
src/ai/runner.ts
@ -14,7 +14,21 @@ import { relayChatToolImages } from './chat-tool-images'
|
||||
import { runCodexAppServerGeneration } from './codex-app-server'
|
||||
import { readCodexConnection } from './codex-connection'
|
||||
import { scopeAiControllerToSlide, type AiEditorController } from './controller'
|
||||
import { collectMessageImages, describeMessageImages } from './describe-images'
|
||||
import { buildHtmlInstructions } from './html-prompt'
|
||||
import {
|
||||
createOpencodeImageDescriber,
|
||||
createOpencodeModel,
|
||||
} from './opencode-client'
|
||||
import {
|
||||
opencodeDialectNeedsChatToolImageRelay,
|
||||
opencodeSelectionDialect,
|
||||
} from './opencode-dialects'
|
||||
import {
|
||||
isOpencodeProviderId,
|
||||
opencodeRequestModelId,
|
||||
pickOpencodeVisionModel,
|
||||
} from './opencode-models'
|
||||
import { buildInstructions, buildUserMessage } from './prompt'
|
||||
import {
|
||||
buildStreamRequestOptions,
|
||||
@ -26,6 +40,8 @@ import {
|
||||
getAiModel,
|
||||
getAiProvider,
|
||||
getAiSdkReasoningEffort,
|
||||
getDynamicProviderModels,
|
||||
modelSupportsVision,
|
||||
type AiModelSelection,
|
||||
} from './provider-catalog'
|
||||
import {
|
||||
@ -107,7 +123,9 @@ const buildUserContent = (options: {
|
||||
|
||||
const createAiModel = async (selection: AiModelSelection) => {
|
||||
if (!getAiProviderTransportAvailability()[selection.provider]) {
|
||||
throw new Error('Moonshot requires the local Shotluma CORS proxy and is unavailable on this host')
|
||||
throw new Error(
|
||||
`${getAiProvider(selection.provider).label} requires the local Shotluma CORS proxy and is unavailable on this host`,
|
||||
)
|
||||
}
|
||||
const apiKey = getAiProviderKey(selection.provider)
|
||||
if (!apiKey) {
|
||||
@ -162,6 +180,15 @@ const createAiModel = async (selection: AiModelSelection) => {
|
||||
},
|
||||
}).chat(selection.model)
|
||||
}
|
||||
case 'opencode-zen':
|
||||
case 'opencode-go': {
|
||||
const model = getAiModel(selection)
|
||||
return createOpencodeModel({
|
||||
providerId: selection.provider,
|
||||
modelId: opencodeRequestModelId(model.id, model.providerModelId),
|
||||
apiKey,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -304,7 +331,99 @@ const collectStreamPart = <TOOLS extends ToolSet>(options: {
|
||||
*/
|
||||
export const NARRATION_WORD_DELAY_MS = 18
|
||||
|
||||
export const narrationSmoothing = () => smoothStream<ToolSet>({ delayInMs: NARRATION_WORD_DELAY_MS })
|
||||
export const narrationSmoothing = () =>
|
||||
smoothStream<ToolSet>({ delayInMs: NARRATION_WORD_DELAY_MS })
|
||||
|
||||
/**
|
||||
* OpenCode serves each model on one of four dialects, and only its
|
||||
* chat-completions models reject images inside tool results.
|
||||
*/
|
||||
const needsChatToolImageRelay = (selection: AiModelSelection): boolean => {
|
||||
const dialect = opencodeSelectionDialect(selection)
|
||||
if (dialect) return opencodeDialectNeedsChatToolImageRelay(dialect)
|
||||
return selection.provider === 'moonshot'
|
||||
|| selection.provider === 'openrouter'
|
||||
|| selection.provider === 'qwen'
|
||||
}
|
||||
|
||||
export type OpencodeVisionFallback = {
|
||||
describeImage: ReturnType<typeof createOpencodeImageDescriber>
|
||||
visionLabel: string
|
||||
}
|
||||
|
||||
const needsOpencodeVisionFallback = (selection: AiModelSelection): boolean =>
|
||||
isOpencodeProviderId(selection.provider) && !modelSupportsVision(getAiModel(selection))
|
||||
|
||||
const resolveOpencodeVisionDescriber = (options: {
|
||||
selection: AiModelSelection
|
||||
apiKey: string
|
||||
signal?: AbortSignal
|
||||
}): OpencodeVisionFallback | null => {
|
||||
if (!isOpencodeProviderId(options.selection.provider)) return null
|
||||
const model = getAiModel(options.selection)
|
||||
if (modelSupportsVision(model)) return null
|
||||
const catalog = [
|
||||
...getAiProvider(options.selection.provider).models,
|
||||
...getDynamicProviderModels(options.selection.provider),
|
||||
]
|
||||
const vision = pickOpencodeVisionModel(
|
||||
options.selection.provider,
|
||||
catalog,
|
||||
options.selection.visionModel,
|
||||
)
|
||||
if (!vision) {
|
||||
throw new Error(
|
||||
`${model.label} cannot see images, and no OpenCode vision model is available to describe them.`,
|
||||
)
|
||||
}
|
||||
return {
|
||||
visionLabel: vision.label,
|
||||
describeImage: createOpencodeImageDescriber({
|
||||
providerId: options.selection.provider,
|
||||
visionModelId: opencodeRequestModelId(vision.id, vision.providerModelId),
|
||||
apiKey: options.apiKey,
|
||||
...(options.signal ? { signal: options.signal } : {}),
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The resolver runs only once a prepared step actually contains an image, so
|
||||
* a text-only OpenCode model without any vision fallback in the catalog can
|
||||
* still run image-free requests; the missing-fallback error surfaces the
|
||||
* moment a screenshot or slide preview enters the conversation.
|
||||
*/
|
||||
export const prepareGenerationMessages = async (options: {
|
||||
messages: ModelMessage[]
|
||||
movingCacheProvider: 'anthropic' | 'alibaba' | null
|
||||
needsChatImageRelay: boolean
|
||||
resolveVisionFallback: (() => OpencodeVisionFallback) | null
|
||||
onStatus?: (message: string) => void
|
||||
visionAnnouncement: { sent: boolean }
|
||||
descriptionCache: Map<string, string>
|
||||
}): Promise<ModelMessage[]> => {
|
||||
// Text-only models reject images anywhere, so the vision fallback also
|
||||
// needs tool-result images relayed into user messages before describing.
|
||||
const relayed = options.needsChatImageRelay || options.resolveVisionFallback !== null
|
||||
? relayChatToolImages(options.messages)
|
||||
: options.messages
|
||||
let next = relayed
|
||||
if (options.resolveVisionFallback && collectMessageImages(relayed).length > 0) {
|
||||
const visionFallback = options.resolveVisionFallback()
|
||||
if (!options.visionAnnouncement.sent) {
|
||||
options.visionAnnouncement.sent = true
|
||||
options.onStatus?.(`Describing images with ${visionFallback.visionLabel}…`)
|
||||
}
|
||||
next = await describeMessageImages(
|
||||
relayed,
|
||||
visionFallback.describeImage,
|
||||
options.descriptionCache,
|
||||
)
|
||||
}
|
||||
return options.movingCacheProvider
|
||||
? withMovingCacheBreakpoint(next, options.movingCacheProvider)
|
||||
: next
|
||||
}
|
||||
|
||||
const openAiGenerationStream = (options: {
|
||||
model: Awaited<ReturnType<typeof createAiModel>>
|
||||
@ -317,6 +436,7 @@ const openAiGenerationStream = (options: {
|
||||
signal?: AbortSignal
|
||||
onActivity?: (activity: AiToolActivity) => void
|
||||
onStepUsage?: (usage: LanguageModelUsage, toolNames: string[]) => void
|
||||
onStatus?: (message: string) => void
|
||||
}) => {
|
||||
const {
|
||||
model,
|
||||
@ -329,6 +449,7 @@ const openAiGenerationStream = (options: {
|
||||
signal,
|
||||
onActivity,
|
||||
onStepUsage,
|
||||
onStatus,
|
||||
} = options
|
||||
const runController = targetSlideId
|
||||
? scopeAiControllerToSlide(controller, targetSlideId)
|
||||
@ -352,9 +473,23 @@ const openAiGenerationStream = (options: {
|
||||
const movingCacheProvider = selection.provider === 'anthropic'
|
||||
? 'anthropic'
|
||||
: selection.provider === 'qwen' ? 'alibaba' : null
|
||||
const needsChatImageRelay = selection.provider === 'moonshot'
|
||||
|| selection.provider === 'openrouter'
|
||||
|| selection.provider === 'qwen'
|
||||
const needsChatImageRelay = needsChatToolImageRelay(selection)
|
||||
let cachedVisionFallback: OpencodeVisionFallback | null = null
|
||||
const resolveVisionFallback = needsOpencodeVisionFallback(selection)
|
||||
? (): OpencodeVisionFallback => {
|
||||
cachedVisionFallback ??= resolveOpencodeVisionDescriber({
|
||||
selection,
|
||||
apiKey: getAiProviderKey(selection.provider),
|
||||
...(signal ? { signal } : {}),
|
||||
})
|
||||
if (!cachedVisionFallback) {
|
||||
throw new Error(`${getAiModel(selection).label} has no vision fallback to describe images.`)
|
||||
}
|
||||
return cachedVisionFallback
|
||||
}
|
||||
: null
|
||||
const visionAnnouncement = { sent: false }
|
||||
const descriptionCache = new Map<string, string>()
|
||||
|
||||
return streamText({
|
||||
model,
|
||||
@ -375,18 +510,19 @@ const openAiGenerationStream = (options: {
|
||||
...requestOptions,
|
||||
// Anthropic and Alibaba require explicit message breakpoints. Keep one
|
||||
// moving forward so each step can reuse the growing tool-loop prefix.
|
||||
...(movingCacheProvider || needsChatImageRelay
|
||||
...(movingCacheProvider || needsChatImageRelay || resolveVisionFallback
|
||||
? {
|
||||
prepareStep: ({ messages }: { messages: ModelMessage[] }) => {
|
||||
const relayed = needsChatImageRelay
|
||||
? relayChatToolImages(messages)
|
||||
: messages
|
||||
return {
|
||||
messages: movingCacheProvider
|
||||
? withMovingCacheBreakpoint(relayed, movingCacheProvider)
|
||||
: relayed,
|
||||
}
|
||||
},
|
||||
prepareStep: async ({ messages }: { messages: ModelMessage[] }) => ({
|
||||
messages: await prepareGenerationMessages({
|
||||
messages,
|
||||
movingCacheProvider,
|
||||
needsChatImageRelay,
|
||||
resolveVisionFallback,
|
||||
...(onStatus ? { onStatus } : {}),
|
||||
visionAnnouncement,
|
||||
descriptionCache,
|
||||
}),
|
||||
}),
|
||||
}
|
||||
: {}),
|
||||
...(signal ? { abortSignal: signal } : {}),
|
||||
@ -499,6 +635,9 @@ const runDirectProviderGeneration = async (options: {
|
||||
...(htmlMode ? { htmlMode: true } : {}),
|
||||
...(signal ? { signal } : {}),
|
||||
...(onActivity ? { onActivity } : {}),
|
||||
onStatus: (message) => {
|
||||
onEvent({ type: 'status', message })
|
||||
},
|
||||
onStepUsage: (usage, toolNames) => {
|
||||
accumulator.steps.push({
|
||||
usage: toAiRunTokenUsage(usage),
|
||||
|
||||
@ -50,7 +50,8 @@ describe('AI API keys dialog', () => {
|
||||
expect(markup).toContain('Qwen')
|
||||
expect(markup).toContain('OpenAI')
|
||||
expect(markup).toContain('Anthropic')
|
||||
expect(markup).toContain('xAI')
|
||||
expect(markup).toContain('OpenRouter')
|
||||
expect(markup).toContain('OpenCode')
|
||||
expect(markup).toContain('type="password"')
|
||||
expect(markup).toContain('<form')
|
||||
expect(markup).toContain('type="submit"')
|
||||
|
||||
@ -5,7 +5,7 @@ import {
|
||||
type SyntheticEvent,
|
||||
} from 'react'
|
||||
import {
|
||||
AI_PROVIDERS,
|
||||
getAiProviderKeyFields,
|
||||
type AiProviderId,
|
||||
} from '../ai/provider-catalog'
|
||||
import {
|
||||
@ -51,9 +51,12 @@ export const AiApiKeysDialog = ({
|
||||
if (!open) return
|
||||
const frame = window.requestAnimationFrame(() => {
|
||||
const targetId = focusProviderId
|
||||
?? AI_PROVIDERS.find((provider) => provider.auth === 'apiKey')?.id
|
||||
?? getAiProviderKeyFields()[0]?.providerIds[0]
|
||||
if (!targetId) return
|
||||
const node = document.getElementById(`${formId}-${targetId}`)
|
||||
const field = getAiProviderKeyFields().find((entry) =>
|
||||
entry.providerIds.includes(targetId),
|
||||
)
|
||||
const node = document.getElementById(`${formId}-${field?.id ?? targetId}`)
|
||||
if (!(node instanceof HTMLInputElement)) return
|
||||
node.focus()
|
||||
node.select()
|
||||
@ -102,18 +105,24 @@ export const AiApiKeysDialog = ({
|
||||
</summary>
|
||||
<p>
|
||||
Keys are stored unencrypted in this browser. AI requests send a key only
|
||||
to the provider used for that request; Moonshot passes through the local proxy,
|
||||
and optional overlay assets use OpenAI. Use dedicated keys with strict limits.
|
||||
to the provider used for that request; Moonshot and OpenCode pass through
|
||||
a same-origin CORS proxy, and optional overlay assets use OpenAI. Use
|
||||
dedicated keys with strict limits.
|
||||
</p>
|
||||
</details>
|
||||
|
||||
<div className="ai-keys-dialog-fields">
|
||||
{AI_PROVIDERS.filter((provider) => provider.auth === 'apiKey').map((provider) => {
|
||||
const inputId = `${formId}-${provider.id}`
|
||||
const hasEnvFallback = envAvailability[provider.id] && !draft[provider.id]
|
||||
{getAiProviderKeyFields().map((field) => {
|
||||
const inputId = `${formId}-${field.id}`
|
||||
const value = field.providerIds
|
||||
.map((providerId) => draft[providerId])
|
||||
.find((entry) => entry.length > 0) ?? ''
|
||||
const hasEnvFallback = field.providerIds.some((providerId) =>
|
||||
envAvailability[providerId] && !draft[providerId],
|
||||
)
|
||||
return (
|
||||
<label className="ai-keys-dialog-field" htmlFor={inputId} key={provider.id}>
|
||||
<span>{provider.label}</span>
|
||||
<label className="ai-keys-dialog-field" htmlFor={inputId} key={field.id}>
|
||||
<span>{field.label}</span>
|
||||
<Input
|
||||
id={inputId}
|
||||
type="password"
|
||||
@ -121,16 +130,22 @@ export const AiApiKeysDialog = ({
|
||||
autoCapitalize="none"
|
||||
spellCheck={false}
|
||||
placeholder={hasEnvFallback ? 'Using .env.local' : 'Paste API key'}
|
||||
value={draft[provider.id]}
|
||||
value={value}
|
||||
onKeyDown={(event) => {
|
||||
if (event.key !== 'Enter' || event.nativeEvent.isComposing) return
|
||||
event.preventDefault()
|
||||
saveDraft()
|
||||
}}
|
||||
onChange={(event) => {
|
||||
const value = event.target.value
|
||||
const nextValue = event.target.value
|
||||
setSaveError(null)
|
||||
setDraft((current) => ({ ...current, [provider.id]: value }))
|
||||
setDraft((current) => {
|
||||
const next = { ...current }
|
||||
for (const providerId of field.providerIds) {
|
||||
next[providerId] = nextValue
|
||||
}
|
||||
return next
|
||||
})
|
||||
}}
|
||||
/>
|
||||
</label>
|
||||
|
||||
@ -5,9 +5,16 @@ import {
|
||||
type DragEvent,
|
||||
type RefObject,
|
||||
} from 'react'
|
||||
import {
|
||||
isOpencodeProviderId,
|
||||
pickOpencodeVisionModel,
|
||||
} from '../ai/opencode-models'
|
||||
import {
|
||||
clampAiReasoningEffort,
|
||||
findAiModelById,
|
||||
getAiModel,
|
||||
getAiProvider,
|
||||
getDynamicProviderModels,
|
||||
modelSupportsVision,
|
||||
type AiModelSelection,
|
||||
type AiProviderId,
|
||||
} from '../ai/provider-catalog'
|
||||
@ -54,6 +61,28 @@ const toImageDrafts = async (files: File[], idPrefix: 'logo' | 'shot'): Promise<
|
||||
dataUrl: await fileToDataUrl(file),
|
||||
})))
|
||||
|
||||
const nextAiSelection = (
|
||||
provider: AiProviderId,
|
||||
modelId: string,
|
||||
current: AiModelSelection,
|
||||
): AiModelSelection => {
|
||||
const model = getAiModel({ provider, model: modelId })
|
||||
const reasoningEffort = clampAiReasoningEffort(model, current.reasoningEffort)
|
||||
const visionModel = isOpencodeProviderId(provider) && !modelSupportsVision(model)
|
||||
? pickOpencodeVisionModel(
|
||||
provider,
|
||||
[...getAiProvider(provider).models, ...getDynamicProviderModels(provider)],
|
||||
current.provider === provider ? current.visionModel : undefined,
|
||||
)?.id
|
||||
: undefined
|
||||
return {
|
||||
provider,
|
||||
model: modelId,
|
||||
...(reasoningEffort ? { reasoningEffort } : {}),
|
||||
...(visionModel ? { visionModel } : {}),
|
||||
}
|
||||
}
|
||||
|
||||
export type AiGenerateModalProps = {
|
||||
open: boolean
|
||||
onClose: () => void
|
||||
@ -320,6 +349,7 @@ const ModalFooter = ({
|
||||
transportAvailability,
|
||||
onModelSelect,
|
||||
onReasoningEffortChange,
|
||||
onVisionModelChange,
|
||||
onManageKeys,
|
||||
onConnectCodex,
|
||||
onGenerate,
|
||||
@ -331,6 +361,7 @@ const ModalFooter = ({
|
||||
transportAvailability: AiProviderAvailability
|
||||
onModelSelect: (provider: AiProviderId, modelId: string) => void
|
||||
onReasoningEffortChange: (reasoningEffort: NonNullable<AiModelSelection['reasoningEffort']>) => void
|
||||
onVisionModelChange: (visionModel: string) => void
|
||||
onManageKeys: (providerId: AiProviderId) => void
|
||||
onConnectCodex: () => void
|
||||
onGenerate: () => void
|
||||
@ -342,6 +373,7 @@ const ModalFooter = ({
|
||||
transportAvailability={transportAvailability}
|
||||
onModelSelect={onModelSelect}
|
||||
onReasoningEffortChange={onReasoningEffortChange}
|
||||
onVisionModelChange={onVisionModelChange}
|
||||
onManageKeys={onManageKeys}
|
||||
onConnectCodex={onConnectCodex}
|
||||
/>
|
||||
@ -452,19 +484,15 @@ export const AiGenerateModal = ({ open, onClose, controller, targetSlide, onPrep
|
||||
const removeScreenshot = (id: string) => setScreenshots((current) => current.filter((shot) => shot.id !== id))
|
||||
|
||||
const handleModelSelect = (provider: AiProviderId, modelId: string) => {
|
||||
const { model } = findAiModelById(modelId)
|
||||
const reasoningEffort = clampAiReasoningEffort(model, selection.reasoningEffort)
|
||||
setSelection({
|
||||
provider,
|
||||
model: modelId,
|
||||
...(reasoningEffort ? { reasoningEffort } : {}),
|
||||
})
|
||||
setSelection(nextAiSelection(provider, modelId, selection))
|
||||
}
|
||||
|
||||
const handleEvent = (event: AiRunEvent) => {
|
||||
if (cancelledRef.current) return
|
||||
if (event.type === 'tool') {
|
||||
setLatestActivity(event.detail)
|
||||
} else if (event.type === 'status') {
|
||||
setLatestActivity(event.message)
|
||||
} else if (event.type === 'plan') setPlan(event.screens)
|
||||
// `index` is the count of screens finished before this one, which is exactly the
|
||||
// rail's "built" count while this screen is in progress.
|
||||
@ -483,7 +511,7 @@ export const AiGenerateModal = ({ open, onClose, controller, targetSlide, onPrep
|
||||
setPhase('done')
|
||||
onActivity?.(null)
|
||||
onFinished(event.slidesCreated)
|
||||
} else if (event.type === 'error') {
|
||||
} else {
|
||||
setNarration(flushNarration)
|
||||
setErrorMessage(event.message)
|
||||
setPhase('error')
|
||||
@ -627,6 +655,9 @@ export const AiGenerateModal = ({ open, onClose, controller, targetSlide, onPrep
|
||||
onReasoningEffortChange={(reasoningEffort) => {
|
||||
setSelection((current) => ({ ...current, reasoningEffort }))
|
||||
}}
|
||||
onVisionModelChange={(visionModel) => {
|
||||
setSelection((current) => ({ ...current, visionModel }))
|
||||
}}
|
||||
onManageKeys={handleManageKeys}
|
||||
onConnectCodex={handleConnectCodex}
|
||||
onGenerate={() => void handleGenerate()}
|
||||
|
||||
@ -42,6 +42,8 @@ const availableTransports = {
|
||||
anthropic: true,
|
||||
xai: true,
|
||||
openrouter: true,
|
||||
'opencode-zen': true,
|
||||
'opencode-go': true,
|
||||
}
|
||||
|
||||
vi.mock('./ui/button', () => ({
|
||||
@ -72,10 +74,13 @@ describe('AI provider controls', () => {
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: false,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
}}
|
||||
transportAvailability={availableTransports}
|
||||
onModelSelect={() => undefined}
|
||||
onReasoningEffortChange={() => undefined}
|
||||
onVisionModelChange={() => undefined}
|
||||
onManageKeys={() => undefined}
|
||||
onConnectCodex={() => undefined}
|
||||
/>,
|
||||
@ -100,10 +105,13 @@ describe('AI provider controls', () => {
|
||||
anthropic: true,
|
||||
xai: true,
|
||||
openrouter: true,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
}}
|
||||
transportAvailability={availableTransports}
|
||||
onModelSelect={() => undefined}
|
||||
onReasoningEffortChange={() => undefined}
|
||||
onVisionModelChange={() => undefined}
|
||||
onManageKeys={() => undefined}
|
||||
onConnectCodex={() => undefined}
|
||||
/>,
|
||||
@ -135,10 +143,13 @@ describe('AI provider controls', () => {
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: false,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
}}
|
||||
transportAvailability={availableTransports}
|
||||
onModelSelect={() => undefined}
|
||||
onReasoningEffortChange={() => undefined}
|
||||
onVisionModelChange={() => undefined}
|
||||
onManageKeys={() => undefined}
|
||||
onConnectCodex={() => undefined}
|
||||
/>,
|
||||
@ -168,10 +179,13 @@ describe('AI provider controls', () => {
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: false,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
}}
|
||||
transportAvailability={availableTransports}
|
||||
onModelSelect={() => undefined}
|
||||
onReasoningEffortChange={() => undefined}
|
||||
onVisionModelChange={() => undefined}
|
||||
onManageKeys={() => undefined}
|
||||
onConnectCodex={() => undefined}
|
||||
/>,
|
||||
@ -202,10 +216,13 @@ describe('AI provider controls', () => {
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: false,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
}}
|
||||
transportAvailability={availableTransports}
|
||||
onModelSelect={() => undefined}
|
||||
onReasoningEffortChange={() => undefined}
|
||||
onVisionModelChange={() => undefined}
|
||||
onManageKeys={() => undefined}
|
||||
onConnectCodex={() => undefined}
|
||||
/>,
|
||||
@ -238,10 +255,13 @@ describe('AI provider controls', () => {
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: true,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
}}
|
||||
transportAvailability={availableTransports}
|
||||
onModelSelect={() => undefined}
|
||||
onReasoningEffortChange={() => undefined}
|
||||
onVisionModelChange={() => undefined}
|
||||
onManageKeys={() => undefined}
|
||||
onConnectCodex={() => undefined}
|
||||
/>,
|
||||
@ -265,10 +285,13 @@ describe('AI provider controls', () => {
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: false,
|
||||
'opencode-zen': false,
|
||||
'opencode-go': false,
|
||||
}}
|
||||
transportAvailability={{ ...availableTransports, moonshot: false }}
|
||||
onModelSelect={() => undefined}
|
||||
onReasoningEffortChange={() => undefined}
|
||||
onVisionModelChange={() => undefined}
|
||||
onManageKeys={() => undefined}
|
||||
onConnectCodex={() => undefined}
|
||||
/>,
|
||||
@ -278,4 +301,37 @@ describe('AI provider controls', () => {
|
||||
expect(markup).toContain('Moonshot · local proxy unavailable')
|
||||
expect(markup).not.toContain('Enter API key')
|
||||
})
|
||||
|
||||
it('offers a vision fallback picker for OpenCode models without native vision', () => {
|
||||
const markup = renderToStaticMarkup(
|
||||
<AiProviderControls
|
||||
selection={{
|
||||
provider: 'opencode-go',
|
||||
model: 'opencode-go/glm-5.3',
|
||||
reasoningEffort: 'high',
|
||||
}}
|
||||
availability={{
|
||||
...availableTransports,
|
||||
moonshot: false,
|
||||
google: false,
|
||||
qwen: false,
|
||||
openai: false,
|
||||
anthropic: false,
|
||||
xai: false,
|
||||
openrouter: false,
|
||||
}}
|
||||
transportAvailability={availableTransports}
|
||||
onModelSelect={() => undefined}
|
||||
onReasoningEffortChange={() => undefined}
|
||||
onVisionModelChange={() => undefined}
|
||||
onManageKeys={() => undefined}
|
||||
onConnectCodex={() => undefined}
|
||||
/>,
|
||||
)
|
||||
|
||||
expect(markup).toContain('OpenCode Go · GLM 5.3')
|
||||
expect(markup).toContain('Vision fallback')
|
||||
expect(markup).toContain('cannot see images')
|
||||
expect(markup).toContain('Other models…')
|
||||
})
|
||||
})
|
||||
|
||||
@ -1,4 +1,13 @@
|
||||
import { useState } from 'react'
|
||||
import { useEffect, useState } from 'react'
|
||||
import {
|
||||
isOpencodeProviderId,
|
||||
listOpencodeVisionModels,
|
||||
loadAllOpencodeModels,
|
||||
loadOpencodeModels,
|
||||
pickOpencodeVisionModel,
|
||||
type OpencodeProviderId,
|
||||
} from '../ai/opencode-models'
|
||||
import { loadOpenRouterModels } from '../ai/openrouter-models'
|
||||
import {
|
||||
AI_PROVIDERS,
|
||||
AI_REASONING_EFFORT_LABELS,
|
||||
@ -6,11 +15,16 @@ import {
|
||||
findAiModelById,
|
||||
getAiModel,
|
||||
getAiProvider,
|
||||
getDynamicProviderModels,
|
||||
modelSupportsVision,
|
||||
type AiModelOption,
|
||||
type AiModelSelection,
|
||||
type AiProviderId,
|
||||
type AiProviderOption,
|
||||
type AiReasoningEffort,
|
||||
} from '../ai/provider-catalog'
|
||||
import {
|
||||
AiGenerative,
|
||||
AiNetwork,
|
||||
AlertCircle,
|
||||
ChatGpt,
|
||||
@ -21,7 +35,7 @@ import {
|
||||
KimiAi,
|
||||
Qwen,
|
||||
} from './icons'
|
||||
import { OpenRouterModelBrowser } from './OpenRouterModelBrowser'
|
||||
import { AiCatalogModelBrowser } from './OpenRouterModelBrowser'
|
||||
import { Button } from './ui/button'
|
||||
import {
|
||||
Popover,
|
||||
@ -45,7 +59,24 @@ import type { ComponentType } from 'react'
|
||||
* browser instead of selecting a model. Real OpenRouter ids use `vendor/model`,
|
||||
* so the `:` namespace cannot collide.
|
||||
*/
|
||||
const OPENROUTER_BROWSE_VALUE = 'openrouter:browse-all'
|
||||
const CATALOG_BROWSE_SUFFIX = ':browse-all'
|
||||
|
||||
const catalogBrowseValue = (providerId: AiProviderId): string =>
|
||||
`${providerId}${CATALOG_BROWSE_SUFFIX}`
|
||||
|
||||
const catalogProviderFromBrowseValue = (value: string): AiProviderId | null => {
|
||||
if (!value.endsWith(CATALOG_BROWSE_SUFFIX)) return null
|
||||
const providerId = value.slice(0, -CATALOG_BROWSE_SUFFIX.length)
|
||||
return AI_PROVIDERS.some((provider) => provider.id === providerId)
|
||||
? providerId as AiProviderId
|
||||
: null
|
||||
}
|
||||
|
||||
const CATALOG_BROWSE_PROVIDERS = new Set<AiProviderId>([
|
||||
'openrouter',
|
||||
'opencode-zen',
|
||||
'opencode-go',
|
||||
])
|
||||
|
||||
const AI_PROVIDER_ICONS: Record<AiProviderId, ComponentType<{ className?: string; size?: number }>> = {
|
||||
codex: ChatGpt,
|
||||
@ -56,6 +87,8 @@ const AI_PROVIDER_ICONS: Record<AiProviderId, ComponentType<{ className?: string
|
||||
anthropic: Claude,
|
||||
xai: Grok,
|
||||
openrouter: AiNetwork,
|
||||
'opencode-zen': AiGenerative,
|
||||
'opencode-go': AiGenerative,
|
||||
}
|
||||
|
||||
export type AiProviderControlsProps = {
|
||||
@ -64,6 +97,7 @@ export type AiProviderControlsProps = {
|
||||
transportAvailability: AiProviderAvailability
|
||||
onModelSelect: (providerId: AiProviderId, modelId: string) => void
|
||||
onReasoningEffortChange: (reasoningEffort: AiReasoningEffort) => void
|
||||
onVisionModelChange: (visionModel: string) => void
|
||||
onManageKeys: (providerId: AiProviderId) => void
|
||||
onConnectCodex: () => void
|
||||
}
|
||||
@ -76,22 +110,192 @@ const triggerLabel = (
|
||||
return `${modelLabel} · ${AI_REASONING_EFFORT_LABELS[reasoningEffort]}`
|
||||
}
|
||||
|
||||
const uniqueVisionModels = (providerId: OpencodeProviderId): AiModelOption[] => {
|
||||
const models = [
|
||||
...getAiProvider(providerId).models,
|
||||
...getDynamicProviderModels(providerId),
|
||||
]
|
||||
const seen = new Set<string>()
|
||||
return listOpencodeVisionModels(models).filter((model) => {
|
||||
if (seen.has(model.id)) return false
|
||||
seen.add(model.id)
|
||||
return true
|
||||
})
|
||||
}
|
||||
|
||||
const providerGroupCaption = (
|
||||
option: AiProviderOption,
|
||||
availability: AiProviderAvailability,
|
||||
transportAvailability: AiProviderAvailability,
|
||||
): string => {
|
||||
if (availability[option.id]) return option.label
|
||||
if (option.id === 'codex') return `${option.label} · connect`
|
||||
if (transportAvailability[option.id]) return `${option.label} · key missing`
|
||||
return `${option.label} · local proxy unavailable`
|
||||
}
|
||||
|
||||
const CatalogModelBrowsers = ({
|
||||
browsingProvider,
|
||||
selectedModelId,
|
||||
onModelSelect,
|
||||
}: {
|
||||
browsingProvider: AiProviderId | null
|
||||
selectedModelId: string
|
||||
onModelSelect: (providerId: AiProviderId, modelId: string) => void
|
||||
}) => {
|
||||
// The key remounts the browser per provider — its load-once effect would
|
||||
// otherwise keep the previous provider's catalog when React reuses the instance.
|
||||
if (browsingProvider === 'openrouter') {
|
||||
return (
|
||||
<AiCatalogModelBrowser
|
||||
key="openrouter"
|
||||
heading="All OpenRouter models"
|
||||
selectedModelId={selectedModelId}
|
||||
loadModels={loadOpenRouterModels}
|
||||
onModelSelect={(modelId) => onModelSelect('openrouter', modelId)}
|
||||
/>
|
||||
)
|
||||
}
|
||||
if (!isOpencodeProviderId(browsingProvider)) return null
|
||||
return (
|
||||
<AiCatalogModelBrowser
|
||||
key={browsingProvider}
|
||||
heading={`All ${getAiProvider(browsingProvider).label} models`}
|
||||
selectedModelId={selectedModelId}
|
||||
loadModels={() => loadOpencodeModels(browsingProvider)}
|
||||
onModelSelect={(modelId) => onModelSelect(browsingProvider, modelId)}
|
||||
/>
|
||||
)
|
||||
}
|
||||
|
||||
const VisionFallbackPicker = ({
|
||||
codingModelLabel,
|
||||
visionModel,
|
||||
visionModels,
|
||||
onVisionModelChange,
|
||||
}: {
|
||||
codingModelLabel: string
|
||||
visionModel: AiModelOption
|
||||
visionModels: readonly AiModelOption[]
|
||||
onVisionModelChange: (visionModel: string) => void
|
||||
}) => (
|
||||
<div className="ai-model-picker-section">
|
||||
<span className="ai-model-picker-section__label" id="ai-vision-label">
|
||||
Vision fallback
|
||||
</span>
|
||||
<Select
|
||||
value={visionModel.id}
|
||||
onValueChange={(value) => {
|
||||
if (typeof value === 'string') onVisionModelChange(value)
|
||||
}}
|
||||
>
|
||||
<SelectTrigger
|
||||
id="ai-vision-trigger"
|
||||
className="ai-provider-trigger"
|
||||
aria-labelledby="ai-vision-label"
|
||||
aria-label="Vision fallback model"
|
||||
>
|
||||
<SelectValue>
|
||||
<span>{visionModel.label}</span>
|
||||
</SelectValue>
|
||||
</SelectTrigger>
|
||||
<SelectContent align="start" className="ai-modal-select-content">
|
||||
{visionModels.map((option) => (
|
||||
<SelectItem key={option.id} value={option.id}>
|
||||
{option.label}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
<small className="ai-provider-description">
|
||||
{codingModelLabel} cannot see images. {visionModel.label} will describe
|
||||
screenshots and previews first.
|
||||
</small>
|
||||
</div>
|
||||
)
|
||||
|
||||
const ProviderSetupWarning = ({
|
||||
providerId,
|
||||
keyLabel,
|
||||
isTransportAvailable,
|
||||
onManageKeys,
|
||||
onConnectCodex,
|
||||
}: {
|
||||
providerId: AiProviderId
|
||||
keyLabel: string
|
||||
isTransportAvailable: boolean
|
||||
onManageKeys: (providerId: AiProviderId) => void
|
||||
onConnectCodex: () => void
|
||||
}) => {
|
||||
const isCodex = providerId === 'codex'
|
||||
return (
|
||||
<div className="ai-provider-warning" role="alert">
|
||||
<AlertCircle size={15} />
|
||||
<div className="ai-provider-warning-body">
|
||||
<span>
|
||||
{isCodex
|
||||
? <><b>Codex isn’t connected.</b> Use your ChatGPT plan without an API key.</>
|
||||
: isTransportAvailable
|
||||
? <><b>API key missing.</b> Add your {keyLabel} key to generate with this model.</>
|
||||
: <><b>Local proxy unavailable.</b> Moonshot can be used only from localhost.</>}
|
||||
</span>
|
||||
{(isCodex || isTransportAvailable) && (
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
className="ai-provider-warning-action"
|
||||
onClick={() => {
|
||||
if (isCodex) onConnectCodex()
|
||||
else onManageKeys(providerId)
|
||||
}}
|
||||
>
|
||||
{isCodex ? 'Connect Codex' : 'Enter API key'}
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export const AiProviderControls = ({
|
||||
selection,
|
||||
availability,
|
||||
transportAvailability,
|
||||
onModelSelect,
|
||||
onReasoningEffortChange,
|
||||
onVisionModelChange,
|
||||
onManageKeys,
|
||||
onConnectCodex,
|
||||
}: AiProviderControlsProps) => {
|
||||
const [isBrowsingOpenRouter, setBrowsingOpenRouter] = useState(false)
|
||||
const [browsingProvider, setBrowsingProvider] = useState<AiProviderId | null>(null)
|
||||
const [catalogRevision, setCatalogRevision] = useState(0)
|
||||
const provider = getAiProvider(selection.provider)
|
||||
const model = getAiModel(selection)
|
||||
const isConfigured = availability[selection.provider]
|
||||
const isTransportAvailable = transportAvailability[selection.provider]
|
||||
const reasoningEffort = clampAiReasoningEffort(model, selection.reasoningEffort)
|
||||
const SelectedIcon = AI_PROVIDER_ICONS[selection.provider]
|
||||
const keyLabel = provider.keyGroup?.label ?? provider.label
|
||||
const needsVisionFallback = isOpencodeProviderId(selection.provider)
|
||||
&& !modelSupportsVision(model)
|
||||
void catalogRevision
|
||||
const visionModels = isOpencodeProviderId(selection.provider)
|
||||
? uniqueVisionModels(selection.provider)
|
||||
: []
|
||||
const visionModel = needsVisionFallback && isOpencodeProviderId(selection.provider)
|
||||
? pickOpencodeVisionModel(selection.provider, visionModels, selection.visionModel)
|
||||
: undefined
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false
|
||||
void loadAllOpencodeModels().then(() => {
|
||||
if (!cancelled) setCatalogRevision((current) => current + 1)
|
||||
})
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
}, [])
|
||||
|
||||
return (
|
||||
<Popover>
|
||||
@ -129,11 +333,12 @@ export const AiProviderControls = ({
|
||||
value={selection.model}
|
||||
onValueChange={(value) => {
|
||||
if (typeof value !== 'string') return
|
||||
if (value === OPENROUTER_BROWSE_VALUE) {
|
||||
setBrowsingOpenRouter(true)
|
||||
const browseProvider = catalogProviderFromBrowseValue(value)
|
||||
if (browseProvider) {
|
||||
setBrowsingProvider(browseProvider)
|
||||
return
|
||||
}
|
||||
setBrowsingOpenRouter(false)
|
||||
setBrowsingProvider(null)
|
||||
const match = findAiModelById(value)
|
||||
onModelSelect(match.provider.id, match.model.id)
|
||||
}}
|
||||
@ -156,13 +361,7 @@ export const AiProviderControls = ({
|
||||
<SelectLabel>
|
||||
<ProviderIcon className="ai-provider-icon" size={14} />
|
||||
<span>
|
||||
{availability[option.id]
|
||||
? option.label
|
||||
: option.id === 'codex'
|
||||
? `${option.label} · connect`
|
||||
: transportAvailability[option.id]
|
||||
? `${option.label} · key missing`
|
||||
: `${option.label} · local proxy unavailable`}
|
||||
{providerGroupCaption(option, availability, transportAvailability)}
|
||||
</span>
|
||||
</SelectLabel>
|
||||
{option.models.map((modelOption) => (
|
||||
@ -170,8 +369,8 @@ export const AiProviderControls = ({
|
||||
{modelOption.label}
|
||||
</SelectItem>
|
||||
))}
|
||||
{option.id === 'openrouter' && (
|
||||
<SelectItem value={OPENROUTER_BROWSE_VALUE}>
|
||||
{CATALOG_BROWSE_PROVIDERS.has(option.id) && (
|
||||
<SelectItem value={catalogBrowseValue(option.id)}>
|
||||
Other models…
|
||||
</SelectItem>
|
||||
)}
|
||||
@ -182,10 +381,18 @@ export const AiProviderControls = ({
|
||||
</Select>
|
||||
</div>
|
||||
|
||||
{isBrowsingOpenRouter && (
|
||||
<OpenRouterModelBrowser
|
||||
selectedModelId={selection.model}
|
||||
onModelSelect={(modelId) => onModelSelect('openrouter', modelId)}
|
||||
<CatalogModelBrowsers
|
||||
browsingProvider={browsingProvider}
|
||||
selectedModelId={selection.model}
|
||||
onModelSelect={onModelSelect}
|
||||
/>
|
||||
|
||||
{needsVisionFallback && visionModel && (
|
||||
<VisionFallbackPicker
|
||||
codingModelLabel={model.label}
|
||||
visionModel={visionModel}
|
||||
visionModels={visionModels}
|
||||
onVisionModelChange={onVisionModelChange}
|
||||
/>
|
||||
)}
|
||||
|
||||
@ -230,32 +437,13 @@ export const AiProviderControls = ({
|
||||
</div>
|
||||
|
||||
{!isConfigured && (
|
||||
<div className="ai-provider-warning" role="alert">
|
||||
<AlertCircle size={15} />
|
||||
<div className="ai-provider-warning-body">
|
||||
<span>
|
||||
{selection.provider === 'codex'
|
||||
? <><b>Codex isn’t connected.</b> Use your ChatGPT plan without an API key.</>
|
||||
: isTransportAvailable
|
||||
? <><b>API key missing.</b> Add your {provider.label} key to generate with this model.</>
|
||||
: <><b>Local proxy unavailable.</b> Moonshot can be used only from localhost.</>}
|
||||
</span>
|
||||
{(selection.provider === 'codex' || isTransportAvailable) && (
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
className="ai-provider-warning-action"
|
||||
onClick={() => {
|
||||
if (selection.provider === 'codex') onConnectCodex()
|
||||
else onManageKeys(selection.provider)
|
||||
}}
|
||||
>
|
||||
{selection.provider === 'codex' ? 'Connect Codex' : 'Enter API key'}
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
<ProviderSetupWarning
|
||||
providerId={selection.provider}
|
||||
keyLabel={keyLabel}
|
||||
isTransportAvailable={isTransportAvailable}
|
||||
onManageKeys={onManageKeys}
|
||||
onConnectCodex={onConnectCodex}
|
||||
/>
|
||||
)}
|
||||
</PopoverContent>
|
||||
</Popover>
|
||||
|
||||
@ -1,17 +1,21 @@
|
||||
import { useEffect, useId, useMemo, useState } from 'react'
|
||||
import {
|
||||
loadOpenRouterModels,
|
||||
type OpenRouterModelsResult,
|
||||
} from '../ai/openrouter-models'
|
||||
import { Search01 } from './icons'
|
||||
import { Input } from './ui/input'
|
||||
import type { AiModelOption } from '../ai/provider-catalog'
|
||||
|
||||
const VISIBLE_MODEL_LIMIT = 60
|
||||
|
||||
export type OpenRouterModelBrowserProps = {
|
||||
export type CatalogModelsResult = {
|
||||
models: readonly AiModelOption[]
|
||||
source: 'remote' | 'cache' | 'fallback'
|
||||
}
|
||||
|
||||
export type AiCatalogModelBrowserProps = {
|
||||
heading: string
|
||||
selectedModelId: string
|
||||
onModelSelect: (modelId: string) => void
|
||||
loadModels: () => Promise<CatalogModelsResult>
|
||||
fallbackNote?: string
|
||||
}
|
||||
|
||||
const filterModels = (
|
||||
@ -26,26 +30,30 @@ const filterModels = (
|
||||
}
|
||||
|
||||
/**
|
||||
* Searchable list over the full OpenRouter catalog (vision + tools capable
|
||||
* models only). Shown in the model picker once the OpenRouter provider is
|
||||
* selected; the curated shortlist stays in the grouped select above.
|
||||
* Searchable list over a runtime provider catalog. Shown in the model picker
|
||||
* once the user chooses "Other models…".
|
||||
*/
|
||||
export const OpenRouterModelBrowser = ({
|
||||
export const AiCatalogModelBrowser = ({
|
||||
heading,
|
||||
selectedModelId,
|
||||
onModelSelect,
|
||||
}: OpenRouterModelBrowserProps) => {
|
||||
loadModels,
|
||||
fallbackNote = 'Model list could not be loaded — showing the built-in shortlist.',
|
||||
}: AiCatalogModelBrowserProps) => {
|
||||
const labelId = useId()
|
||||
const [query, setQuery] = useState('')
|
||||
const [result, setResult] = useState<OpenRouterModelsResult | null>(null)
|
||||
const [result, setResult] = useState<CatalogModelsResult | null>(null)
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false
|
||||
void loadOpenRouterModels().then((loaded) => {
|
||||
void loadModels().then((loaded) => {
|
||||
if (!cancelled) setResult(loaded)
|
||||
})
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
// Load once when this catalog panel mounts; the parent remounts it per provider.
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [])
|
||||
|
||||
const models = useMemo(
|
||||
@ -58,7 +66,7 @@ export const OpenRouterModelBrowser = ({
|
||||
return (
|
||||
<div className="ai-model-picker-section">
|
||||
<span className="ai-model-picker-section__label" id={labelId}>
|
||||
{result ? `All OpenRouter models (${models.length})` : 'All OpenRouter models'}
|
||||
{result ? `${heading} (${models.length})` : heading}
|
||||
</span>
|
||||
<div className="ai-openrouter-search">
|
||||
<Search01 size={13} aria-hidden="true" />
|
||||
@ -72,7 +80,7 @@ export const OpenRouterModelBrowser = ({
|
||||
</div>
|
||||
{result?.source === 'fallback' && (
|
||||
<small className="ai-openrouter-note" role="status">
|
||||
Model list could not be loaded — showing the built-in shortlist.
|
||||
{fallbackNote}
|
||||
</small>
|
||||
)}
|
||||
<div className="ai-openrouter-list" role="listbox" aria-labelledby={labelId}>
|
||||
|
||||
@ -39,6 +39,11 @@ describe('deviceElementFromAttributes', () => {
|
||||
expect(deviceElementFromAttributes({ screenshot: 'https://evil.example/shot.png' }, 0).screenshot).toBeUndefined()
|
||||
})
|
||||
|
||||
it('accepts an object URL resolved from the Blob asset store', () => {
|
||||
const screenshot = 'blob:http://127.0.0.1:4173/0b1c2d3e'
|
||||
expect(deviceElementFromAttributes({ screenshot }, 0).screenshot).toBe(screenshot)
|
||||
})
|
||||
|
||||
it('clamps shadow into the editor range', () => {
|
||||
expect(deviceElementFromAttributes({ shadow: '400' }, 0).shadow).toBe(100)
|
||||
expect(deviceElementFromAttributes({ shadow: '-4' }, 0).shadow).toBe(0)
|
||||
|
||||
@ -27,9 +27,13 @@ export const deviceElementFromAttributes = (
|
||||
const mockup = attributes.mockup ?? ''
|
||||
const theme = attributes.theme ?? ''
|
||||
const shadow = Number(attributes.shadow ?? '')
|
||||
// Only an already-resolved upload (data URL) may reach the screen; an
|
||||
// Only an already-resolved upload may reach the screen: an object URL from
|
||||
// the Blob asset store, or a data URL from its storage fallback. An
|
||||
// unresolved asset: reference or anything else falls back to the fake screen.
|
||||
const screenshot = attributes.screenshot?.startsWith('data:image/') ? attributes.screenshot : undefined
|
||||
const rawScreenshot = attributes.screenshot
|
||||
const screenshot = rawScreenshot && (rawScreenshot.startsWith('blob:') || rawScreenshot.startsWith('data:image/'))
|
||||
? rawScreenshot
|
||||
: undefined
|
||||
|
||||
return {
|
||||
id: `html-device-${index}`,
|
||||
|
||||
1
src/vite-env.d.ts
vendored
1
src/vite-env.d.ts
vendored
@ -10,6 +10,7 @@ interface ImportMetaEnv {
|
||||
readonly VITE_ANTHROPIC_API_KEY?: string
|
||||
readonly VITE_XAI_API_KEY?: string
|
||||
readonly VITE_OPENROUTER_API_KEY?: string
|
||||
readonly VITE_OPENCODE_API_KEY?: string
|
||||
}
|
||||
|
||||
interface ImportMeta {
|
||||
|
||||
@ -10,12 +10,24 @@ export default defineConfig(({ command, mode }) => {
|
||||
const env = loadEnv(mode, process.cwd(), '')
|
||||
const aiLoggingValue = env['SHOTLUMA_AI_LOGGING'] ?? env['FRAMEFLOW_AI_LOGGING']
|
||||
const isAiLoggingEnabled = aiLoggingValue?.trim().toLowerCase() === 'true'
|
||||
const moonshotProxy = {
|
||||
const moonshotAndOpencodeProxy = {
|
||||
'/api/moonshot': {
|
||||
target: 'https://api.moonshot.ai',
|
||||
changeOrigin: true,
|
||||
rewrite: (requestPath: string) => requestPath.replace(/^\/api\/moonshot/, ''),
|
||||
},
|
||||
'/api/opencode/go': {
|
||||
target: 'https://opencode.ai',
|
||||
changeOrigin: true,
|
||||
rewrite: (requestPath: string) =>
|
||||
requestPath.replace(/^\/api\/opencode\/go/, '/zen/go') || '/zen/go',
|
||||
},
|
||||
'/api/opencode/zen': {
|
||||
target: 'https://opencode.ai',
|
||||
changeOrigin: true,
|
||||
rewrite: (requestPath: string) =>
|
||||
requestPath.replace(/^\/api\/opencode\/zen/, '/zen') || '/zen',
|
||||
},
|
||||
}
|
||||
|
||||
return {
|
||||
@ -40,10 +52,10 @@ export default defineConfig(({ command, mode }) => {
|
||||
server: {
|
||||
host: '127.0.0.1',
|
||||
port: Number(process.env['PORT']) || 4173,
|
||||
proxy: moonshotProxy,
|
||||
proxy: moonshotAndOpencodeProxy,
|
||||
},
|
||||
preview: {
|
||||
proxy: moonshotProxy,
|
||||
proxy: moonshotAndOpencodeProxy,
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
@ -10,7 +10,9 @@ export default defineConfig({
|
||||
reporter: ['text', 'lcov'],
|
||||
include: [
|
||||
'src/ai/chroma-key.ts',
|
||||
'src/ai/describe-images.ts',
|
||||
'src/ai/normalize-copy.ts',
|
||||
'src/ai/opencode-models.ts',
|
||||
'src/ai/overlay-asset-prompt.ts',
|
||||
'src/ai/provider-config.ts',
|
||||
'src/ai/provider-catalog.ts',
|
||||
|
||||
@ -2,9 +2,12 @@
|
||||
"$schema": "./node_modules/wrangler/config-schema.json",
|
||||
"name": "shotluma-app",
|
||||
"compatibility_date": "2026-08-03",
|
||||
"main": "scripts/shotluma-worker.ts",
|
||||
"assets": {
|
||||
"directory": "./dist",
|
||||
"not_found_handling": "single-page-application"
|
||||
"binding": "ASSETS",
|
||||
"not_found_handling": "single-page-application",
|
||||
"run_worker_first": ["/api/opencode/*"]
|
||||
},
|
||||
"routes": [
|
||||
{
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user