Compare commits

...

59 Commits

Author SHA1 Message Date
Doruk Kavcioglu
749eb018cd
Offer Tidy up and media actions in the canvas right-click menu
- Right-clicking a multi-selection puts Tidy up (⌃⌥T) at the top of the menu.
- A Media actions submenu, in its own group after Fork, lists the action bar's actions for the selection under the same conditions: transforms with a check when on, Edit with prompt, Make video, Rerun and Reuse for generated media, and Auto-Captions for scenes.
2026-10-04 18:04:20 +03:00
Doruk Kavcioglu
89e29285e0
Move the action bar's actions into a shared hook
- useObjectActions holds the selection's conditions and handlers (transforms, rerun, reuse, make video, edit with prompt, fork, tidy up, auto-captions), so other surfaces can offer the same actions; the action bar renders as before.
2026-10-04 18:04:20 +03:00
Doruk Kavcioglu
6909d3f586
Offer Tidy up in the prompt's More options menu
- With more than one node selected, the toolbar prompt's More options menu lists Tidy up (⌃⌥T) alongside the transforms, like the action bar's Tidy up button.
2026-10-04 17:49:01 +03:00
Doruk Kavcioglu
ae7efbd279
Submit the toolbar prompt with Cmd+Enter and let Enter add a line
- Enter inserts a new line in the toolbar prompt and ⌘Enter generates, the same as in canvas prompt boxes; the Generate tooltip shows ⌘↵.
2026-10-04 17:44:23 +03:00
Doruk Kavcioglu
6e5566a543
Recall prompts only from an empty box and stop once the text is touched
- ↑ starts walking recent prompts only when the prompt box is empty; while a recall is untouched, ↑ and ↓ keep stepping, and ↓ past the newest returns to an empty box.
- Moving the caret, selecting, typing, any other key or a click ends browsing, and the arrows move through the text again until the box is empty, so text being read or edited is never swapped out.
2026-10-04 17:43:05 +03:00
Doruk Kavcioglu
55dd7b393c
Recall earlier prompts with the arrow keys in prompt boxes
- In the canvas prompt boxes and the toolbar prompt, ↑ from the first line steps back through recent prompts and ↓ from the last line steps forward, back to the draft that was being typed.
- Canvas prompts are remembered on submit too, in the same recent prompts list the toolbar's / menu shows.
2026-10-04 17:36:35 +03:00
Doruk Kavcioglu
373d6e652d
Fix two type errors the web type check had been missing
- The canvas right-click menu closes after a parent folder row is picked through Kobalte's own outside-press dismissal; ContextMenu takes no controlled open state, and forcing one could leave the menu unable to reopen.
- The prompt attach menu's trigger renders the attach button through Kobalte's polymorphic `as`, which types its icon and label.
2026-10-04 17:36:35 +03:00
Doruk Kavcioglu
c05aca2a01
Float the asset sidebar when revealing with the UI hidden
- With the UI hidden, the left sidebar floats over the canvas while an asset is picked, the way the inspector floats while something is selected, and the canvas is clipped on that side.
- Reveal in assets picks the asset itself, so with the UI hidden the sidebar floats in on its folder instead of bringing the whole UI back; selecting on the canvas closes it again.
- The floating project header steps aside while the sidebar floats.
2026-10-04 17:27:56 +03:00
Doruk Kavcioglu
bc5b35ff5f
Reveal the asset picked in the panel with ⌥⌘O
- ⌥⌘O reveals the asset picked in the asset panel when there is one, and the canvas selection's asset otherwise; the panel's Reveal in finder item shows the shortcut.
2026-10-04 17:24:45 +03:00
Doruk Kavcioglu
2ccfcee536
Reveal an asset in Finder from the asset panel
- An asset's right-click menu offers Reveal in finder on the desktop when the asset has a file, sharing the canvas reveal's helper.
2026-10-04 17:22:52 +03:00
Doruk Kavcioglu
2c9b92542f
Rename Move to folder to Set folder and group it with the reveals
- Set folder sits with Reveal in assets and Reveal in finder in one group, separated from Add prompt and Fork.
2026-10-04 17:22:52 +03:00
Doruk Kavcioglu
f939a99922
Reveal the selection's asset in the asset panel or in Finder
- Right-clicking the selection offers Reveal in assets (⇧⌘O) and Reveal in finder (⌥⌘O), and the shortcuts work from the canvas.
- Reveal in assets switches the sidebar to Assets, opens the asset's folder, clears the search and filter, selects the asset and scrolls to it, bringing the panels back when they are hidden.
- Reveal in finder shows the asset's file in the OS file manager; it is offered on the desktop for assets with a file.
2026-10-04 17:22:51 +03:00
Doruk Kavcioglu
1b25b90f17
Cascade the Move to folder menu and list recent folders first
- Folders open as nested submenus showing their own names; a folder with subfolders can be clicked, or entered with Enter, to move there, and its arrow opens what is inside.
- The last three folders moved into sit at the top above a separator, for this session.
- Canvas right-click items can be separators, and submenus nest to any depth.
2026-10-04 17:09:35 +03:00
Doruk Kavcioglu
132cd67534
Show a media's folder on the canvas and move it from the right-click menu
- A top-level node showing a library asset names the asset's folder at the right of its header, beside the duration; it gives way first when the header runs out of room, and the title keeps today's behavior.
- Right-clicking the selection offers Move to folder: every folder as a breadcrumb path under All assets, the current one checked, moving each distinct asset in the selection like a drag in the asset panel.
- Canvas right-click items can carry a submenu and a checked state.
2026-10-04 17:02:27 +03:00
Doruk Kavcioglu
c4a7afdf59
Set selection opacity by typing digits
- A digit sets the selection's opacity the way Figma does: 1 to 9 are 10% to 90%, 0 is 100%, and two digits typed quickly are that exact value (05 is 5%, 15 is 15%, 00 is 0%); each digit pairs with the one typed just before it.
- Every digit pressed in a frame counts, in order, and a toast that fits its text says what was set.
- Prompts are skipped, values already in place are not rewritten, and keyframed opacity gets a keyframe at the playhead like the inspector's slider.
2026-10-04 16:40:22 +03:00
Doruk Kavcioglu
8fccec5aa6
Show estimated progress while a generation runs
- A generating node's label starts as "Generating...", dissolves into "N% done" as time passes against a rough per-model estimate, and turns to "Almost done..." once the estimate runs out; the name returns when the asset lands.
- Estimates live in one table per model and transform, ready to be replaced by measured timings or the API's own ETA.
2026-10-04 16:28:56 +03:00
Doruk Kavcioglu
db1c12822f
Draw selection chrome above prompt boxes
- The hover outlines, selection outline and handles, size badge, marquee and snap guides are drawn in one layer above all prompt boxes, mirroring what the canvas draws, so no prompt box can cover them.
- Prompt boxes no longer draw their own ring or handles; a prompt's hover and selection follow the same rules as any canvas object, with the box under the pointer standing in for the canvas hover.
2026-10-04 16:16:36 +03:00
Doruk Kavcioglu
36a8aff0c2
End picking from the canvas with Enter as well as Esc
- Enter keeps what was picked, closes the picker and returns focus to the prompt's text box, without adding a new line or reaching canvas shortcuts.
2026-10-04 07:34:32 +03:00
Doruk Kavcioglu
a8d55c3e23
Show an eyedropper cursor while picking from the canvas
- A pick cursor sits with the trim cursors in use-cursor, as a self-contained SVG with its click point at the tip.
- The picking layer stays mounted and hides between picks, so it takes the cursor once.
2026-10-04 07:33:01 +03:00
Doruk Kavcioglu
16b6ba5f59
Focus the prompt text box when picking from the canvas ends
- Reaching the reference limit, filling a frame, Esc, or a click on anything that is not an image puts focus back in that prompt's text box.
2026-10-04 07:17:47 +03:00
Doruk Kavcioglu
efd87953be
Pick prompt image references and video frames from the canvas
- The prompt box's Image references, Start frame and End frame buttons open a menu: Pick from canvas or Upload from computer.
- Picking shows a crosshair; clicking images adds them as references until the limit, a frame takes one image, and Esc or a click on anything that is not an image stops; the canvas still pans and zooms meanwhile.
- PromptInputAttachButton passes other button props through, and the media lookup behind the selection is exported as resolveMedia.
2026-10-04 07:15:36 +03:00
Doruk Kavcioglu
5cd127cc7b
Fork an AI-generated object into a prompt
- Fork creates a prompt whose template is the object's src recipe without the seed, keeping any transform, placed in free space beside the object, selected and focused.
- It is in the More menu for generated images and videos, in the right-click menu on a selected generated object, and on Shift+N; N alone still adds an empty prompt.
- insertPrompt takes the template to start from.
2026-10-04 06:50:59 +03:00
Doruk Kavcioglu
baf762da98
Pick models and voices with the arrow keys while the search box keeps focus
- Up and Down move a highlight through the list, wrapping and scrolling it into view, and Enter picks it; hovering moves the same highlight, and clicking a row picks it without taking focus from the search box, so typing always filters.
- The model dropdown uses a popover instead of a menu, whose items take focus on hover, with the same look as before.
- SearchInput passes key presses to an optional handler.
2026-10-04 06:32:29 +03:00
Doruk Kavcioglu
dcecee87c7
Focus the search box when a model or voice dropdown opens
- The model and voice dropdowns put focus in their search box on open, after the menu's own focus, so typing filters right away; this applies to the canvas prompt box and the toolbar prompt, which share them.
- SearchInput takes an optional ref for its input.
2026-10-04 06:23:09 +03:00
Doruk Kavcioglu
1c87058c16
Focus a prompt added from the context menu once the menu is gone
- A new prompt takes its focus only when no menu is left on the page; a fading menu item takes focus back when hovered, which left the text box unfocused after Add prompt.
2026-10-04 06:14:42 +03:00
Doruk Kavcioglu
6194ddbf85
Keep a new prompt's box in place so it keeps focus
- A prompt box without a hit region of its own stacks with its group, and one with neither goes on top, where new elements land; before, it sorted to the bottom.
- A prompt added with N has no hit region until the next frame, so its box used to move to the top one frame after taking focus, and the move dropped the focus.
2026-10-04 06:14:42 +03:00
Doruk Kavcioglu
435c935297
Stop a resized prompt at its minimum size without sliding
- The minimum now limits the drag itself, so a prompt shrinks around its fixed edge and stops there instead of sliding once it reaches the minimum; Shift keeps the ratio and Alt stops about the center.
- It covers prompts directly inside a resized group, and the whole selection stops when any prompt reaches its minimum.
- The earlier check on the resized size is gone, so the rule lives in one place; the resize loop and the check share one list of what gets resized.
2026-10-04 06:03:31 +03:00
Doruk Kavcioglu
8c64c075bd
Store a prompt's settings as a generate template
- A prompt keeps its settings as template={generate.<type>({...})} plus count: the same recipe generated media carries in src, without a seed.
- Running copies the template with a fresh seed per result, keeping any transform around it; typing changes only the prompt inside it.
- Prompts with the older flat props (mode or type, prompt, model and the rest) still load and switch to the template on their next edit in the box.
- A template restored by undo as plain data is read back as a recipe.
2026-10-04 05:54:01 +03:00
Doruk Kavcioglu
127b20a4fe
Check for an empty prompt when generating, not when declaring
- generate.* accept any string prompt, so a recipe can be written down before it has text; a prompt that is not a string still throws.
- Generation refuses a blank prompt with "Needs a prompt" before anything is reserved, sent or charged; the failure lands on the element and in check.
2026-10-04 05:53:57 +03:00
Doruk Kavcioglu
31edd1731d
Hide keyframe diamonds on prompt transform and layout fields
- TransformSettings and LayoutPanel take an optional keyframes flag, passed down to the rotate, offset and scale rows; without it a field leaves its keyframe slot empty.
- The inspector turns it off for prompts, which are not animated.
2026-10-04 04:50:56 +03:00
Doruk Kavcioglu
fc724b1ff5
Give the prompt content field a placeholder and two-line minimum
- GrowingTextArea takes an optional placeholder and minRows; only the Prompt section uses them, so text content is unchanged.
- The prompt's Content field shows the prompt box placeholder and starts two lines tall.
2026-10-04 04:48:35 +03:00
Doruk Kavcioglu
cc2b8484bd
Keep Content labels at the top of growing fields
- The Content label in Typography and Prompt aligns to the first line of its text area instead of centering as it grows; a single line keeps the same spacing.
2026-10-04 04:45:40 +03:00
Doruk Kavcioglu
70777fa731
Show a prompt's content in the inspector
- A Prompt section with a Content field edits the prompt text, using the same growing text area as text content.
2026-10-04 04:45:40 +03:00
Doruk Kavcioglu
352d2c6925
Bring back prompt transform and layout panels
- A selected prompt shows the transform and layout panels in the inspector again; the collapsed view still opens no floating inspector for prompts.
2026-10-04 04:45:40 +03:00
Doruk Kavcioglu
4fa1a6372e
Generate from a prompt box with Cmd+Enter
- Enter adds a new line in the prompt box; Cmd+Enter (Ctrl+Enter on Windows) generates, and the Generate tooltip shows it.
2026-10-04 04:36:38 +03:00
Doruk Kavcioglu
58fa8011e1
Draw prompt selection handles and stop selection flicker
- A prompt that is the only selection draws its corner handles inside its own box, matching the canvas handles, so they are not half hidden behind it.
- The handles stay up while a press is held and the ring keeps its selected width, so selecting no longer blinks.
- A press only pre-selects a prompt when the canvas would hit something else there, and then marks the HUD as moving so the size label does not flash; otherwise the press goes straight to the canvas.
2026-10-04 04:34:42 +03:00
Doruk Kavcioglu
1692775bf0
Give prompts no inspector panels or floating sidebar
- A selected prompt shows no properties in the inspector.
- In the collapsed view the floating inspector opens only when the selection holds something other than prompts.
2026-10-04 04:28:11 +03:00
Doruk Kavcioglu
f69f709109
Make prompt boxes opaque to the pointer and selectable over a selection
- A prompt box takes the pointer over its whole area, label row included, so controls of prompts behind it no longer catch the cursor or clicks; presses and double-clicks on its empty areas are forwarded to the canvas, and the layer mirrors the canvas cursor.
- Pressing an unselected prompt selects it and forwards the press a frame later, once the canvas has its selection mask, so a covering selection no longer swallows it; a release before then flushes the press first.
- A prompt shows its ring while the pointer is over its box.
2026-10-04 04:26:21 +03:00
Doruk Kavcioglu
4c62068a32
Stack prompt boxes in canvas order and give them their own label
- Prompt boxes are ordered by the canvas hit regions, so the box that looks on top is the one hover, clicks and drags reach.
- Each top-level prompt draws its name label inside its own box, so a prompt in front keeps its label over the one behind; the HUD keeps the label's hit region but no longer paints prompt labels.
2026-10-04 04:13:18 +03:00
Doruk Kavcioglu
2eabc8548d
Keep canvas hover current over prompt controls
- Pointer moves over a prompt box's controls are forwarded to the canvas, so hover follows the cursor instead of sticking on the last item the canvas saw.
2026-10-04 04:05:12 +03:00
Doruk Kavcioglu
cca172e05e
Fix prompt boxes during canvas drags and draw their own ring
- While a press that began on the canvas is held, prompt boxes ignore the pointer, so a drag crossing another prompt keeps receiving moves.
- A selected or hovered prompt draws its selection ring as part of its own box, matching the canvas ring, so overlapping prompts cannot hide it; the box border stays neutral.
2026-10-04 04:02:07 +03:00
Doruk Kavcioglu
8014aa1f48
Back prompt boxes with the canvas background
- The prompt layer reads the stage background from the document through useTrait, so it updates only when the color changes.
- Each prompt box paints that color under the translucent input color, so canvas items behind it no longer show through.
2026-10-04 03:46:22 +03:00
Doruk Kavcioglu
2b86154650
Default new prompts to 1088x496
- New prompt boxes start at 544x248 at the 2x scale; the minimum size is unchanged.
2026-10-04 03:43:20 +03:00
Doruk Kavcioglu
504632b8a7
Draw prompt boxes at 2x scale
- The prompt box lays out at its design size and is drawn twice as large on the canvas, so a new prompt (1152x768) sits in proportion to the 1920x1080 results it makes.
- Default and minimum sizes are in canvas units at that scale; resizing still reflows rather than scaling the text.
2026-10-04 03:38:12 +03:00
Doruk Kavcioglu
8d8814b47c
Keep a scroll gesture where it started over prompt boxes
- A wheel gesture belongs to what it began on until it pauses for 250ms: a pan that slides a prompt under the cursor keeps panning, and a scroll begun over overflowing prompt text stays in the text.
- Pinch and modifier zoom always go to the canvas; only prompt box text areas take part.
2026-10-04 03:29:45 +03:00
Doruk Kavcioglu
97aaffa9d6
Let the canvas pan and zoom over prompt boxes
- Wheel and pinch over a prompt box go to the canvas, except a vertical scroll over prompt text that overflows its box, which scrolls the text and stays there.
- Middle-button drags, and drags with the hand tool, pan the canvas over a prompt box too.
2026-10-04 03:24:52 +03:00
Doruk Kavcioglu
91ea5231aa
Clip prompt boxes to the canvas
- The prompt boxes sit in a canvas-sized container that hides overflow, so a box panned past the edge no longer draws over the sidebars.
2026-10-04 03:16:02 +03:00
Doruk Kavcioglu
ad7686bb99
Focus a new prompt's text box when it is added
- insertPrompt leaves a one-time focus request; the prompt box takes it on its next frame, without scrolling.
- The context menu keeps focus where Add prompt put it instead of restoring it to the trigger.
2026-10-04 03:14:56 +03:00
Doruk Kavcioglu
a8dfe89e89
Add N shortcut for adding a prompt
- N adds a prompt at the pointer, or in the middle of the view when the pointer is off the canvas.
- insertPrompt moves into the engine so the shortcut and the context menu share it; new prompts leave the model to its default.
- The context menu shows N next to Add prompt.
2026-10-04 03:09:18 +03:00
Doruk Kavcioglu
7fb6edbcc2
Add a prompt element to the canvas
- <prompt> is a composition element that lives in the project file: position, size and generation settings (mode, prompt, model, aspect ratio, count, refs, frames, duration, audio, voice) as props. It draws nothing in renders.
- The canvas shows each prompt as the prompt box design, following the camera; edits write back to its props and running it places results beside it.
- Right-clicking the canvas adds "Add prompt" to the app context menu.
- Prompts resize no smaller than 400x200 and show only transform and layout in the inspector.
- Generate hooks take an optional box to place results beside; the toolbar prompt box is unchanged.
2026-10-04 02:57:14 +03:00
Doruk Kavcioglu
0bb9890d5b
Remove comments added in the collapsed inspector and Tidy up work
- Restore the align.ts header to its original wording.
2026-10-04 00:59:05 +03:00
Doruk Kavcioglu
d50e02b588
Tidy up: group by aspect ratio and pack mixed layouts
- Alike items also need aspect ratios within 15%, so 16:9, 9:16 and 1:1 group apart.
- Mixed groups and odd ones out are shelf-packed, tallest first, at the width that best fits a 16:9 screen.
- Mixed layouts use one gap throughout, 20% of the items' average shorter side.
2026-10-04 00:56:48 +03:00
Doruk Kavcioglu
3176706068
Tidy up: one gap, both-way columns, and like-with-like groups
- Rough grids take the wider of their longest row and tallest column, so one offset item cannot add a column.
- One gap for both axes, the tighter of the measured two.
- Scatters widen until the last row is more than half full.
- Mixed sizes in two dimensions group alike items (same kind, sizes within 1.5x) and lay the groups out as blocks; rows and columns keep their order.
2026-10-04 00:18:35 +03:00
Doruk Kavcioglu
2b051229b5
Tidy up: ignore gaps an item would fit in and scatter gaps
- In a grid or scatter, a typical gap at least an item wide is an empty cell, not spacing.
- A scatter's gaps are happenstance, so it uses the fallback gap.
- The fallback is 20% of the items' shorter side instead of a fixed 40 units.
2026-10-03 23:59:52 +03:00
Doruk Kavcioglu
e34f53ec3c
Show the ⌃⌥T shortcut in the Tidy up tooltip
- Follows the clip-path and object-mask bars: the label repeated with the shortcut through TooltipContent's shortcut prop.
2026-10-03 23:30:10 +03:00
Doruk Kavcioglu
74e918a201
Add ⌃⌥T shortcut for Tidy up
- Registered for both t and †, since macOS can report ⌥T as the composed character.
2026-10-03 23:26:46 +03:00
Doruk Kavcioglu
1c83d89381
Add Tidy up to the canvas action bar
- tidySelection lays the selection out as an even grid from its top-left corner, in reading order, reading rows, columns and gaps off the current layout.
- The action bar shows Tidy up for any multi-selection, next to Edit with prompt for images.
2026-10-03 23:17:26 +03:00
Doruk Kavcioglu
f0d79e8d32
Keep vibrancy behind the floating inspector
- Clip the canvas under the inspector overlay in collapsed view so the transparent page lets macOS vibrancy show through, matching the docked sidebar.
- Canvas accepts an optional style prop for the clip.
2026-10-03 23:08:07 +03:00
Doruk Kavcioglu
303884ef27
Show inspector on canvas selection in collapsed view
- With the sidebars hidden, the inspector overlays the right edge while canvas nodes are selected and hides when the selection clears.
- Overlaid rather than a grid column so the canvas never resizes under the pointer.
2026-10-03 23:06:23 +03:00
49 changed files with 2384 additions and 176 deletions

View File

@ -2,6 +2,7 @@
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { For, Show, createSignal } from "solid-js"
import type { JSX } from "solid-js"
import {
ContextMenu,
@ -10,10 +11,19 @@ import {
ContextMenuPortal,
ContextMenuSeparator,
ContextMenuShortcut,
ContextMenuSub,
ContextMenuSubContent,
ContextMenuSubTrigger,
ContextMenuTrigger,
} from "@/components/ui/context-menu"
import { Icon } from "@/components/ui/icon"
import { takeContextMenuItems } from "@/components/context-menu-extras"
import type { ContextMenuExtra } from "@/components/context-menu-extras"
export function AppContextMenu(props: { children: JSX.Element }) {
const [extras, setExtras] = createSignal<ContextMenuExtra[]>([])
let keepFocus = false
const handleUndo = () => {
document.execCommand("undo")
}
@ -55,14 +65,82 @@ export function AppContextMenu(props: { children: JSX.Element }) {
window.location.reload()
}
const select = (item: { onSelect?(): void }) => {
keepFocus = true
item.onSelect?.()
}
const selectFolderRow = (item: { onSelect?(): void }) => {
if (!item.onSelect) return
select(item)
document.body.dispatchEvent(new PointerEvent("pointerdown", { bubbles: true, button: 0, pointerType: "mouse" }))
}
const checkCell = (checked: boolean | undefined) => (
<Show when={checked !== undefined}>
<span class="flex h-7 w-6 shrink-0 items-center justify-center">
<Show when={checked}>
<Icon name="confirm-check" class="size-6" />
</Show>
</span>
</Show>
)
const renderExtra = (entry: ContextMenuExtra): JSX.Element => {
if ("separator" in entry) return <ContextMenuSeparator />
const item = entry
if (item.items) {
return (
<ContextMenuSub>
<ContextMenuSubTrigger
onClick={() => selectFolderRow(item)}
onKeyDown={(event: KeyboardEvent) => {
if (event.key === "Enter") selectFolderRow(item)
}}
>
<span class="flex w-full items-center">
<span class="min-w-0 flex-1 truncate">{item.label}</span>
{checkCell(item.checked)}
</span>
</ContextMenuSubTrigger>
<ContextMenuPortal>
<ContextMenuSubContent class="min-w-[200px] max-w-[320px]">
<For each={item.items}>{renderExtra}</For>
</ContextMenuSubContent>
</ContextMenuPortal>
</ContextMenuSub>
)
}
return (
<ContextMenuItem classList={{ "pr-0!": item.checked !== undefined }} onSelect={() => select(item)}>
<span class="min-w-0 flex-1 truncate">{item.label}</span>
<Show when={item.shortcut}>
<ContextMenuShortcut>{item.shortcut}</ContextMenuShortcut>
</Show>
{checkCell(item.checked)}
</ContextMenuItem>
)
}
return (
<ContextMenu>
<ContextMenuTrigger class="contents">
<ContextMenuTrigger class="contents" onContextMenu={(event: MouseEvent) => setExtras(takeContextMenuItems(event))}>
{props.children}
</ContextMenuTrigger>
<ContextMenuPortal>
<ContextMenuContent class="w-[200px]">
<ContextMenuContent
class="w-[200px]"
onCloseAutoFocus={(event: Event) => {
if (keepFocus) event.preventDefault()
keepFocus = false
}}
>
<Show when={extras().length > 0}>
<For each={extras()}>{renderExtra}</For>
<ContextMenuSeparator />
</Show>
<ContextMenuItem onSelect={handleUndo}>
Undo
<ContextMenuShortcut>⌘Z</ContextMenuShortcut>

View File

@ -3,21 +3,77 @@
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { useWorld } from "@diffusionstudio/koota-solid";
import { findSceneAt, screenToWorld, worldToLocal, Library, Root } from "@diffusionstudio/runtime";
import { findSceneAt, getSelection, isPointerInEntity, screenToWorld, worldToLocal, Library, RenderSurface, Root } from "@diffusionstudio/runtime";
import { CameraController, EngineCanvas } from "@/engine";
import { forkPrompt, forkTemplate, insertPrompt } from "@/engine/prompt";
import { insertAsset } from "@/engine/insert-asset";
import { selectionAssets } from "@/engine/asset-folders";
import { useObjectActions } from "@/components/genai/use-object-actions";
import { usePromptInput } from "@/context/prompt-input";
import { droppedFiles, importFiles } from "@/engine/asset-actions";
import { Toolbar } from "./toolbar";
import { DrawOverlay } from "./draw-overlay";
import { DesktopAppBanner } from "./desktop-app-banner";
import { toast } from "somoto"
import { SceneInitOverlay } from "./scene-init-overlay";
import { PromptNodes } from "./prompt-node";
import { offerContextMenuItems } from "@/components/context-menu-extras";
import { finderPath, revealInAssets, revealInFinder } from "@/components/sidebar-left/reveal";
import { ASSET_DRAG_TYPE } from "@/components/sidebar-left/folder-item";
import type { Asset } from "@diffusionstudio/assets";
import type { JSX } from "solid-js";
import { assetFolder, basename } from "@diffusionstudio/assets";
import type { Asset, AssetLibrary } from "@diffusionstudio/assets";
import type { ContextMenuExtra } from "@/components/context-menu-extras";
export function Canvas() {
type CanvasProps = {
style?: JSX.CSSProperties;
}
let recentFolders: string[] = [];
function setFolder(library: AssetLibrary, assets: Asset[]): ContextMenuExtra {
const current = new Set(assets.map(assetFolder));
const checked = (folder: string) => current.size === 1 && current.has(folder);
const move = (folder: string) => () => {
library.move(assets, folder);
if (folder) recentFolders = [folder, ...recentFolders.filter((recent) => recent !== folder)].slice(0, 3);
};
const branch = (folder: string): ContextMenuExtra => {
const children = library.childrenOf(folder).folders;
return { label: basename(folder), checked: checked(folder), onSelect: move(folder), ...(children.length ? { items: children.map(branch) } : {}) };
};
const recents = recentFolders.filter((folder) => library.folders().has(folder));
return {
label: "Set folder",
items: [
...recents.map((folder) => ({ label: basename(folder), checked: checked(folder), onSelect: move(folder) })),
...(recents.length ? [{ separator: true as const }] : []),
{ label: "All assets", checked: checked(""), onSelect: move("") },
...library.childrenOf("").folders.map(branch),
],
};
}
export function Canvas(props: CanvasProps) {
const world = useWorld();
const actions = useObjectActions(usePromptInput().openPromptInput);
const objectActions = (): ContextMenuExtra[] => {
const media = actions.isImage() || actions.isVideo();
const generated = media && actions.isGenerated();
return [
...(actions.hasScene() ? [{ label: "Auto-Captions", onSelect: actions.autoCaptions }] : []),
...(actions.isImage() ? [{ label: "Remove background", checked: actions.isOn("removeBackground"), onSelect: () => actions.toggle("removeBackground") }] : []),
...(actions.isVideo() ? [{ label: "Add audio", checked: actions.isOn("addAudio"), onSelect: () => actions.toggle("addAudio") }] : []),
...(media ? [{ label: "Upscale", checked: actions.isOn("upscale"), onSelect: () => actions.toggle("upscale") }] : []),
...(actions.isImage() ? [{ label: "Edit with prompt", onSelect: actions.editWithPrompt }] : []),
...(generated && actions.isImage() ? [{ label: "Make video", onSelect: actions.makeVideo }] : []),
...(generated ? [{ label: "Rerun", onSelect: actions.rerun }] : []),
...(generated && actions.isImage() ? [{ label: "Reuse", onSelect: actions.reuse }] : []),
];
};
/**
* Drops onto the canvas: library assets (dragged from the panel) land where
@ -68,18 +124,46 @@ export function Canvas() {
event.stopPropagation();
}
const handleContextMenu = (event: MouseEvent & { currentTarget: HTMLElement }) => {
if (!(event.target instanceof HTMLCanvasElement)) return;
const rect = event.currentTarget.getBoundingClientRect();
const point = screenToWorld(world, event.clientX - rect.left, event.clientY - rect.top);
const bounds = event.target.getBoundingClientRect();
const resolution = world.get(RenderSurface)?.resolution ?? 1;
const device = { x: (event.clientX - bounds.left) * resolution, y: (event.clientY - bounds.top) * resolution };
const forkable = getSelection(world).find((entity) => isPointerInEntity(world, entity, device) && forkTemplate(entity));
const library = world.get(Library);
const onSelection = getSelection(world).some((entity) => isPointerInEntity(world, entity, device));
const assets = onSelection ? selectionAssets(world) : [];
const objectItems = onSelection ? objectActions() : [];
offerContextMenuItems(event, [
...(onSelection && actions.canTidy() ? [{ label: "Tidy up", shortcut: "⌃⌥T", onSelect: actions.tidy }] : []),
{ label: "Add prompt", shortcut: "N", onSelect: () => insertPrompt(world, point) },
...(forkable ? [{ label: "Fork", shortcut: "⇧N", onSelect: () => forkPrompt(world, forkable) }] : []),
...(objectItems.length ? [{ separator: true as const }, { label: "Media actions", items: objectItems }] : []),
...(library && assets.length ? [
{ separator: true as const },
setFolder(library, assets),
{ label: "Reveal in assets", shortcut: "⇧⌘O", onSelect: () => revealInAssets(world) },
...(finderPath(world) ? [{ label: "Reveal in finder", shortcut: "⌥⌘O", onSelect: () => revealInFinder(world) }] : []),
] : []),
]);
};
return (
<div class="relative size-full bg-background">
<div class="relative size-full bg-background" style={props.style}>
<div
class="absolute inset-0"
on:drop={handleDropEvent}
on:dragover={handleDragOver}
onContextMenu={handleContextMenu}
>
<Toolbar />
<DesktopAppBanner />
<DrawOverlay />
<SceneInitOverlay />
<EngineCanvas />
<PromptNodes />
<CameraController />
</div>
</div>

View File

@ -0,0 +1,988 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { For, Match, Show, Switch, createEffect, createMemo, createSignal, on, onCleanup, onMount } from "solid-js";
import { Not, Or } from "koota";
import { useQuery, useTrait, useWorld } from "@diffusionstudio/koota-solid";
import { generate, getAssetSpec } from "@diffusionstudio/jsx";
import { Audio, ImagePaint, Rect, VideoPaint, authoredElement } from "@diffusionstudio/reconciler";
import {
AssetId, Background, Computed, Culled, DEFAULT_BACKGROUND, Geometry, Group, HitRegions, Hidden, Hovering, Name, PaintType, PromptNode,
RenderSurface, Root, Selected, Sequential, Tool, ToolType, colorToHex, entityQuad, entityWorldMat, getEntityBounds, getMaskSelection,
getParentEntity, getSelectionMask, invert2D, isPointerInEntity, multiply2D, pointInQuad, rectToQuad, store, transformPoint, translate2D,
} from "@diffusionstudio/runtime";
import { Button } from "@/components/ui/button";
import {
DropdownMenu, DropdownMenuContent, DropdownMenuItem, DropdownMenuPortal, DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu";
import { Icon } from "@/components/ui/icon";
import { Separator } from "@/components/ui/separator";
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip";
import { ASSET_DRAG_TYPE } from "@/components/sidebar-left/folder-item";
import { useEngineContext } from "@/engine";
import { droppedFiles, importFiles, pickFiles } from "@/engine/asset-actions";
import { useDerived, useEditor } from "@/engine/hooks";
import { useLibrary } from "@/engine/library";
import { useCursor } from "@/hooks/use-cursor";
import { getMountedNameInput } from "@/engine/hud/name-input";
import { AUDIO_SIZE } from "@/engine/insert-asset";
import { PROMPT_SCALE, PROMPT_SIZE, generateRef, generationOf, setPromptText, takePromptFocus, templateOf, withGeneration } from "@/engine/prompt";
import { getMarqueeQuad } from "@/engine/input/snapping";
import { Hud, Keys, Pointer, SnapLines } from "@/engine/traits";
import {
ALL_DURATION_OPTIONS,
ASPECT_RATIO_DIMENSIONS,
ALL_VIDEO_ASPECT_RATIO_OPTIONS,
PROMPT_INPUT_AUDIO_MODEL_OPTIONS,
PROMPT_INPUT_IMAGE_ASPECT_RATIO_OPTIONS,
PROMPT_INPUT_IMAGE_MODEL_OPTIONS,
PROMPT_INPUT_MODE_OPTIONS,
PROMPT_INPUT_VARIANT_COUNT_OPTIONS,
PROMPT_INPUT_VIDEO_MODEL_OPTIONS,
PROMPT_INPUT_VOICE_OPTIONS,
} from "@/components/genai/config";
import {
DEFAULT_BUTTON_CLASS,
MAX_IMAGE_REFERENCES,
ModelMenu,
PromptInputAttachButton,
PromptInputCompactMenu,
PromptInputReferenceImageButton,
VoiceMenu,
createDefaultConfig,
} from "@/components/genai/prompt-input";
import { insertGenerated, randomSeed } from "@/components/genai/insert";
import { toPromptConfig } from "@/components/genai/use-generation-records";
import { resolveMedia } from "@/components/genai/selection";
import { createPromptHistory, rememberPrompt } from "@/components/genai/prompt-history";
import type { AssetLibrary } from "@diffusionstudio/assets";
import type { AssetRef, GenerateSpec } from "@diffusionstudio/jsx";
import type { AABB, Point, Quad, SelectionMask } from "@diffusionstudio/runtime";
import type { Entity, World } from "koota";
import type { PromptInputMode } from "@/components/genai/prompt-input";
import type { AspectRatio, GenerationConfig } from "@/components/genai/schemas";
type Props = Record<string, unknown>;
const COMMIT_DELAY = 400;
const text = (value: unknown) => (typeof value === "string" && value !== "" ? value : undefined);
const number = (value: unknown) => (typeof value === "number" ? value : undefined);
const LEGACY_PROPS = ["mode", "type", "prompt", "model", "aspectRatio", "refs", "startFrame", "endFrame", "duration", "audio", "voice"];
function legacyConfig(props: Props, library: AssetLibrary | undefined): GenerationConfig {
const mode = (text(props.type) ?? text(props.mode) ?? "image").toUpperCase() as PromptInputMode;
const base = createDefaultConfig(mode, text(props.prompt) ?? "");
const idOf = (path: unknown) => {
const value = text(path);
return value ? library?.get(value)?.id : undefined;
};
switch (base.mode) {
case "IMAGE":
return {
...base,
model: text(props.model) ?? base.model,
aspectRatio: (text(props.aspectRatio) as AspectRatio | undefined) ?? base.aspectRatio,
count: number(props.count) ?? base.count,
imageRefIds: (Array.isArray(props.refs) ? props.refs : []).map(idOf).filter((id): id is string => id !== undefined),
};
case "VIDEO":
return {
...base,
model: text(props.model) ?? base.model,
aspectRatio: (text(props.aspectRatio) as AspectRatio | undefined) ?? base.aspectRatio,
duration: number(props.duration) ?? base.duration,
generateAudio: props.audio === true,
startFrameImageId: idOf(props.startFrame),
endFrameImageId: idOf(props.endFrame),
};
case "VOICE":
return { ...base, voice: text(props.voice) ?? base.voice };
case "AUDIO":
return { ...base, model: text(props.model) ?? base.model };
}
}
function toConfig(props: Props, library: AssetLibrary | undefined): GenerationConfig {
const template = templateOf(props.template);
const generation = template && generationOf(template);
if (!generation) return legacyConfig(props, library);
const config = toPromptConfig(getAssetSpec(generation) as GenerateSpec, library);
return config.mode === "IMAGE" ? { ...config, count: number(props.count) ?? 1 } : config;
}
function toGeneration(config: GenerationConfig, library: AssetLibrary | undefined): AssetRef {
const pathOf = (id: string | undefined) => (id ? library?.get(id)?.path : undefined);
switch (config.mode) {
case "IMAGE": {
const refs = (config.imageRefIds ?? []).map(pathOf).filter((path): path is string => path !== undefined);
return generate.image({
prompt: config.prompt,
model: config.model,
aspectRatio: config.aspectRatio,
...(refs.length ? { refs } : {}),
});
}
case "VIDEO": {
const startFrame = pathOf(config.startFrameImageId);
const endFrame = pathOf(config.endFrameImageId);
return generate.video({
prompt: config.prompt,
model: config.model,
aspectRatio: config.aspectRatio,
duration: config.duration,
audio: config.generateAudio ?? false,
...(startFrame ? { startFrame } : {}),
...(endFrame ? { endFrame } : {}),
});
}
case "VOICE":
return generate.voice({ prompt: config.prompt, voice: config.voice });
case "AUDIO":
return generate.audio({ prompt: config.prompt, model: config.model });
}
}
const unsetValue = (value: unknown) => value === undefined || value === false;
const sameValue = (a: unknown, b: unknown) =>
(unsetValue(a) && unsetValue(b)) || JSON.stringify(a) === JSON.stringify(b);
const MIDDLE_BUTTON = 1;
const GESTURE_GAP = 250;
const DOUBLE_CLICK_WINDOW = 500;
const CONTROLS = "textarea, input, button, select, a, [role='button'], [role='menuitem'], [role='option']";
const isControl = (target: EventTarget | null) => target instanceof Element && target.closest(CONTROLS) !== null;
const delayedPresses = new WeakSet<PointerEvent>();
type Picking = { entity: Entity; pick(assetId: string): boolean };
const [picking, setPicking] = createSignal<Picking | null>(null);
const [pointerBox, setPointerBox] = createSignal<Entity | null>(null);
const zooming = (event: WheelEvent) => event.ctrlKey || event.metaKey;
function scrollableText(event: WheelEvent): HTMLTextAreaElement | null {
const target = event.target;
if (!(target instanceof HTMLTextAreaElement) || !target.closest("[data-prompt-node]") || zooming(event)) return null;
return Math.abs(event.deltaY) >= Math.abs(event.deltaX) && target.scrollHeight > target.clientHeight ? target : null;
}
function sizeReadout(world: World, mask: SelectionMask, selection: Entity[]) {
if (selection.length !== 1) {
const inverse = invert2D(entityWorldMat(world, null));
inverse.e = 0;
inverse.f = 0;
const size = transformPoint(inverse, mask.width, mask.height);
return { x: mask.mat.e, y: mask.mat.f + mask.height, rotation: 0, center: mask.width / 2, width: size.x, height: size.y };
}
const eid = selection[0]!.id();
const computed = store(world, Computed);
const [left, right] = [...entityQuad(world, selection[0]!)].sort((a, b) => b.y - a.y).slice(0, 2).sort((a, b) => a.x - b.x);
return {
x: left!.x,
y: left!.y,
rotation: Math.atan2(right!.y - left!.y, right!.x - left!.x),
center: Math.hypot(right!.x - left!.x, right!.y - left!.y) / 2,
width: (computed.width[eid] ?? 0) * Math.abs(computed.scaleX[eid] ?? 1),
height: (computed.height[eid] ?? 0) * Math.abs(computed.scaleY[eid] ?? 1),
};
}
function handleQuads(mask: SelectionMask, resolution: number): Quad[] {
const size = Math.round(8 * resolution);
const offset = Math.round(size / 2);
const corners: [number, number][] = [[-offset, -offset], [mask.width - offset, -offset], [mask.width - offset, mask.height - offset], [-offset, mask.height - offset]];
return corners.map(([x, y]) => rectToQuad(multiply2D(mask.mat, translate2D(x, y)), size, size));
}
export function PromptNodes() {
const world = useWorld();
const library = useLibrary();
const cursor = useCursor();
const prompts = useQuery(PromptNode);
const stacked = useDerived(
() => {
const order = new Map<Entity, number>();
world.get(HitRegions)?.list.forEach((region, index) => {
if (region.target.kind === "entity") order.set(region.target.id, index);
});
const layer = (entity: Entity | null): number => (entity ? order.get(entity) ?? layer(getParentEntity(entity)) : order.size);
return [...prompts()].sort((a, b) => layer(a) - layer(b));
},
(a, b) => a.length === b.length && a.every((entity, index) => entity === b[index]),
);
const background = useTrait(world.get(Root)!, Background);
const pressed = useDerived(() => world.get(Pointer)?.phase === "pressed");
const { frame } = useEngineContext();
let layer!: HTMLDivElement;
let snapGuides!: SVGPathElement;
let hoverOutlines!: SVGPathElement;
let outline!: SVGPathElement;
let handles!: SVGPathElement;
let sizeBadge!: HTMLDivElement;
let marquee!: SVGPathElement;
let lastForwardedPress = -Infinity;
let pendingPress: PointerEvent | null = null;
const canvas = () => {
const target = world.get(RenderSurface)?.canvas;
return target instanceof HTMLCanvasElement ? target : undefined;
};
const flushPress = () => {
const target = canvas();
if (!pendingPress || !target) return;
target.dispatchEvent(new PointerEvent("pointerdown", pendingPress));
pendingPress = null;
};
createEffect(() => {
frame();
layer.style.cursor = canvas()?.style.cursor ?? "";
flushPress();
});
createEffect(() => {
frame();
const resolution = world.get(RenderSurface)?.resolution ?? 1;
const line = Math.round(resolution) / resolution;
const at = (x: number, y: number) => `${x / resolution} ${y / resolution}`;
const path = (quad: Quad) => `M${quad.map((point) => at(point.x, point.y)).join("L")}Z`;
const keys = world.get(Keys)?.held;
const cross = Math.round(3 * resolution);
const crossAt = ({ x, y }: Point) => `M${at(x - cross, y - cross)}L${at(x + cross, y + cross)}M${at(x - cross, y + cross)}L${at(x + cross, y - cross)}`;
const snaps = keys?.has("mod") ? [] : world.get(SnapLines)?.list ?? [];
snapGuides.setAttribute("d", snaps.map(({ from, to }) => `M${at(from.x, from.y)}L${at(to.x, to.y)}${crossAt(from)}${crossAt(to)}`).join(""));
snapGuides.setAttribute("stroke-width", `${line}`);
const panning = world.get(Tool)?.value === ToolType.HAND || (keys?.has(" ") ?? false);
const boxed = pointerBox();
const hovered = panning ? [] : boxed?.isAlive() ? [boxed] : world.query(Hovering, Or(Geometry, Group), Not(Sequential), Not(Culled));
hoverOutlines.setAttribute("d", hovered.map((entity) => path(entityQuad(world, entity))).join(""));
hoverOutlines.setAttribute("stroke-width", `${Math.round(2 * resolution) / resolution}`);
const mask = world.get(Hud)?.mode === "moving" ? null : getSelectionMask(world);
const selection = mask ? getMaskSelection(world) : [];
outline.setAttribute("d", mask ? [...selection.map((entity) => entityQuad(world, entity)), rectToQuad(mask.mat, mask.width, mask.height)].map(path).join("") : "");
outline.setAttribute("stroke-width", `${(selection.length === 1 ? 2 : Math.round(resolution)) / resolution}`);
handles.setAttribute("d", mask ? handleQuads(mask, resolution).map(path).join("") : "");
handles.setAttribute("stroke-width", `${line}`);
const marqueeQuad = getMarqueeQuad(world);
marquee.setAttribute("d", marqueeQuad ? path(marqueeQuad) : "");
marquee.setAttribute("stroke-width", `${line}`);
sizeBadge.style.display = mask ? "" : "none";
if (!mask) return;
const { x, y, rotation, center, width, height } = sizeReadout(world, mask, selection);
const text = `${Math.round(width * 100) / 100}x${Math.round(height * 100) / 100}`;
if (sizeBadge.textContent !== text) sizeBadge.textContent = text;
sizeBadge.style.transform = `translate(${x / resolution}px, ${y / resolution}px) rotate(${rotation}rad) translate(${center / resolution}px, 16px) translate(-50%, -9px)`;
});
onMount(() => {
window.addEventListener("pointerup", flushPress, { capture: true });
window.addEventListener("pointercancel", flushPress, { capture: true });
});
onCleanup(() => {
window.removeEventListener("pointerup", flushPress, { capture: true });
window.removeEventListener("pointercancel", flushPress, { capture: true });
});
let lastWheel = -Infinity;
let textOwner: HTMLTextAreaElement | null = null;
const trackGesture = (event: WheelEvent) => {
if (!event.isTrusted) return;
if (event.timeStamp - lastWheel > GESTURE_GAP) textOwner = scrollableText(event);
lastWheel = event.timeStamp;
if (textOwner && event.target !== textOwner && !zooming(event)) event.stopPropagation();
};
onMount(() => window.addEventListener("wheel", trackGesture, { capture: true, passive: true }));
onCleanup(() => window.removeEventListener("wheel", trackGesture, { capture: true }));
const handleWheel = (event: WheelEvent) => {
const target = canvas();
if (!target || (event.target === textOwner && !zooming(event))) return;
event.preventDefault();
target.dispatchEvent(new WheelEvent(event.type, event));
};
const handlePointerDown = (event: PointerEvent) => {
const target = canvas();
if (!target) return;
const pans = event.button === MIDDLE_BUTTON || world.get(Tool)?.value === ToolType.HAND;
if (!pans && (event.button !== 0 || isControl(event.target))) return;
event.preventDefault();
if (!pans && document.activeElement instanceof HTMLElement) document.activeElement.blur();
lastForwardedPress = event.timeStamp;
if (delayedPresses.has(event)) pendingPress = event;
else target.dispatchEvent(new PointerEvent(event.type, event));
};
const forwardDoubleClick = (event: MouseEvent) => {
const target = canvas();
if (!target || !event.isTrusted || event.target === target) return;
if (event.timeStamp - lastForwardedPress > DOUBLE_CLICK_WINDOW) return;
target.dispatchEvent(new MouseEvent(event.type, event));
};
onMount(() => window.addEventListener("dblclick", forwardDoubleClick));
onCleanup(() => window.removeEventListener("dblclick", forwardDoubleClick));
const handlePointerMove = (event: PointerEvent) => {
canvas()?.dispatchEvent(new PointerEvent(event.type, event));
};
const imageAt = (event: PointerEvent): string | undefined => {
const target = canvas();
if (!target) return undefined;
const rect = target.getBoundingClientRect();
const resolution = world.get(RenderSurface)?.resolution ?? 1;
const point = { x: (event.clientX - rect.left) * resolution, y: (event.clientY - rect.top) * resolution };
const regions = world.get(HitRegions)?.list ?? [];
for (let index = regions.length - 1; index >= 0; index--) {
const { target: hit } = regions[index]!;
if (hit.kind !== "entity" || !hit.id.isAlive() || !isPointerInEntity(world, hit.id, point)) continue;
const { source, paint } = resolveMedia(hit.id);
const assetId = paint === PaintType.IMAGE ? source.get(AssetId)?.value : undefined;
return assetId && library()?.get(assetId)?.type === "IMAGE" ? assetId : undefined;
}
return undefined;
};
const handlePick = (event: PointerEvent) => {
const current = picking();
if (!current || event.button !== 0 || world.get(Tool)?.value === ToolType.HAND) return;
event.preventDefault();
event.stopPropagation();
const assetId = imageAt(event);
if (!assetId || !current.pick(assetId)) setPicking(null);
};
const stopPicking = (event: KeyboardEvent) => {
if ((event.key !== "Escape" && event.key !== "Enter") || !picking()) return;
event.preventDefault();
event.stopPropagation();
setPicking(null);
};
onMount(() => window.addEventListener("keydown", stopPicking, { capture: true }));
onCleanup(() => window.removeEventListener("keydown", stopPicking, { capture: true }));
return (
<div
ref={layer}
class="pointer-events-none absolute inset-0 overflow-hidden"
classList={{ "[&_*]:pointer-events-none!": pressed() }}
style={{ "--canvas-background": colorToHex(background()?.value ?? DEFAULT_BACKGROUND) }}
on:wheel={handleWheel}
on:pointerdown={handlePointerDown}
on:pointermove={handlePointerMove}
>
<For each={stacked()}>
{(entity) => <PromptNodeBox entity={entity} />}
</For>
<svg class="absolute inset-0 z-[1] size-full">
<path ref={snapGuides} fill="none" stroke="#F43535" />
<path ref={hoverOutlines} fill="none" stroke="#008CFF" />
<path ref={outline} fill="none" stroke="#008CFF" />
<path ref={handles} fill="#FFFFFF" stroke="#008CFF" />
</svg>
<div
ref={sizeBadge}
class="absolute left-0 top-0 z-[1] h-4 origin-top-left whitespace-nowrap rounded-[3px] bg-[#008CFF] px-1 text-white"
style={{ font: "11px/18px Inter, sans-serif" }}
/>
<svg class="absolute inset-0 z-[1] size-full">
<path ref={marquee} fill="#008CFF" fill-opacity="0.15" stroke="#008CFF" />
</svg>
<div
ref={(element) => cursor.set(element, "pick")}
class="pointer-events-auto absolute inset-0"
classList={{ hidden: !picking() }}
on:pointerdown={handlePick}
/>
</div>
);
}
function AttachMenu(props: { icon: string; label: string; class: string; onPick(): void; onUpload(): void }) {
return (
<DropdownMenu>
<DropdownMenuTrigger as={PromptInputAttachButton} icon={props.icon} label={props.label} class={props.class} />
<DropdownMenuPortal>
<DropdownMenuContent>
<DropdownMenuItem onSelect={props.onPick}>
<Icon name="tool.color-picker" class="size-6 mr-2 text-foreground" />
Pick from canvas
</DropdownMenuItem>
<DropdownMenuItem onSelect={props.onUpload}>
<Icon name="attachment" class="size-6 mr-2 text-foreground" />
Upload from computer
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenuPortal>
</DropdownMenu>
);
}
function PromptNodeBox(props: { entity: Entity }) {
const world = useWorld();
const editor = useEditor();
const library = useLibrary();
const { frame } = useEngineContext();
let box!: HTMLDivElement;
let label!: HTMLDivElement;
let textarea!: HTMLTextAreaElement;
let commitTimer: ReturnType<typeof setTimeout> | undefined;
const authored = useDerived(
() => ({ ...(authoredElement(props.entity)?.props ?? {}) }) as Props,
(a, b) => JSON.stringify(a) === JSON.stringify(b),
);
const selected = useDerived(() => props.entity.has(Selected));
const name = useDerived(() => (getParentEntity(props.entity) === world.get(Root) ? props.entity.get(Name)?.value ?? "" : ""));
const config = createMemo(() => toConfig(authored(), library()));
const [draft, setDraft] = createSignal(config().prompt);
const [settingsVisible, setSettingsVisible] = createSignal(true);
const [isDragging, setIsDragging] = createSignal(false);
createEffect(on(() => config().prompt, (prompt) => {
if (document.activeElement !== textarea) setDraft(prompt);
}));
createEffect(() => {
frame();
const matrix = entityWorldMat(world, props.entity);
const computed = store(world, Computed);
const eid = props.entity.id();
const resolution = world.get(RenderSurface)?.resolution ?? 1;
const scale = PROMPT_SCALE / resolution;
const screenScale = Math.hypot(matrix.a, matrix.b) * scale;
const layoutWidth = (computed.width[eid] ?? PROMPT_SIZE.width) / PROMPT_SCALE;
box.style.width = `${layoutWidth}px`;
box.style.height = `${(computed.height[eid] ?? PROMPT_SIZE.height) / PROMPT_SCALE}px`;
box.style.transform = `matrix(${matrix.a * scale}, ${matrix.b * scale}, ${matrix.c * scale}, ${matrix.d * scale}, ${matrix.e / resolution}, ${matrix.f / resolution})`;
box.style.display = props.entity.has(Hidden) || props.entity.has(Culled) ? "none" : "";
label.style.transform = `scale(${1 / screenScale}) translateY(-22px)`;
label.style.maxWidth = `${layoutWidth * screenScale + 4}px`;
label.style.visibility = getMountedNameInput()?.entity === props.entity ? "hidden" : "";
if (takePromptFocus(props.entity)) textarea.focus({ preventScroll: true });
});
const templateFor = (next: GenerationConfig): AssetRef => {
const generation = toGeneration(next, library());
const previous = templateOf(authored().template);
if (!previous) return generation;
const current = generationOf(previous);
const sameKind = current !== undefined && getAssetSpec(current).type === next.mode.toLowerCase();
return sameKind ? withGeneration(previous, generation) : generation;
};
const write = (next: GenerationConfig) => {
const current = authored();
const template = templateFor(next);
if (!sameValue(templateOf(current.template), template)) editor.editProperty(props.entity, "template", template);
const count = next.mode === "IMAGE" ? next.count : false;
if (!sameValue(current.count, count)) editor.editProperty(props.entity, "count", count);
for (const name of LEGACY_PROPS) {
if (!unsetValue(current[name])) editor.editProperty(props.entity, name, false);
}
};
const patch = (updates: Partial<GenerationConfig>) => {
write({ ...config(), prompt: draft(), ...updates } as GenerationConfig);
};
const commitPrompt = () => {
clearTimeout(commitTimer);
if (draft() === config().prompt) return;
if (templateOf(authored().template)) setPromptText(editor, props.entity, draft());
else write({ ...config(), prompt: draft() } as GenerationConfig);
};
onCleanup(() => clearTimeout(commitTimer));
onCleanup(() => {
if (picking()?.entity === props.entity) setPicking(null);
});
const leaveBox = () => {
if (pointerBox() === props.entity) setPointerBox(null);
};
onCleanup(leaveBox);
createEffect(on(picking, (current, previous) => {
if (!current && previous?.entity === props.entity) textarea.focus({ preventScroll: true });
}, { defer: true }));
const select = () => {
if (!props.entity.has(Selected)) editor.select([props.entity]);
};
const canvasHitsThis = (event: PointerEvent): boolean => {
const surface = world.get(RenderSurface);
if (!(surface?.canvas instanceof HTMLCanvasElement)) return false;
const rect = surface.canvas.getBoundingClientRect();
const point = { x: (event.clientX - rect.left) * surface.resolution, y: (event.clientY - rect.top) * surface.resolution };
const regions = world.get(HitRegions)?.list ?? [];
for (let index = regions.length - 1; index >= 0; index--) {
const { target } = regions[index]!;
if (target.kind === "entity") {
if (target.id.isAlive() && isPointerInEntity(world, target.id, point)) return target.id === props.entity;
} else if (pointInQuad(point.x, point.y, target.quad)) {
return target.entity === props.entity;
}
}
return false;
};
const handlePress = (event: PointerEvent) => {
if (event.button !== 0 || isControl(event.target) || world.get(Tool)?.value === ToolType.HAND) return;
if (props.entity.has(Selected) || canvasHitsThis(event)) return;
editor.select(props.entity, { extend: event.shiftKey });
world.set(Hud, { mode: "moving" });
delayedPresses.add(event);
};
const imageConfig = () => {
const current = config();
return current.mode === "IMAGE" ? current : undefined;
};
const videoConfig = () => {
const current = config();
return current.mode === "VIDEO" ? current : undefined;
};
const voiceConfig = () => {
const current = config();
return current.mode === "VOICE" ? current : undefined;
};
const currentVideoModel = createMemo(() => PROMPT_INPUT_VIDEO_MODEL_OPTIONS.find((model) => model.id === config().model));
const videoDurationOptions = createMemo(() => {
const model = currentVideoModel();
return model ? ALL_DURATION_OPTIONS.filter((option) => model.durations.includes(option.value)) : [];
});
const videoAspectRatioOptions = createMemo(() => {
const model = currentVideoModel();
return model ? ALL_VIDEO_ASPECT_RATIO_OPTIONS.filter((option) => model.aspectRatios.includes(option.value)) : [];
});
const imageRefIds = () => (imageConfig()?.imageRefIds ?? []).slice(0, MAX_IMAGE_REFERENCES);
const frameId = (frame: "start" | "end") => {
if (!currentVideoModel()?.features.includes(`${frame}-frame`)) return undefined;
return frame === "start" ? videoConfig()?.startFrameImageId : videoConfig()?.endFrameImageId;
};
const frameAsset = (frame: "start" | "end") => {
const id = frameId(frame);
return id ? library()?.get(id) : undefined;
};
const handleModeChange = (value: string) => {
const mode = value as PromptInputMode;
if (mode === config().mode) return;
write(createDefaultConfig(mode, draft()));
};
const handleVideoModelChange = (id: string) => {
const model = PROMPT_INPUT_VIDEO_MODEL_OPTIONS.find((option) => option.id === id);
const current = videoConfig();
if (!model || !current) return;
write({
...current,
prompt: draft(),
model: id,
endFrameImageId: model.features.includes("end-frame") ? current.endFrameImageId : undefined,
generateAudio: model.features.includes("audio") ? current.generateAudio : false,
duration: model.durations.includes(`${current.duration}s`) ? current.duration : parseInt(model.durations[0], 10),
aspectRatio: model.aspectRatios.includes(current.aspectRatio) ? current.aspectRatio : (model.aspectRatios[0] as AspectRatio),
});
};
const addImages = (ids: string[]) => {
const current = videoConfig();
if (current) {
const [first, second] = ids;
patch({
startFrameImageId: current.startFrameImageId ?? first,
endFrameImageId: current.endFrameImageId ?? (current.startFrameImageId ? first : second),
});
return;
}
if (imageConfig()) patch({ imageRefIds: [...imageRefIds(), ...ids].slice(0, MAX_IMAGE_REFERENCES) });
};
const pickImages = async (multiple = true) => {
const lib = library();
if (!lib) return [];
const files = await pickFiles({ accept: "image/*", multiple });
if (files.length === 0) return [];
return (await importFiles(lib, files, "")).filter((asset) => asset.type === "IMAGE").map((asset) => asset.id);
};
const openImageReferencesPicker = async () => {
addImages(await pickImages());
};
const openVideoFramePicker = async (frame: "start" | "end") => {
const [id] = await pickImages(false);
if (id) patch(frame === "start" ? { startFrameImageId: id } : { endFrameImageId: id });
};
const pickReferencesFromCanvas = () => setPicking({
entity: props.entity,
pick: (assetId) => {
if (!imageConfig()) return false;
const refs = imageRefIds();
const next = refs.includes(assetId) ? refs : [...refs, assetId];
patch({ imageRefIds: next });
return next.length < MAX_IMAGE_REFERENCES;
},
});
const pickFrameFromCanvas = (frame: "start" | "end") => setPicking({
entity: props.entity,
pick: (assetId) => {
if (videoConfig()) patch(frame === "start" ? { startFrameImageId: assetId } : { endFrameImageId: assetId });
return false;
},
});
const handleDrop = async (event: DragEvent) => {
event.preventDefault();
event.stopPropagation();
setIsDragging(false);
const lib = library();
if (!lib || (config().mode !== "IMAGE" && config().mode !== "VIDEO")) return;
const assetIds = event.dataTransfer?.getData(ASSET_DRAG_TYPE)?.split(",").filter(Boolean) ?? [];
const files = droppedFiles(event).filter((file) => file.type.startsWith("image/"));
const ids = assetIds.length > 0 ? assetIds : (await importFiles(lib, files, "")).map((asset) => asset.id);
addImages(ids.filter((id) => lib.get(id)?.type === "IMAGE"));
};
const handleDragOver = (event: DragEvent) => {
event.preventDefault();
event.stopPropagation();
setIsDragging(true);
};
const editDraft = (text: string) => {
setDraft(text);
clearTimeout(commitTimer);
commitTimer = setTimeout(commitPrompt, COMMIT_DELAY);
};
const history = createPromptHistory(editDraft);
const handleInput = (event: InputEvent & { currentTarget: HTMLTextAreaElement }) => {
history.reset();
editDraft(event.currentTarget.value);
};
const handleSubmit = () => {
commitPrompt();
const next = { ...config(), prompt: draft().trim() } as GenerationConfig;
if (!next.prompt) return;
rememberPrompt(next.prompt);
history.reset();
if (next.mode === "IMAGE") next.imageRefIds = imageRefIds();
if (next.mode === "VIDEO") {
next.startFrameImageId = frameId("start");
next.endFrameImageId = frameId("end");
}
const template = templateFor(next);
const generation = generationOf(template);
if (!generation) return;
const spec = getAssetSpec(generation) as GenerateSpec;
const count = next.mode === "IMAGE" ? next.count : 1;
const sources = Array.from({ length: count }, () => withGeneration(template, generateRef({ ...spec, seed: randomSeed() })));
const bounds = getEntityBounds(world, [props.entity]);
const near: AABB | undefined = bounds
? { minX: bounds.x, minY: bounds.y, maxX: bounds.x + bounds.width, maxY: bounds.y + bounds.height }
: undefined;
if (spec.type === "image" || spec.type === "video") {
const size = ASPECT_RATIO_DIMENSIONS[spec.aspectRatio ?? "16:9"] ?? { width: 1920, height: 1080 };
insertGenerated(world, editor, size, (box, index) => (
<Rect keepAspectRatio x={box.x} y={box.y} width={box.width} height={box.height}>
{spec.type === "image" ? <ImagePaint src={sources[index]} /> : <VideoPaint src={sources[index]} />}
</Rect>
), count, sources, near);
return;
}
insertGenerated(world, editor, AUDIO_SIZE, (box, index) => (
<Audio src={sources[index]} x={box.x} y={box.y} width={box.width} height={box.height} />
), count, sources, near);
};
const handleKeyDown = (event: KeyboardEvent) => {
if (history.handleKeyDown(event)) return;
if (event.key === "Enter" && (event.metaKey || event.ctrlKey)) {
event.preventDefault();
handleSubmit();
} else if (event.key === "Escape") {
textarea.blur();
}
};
return (
<div
ref={box}
data-prompt-node
class="pointer-events-auto absolute left-0 top-0 z-[1] flex origin-top-left flex-col gap-2 rounded-xl border border-border p-2 [background:linear-gradient(var(--input),var(--input)),var(--canvas-background)]"
on:pointerdown={handlePress}
on:pointerenter={() => setPointerBox(props.entity)}
on:pointerleave={leaveBox}
>
<div
ref={label}
class="absolute -left-px -top-px h-[22px] w-max origin-top-left truncate pt-[3px] pr-1"
classList={{ hidden: !name() }}
style={{ font: "350 11px/11px Inter, sans-serif", color: selected() ? "#cce8ff" : "rgba(242, 242, 242, 0.64)" }}
>
{name()}
</div>
<Switch>
<Match when={imageConfig()}>
<div class="pointer-events-auto flex w-fit max-w-full items-start gap-2 overflow-x-auto">
<For each={imageRefIds()}>
{(assetId) => (
<Show when={library()?.get(assetId)}>
{(asset) => (
<PromptInputReferenceImageButton
asset={asset()}
cache={library()!.cache}
class="size-16"
size={{ width: 64, height: 64 }}
onRemove={() => patch({ imageRefIds: imageRefIds().filter((id) => id !== assetId) })}
/>
)}
</Show>
)}
</For>
<Show when={imageRefIds().length < MAX_IMAGE_REFERENCES}>
<AttachMenu
icon="attachment"
label="Image references"
class="w-36"
onPick={pickReferencesFromCanvas}
onUpload={openImageReferencesPicker}
/>
</Show>
</div>
</Match>
<Match when={videoConfig()}>
<div class="pointer-events-auto flex w-fit max-w-full items-center gap-2 overflow-x-auto">
<For each={["start", "end"] as const}>
{(frame) => (
<Show when={currentVideoModel()?.features.includes(`${frame}-frame`)}>
<Show
when={frameAsset(frame)}
fallback={
<AttachMenu
icon="image"
label={frame === "start" ? "Start frame" : "End frame"}
class="w-28"
onPick={() => pickFrameFromCanvas(frame)}
onUpload={() => openVideoFramePicker(frame)}
/>
}
>
{(asset) => (
<PromptInputReferenceImageButton
asset={asset()}
cache={library()!.cache}
class="h-16 w-28"
size={{ width: 112, height: 64 }}
onRemove={() => patch(frame === "start" ? { startFrameImageId: undefined } : { endFrameImageId: undefined })}
/>
)}
</Show>
</Show>
)}
</For>
</div>
</Match>
</Switch>
<div class="flex min-h-14 w-full flex-1 p-1">
<textarea
ref={textarea}
class="pointer-events-auto min-h-8 size-full resize-none overflow-y-auto overscroll-contain bg-transparent px-1 py-1 text-xs text-foreground placeholder:text-muted-foreground outline-none"
placeholder="Describe what you want to create."
value={draft()}
onInput={handleInput}
onBlur={commitPrompt}
onFocus={select}
onKeyDown={handleKeyDown}
onPointerDown={history.reset}
onDragOver={handleDragOver}
onDragLeave={() => setIsDragging(false)}
onDrop={handleDrop}
/>
</div>
<div class="flex min-h-4 w-full items-center justify-between">
<div class="pointer-events-auto flex min-w-0 items-center gap-1 overflow-x-auto pr-1">
<PromptInputCompactMenu
aria-label="Select prompt mode"
menuLabel="Generate"
value={() => config().mode}
options={PROMPT_INPUT_MODE_OPTIONS}
onChange={handleModeChange}
/>
<Tooltip>
<TooltipTrigger
as={Button}
variant="ghost"
size="icon-square"
class="text-muted-foreground"
onClick={() => setSettingsVisible(!settingsVisible())}
>
<Icon name="preferences-adjust" class="size-6" />
</TooltipTrigger>
<TooltipContent>Settings</TooltipContent>
</Tooltip>
<Show when={settingsVisible()}>
<Separator orientation="vertical" class="min-h-5" />
<Switch>
<Match when={imageConfig()}>
{(image) => (
<>
<ModelMenu
searchPlaceholder="Search in image models"
options={PROMPT_INPUT_IMAGE_MODEL_OPTIONS}
value={() => image().model}
onChange={(model) => patch({ model })}
/>
<PromptInputCompactMenu
aria-label="Select aspect ratio"
menuLabel="Aspect ratio"
value={() => image().aspectRatio}
options={PROMPT_INPUT_IMAGE_ASPECT_RATIO_OPTIONS}
onChange={(value) => patch({ aspectRatio: value as AspectRatio })}
triggerIcon="aspect-ratio-16-9"
/>
<PromptInputCompactMenu
aria-label="Select amount of variants"
menuLabel="Amount of variants"
value={() => String(image().count)}
options={PROMPT_INPUT_VARIANT_COUNT_OPTIONS}
onChange={(value) => patch({ count: Number(value) })}
triggerIcon="variants"
/>
</>
)}
</Match>
<Match when={videoConfig()}>
{(video) => (
<>
<ModelMenu
searchPlaceholder="Search in video models"
options={PROMPT_INPUT_VIDEO_MODEL_OPTIONS}
value={() => video().model}
onChange={handleVideoModelChange}
/>
<PromptInputCompactMenu
aria-label="Select aspect ratio"
menuLabel="Aspect ratio"
value={() => video().aspectRatio}
options={videoAspectRatioOptions()}
onChange={(value) => patch({ aspectRatio: value as AspectRatio })}
triggerIcon="aspect-ratio-16-9"
/>
<PromptInputCompactMenu
aria-label="Select duration"
menuLabel="Duration"
value={() => `${video().duration}s`}
options={videoDurationOptions()}
onChange={(value) => patch({ duration: parseInt(value, 10) })}
triggerIcon="duration"
/>
<Show when={currentVideoModel()?.features.includes("audio")}>
<Button
variant="ghost"
onClick={() => patch({ generateAudio: !video().generateAudio })}
class={DEFAULT_BUTTON_CLASS}
>
<Icon name={video().generateAudio ? "audio-on" : "audio-off"} class="size-6" />
{video().generateAudio ? "On" : "Off"}
</Button>
</Show>
</>
)}
</Match>
<Match when={voiceConfig()}>
{(voice) => (
<VoiceMenu
options={PROMPT_INPUT_VOICE_OPTIONS}
value={() => voice().voice}
onChange={(value) => patch({ voice: value })}
/>
)}
</Match>
<Match when={config().mode === "AUDIO"}>
<ModelMenu
searchPlaceholder="Search in audio models"
options={PROMPT_INPUT_AUDIO_MODEL_OPTIONS}
value={() => config().model}
onChange={(model) => patch({ model })}
/>
</Match>
</Switch>
</Show>
</div>
<Tooltip>
<TooltipTrigger
as={Button}
variant="default"
size="icon-square"
class="pointer-events-auto"
disabled={!draft().trim()}
onClick={handleSubmit}
>
<Icon name="arrow-right" class="-rotate-90 size-6" />
</TooltipTrigger>
<TooltipContent shortcut="⌘↵">Generate</TooltipContent>
</Tooltip>
</div>
<Show when={isDragging()}>
<div class="absolute inset-0 z-20 overflow-hidden rounded-xl border border-primary bg-background p-2">
<div class="absolute inset-0 rounded-xl bg-accent/40" />
<div class="flex size-full items-center justify-center rounded-md border border-dashed border-border-input">
<Icon name="attachment" class="size-6 text-muted-foreground" />
<span class="text-xs font-450 text-muted-foreground">Drop images here</span>
</div>
</div>
</Show>
</div>
);
}

View File

@ -0,0 +1,19 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
export type ContextMenuExtra =
| { label: string; shortcut?: string; checked?: boolean; items?: ContextMenuExtra[]; onSelect?(): void }
| { separator: true };
let offered: { event: Event; items: ContextMenuExtra[] } | null = null;
export function offerContextMenuItems(event: Event, items: ContextMenuExtra[]): void {
offered = { event, items };
}
export function takeContextMenuItems(event: Event): ContextMenuExtra[] {
const items = offered?.event === event ? offered.items : [];
offered = null;
return items;
}

View File

@ -5,6 +5,7 @@
import { Button } from "@/components/ui/button";
import { Icon } from "@/components/ui/icon";
import { Separator } from "@/components/ui/separator";
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip";
import {
DropdownMenu,
DropdownMenuContent,
@ -14,16 +15,9 @@ import {
DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu";
import { createMemo, Show } from "solid-js";
import { useGenerationRecords } from "./use-generation-records";
import { useGenerateImage } from "./use-generate-image";
import { useGenerateVideo } from "./use-generate-video";
import { useGenerateVoice } from "./use-generate-voice";
import { useGenerateAudio } from "./use-generate-audio";
import { useAutoCaptions } from "./use-auto-captions";
import { useMediaSelection } from "./selection";
import { useTransforms } from "./use-transforms";
import { createDefaultConfig } from "./prompt-input";
import { toast } from "somoto";
import { useWorld } from "@diffusionstudio/koota-solid";
import { tidySelection } from "@/engine/align";
import { useObjectActions } from "./use-object-actions";
import type { GenerationConfig } from "./schemas";
@ -31,68 +25,30 @@ interface ActionBarProps {
openPromptInput?(config: GenerationConfig): void;
}
function TidyUpButton() {
const world = useWorld();
return (
<Tooltip>
<TooltipTrigger as={Button} variant="ghost" class="gap-0 pl-0.5 text-muted-foreground" onClick={() => tidySelection(world)}>
<Icon name="view.grid" />
Tidy up
</TooltipTrigger>
<TooltipContent shortcut="⌃⌥T">Tidy up</TooltipContent>
</Tooltip>
);
}
export function ActionBar(props: ActionBarProps) {
const { imageNodes, videoNodes } = useMediaSelection();
const { isOn, toggle } = useTransforms();
const { generate: generateImage } = useGenerateImage();
const { generate: generateVideo } = useGenerateVideo();
const { generate: generateVoice } = useGenerateVoice();
const { generate: generateAudio } = useGenerateAudio();
const { generate: autoCaptions, hasScene } = useAutoCaptions();
const { isGenerated, totalCredits, firstConfig } = useGenerationRecords();
const isImage = createMemo(() => imageNodes().length > 0);
const isVideo = createMemo(() => videoNodes().length > 0);
const {
isImage, isVideo, hasScene, canTidy, isGenerated, totalCredits, isOn, toggle,
autoCaptions, rerun, editWithPrompt, makeVideo, reuse, fork,
} = useObjectActions(props.openPromptInput);
const visible = createMemo(() => {
return isImage() || isVideo() || hasScene();
return isImage() || isVideo() || hasScene() || canTidy();
})
const handleRerun = () => {
const config = firstConfig();
if (!config) {
toast("No generation config found", { description: "This asset wasn't generated with a prompt." });
return;
}
const promise = (() => {
switch (config.mode) {
case "IMAGE":
return generateImage(config);
case "VIDEO":
return generateVideo(config);
case "VOICE":
return generateVoice(config);
case "AUDIO":
return generateAudio(config);
}
})();
promise.catch((err) => {
toast("Rerun failed", {
description: err instanceof Error ? err.message : String(err),
});
});
};
const handleEditWithPrompt = () => {
props.openPromptInput?.(createDefaultConfig("IMAGE"));
};
const handleMakeVideo = () => {
props.openPromptInput?.(createDefaultConfig("VIDEO"));
};
const handleReuse = () => {
const config = firstConfig();
if (!config) {
toast("No generation config found", { description: "This asset wasn't generated with a prompt." });
return;
}
props.openPromptInput?.(config);
};
return (
<>
<Show when={visible()}>
@ -126,10 +82,13 @@ export function ActionBar(props: ActionBarProps) {
</div>
<Separator orientation="vertical" class="min-h-5" />
<div class="flex gap-1 items-center">
<Button variant="ghost" class="gap-0 pl-0.5 text-muted-foreground" onClick={handleEditWithPrompt}>
<Button variant="ghost" class="gap-0 pl-0.5 text-muted-foreground" onClick={editWithPrompt}>
<Icon name="ai-generate" />
Edit with prompt
</Button>
<Show when={canTidy()}>
<TidyUpButton />
</Show>
<Show when={isGenerated()}>
<DropdownMenu placement="right">
<DropdownMenuTrigger<typeof Button>
@ -148,18 +107,22 @@ export function ActionBar(props: ActionBarProps) {
</div>
<Separator class="my-1" />
<DropdownMenuGroup>
<DropdownMenuItem onSelect={handleMakeVideo}>
<DropdownMenuItem onSelect={makeVideo}>
<Icon name="film-video-export" class="size-6 mr-2 text-foreground" />
Make video
</DropdownMenuItem>
<DropdownMenuItem onSelect={handleRerun}>
<DropdownMenuItem onSelect={rerun}>
<Icon name="rerun" class="size-6 mr-2 text-foreground" />
Rerun
</DropdownMenuItem>
<DropdownMenuItem onSelect={handleReuse}>
<DropdownMenuItem onSelect={reuse}>
<Icon name="reuse-settings" class="size-6 mr-2 text-foreground" />
Reuse
</DropdownMenuItem>
<DropdownMenuItem onSelect={fork}>
<Icon name="vary" class="size-6 mr-2 text-foreground" />
Fork
</DropdownMenuItem>
</DropdownMenuGroup>
</DropdownMenuContent>
</DropdownMenuPortal>
@ -206,10 +169,14 @@ export function ActionBar(props: ActionBarProps) {
</div>
<Separator class="my-1" />
<DropdownMenuGroup>
<DropdownMenuItem onSelect={handleRerun}>
<DropdownMenuItem onSelect={rerun}>
<Icon name="rerun" class="size-6 mr-2 text-foreground" />
Rerun
</DropdownMenuItem>
<DropdownMenuItem onSelect={fork}>
<Icon name="vary" class="size-6 mr-2 text-foreground" />
Fork
</DropdownMenuItem>
</DropdownMenuGroup>
</DropdownMenuContent>
</DropdownMenuPortal>
@ -217,6 +184,12 @@ export function ActionBar(props: ActionBarProps) {
</Show>
</div>
</Show>
<Show when={canTidy() && !isImage()}>
<Show when={isVideo() || hasScene()}>
<Separator orientation="vertical" class="min-h-5" />
</Show>
<TidyUpButton />
</Show>
</div>
</Show>
</>

View File

@ -8,6 +8,7 @@ import { findEmptyPlacement } from "@/engine/placement";
import { toast } from "somoto";
import type { AssetRef } from "@diffusionstudio/jsx";
import type { AABB } from "@diffusionstudio/runtime";
import type { DocumentEditor } from "@/engine/editor";
import type { Entity, World } from "koota";
@ -44,13 +45,14 @@ export function insertGenerated(
element: (box: GenerationBox, index: number) => unknown,
count = 1,
sources: AssetRef[] = [],
near?: AABB,
): Entity[] {
const root = world.get(Root);
const inserted: Entity[] = [];
if (root) {
const totalWidth = count * size.width + (count - 1) * GAP;
const center = findEmptyPlacement(world, totalWidth, size.height, GAP);
const center = findEmptyPlacement(world, totalWidth, size.height, GAP, near);
const left = center.x - totalWidth / 2;
const y = Math.round(center.y - size.height / 2);

View File

@ -0,0 +1,63 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { createStoredSignal } from "@/lib/store";
import { store } from "@/init";
const MAX_RECENT_PROMPTS = 10;
const [recentPrompts, setRecentPrompts] = createStoredSignal(
store.define<string[]>("prompt-input.recent-prompts", []),
);
export { recentPrompts };
export function rememberPrompt(prompt: string): void {
const text = prompt.trim();
if (!text) return;
setRecentPrompts([text, ...recentPrompts().filter((recent) => recent !== text)].slice(0, MAX_RECENT_PROMPTS));
}
export function clearRecentPrompts(): void {
setRecentPrompts([]);
}
const MODIFIER_KEYS = new Set(["Shift", "Meta", "Alt", "Control", "CapsLock"]);
export function createPromptHistory(write: (text: string) => void) {
let index = -1;
let shown = "";
const reset = () => {
index = -1;
};
const handleKeyDown = (event: KeyboardEvent): boolean => {
const arrow = event.key === "ArrowUp" || event.key === "ArrowDown";
if (!arrow || event.altKey || event.metaKey || event.ctrlKey || event.shiftKey) {
if (!MODIFIER_KEYS.has(event.key)) reset();
return false;
}
const textarea = event.target as HTMLTextAreaElement;
const { selectionStart, selectionEnd, value } = textarea;
const browsing = index >= 0 && selectionStart === selectionEnd && selectionStart === value.length && value === shown;
if (!browsing) reset();
if (!browsing && value.trim()) return false;
event.preventDefault();
const history = recentPrompts();
const next = index + (event.key === "ArrowUp" ? 1 : -1);
if (next < -1 || next >= history.length) return true;
index = next;
shown = index === -1 ? "" : history[index]!;
write(shown);
textarea.value = shown;
textarea.setSelectionRange(shown.length, shown.length);
return true;
};
return { handleKeyDown, reset };
}

View File

@ -11,11 +11,15 @@ import {
DropdownMenuGroup,
DropdownMenuItem,
DropdownMenuPortal,
DropdownMenuShortcut,
DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu";
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip";
import { createMemo, Show } from "solid-js";
import { toast } from "somoto";
import { useWorld } from "@diffusionstudio/koota-solid";
import { tidySelection } from "@/engine/align";
import { useSelection } from "@/engine/hooks";
import { useGenerationRecords } from "./use-generation-records";
import { useGenerateImage } from "./use-generate-image";
@ -28,6 +32,8 @@ import { useTransforms } from "./use-transforms";
import type { TransformType } from "@diffusionstudio/jsx";
export function PromptInputActions() {
const world = useWorld();
const { nodes } = useSelection();
const { isGenerated, totalCredits, firstConfig } = useGenerationRecords();
const { imageNodes, videoNodes } = useMediaSelection();
const { isOn, toggle } = useTransforms();
@ -39,6 +45,7 @@ export function PromptInputActions() {
const hasImageSelection = createMemo(() => imageNodes().length > 0);
const hasVideoSelection = createMemo(() => videoNodes().length > 0);
const canTidy = createMemo(() => nodes().length > 1);
const handleRerun = () => {
const config = firstConfig();
@ -120,6 +127,13 @@ export function PromptInputActions() {
<Separator class="my-1" />
</Show>
<DropdownMenuGroup>
<Show when={canTidy()}>
<DropdownMenuItem onSelect={() => tidySelection(world)}>
<Icon name="view.grid" class="mr-2 text-foreground" />
<span class="flex-1">Tidy up</span>
<DropdownMenuShortcut>⌃⌥T</DropdownMenuShortcut>
</DropdownMenuItem>
</Show>
<Show when={hasImageSelection() || hasVideoSelection()}>
<TransformItem name="upscale" icon="arrow-scale" label="Upscale" isOn={isOn} toggle={toggle} />
</Show>

View File

@ -26,7 +26,9 @@ import {
createMemo,
createSignal,
onCleanup,
splitProps,
type Accessor,
type JSX,
} from "solid-js";
import {
@ -44,6 +46,7 @@ import {
} from "./config";
import { createStoredSignal } from "@/lib/store";
import { store } from "@/init";
import { clearRecentPrompts, createPromptHistory, recentPrompts, rememberPrompt } from "./prompt-history";
import { toast } from "somoto";
import { useGenerateImage } from "./use-generate-image";
import { useGenerateVideo } from "./use-generate-video";
@ -70,8 +73,8 @@ import type { ThumbnailAsset } from "@/components/ui/asset-thumbnail";
export type PromptInputMode = "IMAGE" | "VIDEO" | "VOICE" | "AUDIO";
const DEFAULT_BUTTON_CLASS = "text-muted-foreground gap-0 pr-2 pl-0";
const MAX_IMAGE_REFERENCES = 5;
export const DEFAULT_BUTTON_CLASS = "text-muted-foreground gap-0 pr-2 pl-0";
export const MAX_IMAGE_REFERENCES = 5;
type GenerationConfgs = {
IMAGE: ImageGenerationConfig;
@ -204,9 +207,6 @@ export function PromptInput(props: PromptInputProps) {
const effectiveStartFrameId = createMemo(() => frameId("start"));
const effectiveEndFrameId = createMemo(() => frameId("end"));
const [recentPrompts, setRecentPrompts] = createStoredSignal(
store.define<string[]>("prompt-input.recent-prompts", []),
);
const [slashMenuDismissed, setSlashMenuDismissed] = createSignal(false);
const [slashMenuIndex, setSlashMenuIndex] = createSignal(-1);
const [settingsVisible, setSettingsVisible] = createStoredSignal(
@ -217,7 +217,13 @@ export function PromptInput(props: PromptInputProps) {
const showSlashMenu = () =>
prompt().startsWith("/") && !slashMenuDismissed() && recentPrompts().length > 0;
const history = createPromptHistory((text) => {
patch({ prompt: text });
resizeTextarea();
});
const handlePromptInput = (event: InputEvent & { currentTarget: HTMLTextAreaElement }) => {
history.reset();
patch({ prompt: event.currentTarget.value });
setSlashMenuDismissed(false);
setSlashMenuIndex(-1);
@ -300,13 +306,14 @@ export function PromptInput(props: PromptInputProps) {
};
const handleSelectRecentPrompt = (prompt: string) => {
history.reset();
patch({ prompt });
setSlashMenuDismissed(true);
resizeTextarea();
};
const handleClearHistory = () => {
setRecentPrompts([]);
clearRecentPrompts();
};
const handleSubmit = async () => {
@ -314,10 +321,8 @@ export function PromptInput(props: PromptInputProps) {
const currentConfig = { ...config(), prompt: prompt().trim() };
// Add to recent prompts
const filtered = recentPrompts().filter((p) => p !== currentConfig.prompt);
setRecentPrompts([currentConfig.prompt, ...filtered].slice(0, 10));
rememberPrompt(currentConfig.prompt);
history.reset();
patch({ prompt: "" });
resizeTextarea();
@ -366,7 +371,8 @@ export function PromptInput(props: PromptInputProps) {
}
return;
}
if (e.key === "Enter" && !e.shiftKey) {
if (history.handleKeyDown(e)) return;
if (e.key === "Enter" && (e.metaKey || e.ctrlKey)) {
e.preventDefault();
handleSubmit();
}
@ -604,6 +610,7 @@ export function PromptInput(props: PromptInputProps) {
value={prompt()}
onInput={handlePromptInput}
onKeyDown={handleKeyDown}
onPointerDown={history.reset}
/>
</div>
<div class="flex min-h-4 w-full items-center justify-between">
@ -731,7 +738,7 @@ export function PromptInput(props: PromptInputProps) {
>
<Icon name="arrow-right" class="-rotate-90 size-6" />
</TooltipTrigger>
<TooltipContent shortcut="↵">Generate</TooltipContent>
<TooltipContent shortcut="⌘↵">Generate</TooltipContent>
</Tooltip>
</div>
<Show when={isDragging()}>
@ -756,7 +763,7 @@ type PromptInputCompactMenuProps = {
triggerIcon?: string;
}
function PromptInputCompactMenu(props: PromptInputCompactMenuProps) {
export function PromptInputCompactMenu(props: PromptInputCompactMenuProps) {
const selectedOption = () =>
props.options.find((option) => option.value === props.value()) ?? props.options[0];
const selectedIcon = () => selectedOption()?.icon ?? props.triggerIcon ?? "chevron-down";
@ -824,6 +831,25 @@ function PromptInputCompactMenu(props: PromptInputCompactMenuProps) {
);
}
function focusSearch(event: Event, search: HTMLInputElement): void {
event.preventDefault();
setTimeout(() => search.focus());
}
function moveActive(event: KeyboardEvent, list: HTMLElement, count: number, active: number, setActive: (index: number) => void, choose: (index: number) => void): void {
if (count === 0) return;
if (event.key === "Enter") {
event.preventDefault();
choose(active);
return;
}
if (event.key !== "ArrowDown" && event.key !== "ArrowUp") return;
event.preventDefault();
const next = (active + (event.key === "ArrowDown" ? 1 : count - 1)) % count;
setActive(next);
list.children[next]?.scrollIntoView({ block: "nearest" });
}
type ModelMenuProps = {
searchPlaceholder: string;
options: { id: string; name: string; description: string; icon: string }[];
@ -831,8 +857,12 @@ type ModelMenuProps = {
onChange(value: string): void;
}
function ModelMenu(props: ModelMenuProps) {
export function ModelMenu(props: ModelMenuProps) {
const [open, setOpen] = createSignal(false);
const [query, setQuery] = createSignal("");
const [active, setActive] = createSignal(0);
let search!: HTMLInputElement;
let list!: HTMLDivElement;
const filteredOptions = createMemo(() => {
const q = query().trim().toLowerCase();
@ -842,8 +872,15 @@ function ModelMenu(props: ModelMenuProps) {
);
});
const handleOpenChange = (open: boolean) => {
if (!open) setQuery("");
const handleOpenChange = (isOpen: boolean) => {
setOpen(isOpen);
setActive(0);
if (!isOpen) setQuery("");
};
const select = (id: string) => {
props.onChange(id);
setOpen(false);
};
const icon = createMemo(() => {
@ -857,8 +894,8 @@ function ModelMenu(props: ModelMenuProps) {
});
return (
<DropdownMenu placement="top-start" onOpenChange={handleOpenChange}>
<DropdownMenuTrigger<typeof Button>
<Popover placement="top-start" open={open()} onOpenChange={handleOpenChange}>
<PopoverTrigger<typeof Button>
as={(triggerProps) => (
<Button
{...triggerProps}
@ -870,26 +907,36 @@ function ModelMenu(props: ModelMenuProps) {
</Button>
)}
/>
<DropdownMenuPortal>
<DropdownMenuContent class="w-[340px] p-0">
<PopoverPortal>
<PopoverContent
class="z-[10000] flex max-h-[var(--kb-popper-content-available-height)] w-[340px] min-w-[8rem] flex-col gap-2 overflow-x-hidden overflow-y-auto rounded-xl border-border p-0 shadow-[0px_0px_1px_2px_rgba(0,0,0,0.12),0px_4px_12px_8px_rgba(0,0,0,0.12),0px_12px_16px_0px_rgba(0,0,0,0.16)] outline-none"
onOpenAutoFocus={(event: Event) => focusSearch(event, search)}
>
<SearchInput
ref={search}
placeholder={props.searchPlaceholder}
value={query()}
onValue={setQuery}
onValue={(value) => {
setQuery(value);
setActive(0);
}}
onKeyDown={(event) => moveActive(event, list, filteredOptions().length, active(), setActive, (index) => select(filteredOptions()[index]!.id))}
/>
<div class="flex flex-col gap-2 px-2 py-1">
<div ref={list} class="flex flex-col gap-2 px-2 py-1">
<For each={filteredOptions()}>
{(option) => {
{(option, index) => {
const selected = props.value() === option.id;
return (
<DropdownMenuItem
tone="neutral"
class="h-[42px] gap-2 rounded-md px-1 py-1 group"
onSelect={() => props.onChange(option.id)}
<div
class="group relative flex h-[42px] min-h-7 w-full cursor-default select-none items-center gap-2 rounded-md bg-popover px-1 py-1 text-xs outline-hidden [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4 [&_svg:not([class*='text-'])]:text-muted-foreground data-[highlighted]:bg-input data-[highlighted]:text-foreground data-[highlighted]:[&_svg]:text-foreground"
classList={{
"data-highlighted:bg-muted": selected,
}}
data-highlighted={index() === active() ? "" : undefined}
onPointerMove={() => setActive(index())}
onMouseDown={(event) => event.preventDefault()}
onClick={() => select(option.id)}
>
<div class="grid size-8 shrink-0 place-items-center overflow-hidden rounded-sm" >
<Icon name={option.icon!} class="size-6 text-muted-foreground" />
@ -903,14 +950,14 @@ function ModelMenu(props: ModelMenuProps) {
<Icon name="confirm-check" class="size-6 text-foreground" />
</Show>
</span>
</DropdownMenuItem>
</div>
)
}}
</For>
</div>
</DropdownMenuContent>
</DropdownMenuPortal>
</DropdownMenu>
</PopoverContent>
</PopoverPortal>
</Popover>
);
}
@ -921,12 +968,15 @@ type VoiceMenuProps = {
onChange(value: string): void;
}
function VoiceMenu(props: VoiceMenuProps) {
export function VoiceMenu(props: VoiceMenuProps) {
const [open, setOpen] = createSignal(false);
const [query, setQuery] = createSignal("");
const [active, setActive] = createSignal(0);
const [playingVoice, setPlayingVoice] = createSignal<string | null>(null);
let audioRef: HTMLAudioElement | undefined;
let search!: HTMLInputElement;
let list!: HTMLDivElement;
const selectedLabel = () =>
props.options.find((o) => o.value === props.value())?.label ?? props.value();
@ -971,6 +1021,7 @@ function VoiceMenu(props: VoiceMenuProps) {
const handleOpenChange = (isOpen: boolean) => {
setOpen(isOpen);
setActive(0);
if (!isOpen) {
setQuery("");
stopPlayback();
@ -994,23 +1045,31 @@ function VoiceMenu(props: VoiceMenuProps) {
)}
/>
<PopoverPortal>
<PopoverContent class="w-[340px] p-0 rounded-xl">
<PopoverContent class="w-[340px] p-0 rounded-xl" onOpenAutoFocus={(event: Event) => focusSearch(event, search)}>
<SearchInput
ref={search}
placeholder="Search in voices"
value={query()}
onValue={setQuery}
onValue={(value) => {
setQuery(value);
setActive(0);
}}
onKeyDown={(event) => moveActive(event, list, filteredOptions().length, active(), setActive, (index) => selectVoice(filteredOptions()[index]!.value))}
/>
<div class="flex flex-col gap-2 p-2 max-h-[320px] overflow-y-auto">
<div ref={list} class="flex flex-col gap-2 p-2 max-h-[320px] overflow-y-auto">
<For each={filteredOptions()}>
{(option) => {
{(option, index) => {
const selected = () => props.value() === option.value;
const isPlaying = () => playingVoice() === option.value;
return (
<button
type="button"
class="flex items-center gap-2 rounded-md px-1 py-1 h-[42px] w-full text-left group hover:bg-accent transition-colors"
class="flex items-center gap-2 rounded-md px-1 py-1 h-[42px] w-full text-left group data-[highlighted]:bg-accent transition-colors"
classList={{ "bg-muted": selected() }}
data-highlighted={index() === active() ? "" : undefined}
onPointerMove={() => setActive(index())}
onMouseDown={(event) => event.preventDefault()}
onClick={() => selectVoice(option.value)}
>
<div
@ -1033,7 +1092,7 @@ function VoiceMenu(props: VoiceMenuProps) {
</div>
<div class="min-w-0 flex-1 text-muted-foreground">
<div
class="truncate text-base font-450 group-hover:text-foreground"
class="truncate text-base font-450 group-data-[highlighted]:text-foreground"
classList={{ "text-foreground": selected() }}
>
{option.label}
@ -1064,7 +1123,7 @@ type PromptInputReferenceImageButtonProps = {
onRemove(): void;
}
function PromptInputReferenceImageButton(props: PromptInputReferenceImageButtonProps) {
export function PromptInputReferenceImageButton(props: PromptInputReferenceImageButtonProps) {
return (
<div class={cx("group relative shrink-0", props.class)}>
<div class="size-full overflow-hidden rounded-md bg-input outline-none">
@ -1081,26 +1140,26 @@ function PromptInputReferenceImageButton(props: PromptInputReferenceImageButtonP
);
}
type PromptInputAttachButtonProps = {
type PromptInputAttachButtonProps = JSX.ButtonHTMLAttributes<HTMLButtonElement> & {
icon: string;
label: string;
class?: string;
onClick(): void;
}
export function PromptInputAttachButton(props: PromptInputAttachButtonProps) {
const [local, rest] = splitProps(props, ["icon", "label", "class"]);
return (
<button
type="button"
{...rest}
class={cx(
"flex h-16 shrink-0 items-center justify-center gap-1 overflow-clip rounded-md bg-secondary active:bg-secondary-pressing pl-3 pr-4 py-2 text-muted-foreground transition-colors hover:bg-muted",
props.class,
local.class,
)}
onClick={props.onClick}
>
<Icon name={props.icon} class="size-6" />
<Icon name={local.icon} class="size-6" />
<span class="whitespace-nowrap text-xs">
{props.label}
{local.label}
</span>
</button>
);

View File

@ -34,7 +34,7 @@ function isMediaPaint(paint: PaintType | undefined): paint is MediaPaint {
* else its first media fill. A node that paints no media at all is still its
* own source element — an `<audio>` carries its `src` like any other.
*/
function resolve(entity: Entity): SelectedNode {
export function resolveMedia(entity: Entity): SelectedNode {
const intrinsic = getIntrinsicPaint(entity);
if (isMediaPaint(intrinsic)) return { entity, source: entity, paint: intrinsic };
@ -50,7 +50,7 @@ export function useMediaSelection() {
const library = useLibrary();
const { nodes } = useSelection();
const selected = createMemo(() => nodes().map(resolve));
const selected = createMemo(() => nodes().map(resolveMedia));
const painted = (paint: MediaPaint) =>
createMemo(() => selected().filter((node) => node.paint === paint).map((node) => node.source));

View File

@ -13,12 +13,13 @@ import { AUDIO_SIZE } from "@/engine/insert-asset";
import { insertGenerated, randomSeed } from "./insert";
import type { AudioGenerationConfig } from "./schemas";
import type { AABB } from "@diffusionstudio/runtime";
export function useGenerateAudio() {
const world = useWorld();
const editor = useEditor();
const run = async (config: AudioGenerationConfig) => {
const run = async (config: AudioGenerationConfig, near?: AABB) => {
const src = generate.audio({
prompt: config.prompt,
model: config.model,
@ -27,7 +28,7 @@ export function useGenerateAudio() {
insertGenerated(world, editor, AUDIO_SIZE, (box) => (
<Audio src={src} x={box.x} y={box.y} width={box.width} height={box.height} />
), 1, [src]);
), 1, [src], near);
};
return { generate: run } as const;

View File

@ -12,12 +12,13 @@ import { ASPECT_RATIO_DIMENSIONS } from "./config";
import { insertGenerated, randomSeed } from "./insert";
import type { ImageGenerationConfig } from "./schemas";
import type { AABB } from "@diffusionstudio/runtime";
export function useGenerateImage() {
const world = useWorld();
const editor = useEditor();
const run = async (config: ImageGenerationConfig) => {
const run = async (config: ImageGenerationConfig, near?: AABB) => {
const library = world.get(Library);
// References travel by library path: durable in the file, and readable
@ -42,7 +43,7 @@ export function useGenerateImage() {
<Rect keepAspectRatio x={box.x} y={box.y} width={box.width} height={box.height}>
<ImagePaint src={sources[index]} />
</Rect>
), config.count, sources);
), config.count, sources, near);
};
return { generate: run } as const;

View File

@ -11,12 +11,13 @@ import { ASPECT_RATIO_DIMENSIONS } from "./config";
import { insertGenerated, randomSeed } from "./insert";
import type { VideoGenerationConfig } from "./schemas";
import type { AABB } from "@diffusionstudio/runtime";
export function useGenerateVideo() {
const world = useWorld();
const editor = useEditor();
const run = async (config: VideoGenerationConfig) => {
const run = async (config: VideoGenerationConfig, near?: AABB) => {
const library = world.get(Library);
const frame = (id: string | undefined) => (id ? library?.get(id)?.path : undefined);
@ -39,7 +40,7 @@ export function useGenerateVideo() {
<Rect keepAspectRatio x={box.x} y={box.y} width={box.width} height={box.height}>
<VideoPaint src={src} />
</Rect>
), 1, [src]);
), 1, [src], near);
};
return { generate: run } as const;

View File

@ -14,12 +14,13 @@ import { AUDIO_SIZE } from "@/engine/insert-asset";
import { insertGenerated, randomSeed } from "./insert";
import type { VoiceGenerationConfig } from "./schemas";
import type { AABB } from "@diffusionstudio/runtime";
export function useGenerateVoice() {
const world = useWorld();
const editor = useEditor();
const run = async (config: VoiceGenerationConfig) => {
const run = async (config: VoiceGenerationConfig, near?: AABB) => {
const src = generate.voice({
prompt: config.prompt,
voice: config.voice,
@ -28,7 +29,7 @@ export function useGenerateVoice() {
insertGenerated(world, editor, AUDIO_SIZE, (box) => (
<Audio src={src} x={box.x} y={box.y} width={box.width} height={box.height} />
), 1, [src]);
), 1, [src], near);
};
return { generate: run } as const;

View File

@ -62,9 +62,9 @@ export function toClientConfig(stored: unknown): GenerationConfig | undefined {
* "Rerun" and "Reuse" work from. Inputs are named by library path in the
* file and by asset id in the prompt box, so they are looked up on the way.
*/
function toPromptConfig(spec: GenerateSpec, library: AssetLibrary): GenerationConfig {
export function toPromptConfig(spec: GenerateSpec, library: AssetLibrary | undefined): GenerationConfig {
const idOf = (input: AssetInput | undefined): string | undefined =>
typeof input === "string" ? library.get(input)?.id : undefined;
typeof input === "string" ? library?.get(input)?.id : undefined;
switch (spec.type) {
case "image":

View File

@ -0,0 +1,95 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { createMemo } from "solid-js";
import { toast } from "somoto";
import { useWorld } from "@diffusionstudio/koota-solid";
import { useSelection } from "@/engine/hooks";
import { tidySelection } from "@/engine/align";
import { forkSelection } from "@/engine/prompt";
import { useGenerationRecords } from "./use-generation-records";
import { useGenerateImage } from "./use-generate-image";
import { useGenerateVideo } from "./use-generate-video";
import { useGenerateVoice } from "./use-generate-voice";
import { useGenerateAudio } from "./use-generate-audio";
import { useAutoCaptions } from "./use-auto-captions";
import { useMediaSelection } from "./selection";
import { useTransforms } from "./use-transforms";
import { createDefaultConfig } from "./prompt-input";
import type { GenerationConfig } from "./schemas";
export function useObjectActions(openPromptInput?: (config: GenerationConfig) => void) {
const world = useWorld();
const { imageNodes, videoNodes } = useMediaSelection();
const { nodes } = useSelection();
const { isOn, toggle } = useTransforms();
const { generate: generateImage } = useGenerateImage();
const { generate: generateVideo } = useGenerateVideo();
const { generate: generateVoice } = useGenerateVoice();
const { generate: generateAudio } = useGenerateAudio();
const { generate: autoCaptions, hasScene } = useAutoCaptions();
const { isGenerated, totalCredits, firstConfig } = useGenerationRecords();
const isImage = createMemo(() => imageNodes().length > 0);
const isVideo = createMemo(() => videoNodes().length > 0);
const canTidy = createMemo(() => nodes().length > 1);
const rerun = () => {
const config = firstConfig();
if (!config) {
toast("No generation config found", { description: "This asset wasn't generated with a prompt." });
return;
}
const promise = (() => {
switch (config.mode) {
case "IMAGE":
return generateImage(config);
case "VIDEO":
return generateVideo(config);
case "VOICE":
return generateVoice(config);
case "AUDIO":
return generateAudio(config);
}
})();
promise.catch((err) => {
toast("Rerun failed", {
description: err instanceof Error ? err.message : String(err),
});
});
};
const editWithPrompt = () => {
openPromptInput?.(createDefaultConfig("IMAGE"));
};
const makeVideo = () => {
openPromptInput?.(createDefaultConfig("VIDEO"));
};
const reuse = () => {
const config = firstConfig();
if (!config) {
toast("No generation config found", { description: "This asset wasn't generated with a prompt." });
return;
}
openPromptInput?.(config);
};
const fork = () => {
if (forkSelection(world)) return;
toast("No generation config found", { description: "This asset wasn't generated with a prompt." });
};
const tidy = () => tidySelection(world);
return {
isImage, isVideo, hasScene, canTidy, isGenerated, totalCredits, isOn, toggle,
autoCaptions, rerun, editWithPrompt, makeVideo, reuse, fork, tidy,
};
}

View File

@ -20,6 +20,7 @@ import { AssetThumbnail } from "../ui/asset-thumbnail";
import { formatAssetDuration } from "@/utils";
import { useLibrary } from "@/engine/library";
import { ASSET_DRAG_TYPE } from "./folder-item";
import { assetFinderPath, revealAssetInFinder } from "./reveal";
import type { Asset } from "@diffusionstudio/assets";
@ -99,6 +100,7 @@ export function LazyAssetItem(props: LazyAssetItemProps) {
};
const handleSaveAs = () => saveAssetAs(props.asset);
const handleRevealInFinder = () => revealAssetInFinder(world, props.asset);
const handleInsertToTimeline = () => insertAssetAtPlayhead(world, props.asset);
return (
@ -167,6 +169,12 @@ export function LazyAssetItem(props: LazyAssetItemProps) {
<ContextMenuItem onSelect={handleSaveAs}>
Save as
</ContextMenuItem>
<Show when={assetFinderPath(world, props.asset)}>
<ContextMenuItem onSelect={handleRevealInFinder}>
Reveal in finder
<ContextMenuShortcut>⌥⌘O</ContextMenuShortcut>
</ContextMenuItem>
</Show>
<ContextMenuSeparator />
<ContextMenuItem onSelect={handleDeleteAsset}>
Delete

View File

@ -2,7 +2,7 @@
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { basename, dirname } from "@diffusionstudio/assets";
import { assetFolder, basename, dirname } from "@diffusionstudio/assets";
import { usePromptInput } from "@/context/prompt-input";
import { createDefaultConfig } from "@/components/genai/prompt-input";
import { createEffect, createMemo, createSignal, For, onCleanup, onMount, Show } from "solid-js";
@ -30,6 +30,7 @@ import {
import { LazyAssetItem } from "./asset-item";
import { PartialAssetItem } from "./partial-asset-item";
import { FolderItem, handleFolderDrop, isAssetOrFolderDrag, ASSET_DRAG_TYPE, FOLDER_DRAG_TYPE } from "./folder-item";
import { assetReveal, clearAssetReveal } from "./reveal";
import { useLibrary } from "@/engine/library";
import { useAssetSelection } from "@/engine/hooks";
import { droppedFiles, importFiles, pickAndImport } from "@/engine/asset-actions";
@ -124,6 +125,15 @@ export function Assets() {
setCurrentFolder(folder);
});
createEffect(() => {
const asset = assetReveal();
if (!asset) return;
clearAssetReveal();
setAssetFilter("ALL");
openFolder(assetFolder(asset));
requestAnimationFrame(() => root?.querySelector(`[data-asset-id="${asset.id}"]`)?.scrollIntoView({ block: "nearest" }));
});
const withLibrary = (run: (library: AssetLibrary) => void | Promise<void>) => {
const lib = library();
if (!lib) {

View File

@ -0,0 +1,60 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { createSignal } from "solid-js";
import { Library } from "@diffusionstudio/runtime";
import { toast } from "somoto";
import { setSidebarTab } from "@/agent-chat/store";
import { selectionAssets } from "@/engine/asset-folders";
import { assetFilePath } from "@/engine/library";
import { AssetSelection } from "@/engine/traits";
import { revealPath } from "@/lib/shell";
import type { Asset } from "@diffusionstudio/assets";
import type { World } from "koota";
const [assetReveal, setAssetReveal] = createSignal<Asset | null>(null);
export { assetReveal };
export function clearAssetReveal(): void {
setAssetReveal(null);
}
export function revealInAssets(world: World): void {
const [asset] = selectionAssets(world);
if (!asset) return;
setSidebarTab("assets");
world.set(AssetSelection, { id: asset.id });
setAssetReveal(asset);
}
export function assetFinderPath(world: World, asset: Asset): string | undefined {
return window.desktop ? assetFilePath(world, asset) : undefined;
}
export function revealAssetInFinder(world: World, asset: Asset): void {
const path = assetFinderPath(world, asset);
if (!path) return;
revealPath(path).catch((error: unknown) => {
toast.error("Failed to reveal in finder", { description: (error as Error).message });
});
}
export function finderPath(world: World): string | undefined {
const [asset] = selectionAssets(world);
return asset && assetFinderPath(world, asset);
}
export function revealInFinder(world: World): void {
const [asset] = selectionAssets(world);
if (asset) revealAssetInFinder(world, asset);
}
export function revealPickedInFinder(world: World): void {
const picked = world.get(AssetSelection)?.id;
if (!picked) return revealInFinder(world);
const asset = world.get(Library)?.get(picked);
if (asset) revealAssetInFinder(world, asset);
}

View File

@ -8,6 +8,7 @@ import {
ToolType,
getParentNode,
isAdjustmentLayer,
isPrompt,
isAudio,
isCaption,
isGroup,
@ -31,6 +32,7 @@ import { LayoutPanel } from "./layout";
import { TransformSettings } from "./transform";
import { CaptionSettings } from "./caption-settings";
import { TextPanel } from "./text";
import { PromptPanel } from "./prompt";
import { FillsSettings } from "./fills";
import { StrokesSettings } from "./strokes";
import { ShadowsSettings } from "./shadows";
@ -55,6 +57,7 @@ export type SelectionTarget =
| "adjustment"
| "text"
| "shape"
| "prompt"
| "group"
| "stage";
@ -66,6 +69,7 @@ function classifyNode(entity: Entity): SelectionTarget {
if (isAudio(entity)) return "audio";
if (isAdjustmentLayer(entity)) return "adjustment";
if (isText(entity)) return "text";
if (isPrompt(entity)) return "prompt";
if (isShape(entity)) return "shape";
if (isGroup(entity)) return "group";
return "stage";
@ -131,12 +135,16 @@ export function Inspector() {
<TimeSettings selection={nodes()} />
</Show>
<Show when={includesTarget("shape", "text", "audio", "scene", "caption", "group", "clip-path", "adjustment")}>
<TransformSettings selection={nodes()} />
<Show when={includesTarget("shape", "text", "audio", "scene", "caption", "group", "clip-path", "adjustment", "prompt")}>
<TransformSettings selection={nodes()} keyframes={!includesTarget("prompt")} />
</Show>
<Show when={includesTarget("shape", "text", "audio", "scene", "clip-path")}>
<LayoutPanel selection={nodes()} />
<Show when={includesTarget("shape", "text", "audio", "scene", "clip-path", "prompt")}>
<LayoutPanel selection={nodes()} keyframes={!includesTarget("prompt")} />
</Show>
<Show when={includesTarget("prompt")}>
<PromptPanel selection={nodes()} />
</Show>
<Show when={includesTarget("shape", "text", "scene", "caption", "group", "audio", "clip-path")}>

View File

@ -38,6 +38,7 @@ import type { Entity } from "koota";
type LayoutPanelProps = {
selection: Entity[];
keyframes?: boolean;
};
const DEFAULT_WIDTH = 1920;
@ -257,7 +258,7 @@ export function LayoutPanel(props: LayoutPanelProps) {
limitEvents
skipEmpty
sliderEnabled
keyframe={<Keyframe target={entity()} property="width" />}
keyframe={props.keyframes === false ? undefined : <Keyframe target={entity()} property="width" />}
/>
<ControlledTextField
class="group"
@ -269,7 +270,7 @@ export function LayoutPanel(props: LayoutPanelProps) {
limitEvents
skipEmpty
sliderEnabled
keyframe={<Keyframe target={entity()} property="height" />}
keyframe={props.keyframes === false ? undefined : <Keyframe target={entity()} property="height" />}
/>
</ControlRow>

View File

@ -0,0 +1,36 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { ControlRow } from "@/components/ui/control-group";
import { PanelSection } from "@/components/ui/panel-section";
import { useDerived, useEditor } from "@/engine/hooks";
import { promptText, setPromptText } from "@/engine/prompt";
import { GrowingTextArea } from "./text";
import type { Entity } from "koota";
type PromptPanelProps = {
selection: Entity[];
};
export function PromptPanel(props: PromptPanelProps) {
const editor = useEditor();
const entity = () => props.selection[0]!;
const prompt = useDerived(() => promptText(entity()));
return (
<PanelSection title="Prompt">
<ControlRow label="Content" class="items-start" labelClass="pt-1 leading-[18px]" contentClass="flex flex-col">
<GrowingTextArea
value={prompt()}
placeholder="Describe what you want to create."
minRows={2}
maxRows={8}
onInput={(value) => setPromptText(editor, entity(), value)}
/>
</ControlRow>
</PanelSection>
);
}

View File

@ -143,7 +143,7 @@ export function TextPanel(props: TextPanelProps) {
return (
<PanelSection title="Typography">
<Show when={textSelected()}>
<ControlRow label="Content" contentClass="flex flex-col">
<ControlRow label="Content" class="items-start" labelClass="pt-1 leading-[18px]" contentClass="flex flex-col">
<GrowingTextArea
value={chars()?.value ?? ''}
maxRows={8}
@ -262,6 +262,8 @@ export function TextPanel(props: TextPanelProps) {
type GrowingTextAreaProps = {
value: string;
onInput(value: string): void;
placeholder?: string;
minRows?: number;
maxRows?: number;
focused?: boolean;
onBlur?(): void;
@ -277,7 +279,8 @@ export function GrowingTextArea(props: GrowingTextAreaProps) {
if (!ref) return;
ref.style.height = 'auto';
const max = lineHeight * maxRows() + 8; // 8px for py-1
ref.style.height = `${Math.min(ref.scrollHeight, max)}px`;
const min = props.minRows ? lineHeight * props.minRows + 8 : 0;
ref.style.height = `${Math.max(min, Math.min(ref.scrollHeight, max))}px`;
};
// Resize when value changes externally (e.g. undo/redo, selection change)
@ -313,6 +316,7 @@ export function GrowingTextArea(props: GrowingTextAreaProps) {
<textarea
ref={ref!}
value={props.value}
placeholder={props.placeholder}
onInput={handleInput}
onKeyDown={handleKeyDown}
onFocus={props.onFocus}

View File

@ -22,6 +22,7 @@ import type { Entity } from "koota";
export type OffsetRowProps = {
node: Entity;
keyframes?: boolean;
onRemoveAddon(): void;
};
@ -59,7 +60,7 @@ export function OffsetRow(props: OffsetRowProps) {
step={1}
autoSelect
sliderEnabled
keyframe={<Keyframe target={props.node} property="offsetX" />}
keyframe={props.keyframes === false ? undefined : <Keyframe target={props.node} property="offsetX" />}
/>
<ControlledTextField
icon={<Icon name="prop-y-position" />}
@ -68,7 +69,7 @@ export function OffsetRow(props: OffsetRowProps) {
step={1}
autoSelect
sliderEnabled
keyframe={<Keyframe target={props.node} property="offsetY" />}
keyframe={props.keyframes === false ? undefined : <Keyframe target={props.node} property="offsetY" />}
/>
</ContextMenuTrigger>
<ContextMenuContent>

View File

@ -23,6 +23,7 @@ import type { Entity } from 'koota';
type RotateRowProps = {
node: Entity;
keyframes?: boolean;
onRemoveAddon(): void;
};
@ -74,7 +75,7 @@ export function RotateRow(props: RotateRowProps) {
autoSelect
sliderEnabled
limitEvents
keyframe={<Keyframe target={props.node} property="rotation" />}
keyframe={props.keyframes === false ? undefined : <Keyframe target={props.node} property="rotation" />}
/>
<div class="flex flex-1 gap-px rounded-md overflow-hidden">
<Tooltip>

View File

@ -23,6 +23,7 @@ import type { Entity } from "koota";
export type ScaleRowProps = {
node: Entity;
keyframes?: boolean;
onRemoveAddon(): void;
};
@ -115,7 +116,7 @@ export function ScaleRow(props: ScaleRowProps) {
autoSelect
sliderEnabled
limitEvents
keyframe={<Keyframe target={props.node} property="scale" />}
keyframe={props.keyframes === false ? undefined : <Keyframe target={props.node} property="scale" />}
/>
<SegmentedIconTabs
value={() => scaleMode()}
@ -132,7 +133,7 @@ export function ScaleRow(props: ScaleRowProps) {
autoSelect
sliderEnabled
limitEvents
keyframe={<Keyframe target={props.node} property="scaleX" />}
keyframe={props.keyframes === false ? undefined : <Keyframe target={props.node} property="scaleX" />}
/>
<ControlledTextField
icon={<Icon name="prop-y-position" />}
@ -143,7 +144,7 @@ export function ScaleRow(props: ScaleRowProps) {
autoSelect
sliderEnabled
limitEvents
keyframe={<Keyframe target={props.node} property="scaleY" />}
keyframe={props.keyframes === false ? undefined : <Keyframe target={props.node} property="scaleY" />}
/>
</Show>
</ContextMenuTrigger>

View File

@ -39,6 +39,7 @@ import type { Entity } from "koota";
type TransformSettingsProps = {
selection: Entity[];
keyframes?: boolean;
};
type TransformAddon = 'rotate' | 'anchor' | 'offset' | 'scale' | 'skew' | 'constraints';
@ -158,7 +159,7 @@ export function TransformSettings(props: TransformSettingsProps) {
<div class="grid grid-cols-2 gap-2">
<ControlledTextField
icon={<Icon name="prop-x-position" />}
keyframe={<Keyframe target={entity()} property="x" />}
keyframe={props.keyframes === false ? undefined : <Keyframe target={entity()} property="x" />}
value={positionX()}
onNumber={updatePositionX}
step={1}
@ -168,7 +169,7 @@ export function TransformSettings(props: TransformSettingsProps) {
/>
<ControlledTextField
icon={<Icon name="prop-y-position" />}
keyframe={<Keyframe target={entity()} property="y" />}
keyframe={props.keyframes === false ? undefined : <Keyframe target={entity()} property="y" />}
value={positionY()}
onNumber={updatePositionY}
step={1}
@ -184,7 +185,7 @@ export function TransformSettings(props: TransformSettingsProps) {
</Show>
<Show when={showAddon('rotate')}>
<RotateRow node={entity()} onRemoveAddon={() => toggleAddon('rotate', false)} />
<RotateRow node={entity()} keyframes={props.keyframes} onRemoveAddon={() => toggleAddon('rotate', false)} />
</Show>
<Show when={showAddon('anchor')}>
@ -192,11 +193,11 @@ export function TransformSettings(props: TransformSettingsProps) {
</Show>
<Show when={showAddon('offset')}>
<OffsetRow node={entity()} onRemoveAddon={() => toggleAddon('offset', false)} />
<OffsetRow node={entity()} keyframes={props.keyframes} onRemoveAddon={() => toggleAddon('offset', false)} />
</Show>
<Show when={showAddon('scale')}>
<ScaleRow node={entity()} onRemoveAddon={() => toggleAddon('scale', false)} />
<ScaleRow node={entity()} keyframes={props.keyframes} onRemoveAddon={() => toggleAddon('scale', false)} />
</Show>
<Show when={showAddon('skew')}>

View File

@ -8,6 +8,8 @@ export type SearchInputProps = {
placeholder: string;
value: string;
onValue: (value: string) => void;
onKeyDown?: (event: KeyboardEvent) => void;
ref?: HTMLInputElement | ((element: HTMLInputElement) => void);
};
export function SearchInput(props: SearchInputProps) {
@ -15,10 +17,12 @@ export function SearchInput(props: SearchInputProps) {
<div class="mx-2 relative flex h-11 items-center border-b border-border">
<Icon name="search" class="text-muted-foreground" />
<input
ref={props.ref}
type="text"
value={props.value}
onInput={(e) => props.onValue(e.currentTarget.value)}
onKeyDown={(e) => {
props.onKeyDown?.(e);
if (e.key === "Escape") return;
e.stopPropagation();
}}

View File

@ -15,9 +15,11 @@ import {
Geometry,
Group,
Position,
RenderSurface,
Selected,
WorldBounds,
entityWorldMat,
getCameraScale,
getParentEntity,
getParentNode,
invert2D,
@ -43,6 +45,18 @@ export type Axis = "x" | "y";
type Box = { minX: number; minY: number; maxX: number; maxY: number };
type Sized = { box: Box };
type Item = { entity: Entity; box: Box };
const FALLBACK_GAP = 0.2;
const SIMILAR_SIZE = 1.5;
const SIMILAR_ASPECT = 1.15;
const PACK_ASPECT = 16 / 9;
/** The selected nodes alignment moves: top-level ones and direct children of scenes. */
export function getAlignableSelection(world: World): Entity[] {
return [...world.query(Selected, Or(Geometry, Group))].filter((entity) => {
@ -159,3 +173,223 @@ export function distributeSelection(world: World, axis: Axis): void {
cursor += size(box) + gap;
}
}
function bands<T extends Sized>(items: T[], axis: Axis): T[][] {
const min = (box: Box) => (axis === "x" ? box.minX : box.minY);
const max = (box: Box) => (axis === "x" ? box.maxX : box.maxY);
const result: { items: T[]; end: number }[] = [];
for (const item of [...items].sort((a, b) => min(a.box) - min(b.box))) {
const center = (min(item.box) + max(item.box)) / 2;
const band = result[result.length - 1];
if (band && center < band.end) band.items.push(item);
else result.push({ items: [item], end: max(item.box) });
}
return result.map((band) => band.items);
}
function lowerMedian(values: number[]): number | undefined {
const sorted = [...values].sort((a, b) => a - b);
return sorted[Math.floor((sorted.length - 1) / 2)];
}
function typicalGap(values: number[]): number | undefined {
return lowerMedian(values.filter((value) => value > 0));
}
function squareColumns(count: number): number {
let columns = Math.ceil(Math.sqrt(count));
while ((count % columns || columns) <= columns / 2) columns++;
return columns;
}
const widthOf = (box: Box) => box.maxX - box.minX;
const heightOf = (box: Box) => box.maxY - box.minY;
const sizeRatio = (a: number, b: number) => Math.max(a, b) / Math.min(a, b);
type Arrangement<T> = { placed: { item: T; x: number; y: number }[]; width: number; height: number; gap: number };
function fallbackGap(items: Sized[], scale: number): number {
const side = items.reduce((sum, { box }) => sum + Math.min(widthOf(box), heightOf(box)), 0) / items.length;
return Math.round((side * FALLBACK_GAP) / scale) * scale;
}
function arrange<T extends Sized>(items: T[], scale: number, fixedGap?: number): Arrangement<T> {
const rows = bands(items, "y").map((row) => row.sort((a, b) => a.box.minX - b.box.minX));
const columnBands = bands(items, "x");
const count = items.length;
const longest = (groups: T[][]) => Math.max(...groups.map((group) => group.length));
const structured = rows.length === 1 || rows.length * columnBands.length <= 2 * count;
const columns =
rows.length === 1 ? count
: structured ? Math.max(longest(rows), Math.ceil(count / longest(columnBands)))
: squareColumns(count);
const planar = rows.length > 1 && columnBands.length > 1;
const width = lowerMedian(items.map(({ box }) => widthOf(box)))!;
const height = lowerMedian(items.map(({ box }) => heightOf(box)))!;
const spacing = (gap: number | undefined, size: number) =>
!structured || gap === undefined || (planar && gap >= size) ? undefined : Math.round(gap / scale) * scale;
const gapX = spacing(typicalGap(rows.flatMap((row) => row.slice(1).map((item, index) => item.box.minX - row[index]!.box.maxX))), width);
const gapY = spacing(typicalGap(rows.slice(1).map((row, index) =>
Math.min(...row.map((item) => item.box.minY)) - Math.max(...rows[index]!.map((item) => item.box.maxY)))), height);
const measured = [gapX, gapY].filter((gap) => gap !== undefined);
const gap = fixedGap ?? (measured.length ? Math.min(...measured) : fallbackGap(items, scale));
const order = rows.flat();
const widths = new Array<number>(columns).fill(0);
order.forEach(({ box }, index) => {
widths[index % columns] = Math.max(widths[index % columns]!, widthOf(box));
});
const placed: Arrangement<T>["placed"] = [];
let top = 0;
for (let start = 0; start < count; start += columns) {
const line = order.slice(start, start + columns);
let left = 0;
line.forEach((item, column) => {
placed.push({ item, x: left, y: top });
left += widths[column]! + gap;
});
top += Math.max(...line.map(({ box }) => heightOf(box))) + gap;
}
return { placed, width: widths.reduce((sum, size) => sum + size, 0) + gap * (columns - 1), height: top - gap, gap };
}
type Column<T> = { items: T[]; width: number; height: number };
type Shelf<T> = { height: number; columns: Column<T>[] };
const shelfWidth = (shelf: Shelf<unknown>, gap: number) =>
shelf.columns.reduce((sum, column) => sum + column.width, 0) + gap * (shelf.columns.length - 1);
function shelve<T extends Sized>(sorted: T[], gap: number, limit: number): Shelf<T>[] {
const shelves: Shelf<T>[] = [];
for (const item of sorted) {
const width = widthOf(item.box);
const height = heightOf(item.box);
let fit: { column: Column<T>; room: number } | undefined;
for (const shelf of shelves) {
for (const column of shelf.columns) {
const room = shelf.height - column.height - gap - height;
const widened = shelfWidth(shelf, gap) - column.width + Math.max(column.width, width);
if (room >= 0 && widened <= limit && (!fit || room < fit.room)) fit = { column, room };
}
}
if (fit) {
fit.column.items.push(item);
fit.column.height += gap + height;
fit.column.width = Math.max(fit.column.width, width);
continue;
}
const shelf = shelves.find((candidate) => shelfWidth(candidate, gap) + gap + width <= limit);
if (shelf) shelf.columns.push({ items: [item], width, height });
else shelves.push({ height, columns: [{ items: [item], width, height }] });
}
return shelves;
}
function pack<T extends Sized>(blocks: T[], gap: number): Arrangement<T> {
const sorted = [...blocks].sort((a, b) =>
heightOf(b.box) - heightOf(a.box) || widthOf(b.box) - widthOf(a.box) || a.box.minY - b.box.minY || a.box.minX - b.box.minX);
const limits = [Math.max(...sorted.map(({ box }) => widthOf(box)))];
sorted.reduce((sum, { box }) => {
limits.push(sum + widthOf(box));
return sum + widthOf(box) + gap;
}, 0);
let best: { shelves: Shelf<T>[]; width: number; height: number; zoom: number } | undefined;
for (const limit of limits) {
const shelves = shelve(sorted, gap, limit);
const width = Math.max(...shelves.map((shelf) => shelfWidth(shelf, gap)));
const height = shelves.reduce((sum, shelf) => sum + shelf.height, 0) + gap * (shelves.length - 1);
const zoom = Math.max(width, height * PACK_ASPECT);
if (!best || zoom < best.zoom) best = { shelves, width, height, zoom };
}
const placed: Arrangement<T>["placed"] = [];
let top = 0;
for (const shelf of best!.shelves) {
let left = 0;
for (const column of shelf.columns) {
let y = top;
for (const item of column.items) {
placed.push({ item, x: left, y });
y += heightOf(item.box) + gap;
}
left += column.width + gap;
}
top += shelf.height + gap;
}
return { placed, width: best!.width, height: best!.height, gap };
}
function alike(a: Item, b: Item): boolean {
return isScene(a.entity) === isScene(b.entity)
&& sizeRatio(widthOf(a.box), widthOf(b.box)) <= SIMILAR_SIZE
&& sizeRatio(heightOf(a.box), heightOf(b.box)) <= SIMILAR_SIZE
&& sizeRatio(widthOf(a.box) / heightOf(a.box), widthOf(b.box) / heightOf(b.box)) <= SIMILAR_ASPECT;
}
function groupsOf(items: Item[]): Item[][] {
const groups: Item[][] = [];
const seen = new Set<Item>();
for (const start of items) {
if (seen.has(start)) continue;
seen.add(start);
const group = [start];
for (let index = 0; index < group.length; index++) {
for (const other of items) {
if (seen.has(other) || !alike(group[index]!, other)) continue;
seen.add(other);
group.push(other);
}
}
groups.push(group);
}
return groups;
}
export function tidySelection(world: World): void {
const entities = getAlignableSelection(world);
if (entities.length < 2) return;
const items = entities.map((entity) => ({ entity, box: boundsOf(world, entity) }));
const scale = getCameraScale(world) * (world.get(RenderSurface)?.resolution ?? 1);
const linear = bands(items, "y").length === 1 || bands(items, "x").length === 1;
const groups = linear ? [items] : groupsOf(items);
let placed: Arrangement<Item>["placed"];
if (groups.length === 1) {
placed = arrange(items, scale).placed;
} else {
const gap = fallbackGap(items, scale);
const blocks = groups.map((group) => {
const inner = arrange(group, scale, gap);
const at = union(group.map(({ box }) => box));
return { inner, box: { minX: at.minX, minY: at.minY, maxX: at.minX + inner.width, maxY: at.minY + inner.height } };
});
placed = pack(blocks, gap).placed.flatMap(({ item: block, x, y }) =>
block.inner.placed.map((spot) => ({ item: spot.item, x: x + spot.x, y: y + spot.y })));
}
const origin = union(items.map(({ box }) => box));
const settle = (delta: number) => (Math.abs(delta) < scale / 2 ? 0 : delta);
for (const { item, x, y } of placed) {
moveByWorldDelta(world, item.entity, settle(origin.minX + x - item.box.minX), settle(origin.minY + y - item.box.minY));
}
}

View File

@ -0,0 +1,27 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { assetFolder, basename } from '@diffusionstudio/assets';
import { AssetId, Cache, Library, getSelection } from '@diffusionstudio/runtime';
import type { Asset } from '@diffusionstudio/assets';
import type { Entity, World } from 'koota';
export function mediaAsset(world: World, entity: Entity): Asset | undefined {
const id = [entity, ...(entity.get(Cache)?.fills ?? [])]
.map((candidate) => candidate.get(AssetId)?.value)
.find((value) => value);
const asset = id ? world.get(Library)?.get(id) : undefined;
return asset?.transient ? undefined : asset;
}
export function folderLabel(world: World, entity: Entity): string {
const asset = mediaAsset(world, entity);
return asset ? basename(assetFolder(asset)) : '';
}
export function selectionAssets(world: World): Asset[] {
const assets = getSelection(world).map((entity) => mediaAsset(world, entity));
return [...new Set(assets.filter((asset): asset is Asset => asset !== undefined))];
}

View File

@ -0,0 +1,101 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { getAssetSpec, isTransformSpec } from '@diffusionstudio/jsx';
import { authoredElement } from '@diffusionstudio/reconciler';
import { Cache, Generating, Paint, PaintType, PendingSource } from '@diffusionstudio/runtime';
import {
PROMPT_INPUT_AUDIO_MODEL_OPTIONS, PROMPT_INPUT_IMAGE_MODEL_OPTIONS, PROMPT_INPUT_VIDEO_MODEL_OPTIONS,
} from '@/components/genai/config';
import { templateOf } from './prompt';
import type { AssetRef } from '@diffusionstudio/jsx';
import type { Entity } from 'koota';
const HOLD = 1000;
const FADE = 300;
const IMAGE_SECONDS: Record<string, number> = {
'flux-2-turbo': 8,
'gpt-image-2': 60,
'nano-banana-2': 15,
'nano-banana-pro': 30,
'seedream-4.5': 25,
};
const VIDEO_SECONDS: Record<string, { base: number; perSecond: number }> = {
'kling-3-pro': { base: 40, perSecond: 12 },
'kling-o3-pro': { base: 40, perSecond: 12 },
'kling-2.5-turbo': { base: 30, perSecond: 8 },
'seedance-2.0': { base: 30, perSecond: 10 },
'veo-3.1': { base: 40, perSecond: 12 },
'veo-3.1-fast': { base: 20, perSecond: 6 },
'wan-2.6': { base: 40, perSecond: 10 },
'hailuo-2.3': { base: 40, perSecond: 10 },
};
const AUDIO_SECONDS: Record<string, number> = {
'elevenlabs-music': 30,
'elevenlabs-sfx': 10,
};
const VOICE_SECONDS = 8;
const TRANSFORM_SECONDS = { removeBackground: 10, upscale: 20, addAudio: 45 };
const VIDEO_UPSCALE_SECONDS = 120;
type Stage = 'starting' | 'progress' | 'almost';
type Run = { start: number; stage: Stage; text: string; changedAt: number; previous: string };
const runs = new WeakMap<object, Run>();
function expectedSeconds(carrier: Entity, ref: AssetRef): number {
const spec = getAssetSpec(ref);
if (isTransformSpec(spec)) {
const video = carrier.get(Paint)?.value === PaintType.VIDEO;
return spec.type === 'upscale' && video ? VIDEO_UPSCALE_SECONDS : TRANSFORM_SECONDS[spec.type];
}
switch (spec.type) {
case 'image':
return IMAGE_SECONDS[spec.model ?? PROMPT_INPUT_IMAGE_MODEL_OPTIONS[0].id] ?? 20;
case 'video': {
const { base, perSecond } = VIDEO_SECONDS[spec.model ?? PROMPT_INPUT_VIDEO_MODEL_OPTIONS[0].id] ?? { base: 40, perSecond: 12 };
return base + perSecond * (spec.duration ?? 5);
}
case 'voice':
return VOICE_SECONDS;
case 'audio':
return AUDIO_SECONDS[spec.model ?? PROMPT_INPUT_AUDIO_MODEL_OPTIONS[0].id] ?? 20;
}
}
export function generationLabel(entity: Entity): { text: string; opacity: number } {
const carrier = [entity, ...(entity.get(Cache)?.fills ?? [])].find((candidate) => candidate.has(Generating));
const token = carrier?.get(PendingSource)?.value;
const ref = carrier && templateOf(authoredElement(carrier)?.props.src);
if (!carrier || typeof token !== 'object' || token === null || !ref) return { text: 'Generating...', opacity: 1 };
const now = performance.now();
let run = runs.get(token);
if (!run) {
run = { start: now, stage: 'starting', text: 'Generating...', changedAt: now - FADE, previous: '' };
runs.set(token, run);
}
const elapsed = now - run.start;
const percent = Math.floor((elapsed / (expectedSeconds(carrier, ref) * 1000)) * 100);
const stage: Stage = elapsed < HOLD ? 'starting' : percent < 100 ? 'progress' : 'almost';
const text = stage === 'starting' ? 'Generating...' : stage === 'progress' ? `${Math.max(1, percent)}% done` : 'Almost done...';
if (stage !== run.stage) {
run.previous = run.text;
run.stage = stage;
run.changedAt = now;
}
run.text = text;
const fade = (now - run.changedAt) / (FADE / 2);
return fade < 1 ? { text: run.previous, opacity: 1 - fade } : { text, opacity: Math.min(1, fade - 1) };
}

View File

@ -8,7 +8,7 @@ import {
HitRegions, Hovering, Name, Playback, RenderSurface, Root, Selected,
Sequential, ChildOf,
entityQuad, entityWorldMat, getMaskSelection, getSelectionMask, getSourceFailure,
invert2D, isGenerating,
invert2D, isGenerating, isPrompt,
multiply2D, rectToQuad, rotate2D, scale2D, store, transformPoint,
translate2D,
} from '@diffusionstudio/runtime';
@ -21,6 +21,8 @@ import {
import { getMarqueeQuad } from '../input/snapping';
import { getMountedNameInput } from './name-input';
import { drawObjectMasks } from '../object-mask';
import { generationLabel } from '../generation-progress';
import { folderLabel } from '../asset-folders';
import type { Entity, World } from 'koota';
import type { Mat2D } from '@diffusionstudio/runtime';
@ -31,6 +33,7 @@ const ERROR_COLOR = '#FF8A8A';
const HEADER_FONT = '350 11px Inter, sans-serif';
const HEADER_HEIGHT = 22;
const ACTIVE_BADGE_WIDTH = 60;
const HEADER_GAP = 12;
type Ctx2D = CanvasRenderingContext2D | OffscreenCanvasRenderingContext2D;
@ -110,7 +113,8 @@ function drawHoverOutlines(world: World, ctx: Ctx2D, resolution: number): void {
*/
function drawHeader(world: World, ctx: Ctx2D, entity: Entity, resolution: number): void {
const failure = getSourceFailure(entity);
const label = failure || (isGenerating(entity) ? 'Generating...' : entity.get(Name)?.value);
const progress = !failure && isGenerating(entity) ? generationLabel(entity) : undefined;
const label = failure || progress?.text || entity.get(Name)?.value;
if (!label) return;
const header = getHeaderLayout(world, entity, resolution);
@ -154,7 +158,9 @@ function drawHeader(world: World, ctx: Ctx2D, entity: Entity, resolution: number
if (fitted) {
ctx.fillStyle = foreground;
ctx.fillText(fitted.value, 0, 0);
ctx.globalAlpha = progress?.opacity ?? 1;
if (!isPrompt(entity)) ctx.fillText(fitted.value, 0, 0);
ctx.globalAlpha = 1;
regions.push({
target: {
@ -195,6 +201,8 @@ function drawHeader(world: World, ctx: Ctx2D, entity: Entity, resolution: number
ctx.fillText('Active', 0, 0);
}
let right = header.width;
if (playback && (fitted?.width ?? 0) + badgeWidth + 60 < header.width) {
ctx.setTransform(header.mat.a, header.mat.b, header.mat.c, header.mat.d, header.mat.e, header.mat.f);
ctx.translate(header.width, 3);
@ -215,7 +223,29 @@ function drawHeader(world: World, ctx: Ctx2D, entity: Entity, resolution: number
ctx.fillText(`${minutes}:${rest}`, 0, 0);
ctx.globalAlpha = 1;
right -= ctx.measureText(`${minutes}:${rest}`).width + HEADER_GAP;
}
const folder = progress || failure ? '' : folderLabel(world, entity);
if (!folder) return;
ctx.font = HEADER_FONT;
if (labelStart + (fitted?.width ?? 0) + badgeWidth + HEADER_GAP + ctx.measureText(folder).width > right) return;
ctx.setTransform(header.mat.a, header.mat.b, header.mat.c, header.mat.d, header.mat.e, header.mat.f);
ctx.translate(right, 3);
ctx.textAlign = 'right';
ctx.textBaseline = 'top';
if (selected) {
ctx.globalAlpha = 0.7;
ctx.fillStyle = '#cce8ff';
} else {
ctx.fillStyle = 'rgba(121, 121, 121, 1)';
}
ctx.fillText(folder, 0, 0);
ctx.globalAlpha = 1;
}
/**

View File

@ -23,7 +23,7 @@ import {
computeGroupBounds, computeLocalMatrix, decompose2D, entityAnchor,
entityOffset, entityQuad, entityWorldMat, enterEntity,
findKeyframeTrackEntity, getParentEntity, getParentNode, getSceneAncestor,
getSelection, getSelectionMask, identity2D, invert2D, isPointerInEntity,
getSelection, getSelectionMask, identity2D, invert2D, isPointerInEntity, isPrompt,
multiply2D, quadCenter, quadContainsQuad, quadsIntersect, rectToQuad,
rotate2D, scale2D,
store, syncInteractiveState, togglePlayback, transformPoint, transformSystem,
@ -32,6 +32,7 @@ import {
import { Not, Or } from 'koota';
import { getDocumentEditor } from '../editor';
import { PROMPT_MIN_SIZE } from '../prompt';
import { syncKeyframe } from '../keyframes';
import { Hud, Keys, Pointer, SnapLines } from '../traits';
import { getToolCursor, updateCursor, type CursorType } from './cursor';
@ -436,6 +437,25 @@ export function handleResizeInteraction(world: World, event: DispatchedPointerEv
}
}
const prompts = selection.flatMap((selected) => resizeTargets(world, selected)).filter(isPrompt);
if (prompts.length > 0) {
const minScale = (axis: 'width' | 'height') => Math.max(...prompts.map((entity) => {
const size = getTransformSnapshot(entity)?.[axis] ?? 0;
return size > 0 ? PROMPT_MIN_SIZE[axis] / size : 0;
}));
let scaleX = minScale('width');
let scaleY = minScale('height');
if (lockAspect) scaleX = scaleY = Math.max(scaleX, scaleY);
const minWidth = snapshot.width * scaleX;
const minHeight = snapshot.height * scaleY;
if (newWidth < minWidth || newHeight < minHeight) {
snapLines.length = 0;
newWidth = Math.max(newWidth, minWidth);
newHeight = Math.max(newHeight, minHeight);
}
}
const localScale = scaleAbout(pivotX, pivotY, newWidth / snapshot.width, newHeight / snapshot.height);
// Groups have no Size of their own, so a group resize transforms its
@ -448,11 +468,7 @@ export function handleResizeInteraction(world: World, event: DispatchedPointerEv
const writeAngles = selection.length > 1 || hasGroup;
for (const selected of selection) {
const targets = selected.has(Group)
? [...world.query(Or(Geometry, Group), ChildOf(selected))]
: [selected];
for (const entity of targets) {
for (const entity of resizeTargets(world, selected)) {
resizeNode(world, entity, oldTr, localScale, writeAngles);
// The walk is parent-first, so a group's box is built from its
@ -478,6 +494,10 @@ export function handleResizeInteraction(world: World, event: DispatchedPointerEv
}
}
function resizeTargets(world: World, selected: Entity): Entity[] {
return selected.has(Group) ? [...world.query(Or(Geometry, Group), ChildOf(selected))] : [selected];
}
/**
* Applies the mask's scale to one node: the delta is composed in world space,
* mapped back through the node's parent, and decomposed into the size,

View File

@ -24,6 +24,7 @@ import {
getParentNode,
getSelection,
isGroupLike,
isPrompt,
isSequence,
setPlayhead,
shuttlePlayback,
@ -32,13 +33,18 @@ import {
togglePlayback,
} from '@diffusionstudio/runtime';
import { Not, Or } from 'koota';
import { toast } from 'somoto';
import { tidySelection } from '../align';
import { zoomBy, zoomTo, zoomToFit, zoomToSelection } from '../camera';
import { applyClipPaths, cancelClipPath } from '../clip-path';
import { getDocumentEditor } from '../editor';
import { groupSelection, ungroupSelection, unwrapSequenceSelection, wrapSelectionInScene, wrapSelectionInSequence } from '../group';
import { getEditHistory } from '../history';
import { syncKeyframe } from '../keyframes';
import { cancelObjectMask, getObjectTrack, trackObjectMask, undoMaskStroke } from '../object-mask';
import { forkSelection, insertPrompt } from '../prompt';
import { revealInAssets, revealPickedInFinder } from '@/components/sidebar-left/reveal';
import { splitAtPlayhead } from '../split';
import { Keys, MODIFIER_KEYS, ObjectMaskTool, Pointer } from '../traits';
import { editTransform } from './interactions';
@ -341,6 +347,40 @@ export function toggleSelectionHidden(world: World): void {
for (const entity of selected) editor.editProperty(entity, 'hidden', hide);
}
const OPACITY_WINDOW = 500;
let opacityTyped = { digits: '', at: -Infinity };
function setSelectionOpacity(world: World, percent: number): boolean {
const editor = getDocumentEditor(world);
const selected = getSelection(world).filter((entity) => !isPrompt(entity));
if (!selected.length) return false;
const value = percent / 100;
for (const entity of selected) {
if (Math.round((entity.get(Computed)?.opacity ?? 1) * 100) === percent) continue;
editor.editProperty(entity, 'opacity', value === 1 ? false : value);
syncKeyframe(world, editor, entity, 'opacity', value);
}
toast(`Opacity set to ${percent}%`, { id: 'opacity', classNames: { toast: 'inset-x-0 mx-auto w-fit!' } });
return true;
}
function typedPercent(digits: string): number {
if (digits.length > 1) return Number(digits);
return digits === '0' ? 100 : Number(digits) * 10;
}
function typeOpacity(world: World): void {
const now = performance.now();
let digits = now - opacityTyped.at < OPACITY_WINDOW ? opacityTyped.digits : '';
for (const key of world.get(Keys)?.pressed ?? []) {
if (!/^\d$/.test(key)) continue;
digits = `${digits}${key}`.slice(-2);
}
opacityTyped = setSelectionOpacity(world, typedPercent(digits)) ? { digits, at: now } : { digits: '', at: -Infinity };
}
/**
* Sends the selection to the front or the back of its siblings. Which of two
* nodes is drawn on top is which of them the file lists last, so a restack is
@ -466,6 +506,8 @@ const PRESSED_SHORTCUTS: readonly Shortcut[] = [
{ keys: ['d', 'mod', '!shift'], action: duplicateSelection },
{ keys: ['g', 'mod', '!shift'], action: groupSelection },
{ keys: ['g', 'mod', 'shift'], action: ungroupSelection },
{ keys: ['t', 'control', 'alt'], action: tidySelection },
{ keys: ['†', 'control', 'alt'], action: tidySelection },
{ keys: ['enter', 'mod', '!shift', '!alt'], action: wrapSelectionInScene },
{ keys: ['enter', 'mod', 'alt', '!shift'], action: wrapSelectionInSequence },
{ keys: ['enter', 'mod', 'alt', 'shift'], action: unwrapSequenceSelection },
@ -481,12 +523,18 @@ const PRESSED_SHORTCUTS: readonly Shortcut[] = [
{ keys: ['0', 'mod'], action: zoomActualSize },
{ keys: ['1', 'mod'], action: zoomToFit },
{ keys: ['2', 'mod'], action: zoomToSelection },
{ keys: ['o', 'mod', 'shift', '!alt'], action: revealInAssets },
{ keys: ['o', 'mod', 'alt', '!shift'], action: revealPickedInFinder },
{ keys: ['ø', 'mod', 'alt', '!shift'], action: revealPickedInFinder },
...[...'0123456789'].map((digit) => ({ keys: [digit, '!mod'], action: typeOpacity })),
{ keys: ['v', '!mod'], action: selectTool(ToolType.MOVE) },
{ keys: ['h', '!mod'], action: selectTool(ToolType.HAND) },
{ keys: ['f', '!mod'], action: selectTool(ToolType.SCENE) },
{ keys: ['t', '!mod'], action: selectTool(ToolType.TEXT) },
{ keys: ['r', '!mod'], action: selectTool(ToolType.RECT) },
{ keys: ['m', '!mod'], action: selectTool(ToolType.OBJECT_MASK) },
{ keys: ['n', 'shift', '!mod'], action: (world) => { forkSelection(world); } },
{ keys: ['n', '!mod'], action: (world) => { insertPrompt(world); } },
{ keys: ['a', '!mod'], action: seekFrames(-1) },
{ keys: ['d', '!mod'], action: seekFrames(1) },
{ keys: ['w', '!mod'], action: seekSeconds(1) },

View File

@ -7,7 +7,7 @@
// edit in the file).
import { AssetId, Library } from '@diffusionstudio/runtime';
import { AssetLibrary, MANIFEST_FILE, ASSETS_DIR } from '@diffusionstudio/assets';
import { AssetLibrary, MANIFEST_FILE, ASSETS_DIR, isUrlSource } from '@diffusionstudio/assets';
import { useTrait, useWorld } from '@diffusionstudio/koota-solid';
import { authoredElement } from '@diffusionstudio/reconciler';
import { isAssetRef, mapAssetInputs } from '@diffusionstudio/jsx';
@ -20,6 +20,8 @@ import type { Asset } from '@diffusionstudio/assets';
import type { AssetInput } from '@diffusionstudio/jsx';
import type { World } from 'koota';
const locations = new WeakMap<World, (source: string) => string>();
/**
* Creates the library of the project at `dir`, attaches it to the world and
* starts loading it. Renames in the library are written through to every
@ -27,7 +29,9 @@ import type { World } from 'koota';
* disposer that flushes the manifest and detaches it.
*/
export function attachLibrary(world: World, dir: string) {
const library = new AssetLibrary(createProjectFS(dir), {
const fs = createProjectFS(dir);
if (fs.absolute) locations.set(world, fs.absolute);
const library = new AssetLibrary(fs, {
onRename: (asset, from) => followRename(world, asset, from),
onRelink: (asset, from) => followRelink(world, asset, from),
});
@ -37,6 +41,10 @@ export function attachLibrary(world: World, dir: string) {
return library;
}
export function assetFilePath(world: World, asset: Asset): string | undefined {
return isUrlSource(asset.source) ? undefined : locations.get(world)?.(asset.source);
}
/** Whether a changed project file is the library's business rather than the JSX's. */
export function isLibraryFile(path: string): boolean {
return path === MANIFEST_FILE || path === ASSETS_DIR || path.startsWith(`${ASSETS_DIR}/`);

View File

@ -66,7 +66,7 @@ const distance = (a: Point, b: Point): number => Math.hypot(a.x - b.x, a.y - b.y
* The middle of the view when the canvas is empty, or when nothing has a free
* side left; the origin when no surface is mounted yet.
*/
export function findEmptyPlacement(world: World, width: number, height: number, gap: number): Point {
export function findEmptyPlacement(world: World, width: number, height: number, gap: number, near?: AABB): Point {
const viewport = viewportBounds(world);
const center = viewport ? centerOf(viewport) : { x: 0, y: 0 };
@ -77,6 +77,7 @@ export function findEmptyPlacement(world: World, width: number, height: number,
const ordered = (inView.length > 0 ? inView : boxes)
.slice()
.sort((a, b) => distance(centerOf(a), center) - distance(centerOf(b), center));
if (near) ordered.unshift(near);
const free = (rect: AABB): boolean => !boxes.some((box) => aabbsIntersect(rect, box));

View File

@ -0,0 +1,163 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { generate, getAssetSpec, isAssetRef, isSerializedAssetRef, isTransformSpec, mapAssetInputs, transform } from '@diffusionstudio/jsx';
import { Prompt, authoredElement } from '@diffusionstudio/reconciler';
import { Cache, RenderSurface, Root, getEntityBounds, getNextName, getSelection, getViewport, screenToWorld } from '@diffusionstudio/runtime';
import { PROMPT_INPUT_IMAGE_MODEL_OPTIONS } from '@/components/genai/config';
import { getDocumentEditor } from './editor';
import { findEmptyPlacement } from './placement';
import { Pointer } from './traits';
import type {
AssetInput, AssetRef, GenerateAudioOptions, GenerateImageOptions, GenerateSpec, GenerateVideoOptions, GenerateVoiceOptions,
SerializedAssetInput, SerializedAssetRef,
} from '@diffusionstudio/jsx';
import type { Point } from '@diffusionstudio/runtime';
import type { Entity, World } from 'koota';
import type { DocumentEditor } from './editor';
export const PROMPT_SCALE = 2;
export const PROMPT_SIZE = { width: 544 * PROMPT_SCALE, height: 248 * PROMPT_SCALE };
export const PROMPT_MIN_SIZE = { width: 400 * PROMPT_SCALE, height: 200 * PROMPT_SCALE };
const PROMPT_GAP = 40;
let focusRequest: Entity | null = null;
export function takePromptFocus(entity: Entity): boolean {
if (focusRequest !== entity || document.querySelector("[role='menu']")) return false;
focusRequest = null;
return true;
}
export function generationOf(ref: AssetRef): AssetRef | undefined {
const spec = getAssetSpec(ref);
if (!isTransformSpec(spec)) return ref;
return isAssetRef(spec.input) ? generationOf(spec.input) : undefined;
}
export function withGeneration(ref: AssetRef, generation: AssetRef): AssetRef {
if (!isTransformSpec(getAssetSpec(ref))) return generation;
return mapAssetInputs(ref, (input) => (isAssetRef(input) ? withGeneration(input, generation) : input));
}
export function generateRef(spec: GenerateSpec): AssetRef {
const { type, ...options } = spec;
switch (type) {
case 'image':
return generate.image(options as GenerateImageOptions);
case 'video':
return generate.video(options as GenerateVideoOptions);
case 'voice':
return generate.voice(options as GenerateVoiceOptions);
case 'audio':
return generate.audio(options as GenerateAudioOptions);
}
}
function reviveInput(input: SerializedAssetInput): AssetInput {
return typeof input === 'string' ? input : reviveRef(input);
}
function reviveRef({ $asset: spec }: SerializedAssetRef): AssetRef {
if ('input' in spec) return transform[spec.type](reviveInput(spec.input));
const options: Record<string, unknown> = {};
for (const [key, value] of Object.entries(spec)) {
options[key] = key === 'refs'
? (value as SerializedAssetInput[]).map(reviveInput)
: key === 'startFrame' || key === 'endFrame' ? reviveInput(value as SerializedAssetInput) : value;
}
return generateRef(options as GenerateSpec);
}
export function templateOf(value: unknown): AssetRef | undefined {
if (isAssetRef(value)) return value;
return isSerializedAssetRef(value) ? reviveRef(value) : undefined;
}
export function promptTemplate(entity: Entity): AssetRef | undefined {
return templateOf(authoredElement(entity)?.props.template);
}
export function promptText(entity: Entity): string {
const template = promptTemplate(entity);
const generation = template && generationOf(template);
const value = generation ? (getAssetSpec(generation) as GenerateSpec).prompt : authoredElement(entity)?.props.prompt;
return typeof value === 'string' ? value : '';
}
export function setPromptText(editor: DocumentEditor, entity: Entity, text: string): void {
const template = promptTemplate(entity);
const generation = template && generationOf(template);
if (!template || !generation) {
editor.editProperty(entity, 'prompt', text);
return;
}
const spec = getAssetSpec(generation) as GenerateSpec;
editor.editProperty(entity, 'template', withGeneration(template, generateRef({ ...spec, prompt: text })));
}
function pointerOrCenter(world: World): Point {
const pointer = world.get(Pointer);
const resolution = world.get(RenderSurface)?.resolution ?? 1;
if (pointer?.over) return screenToWorld(world, pointer.clientX / resolution, pointer.clientY / resolution);
const viewport = getViewport(world);
return screenToWorld(world, (viewport?.width ?? 0) / 2, (viewport?.height ?? 0) / 2);
}
export function forkTemplate(entity: Entity): AssetRef | undefined {
const src = [entity, ...(entity.get(Cache)?.fills ?? [])]
.map((candidate) => templateOf(authoredElement(candidate)?.props.src))
.find((ref) => ref !== undefined);
const generation = src && generationOf(src);
if (!src || !generation) return undefined;
const spec = { ...(getAssetSpec(generation) as GenerateSpec) };
delete spec.seed;
return withGeneration(src, generateRef(spec));
}
export function forkPrompt(world: World, entity: Entity): Entity | undefined {
const template = forkTemplate(entity);
if (!template) return undefined;
const bounds = getEntityBounds(world, [entity]);
const near = bounds ? { minX: bounds.x, minY: bounds.y, maxX: bounds.x + bounds.width, maxY: bounds.y + bounds.height } : undefined;
return insertPrompt(world, findEmptyPlacement(world, PROMPT_SIZE.width, PROMPT_SIZE.height, PROMPT_GAP, near), template);
}
export function forkSelection(world: World): Entity | undefined {
const entity = getSelection(world).find((selected) => forkTemplate(selected));
return entity && forkPrompt(world, entity);
}
export function insertPrompt(
world: World,
at: Point = pointerOrCenter(world),
template: AssetRef = generate.image({ prompt: '', model: PROMPT_INPUT_IMAGE_MODEL_OPTIONS[0].id, aspectRatio: '16:9' }),
): Entity | undefined {
const root = world.get(Root);
if (!root) return undefined;
const editor = getDocumentEditor(world);
const [entity] = editor.insertElement(root, () => (
<Prompt
name={getNextName(world, 'Prompt')}
x={Math.round(at.x - PROMPT_SIZE.width / 2)}
y={Math.round(at.y - PROMPT_SIZE.height / 2)}
width={PROMPT_SIZE.width}
height={PROMPT_SIZE.height}
count={1}
template={template}
/>
));
if (entity) {
editor.select([entity]);
focusRequest = entity;
}
return entity;
}

File diff suppressed because one or more lines are too long

View File

@ -14,6 +14,7 @@ import { RULER_HEIGHT } from "@/engine/timeline";
import { createEffect, onCleanup, untrack } from 'solid-js';
import { toast } from 'somoto';
import { useWorld } from '@diffusionstudio/koota-solid';
import { isPrompt } from '@diffusionstudio/runtime';
import { mount } from '@diffusionstudio/reconciler';
import { getDocumentEditor } from '@/engine/editor';
import { getEditHistory } from '@/engine/history';
@ -28,6 +29,7 @@ import { compileProject, isWindowsDesktop, refreshProject, watchProject } from '
import { captureProjectCover } from '@/projects/cover';
import { useProject } from "@/context/project";
import { useEngineContext } from "@/engine";
import { useAssetSelection, useSelection } from "@/engine/hooks";
import type { Mount } from '@diffusionstudio/reconciler';
import type { EditWriter } from '@/projects/edits';
@ -42,6 +44,10 @@ export function EditorPage() {
const project = useProject();
const world = useWorld();
const engine = useEngineContext();
const { nodes } = useSelection();
const { id: pickedAsset } = useAssetSelection();
const inspectorFloating = () => !uiVisible() && nodes().some((node) => !isPrompt(node));
const assetsFloating = () => !uiVisible() && pickedAsset() !== null;
// Keyed on the folder, not the project: a rename moves it, and everything
// below holds a path — the watcher, the library, the writer — so all of it
@ -254,7 +260,11 @@ export function EditorPage() {
<SidebarLeft />
<div class="bg-border-strong" />
</Show>
<Canvas />
<Canvas
style={inspectorFloating() || assetsFloating()
? { 'clip-path': `inset(0 ${inspectorFloating() ? INSPECTOR_WIDTH + 1 : 0}px 0 ${assetsFloating() ? leftSidebarWidth() + 1 : 0}px)` }
: undefined}
/>
<Show when={uiVisible()}>
<div class="bg-border-strong" />
<Inspector />
@ -287,8 +297,24 @@ export function EditorPage() {
<Soundboard />
</Show>
</Show>
<Show when={inspectorFloating()}>
<div
class="fixed right-0 bottom-0 top-(--titlebar-height) z-30 bg-sidebar border-l border-border-strong"
style={{ width: `${INSPECTOR_WIDTH + 1}px` }}
>
<Inspector />
</div>
</Show>
<Show when={assetsFloating()}>
<div
class="fixed left-0 bottom-0 top-(--titlebar-height) z-30 bg-sidebar border-r border-border-strong"
style={{ width: `${leftSidebarWidth() + 1}px` }}
>
<SidebarLeft />
</div>
</Show>
{/* The Windows title bar stays up with the UI hidden and offers the same. */}
<Show when={!uiVisible() && !isWindowsDesktop()}>
<Show when={!uiVisible() && !isWindowsDesktop() && !assetsFloating()}>
<FloatingProjectHeader />
</Show>
</div>

View File

@ -218,6 +218,8 @@ export class EditorGenAi extends GenAi {
return { type: spec.type, inputId: input.id };
}
if (!spec.prompt.trim()) throw new Error("Needs a prompt");
switch (spec.type) {
case "image": {
const refs = await Promise.all((spec.refs ?? []).map((ref) => this.resolveInput(ref)));

View File

@ -235,8 +235,8 @@ function isSerializedSpec(spec: unknown): spec is SerializedAssetSpec {
// The declarations
function requirePrompt(value: unknown, call: string): string {
if (typeof value !== "string" || value.trim().length === 0) {
throw new Error(`${call} requires a non-empty string prompt`);
if (typeof value !== "string") {
throw new Error(`${call} requires a string prompt`);
}
return value;
}

View File

@ -58,6 +58,7 @@ export { INSPECT_TAG, INSPECT_TYPES, __inspect } from "./inspect";
export type { InspectDeclaration, InspectType, InspectValue } from "./inspect";
export type {
AdjustmentLayerProps,
PromptProps,
AnimatableProperty,
AnimationProps,
AnimationType,

View File

@ -22,6 +22,7 @@ import type { JSX as SolidJSX } from "solid-js";
import type { AssetInput } from "./generate";
import type {
AdjustmentLayerProps,
PromptProps,
AnimationProps,
AudioProps,
CaptionsProps,
@ -90,6 +91,7 @@ export declare namespace JSX {
sequence: SequenceProps & SourceProps;
captions: CaptionsProps & SourceProps;
adjustmentLayer: AdjustmentLayerProps & SourceProps;
prompt: PromptProps & SourceProps;
solidPaint: SolidPaintProps & SourceProps;
linearGradientPaint: GradientPaintProps & SourceProps;
radialGradientPaint: GradientPaintProps & SourceProps;

View File

@ -101,6 +101,7 @@ export const COMPOSITION_TAGS = [
"sequence",
"captions",
"adjustmentLayer",
"prompt",
"solidPaint",
"linearGradientPaint",
"radialGradientPaint",

View File

@ -542,6 +542,16 @@ export type AdjustmentLayerProps =
children?: SolidJSX.Element;
};
export type PromptProps =
& IdentityProps
& PositionProps
& SizeProps
& Pick<CompositeProps, "hidden">
& {
template?: AssetRef;
count?: number;
};
export type RectProps = CommonProps & FillProps & {
/**
* Makes the rect a clip path of its parent: it clips the parent (its fills,

View File

@ -3,7 +3,7 @@
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import { Active, AdjustmentLayer, Animation, AnimationPhase, AnimationType, appendChild, AssetId, Audio, Background, bindAsset, BlendMode, BlendModeType, Blur, Caption, CaptionAlign, CAPTION_PRESET_FILLS, CAPTION_PRESET_STYLES, CaptionType, Chars, ClipHeight, ClipsContent, Computed, Constraint, ConstraintCache, ConstraintType, CornerRadius, createEntity, DEFAULT_BACKGROUND, Color, ColorStop, Delay, Effect, EffectType, Expanded, FontStyle, FramePromises, FrameRate, Generating, GenerationRequest, getActiveEntity, Loop, LoadRequest, Mask, Geometry, GeometryType, getEntityTree, getParentEntity, getParentNode, Hidden, Host, IsClipPath, isText, ItemIndex, KeepAspectRatio, Keyframe, KeyframeTrack, MixedCornerRadius, Mode, Muted, Name, Offset, Opacity, Paint, PaintType, parseColor, PendingSource, PendingSync, Playback, PlaybackRate, Position, removeChild, RenderSurface, resizeEntity, Scale, ScaleMode, ScaleModeType, secondsToFrames, getAsset, getEntityChildren, Group, Sequential, Shader, Size, Stage, Root, Rotation, Scene, Selected, Shadow, Source, SourceFrameRate, setCameraMatrix, setPlayhead, setTimelineView, Stroke, StrokeCap, StrokeJoin, StrokeStyle, SyncRequest, TextAlign, TextBaseline, TextCase, TextRange, TextStyle, TranscriptionRequest, Transition, TransitionType, Trim, UniformScale, Volume, Workarea } from '@diffusionstudio/runtime';
import { Active, AdjustmentLayer, Animation, AnimationPhase, AnimationType, appendChild, AssetId, Audio, Background, bindAsset, BlendMode, BlendModeType, Blur, Caption, CaptionAlign, CAPTION_PRESET_FILLS, CAPTION_PRESET_STYLES, CaptionType, Chars, ClipHeight, ClipsContent, Computed, Constraint, ConstraintCache, ConstraintType, CornerRadius, createEntity, DEFAULT_BACKGROUND, Color, ColorStop, Delay, Effect, EffectType, Expanded, FontStyle, FramePromises, FrameRate, Generating, GenerationRequest, getActiveEntity, Loop, LoadRequest, Mask, Geometry, GeometryType, getEntityTree, getParentEntity, getParentNode, Hidden, Host, IsClipPath, isText, ItemIndex, KeepAspectRatio, Keyframe, KeyframeTrack, MixedCornerRadius, Mode, Muted, Name, Offset, Opacity, Paint, PaintType, parseColor, PendingSource, PendingSync, Playback, PlaybackRate, Position, PromptNode, removeChild, RenderSurface, resizeEntity, Scale, ScaleMode, ScaleModeType, secondsToFrames, getAsset, getEntityChildren, Group, Sequential, Shader, Size, Stage, Root, Rotation, Scene, Selected, Shadow, Source, SourceFrameRate, setCameraMatrix, setPlayhead, setTimelineView, Stroke, StrokeCap, StrokeJoin, StrokeStyle, SyncRequest, TextAlign, TextBaseline, TextCase, TextRange, TextStyle, TranscriptionRequest, Transition, TransitionType, Trim, UniformScale, Volume, Workarea } from '@diffusionstudio/runtime';
import { DEFAULT_MASK_SMOOTHING } from '@diffusionstudio/assets';
import { LOOP_ATTR, parseTime, SOURCE_ATTR } from '@diffusionstudio/jsx';
import { createSignal } from 'solid-js';
@ -525,6 +525,16 @@ export class RuntimeDocument implements ProjectDocument<SceneNode> {
resizeEntity(this.world, entity, { width: 100, height: 100 });
break;
}
case 'prompt': {
entity = createEntity(this.world);
entity.add(Geometry);
entity.set(Geometry, { value: GeometryType.RECT });
entity.add(Position);
entity.set(Position, { x: 0, y: 0 });
entity.add(PromptNode);
resizeEntity(this.world, entity, { width: 576, height: 320 });
break;
}
case 'text': {
// No Size: a text without one sizes itself to its glyphs.
entity = createEntity(this.world);

View File

@ -20,6 +20,7 @@ import type { JSX as SolidJSX } from "solid-js";
import type { AuthoredTree } from "./document";
import type {
AdjustmentLayerProps,
PromptProps,
AnimationProps,
AudioProps,
CaptionsProps,
@ -98,6 +99,7 @@ export const TextRange = hostElement<TextRangeProps>("TextRange");
export const Sequence = hostElement<SequenceProps>("Sequence");
export const Captions = hostElement<CaptionsProps>("Captions");
export const AdjustmentLayer = hostElement<AdjustmentLayerProps>("AdjustmentLayer");
export const Prompt = hostElement<PromptProps>("Prompt");
export const SolidPaint = hostElement<SolidPaintProps>("SolidPaint");
export const LinearGradientPaint = hostElement<GradientPaintProps>("LinearGradientPaint");
export const RadialGradientPaint = hostElement<GradientPaintProps>("RadialGradientPaint");

View File

@ -3,7 +3,7 @@
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
import {
Scene, Group, Sequential, Audio, AdjustmentLayer, Caption, Geometry, IsClipPath,
Scene, Group, Sequential, Audio, AdjustmentLayer, Caption, Geometry, IsClipPath, PromptNode,
Paint, Cache, Stage,
} from '../traits';
import { GeometryType, PaintType } from '../constants';
@ -38,6 +38,10 @@ export function isAdjustmentLayer(entity: Entity): boolean {
return entity.has(AdjustmentLayer);
}
export function isPrompt(entity: Entity): boolean {
return entity.has(PromptNode);
}
export function isCaption(entity: Entity): boolean {
return entity.has(Caption);
}

View File

@ -31,6 +31,8 @@ export const Audio = trait();
export const AdjustmentLayer = trait();
export const PromptNode = trait();
// Tag marking an entity as a clip path (`<rect clipPath>`). Clip paths are
// ChildOf their target; Cache.clipPaths on the target is derived from IsClipPath + ChildOf queries.
export const IsClipPath = trait();