- Updated the CLI to include a new `window` command for managing the application window state (show/hide).
- Improved the background launch logic to prevent launching the app if it is already running in the background.
- Added new tray icons for different application states (busy, idle, window) to enhance user experience.
- Refactored the `agent-chat` module to improve the handling of the agent chat endpoint and its lifecycle.
- Introduced a `WindowHost` class to manage the main window's lifecycle, ensuring it only exists when needed and integrates with the tray functionality.
- Removed the deprecated `headless` module and adjusted related logic to streamline the application flow.
- Updated the DAPI server to support window management, allowing tools to interact with the main window effectively.
- Changed font family from 'Poppins' to 'Outfit' and adjusted font weights in both `guinea.ts` and `spotlight.ts` for consistency in text styling.
- Updated the highlight color in `spotlight.ts` to improve visual distinction in captions.
- Introduced new options for the `fonts` command to filter by provider and popular fonts, enhancing user experience in font selection.
- Updated the documentation to reflect the inclusion of Google Fonts and local fonts, clarifying how to access and utilize them in projects.
- Implemented a new `FontDropdown` component in the web inspector for improved font selection, allowing users to choose from popular and Google Fonts seamlessly.
- Refactored existing components to integrate the new font selection logic, ensuring consistency across the application.
- Updated the animations documentation to clarify the behavior of presets, emphasizing that they scale or shift the node's authored properties rather than replacing them.
- Modified the `applyAnimation` function to ensure that computed values are compounded correctly for overlapping animations, improving the accuracy of visual effects during animation playback.
- Enhanced the `present` function to support background track processing in the `media_segment` tool, allowing for real-time tracking and reporting of mask states.
- Introduced a new `presentContext` function to manage the output of background tracks, including their progress and contact sheets.
- Updated the `context` tool to report the status of ongoing background tracks, including their state and associated images.
- Modified the `media_segment` tool to return tracking states and paths, improving integration with the `<mask>` component in the editor.
- Enhanced documentation to clarify the new behavior and usage of background tracks in the context of media segmentation.
- Added the `ClipPathBar` component to the toolbar for managing clip paths.
- Updated the toolbar logic to clear the clip path target when switching tools.
- Refactored the toolbar's tool selection to use a `Switch` statement for better readability.
- Integrated clip path functionality into the inspector, allowing users to begin clip paths directly from the UI.
- Updated cursor settings to reflect the new clip path tool type.
- Enhanced keyboard shortcuts to support clip path actions, improving user interaction.
- Incremented version numbers in `package.json` and `package-lock.json` for the main editor, CLI, desktop, web applications, and JSX package to reflect the new release.
- Ensured consistency in versioning across all related packages to maintain compatibility.
- Introduced the `media_segment` tool to segment objects in video footage using SAM 2.1, allowing users to track objects and generate mask files.
- Implemented input validation for prompts, ensuring at least one point or a bounding box is provided for segmentation.
- Enhanced the output structure to include mask paths and tracking details, improving integration with the `<mask>` component in the editor.
- Added comprehensive documentation for the new tool, detailing usage, input parameters, and expected outputs.
- Updated related components and tests to support the new functionality, ensuring robust performance and user experience.
- Integrated the `ObjectMaskBar` into the `Toolbar`, allowing for dynamic display based on the selected tool.
- Updated the `Inspector` component to remove the deprecated `ObjectMaskPanel`, streamlining the interface.
- Refactored shortcut handling to include new actions for the object mask tool, improving user interaction.
- Enhanced object mask interaction logic to support dynamic operation modes (add/subtract) based on user input.
- Improved overall code organization and documentation for better clarity and maintainability.
- Updated references from "matte" to "mask" across various components and files to align with new terminology.
- Adjusted related logic in the `EffectsInspector`, `ObjectMaskInspector`, and other components to ensure consistent behavior with the updated mask handling.
- Refactored caching and serialization processes to accommodate the new mask structure, enhancing overall functionality.
- Improved documentation and type definitions to reflect the transition from mattes to masks, ensuring clarity for developers.
- Renamed mask-related components and functions to reflect the new terminology of "clip path" throughout the codebase.
- Introduced a new `ClipPathsSettings` component for managing clip paths in the inspector.
- Updated the `Inspector` to integrate clip path settings and adjusted related logic to accommodate the new structure.
- Enhanced the `ObjectMenu` to change "Set mask target" to "Set clip path target" for clarity.
- Refactored various utility functions and traits to replace mask references with clip path equivalents, ensuring consistent behavior across the application.
- Updated documentation and type definitions to reflect the transition from masks to clip paths, improving clarity for developers.
- Added new `ObjectMaskPanel` component to manage object masks within the effects inspector.
- Introduced functionality for adding and managing object masks, including dropdown menus for selecting masks and controls for editing.
- Enhanced the `EffectsInspector` to support object mask interactions, including displaying existing masks and allowing users to track new objects.
- Implemented brush functionality for painting masks directly onto video clips, improving user experience for object segmentation.
- Updated cursor styles for the object mask tool to provide clearer visual feedback during interactions.
- Refactored related components and utilities to support the new object mask features, ensuring seamless integration with existing functionality.
- Updated `package.json` to include new model export for `./models`.
- Expanded `constants.ts` to define new types and structures for SAM 2.1 models, including `Sam2Model` and `Sam2Geometry`.
- Refactored `gpu.ts` to utilize dynamic image sizes and geometry for frame processing, improving flexibility in GPU operations.
- Enhanced `load.ts` to support model-specific loading and progress tracking, optimizing the user experience during model initialization.
- Updated `mask.ts` to redefine mask handling with logits and improved painting functionality for strokes.
- Refactored `tracker.ts` to integrate new model structures and enhance video tracking capabilities with corrected masks.
- Improved documentation and type definitions across the codebase for better clarity and usability.
- Introduced `readFloat16Initializer` function in `onnx.ts` to read float16 initializers from model bytes, improving model loading capabilities.
- Updated `createSession` function in `sessions.ts` to accept additional options for session configuration, allowing for enhanced flexibility during model execution.
- Modified the `load` function in `load.ts` to utilize the new initializer and pass position tokens to the `Sam2Video` constructor, optimizing video processing.
- Refactored `Sam2Video` class to handle position tokens, ensuring accurate representation of the vision encoder's positional embeddings during video tracking.
- Added the `Object Mask` tool to the toolbar and sidebar menu, allowing users to select and utilize this new functionality.
- Introduced a new SVG icon for the `Object Mask` tool to improve visual representation in the UI.
- Updated the `Inspector` component to display the `ObjectMaskPanel` when the `Object Mask` tool is selected.
- Enhanced the `Effect` options to include `opacity`, allowing for more refined control over visual effects.
- Updated documentation to reflect the new `<mask>` element and its properties, improving clarity for developers.
- Refactored related components to ensure seamless integration of the new tool and its functionalities.
- Updated `export.py` to include model size as a parameter for ONNX graph exports, improving flexibility in model loading.
- Refactored the `load` function to accept a checkpoint string, allowing dynamic model loading based on specified size.
- Introduced new methods in `tracker.ts` for holding and previewing frames, enhancing the user experience during object segmentation.
- Updated type exports in `video.ts` to include `FrameRequest`, facilitating better integration with the tracking system.
- Improved documentation and comments throughout the codebase for clarity on new features and functionality.
- Deleted the `convert_fp16.py` script, which was responsible for converting ONNX models from fp32 to fp16.
- Added `export.py` script to export fixed-shape ONNX graphs for SAM 2.1 video tracking, ensuring compatibility with input parameters for image size and memory frames.
- Updated `constants.ts` to reflect changes in model repository links and fixed shapes for `IMAGE_SIZE` and `MEMORY_FRAMES`.
- Adjusted memory management and constants in various source files to align with the new export functionality.
- Introduced `convert_fp16.py` script for converting ONNX models from fp32 to fp16, optimizing memory usage while maintaining input/output compatibility.
- Updated `MODEL_REPO` in `constants.ts` to point to the new fp16 model repository, reflecting changes in model file sizes.
- Enhanced `weights.ts` to retain model files in the cache, improving file management during model loading.
- Introduced the `@diffusionstudio/sam2` package for video object segmentation and tracking in the browser using ONNX Runtime on WebGPU.
- Added essential files including `package.json`, `tsconfig.json`, and various source files for core functionality, constants, and GPU processing.
- Implemented model file fetching and caching from Hugging Face, along with memory management for tracking objects across frames.
- Included shaders for preprocessing video frames and transposing feature maps for memory attention.
- Established a comprehensive API for loading models, processing frames, and tracking objects with segmentation masks.
- Ensured type definitions and module exports for seamless integration with existing projects.
- Updated `WINDOWS.md` to reflect the successful execution of `dapi fonts` and `dapi screenshot` on Windows, including functionality from minimized windows.
- Improved temporary directory handling in `present.ts` to resolve issues with short names in the `%TEMP%` directory.
- Modified `knownDirs` in `env.ts` to ensure proper paths for local application data on Windows, enhancing compatibility for the agent chat feature.
- Updated the `basename` function to trim trailing slashes from the input path, ensuring more accurate extraction of the base name.
- Enhanced the logic to handle both forward and backward slashes, improving compatibility across different operating systems.
- Deleted the `whoami` command from the CLI, including its associated action and description.
- Updated various documentation files to remove references to the `whoami` tool, ensuring consistency across the project.
- Adjusted related test cases and API handlers to reflect the removal of the `whoami` functionality.
- Added a new `dev-shim.mjs` script for creating and removing a Windows shim for the `dapi` command, ensuring it points to the correct executable.
- Updated the `package.json` to include new npm scripts for creating and removing the shim.
- Refactored the CLI installation logic to support Windows, including the addition of `refreshCliShim` to rewrite the shim on app updates.
- Enhanced the `mcp-config` to accommodate Windows-specific command handling for the `dapi` binary.
- Introduced tests for the Windows CLI installation and cloud sync functionalities to ensure reliability across platforms.
- Updated the caption positioning function to account for `Sequential` traits, ensuring proper handling of parent nodes that are not spatial constructs.
- Improved the logic for retrieving the parent node, enhancing the accuracy of caption placement within the media runtime.
- Added `disallowedTools` property to `ClaudeSession` to restrict usage of "Agent" and "Task".
- Updated the arguments in `spawnAppServer` function in `codex.ts` to disable multi-agent features, enhancing configuration control.
- Added `playhead` property to the scene, allowing the editor to track the playhead's position when the project is opened, improving playback accuracy.
- Introduced `editPlayhead` function to manage playhead updates, ensuring the file reflects the playhead's position during scrubbing and timeline interactions.
- Updated documentation to clarify the role of the `playhead` property in the scene, emphasizing its importance for project state management.
- Enhanced the `scene.md` documentation to clarify the `workarea` property and its implications for playback and export.
- Revised the `stage.md` documentation to provide clearer guidance on the `camera` property, emphasizing the importance of setting a camera for non-standard compositions.
- Updated the default camera settings in the `world.ts` and `types.ts` files to better frame typical project dimensions, ensuring a more intuitive starting point for users.
- Deleted the `fetch` command from the CLI and associated handlers, streamlining the toolset.
- Updated documentation to reflect the removal of the `fetch` command, emphasizing the use of `yt-dlp` directly for downloading videos.
- Adjusted various components and types to remove references to `fetch`, ensuring consistency across the codebase.
- Enhanced error handling to clarify the absence of the `fetch` functionality.
- Introduced a new test suite in `projects.init.test.ts` to validate the behavior of the `initProject` function, ensuring it correctly initializes new projects and handles existing project directories.
- Refactored the `initProject` function in `projects.ts` to streamline project setup, including improved handling of entry files and package.json records.
- Enhanced error handling for invalid paths in the `initProject` function, ensuring only absolute paths are accepted.
- Updated the `scaffold` function to utilize the new `ensureRecord` function, improving code clarity and maintainability.
- Added tests for various scenarios, including creating new projects, preserving existing project data, and handling invalid inputs, to ensure robust project management functionality.
- Bumped version numbers in `package.json` and `package-lock.json` for the main editor and all related applications (CLI, desktop, web, and JSX) to 0.205.0, ensuring consistency across the project.
- Updated the `ProjectInfo` documentation to clarify project folder definitions.
- Changed request parameters in `MainRequestMap` for `PROJECTS_LIST` and `PROJECTS_RESOLVE` to use `dirs` and `dir` respectively, enhancing clarity and consistency.
- Refactored the `listProjects` function to accept an array of directories instead of a single root, improving project listing logic.
- Adjusted the `resolveProject` function to streamline project resolution based on directory input.
- Updated various components to reflect changes in project data handling, ensuring compatibility with the new structure.
- Updated the file writing logic in `main.ts` to ensure that generated asset folders are properly marked to prevent unwanted reloads during writes.
- Introduced detailed logging for model call procedures in `trpc.ts`, capturing input arguments and execution time for better debugging and monitoring.
- Simplified logging in `gen-ai.ts` by removing unnecessary details from generation logs.
- Enhanced asset management in `library.ts` by adding functionality to handle partial assets and ensuring unique paths for stored assets.
- Replaced the `useModifiers` hook with `useTransforms` to better reflect the new transform functionality.
- Updated components to utilize the new `useTransforms` hook, including `ActionBar` and `PromptInputActions`, improving the handling of transformations.
- Introduced new `TransformItem` component for managing transform actions in the UI.
- Refactored asset handling logic to accommodate the new transform types and ensure compatibility with existing features.
- Removed deprecated modifier-related code and documentation, streamlining the codebase and enhancing clarity.
- Updated documentation to reflect changes in transform functionality and usage examples.
- Updated the `Assets` component to include support for partial assets, allowing for better management of assets that are still generating.
- Introduced a new `PartialAssetItem` component to represent partial documents in the asset panel, providing options to retry or delete failed generations.
- Modified the `AssetInfoPanel` to display information about partial assets, including their status and error messages.
- Refactored the `useAssetSelection` hook to accommodate both assets and partial assets, improving the selection logic.
- Updated the `FolderItem` component to count partial assets alongside regular assets, enhancing folder statistics.
- Removed the `sourceErrorSystem` as part of the cleanup, streamlining error handling for asset generations.
- Added documentation updates to reflect changes in asset handling and the introduction of partial assets.
- Removed the `@modelcontextprotocol/sdk` dependency from `package-lock.json` and `package.json`.
- Updated the CLI client to connect via Streamable HTTP instead of a socket, enhancing communication with the MCP server.
- Simplified the `ping` and `call` functions in `cli-client.ts` to utilize the new connection method.
- Removed the `mcp-proxy.ts` file as it is no longer needed with the new HTTP integration.
- Updated documentation to reflect changes in MCP server access and removed references to the socket transport.
- Cleaned up unused imports and refactored related code for improved clarity and maintainability.
- Updated the `ToolItem` component to include support for displaying images associated with tool results.
- Modified the `expandable` logic to account for images in addition to existing detail and output properties.
- Introduced a new `collectResult` function to process tool result parts, allowing for the inclusion of images while managing output text.
- Added tests for the `collectResult` function to ensure proper handling of text and image parts, including limits on the number of images and text truncation.
- Updated the `media_transcribe` command to write transcripts to a JSON file, including an output option for specifying the file path.
- Implemented a new `presentTranscript` function to handle the writing of transcripts and return relevant metadata.
- Added filtering options for the `logs` command, including `since` and `contains`, to improve log retrieval capabilities.
- Introduced tests for the new logging features and updated documentation to reflect changes in the `media_transcribe` tool and logging options.
- Added `agent-chat.ts` to manage the agent chat host process, including spawning and communication with the host.
- Updated `main.ts` and `main-channels.ts` to integrate agent chat endpoint handling and lifecycle management.
- Introduced new components for the agent chat interface, including `ChatPanel`, `Composer`, and `HeaderActions`, enhancing user interaction.
- Implemented file attachment handling in the chat with `attachments.tsx` and related functionality.
- Updated package dependencies to include `@diffusionstudio/agent-chat` and related libraries for improved chat capabilities.
- Refactored project deletion logic to ensure associated chats are properly managed.
- Revised README.md to clarify the app's MCP server functionality and updated prompt usage instructions, replacing `dapi` references with `diffusion` for consistency.
- Refactored MCP server configuration to use `diffusion` as the server name instead of `dapi`, ensuring alignment with the updated terminology.
- Enhanced tests to reflect changes in server naming and configuration, ensuring accurate validation of the new structure.
- Added new skills and prompts for `editor` and `watch`, improving user guidance and interaction with the app's features.
- Replaced all instances of `runsIn` with `environment` across the DAPI codebase to standardize terminology.
- Updated related tests and tool definitions to reflect the new property name, ensuring alignment in functionality and documentation.
- Enhanced error handling in the DapiServer class to maintain robustness with the updated property references.
- Removed Babel-related dependencies from apps/cli/package.json and package-lock.json to streamline the build process.
- Added esbuild as a devDependency in apps/cli/package.json for improved bundling.
- Updated README.md to provide a more detailed description of the dapi CLI tool, clarifying its role as a client of the MCP server and stdio proxy.
- Deleted untyped-modules.d.ts to eliminate unnecessary type declarations for Babel presets.
- Enhanced comments in stage-cli.mjs to clarify the staging process for the dapi CLI within the Electron app.
- Removed fallback parameters from the `field` function in CLI tools to simplify input handling.
- Updated command options for `media_grab`, `media_filmstrip`, and `media_waveform` to remove default descriptions, relying on schema definitions instead.
- Enhanced the `logs` and `fonts` tools to provide clearer descriptions for input fields.
- Improved the `windowFields` schema to include default value descriptions for better user guidance.
- Adjusted the `models` tool to specify model types in the description, enhancing clarity for users.
- Replaced the `time` argument with `times` in the CLI commands for consistency across tools.
- Updated the `capture` and `media_grab` tools to handle multiple timestamps, improving flexibility in frame extraction.
- Refactored input validation to utilize `zod` for better error handling and user feedback.
- Removed the `help.ts` module as its functionality has been integrated into the main CLI structure.
- Updated documentation to reflect changes in command syntax and input handling, ensuring clarity for users.
- Added `zod` as a dependency in package-lock.json and apps/cli/package.json for input validation.
- Introduced new help.ts module to provide descriptions and validation for CLI commands, ensuring consistency with the app's tool catalog.
- Refactored cli-client.ts to improve timeout handling for tool calls, enhancing error management.
- Created mcp-proxy.ts to facilitate communication between agents and the app's MCP server.
- Updated index.ts to streamline command execution and improve error handling.
- Implemented mcp-config.ts and mcp-install.ts for managing MCP server configurations across different agents.
- Enhanced desktop application to support MCP registration from the menu, improving user experience.
- Added `horizontalConstraintName` and `verticalConstraintName` functions to map constraint types to their respective names.
- Updated `ConstraintsRow` component to utilize these functions for setting `constrainX` and `constrainY` properties.
- Introduced `HORIZONTAL_CONSTRAINTS` and `VERTICAL_CONSTRAINTS` constants in the runtime to define available constraints.
- Enhanced documentation to clarify the behavior of `constrainX` and `constrainY` properties in the JSX elements reference.
- Introduced new keyboard shortcuts for shuttle playback (J and L) to control scene speed and direction.
- Added a stop playback shortcut (K) to halt the active scene.
- Enhanced playback functionality with `shuttlePlayback` and `stopPlayback` methods in the runtime actions.
- Updated the playback system to accommodate shuttling speeds and ensure proper audio context management.