hermes update rebuilt and repackaged the Electron app (6-38 min on Windows)
on every commit move, because _packaged_desktop_current_for_head requires the
app's baked commit to be HEAD before the freshness receipt is consulted.
The receipt already content-hashes every desktop build input (sources,
lockfile, build scripts, prepared icons/native trees, renderer env, Node,
platform, arch) and the output. The baked install stamp is the one input it
cannot see. desktop_build_reuse asks whether a build at HEAD would bake a
stamp that differs in anything but commit provenance (commit, commitDate,
dirty, builtAt), using write-build-stamp.mjs's own resolver, and reuses the
packaged app when the receipt is current and it would not.
The reused app keeps naming the commit it was built from: the stamp is never
rewritten (bundle-swap would relaunch into a "swap" forever), and bundle-skew
counts desktop runtime-path commits since it, which identical inputs rule out.
The Nix package had no Webapp bundle, and `hermes webapp` could not build
one in the read-only store. nix/webapp.nix builds it in the npm workspace
the hook prepares (no native rebuild, no Electron), exposed as
`.#webapp`. The agent assembler takes it as a `webapp` frontend: the
wrapper sets HERMES_WEBAPP_DIST to it, as HERMES_WEB_DIST is set for the
dashboard.
hermes_cli/webapp.py serves a HERMES_WEBAPP_DIST bundle as shipped: no
freshness check, no build lock, never rebuilt or marked stale, and
`--force-build` says it cannot rebuild it.
The Webapp recipe installs Desktop's workspace, and node-deps prepared
Electron's binary for any install that selects apps/desktop. The browser
bundle never runs Electron, so prepareNodeDependencies now takes
`electron: false` and build-webapp.mjs passes it. Desktop builds keep the
default.
keepPreviousGeneration skipped a copy that failed for an unexpected reason (a full disk, a permission error) without a word: the build recorded its receipt and published, and tabs still on the previous generation lost that chunk with nothing in the build log saying why.
The build still never fails here, but a failed copy, an unexpected directory error, or a failed cleanup of the partial copy now warns with the path and error code. The expected cases stay quiet: no previous build, a rejected manifest entry, a new file standing where the name or its directory would go (including EEXIST from COPYFILE_EXCL).
A Webapp tab that was already open crashed the next time it opened a
lazy view such as Settings once `npm run build:webapp`, `hermes webapp`
or `hermes update` had published a new renderer. Publication replaces
apps/desktop/dist-webapp, which the gateway reads per request, so the
old entry's hashed lazy chunks 404 and the rejected import reaches the
app's error boundary until the page is reloaded.
Before the receipt is written, the builder now copies into the new
generation the files the previous generation's Vite manifest names that
the new one lacks: only that generation's own files, so a rebuild keeps
exactly one generation back; only while its bytes still match its
receipt (freshness.mjs gains outputIntact, which ignores the marker a
failed `hermes update` leaves in the bundle: that still forces the
rebuild, but no longer costs open tabs their chunks); only regular
files inside it, never through a symlink and never over a new file. A
bad entry is skipped and never fails the build. The builder names the
manifest itself, so it exists whatever the Vite config says.
Refs #141864, #93508
(cherry picked from commit 096996d1a591917b21c96e104d20e9dc54c5d850)
Carried from BearHuddleston's #93508 onto main. The Webapp relay and the
profile-incarnation fence stay out: carried code follows main's
named-profile rule and refuses a missing or deleted profile home.
- Webapp routes: one-use launch tickets, window handoff, and gateway
WebSocket tickets minted for the signed-in browser
- Browser surface policy and the SPA document policy
- A host terminal for the browser that survives reconnects
- Staged browser uploads and file tickets; file.attach takes a staged upload
- hermes sessions adopt
- Webapp bundle build helpers and the renderer freshness check
- Regenerated gateway contracts
Co-authored-by: emozilla <emozilla@nousresearch.com>
From 40.10.3 on, every Electron line unpacks its binary with
@electron-internal/extract-zip, an MSVC-built binding that imports
VCRUNTIME140.dll. A Windows host that never installed the VC++
Redistributable has no copy, so install.js dies with ERR_DLOPEN_FAILED.
This is the failure that made bb8280b7537 roll Electron back to 40.10.2.
Our Python ships vcruntime140.dll. source_build_env() passes its path to
the Node builders as HERMES_VC_RUNTIME, and electron-runtime.mjs copies it
beside the extractor's binding right before it runs install.js. Node
loads bindings with LOAD_WITH_ALTERED_SEARCH_PATH (libuv uv_dlopen), so
Windows finds that copy first. This is the same app-local copy
fdea3e92e01 ships beside ARM64 rg.exe.
Clean Windows Sandbox (ARM64, no redist), Node 26.7.0, electron 44.9.3:
- before: install.js exits 1, "Cannot find native binding", ERR_DLOPEN_FAILED
- after: install.js exits 0, dist/electron.exe --version prints v44.9.3
Electron 42 dropped the postinstall that unpacked its binary into
node_modules/electron/dist. The binary now appears on the first
require('electron'), so parallel vitest workers race to unpack it
("Text file busy" in #128267's CI), and `hermes desktop --source` finds
no path.txt to launch.
electron-runtime.mjs runs Electron's own install.js once: after npm ci
for the desktop workspace, and in CI before the parallel checks. It first
checks the installed package against the desktop pin and the lockfile,
and does nothing when the runtime is already in place. On the current
40.10.2 pin, whose postinstall still unpacks the binary, it is a no-op.
Salvaged from #137648 (the runtime-preparation subset only).
save-python-cache and save-node-cache now default to false on
pull_request events, like save-tools-cache. Their restore-only steps
read the exact keys the saving steps write on main, so a PR gets main's
uv and npm caches and leaves nothing PR-scoped behind.
Restore-only used to mean an explicit-save flow: run- and job-scoped
keys meant for save-pm-cache or a caller's actions/cache/save to write
later. Nothing calls save-pm-cache, and the last caller restoring from
the npm explicit-save key moved to its own cache, so the flow and its
outputs are deleted.
`npm run builder` with an arch flag stages native deps into
apps/desktop/build/native-deps-<platform>-<arch>
(scripts/run-electron-builder.mjs). The product guard in
scripts/build/frontend-common.mjs accepted only apps/desktop/build/native-deps
as a generated destination in the tree, so every source build with an
explicit arch died before staging:
Error: Output must be outside source or in a supported generated
destination: ...\apps\desktop\build\native-deps-win32-x64
Seen on blacksmith-32vcpu-windows-2025 packaging the Desktop for the W3
attach lane (run 39053936919, `npm run builder -- --dir --publish never
--win --x64`). The guard now also accepts exactly
native-deps-(darwin|linux|win32)-(x64|arm64) under apps/desktop/build.
A nested path below one, or another platform/arch, is still refused.
Red -> green: run-electron-builder.test.mjs "source multiarch ..." now
asserts each per-target --out it hands stage-native-deps.mjs passes
productOutput (red on main: "Output must be outside source ..." for
native-deps-darwin-x64). product-builders.test.mjs gains the accepted and
refused per-target names (10 passed).
Refs #134889 (W3)
Refs #140337
Vite inlines every VITE_* variable into the renderer and NODE_ENV selects its mode, but the
desktop receipt recorded only sources and prepared inputs. After `VITE_PERF_PROBE=1 npm run
build`, the next plain build reused (and productCurrent certified) the renderer that still
carries the perf probe; a NODE_ENV change had the same gap.
The desktop build inputs now record the renderer environment (VITE_* plus NODE_ENV, unset
read as vite's production default), so both rendererCurrent and productCurrent require it to
match. Reported by Greptile on #135092.
rendererCurrent and productCurrent share hostReceipt() for the schema/product/host checks. The stamped outputs come from bundleElectronMain's own electronOutputs list, so a renamed or added bundle can never be copied stale from the previous dist. The reuse copy skips node_modules/ and native/, which copyNativeTree recopies from nativeDeps right after, and a --typecheck build never reuses a renderer that skipped tsc.
The dashboard build (Vite 8 + Rolldown + Tailwind v4) saturates every CPU
on small hosts (200%+ top readings, OOMs and frozen SSH sessions on 2–4
vCPU VPS boxes). Rolldown exposes no JS-level worker knob, but its native
binding parallelizes with rayon, which honours RAYON_NUM_THREADS.
Cap both sides of the build, everywhere it runs:
- scripts/build/web.mjs (the npm run build / packaging entry): limit the
native bundler's rayon pool to half the available cores before vite is
imported, and re-exec once with a --max-old-space-size cap (sized from
the cgroup memory limit, 75%, floored at 1 GB, ceiling 4 GB) unless
the user already set one.
- hermes_cli/web_build_limits.py + build_source_web(): the same caps for
the install/update/dashboard-triggered source build path.
User env always wins (existing NODE_OPTIONS heap flag or
RAYON_NUM_THREADS is preserved); HERMES_WEB_BUILD_MAX_OLD_SPACE_SIZE /
HERMES_WEB_BUILD_THREADS tailor the caps, and HERMES_WEB_BUILD_LIGHT=1
tightens them (1 thread, 1 GB heap) for hosts that cannot spare full CPU
during the build — the light-build idea from the closed PR #78131.
The reuse receipt check in node-deps.mjs accepted a tree where every locked
directory existed but package contents had been stripped (partial cleanup or
snapshot restore). npm trusts the hidden lockfile and never repairs such
entries, so every 'hermes update' skipped npm ci, the TUI build died on the
missing esbuild manifest, and the stale receipt kept the loop closed (#128935).
Probe each locked entry's package.json — the same file product builders
resolve tools through — so a gutted tree falls back to a real npm ci.
publishDirectory moved a finished product into place with one renameSync.
On Windows an antivirus scanner opens every freshly written file, so the
rename onto build/native-deps (right after hud-modifier-monitor.exe is
compiled) or hermes_cli/web_dist failed EPERM and threw away a build that
took minutes.
retryHeld retries EPERM/EACCES/EBUSY for about 3s and raises every other
code at once. All three publication renames use it, as does the HUD
helper's staging rename. The previous product's backup now sits beside
the output instead of inside withProduct's scratch tree, so a rollback
that also fails no longer has its only copy deleted by scratch cleanup;
the error names where it is.
Co-authored-by: dskwe <6286305+dskwe@users.noreply.github.com>
Co-authored-by: juliorobertoalvarez <39749815+juliorobertoalvarez@users.noreply.github.com>
Co-authored-by: bighamx <10262931+bighamx@users.noreply.github.com>
Co-authored-by: lzmuhioin <1814826+lzmuhioin@users.noreply.github.com>
WIP for the freshness-guard-os-metadata unit: the UpdateLock claim in
complete_source_checkout plus its tests are complete but unrun; the
Thumbs.db/Desktop.ini skip predicate is in but its test rows are not yet
extended, and the rmTree ENOTEMPTY test has not been written yet.
Opening the checkout in Finder mid-build lets .DS_Store, AppleDouble ._*
sidecars or .localized land inside a hashed input dir (apps/desktop, apps/shared,
assets, scripts/build). treeHash then sees a new input and recordProduct throws
"Build inputs changed during compilation; retry the build", so `hermes update`
rebuilds the desktop app, fails closed, and never prints Update complete.
Finder metadata is never a build input, so skip it as a baseline in treeHash —
covering source, output and prepared trees uniformly rather than only the source
predicate. A real source change is still detected and still names nothing new to
hash, so clean checkouts (CI) hash byte-identically.
macOS's Finder/Spotlight can drop a fresh .DS_Store (or xattr) into a
directory while we're removing it, making rmSync throw ENOTEMPTY on the
first pass. That file is cosmetic garbage — retry the removal so the
race can't abort an otherwise-clean product build.
Salvaged from #123078 (its freshness.mjs hunk is subsumed by the broader
metadata skip; the unrelated CI-workflow deletions in that PR are not
included).
Co-Authored-By: dlshumway <85191357+dlshumway@users.noreply.github.com>
write-build-stamp.mjs rewrites the stamp's `builtAt` clock on every build, so the
whole-file hash moved under a receipt that had just recorded it. Two builds
overlapping on one tree failed with "Build inputs changed during compilation" for
a difference the build machinery itself made, and the caller turned that into
SystemExit(1).
Hash the stamp's provenance identity instead, key-sorted so a cosmetic
re-serialisation is not a change. The clock is excluded from that comparison but
recorded as `stampClock` — the value the bake actually embedded, captured from
the same read that produces __HERMES_INSTALL_STAMP__ — because packaging copies
the live stamp into the bundle after the receipt is written. Re-reading the stamp
at record time would certify a clock the output never contained, and
productCurrent would then report a bundle current whose packaged clock disagrees
with its baked main, which is exactly what detectBundleSwap must never do.
recordProduct still compares the prepared inputs, so a clock-only restamp cannot
throw. productCurrent additionally requires the recorded clock to match, so a
restamp between bake and packaging is a rebuild rather than a silent divergence.
A receipt predating the field compares by hash alone until the next build
rewrites it; the release packaging path never consults productCurrent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
write-build-stamp.mjs rewrites `builtAt` on every build, and build.mjs runs it
before compilation, so the desktop build rewrote a file its own receipt hashed
as a prepared input. Two concurrent `hermes desktop` builds therefore made
recordProduct() fail with "Build inputs changed during compilation" on a change
the build itself caused, and the caller turned that into SystemExit(1).
Hash the stamp's provenance identity instead of its raw bytes. builtAt is the
only field that moves between two stamps (the tree documents it as
"differs on every rebuild"), and it is a build clock, not provenance: the baked
output changes when commit/payload/tag change, never when time passes. A
non-JSON stamp still falls back to the whole-file hash, so content keeps
deciding freshness and only the clock is ignored.
Unlike b3618bda35be, the stamp cannot simply leave the hash set:
bundleElectronMain bakes its bytes into electron-main.mjs, and
bundle-swap.test.ts pins "same commit rebuilt (builtAt moved)" as a real swap,
so freezing the clock would break a committed test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Nix agent derivation builds scripts/build/*.py from a fileset that
does not include pm/, so importing pm.store.ALL_TARGETS there failed
nix flake check with ModuleNotFoundError. Extend the local target
regex with linux-(x64|arm64)-musl instead.
npm_execpath can point through a symlink, and the manifest sits beside
the resolved CLI, never beside the link: resolve the realpath before
looking for package.json. Layouts without a readable manifest now fall
back to the pre-fix child probe instead of aborting with ENOENT.
Move the regression test to tests-js/node-deps.test.mjs (the module's
own suite) and cover the symlinked-execpath and fallback lanes there.
The npm version probe spawned node-under-node before the reuse
short-circuit, so on Windows a Job-Object EBUSY spawn failure aborted
the whole dependency preparation even when the install was already
complete — leaving the pending-completion marker behind and turning
every launch into the same doomed completion pass (#123933).
npm's own package manifest states its version without any process
creation, so the probe lane can no longer fail: the install-receipt
key keeps its exact value, engine checks still run, and completed
installs reuse with zero spawns.
The workspace npm ci printed nothing until 'added N packages': builders
set CI=1, which turns npm's progress off, and the stage name only went
to the desktop UI's status file.
Print a line before npm ci, and pass --progress=true so a terminal gets
npm's spinner back (npm still shows it only on a TTY, so piped output
such as the desktop app's log stays clean). The flag stays out of the
receipt-keyed args, so existing installs are still reused.
cryptography ships no win_arm64 wheel, so every Windows ARM64 venv sync
compiles it from the sdist and needs MSVC, Clang, Rust and static OpenSSL.
Only setup-hermes.ps1 (and so activate.ps1) prepared that environment,
between a `pm install --tools-only` and the real sync. install.ps1,
`hermes update` and repair ran the same sync without it and failed in
openssl-sys.
PM owns the sync, so PM prepares it. pm/native_build.py holds the adapter
(moved from scripts/build/windows_deps.py) plus source_build_environment(),
which prepares only on win32-arm64 when the synced project carries the
provider script. A payload has prebuilt dependencies and needs no compiler.
VenvPackage.apply and build_environment pass the result to uv children
only. It carries the bridged pip index settings, which managed_environment
applies only to the ambient environment. The state root stays the store
parent, so existing vcpkg/OpenSSL builds are reused.
setup-hermes.ps1 collapses to one `pm install`: the tools-only split existed
only for this preparation, and pm install already puts its tools on PATH
before the venv sync (pm/cli.py activate check).
Not yet verified live on Windows ARM64.
Install/update completion rewrites the root install-stamp.json (fresh
builtAt) after products are built, and the stamp was still a shared
web/desktop source input, so every install left its own web_dist
"missing, stale, or damaged" and the post-install probe failed on every
installer E2E leg. Nothing in either build reads the root stamp;
desktop's baked stamp is already tracked as a prepared input.
Merge origin/main at 8e806ae1b2. Keep native helper compilation in
prepareDesktopNativeDependencies and keep bundling/beforePack consume-only.
Bind helper sources and headers into preparation identities and cache keys;
copy admitted executable resources beside node_modules and preserve signing
semantics in product freshness checks.
Verified desktop typecheck, focused native/packaging/UI and gateway/cache
tests, and the real Linux preparation/copy/Xvfb execution path. Incoming
upstream anti-slop findings remain unchanged; no baseline was raised.
First launch of a bundled payload paid a cold-compile stall: the launcher
redirects bytecode writes to a user-level cache (signature-breaking on
macOS, read-only mount on AppImage/MSIX), so every import compiled from
source. Now staging bakes the cache into the payload:
- compileall with the payload's OWN staged 3.14 interpreter, unchecked-
hash pycs: repack mtimes cannot invalidate them, a stale source can
never trigger a rewrite, and read-only pycs mean the macOS signature
never observes a change. Dirs stay writable — in-place rebuilds
rmtree the tree; asserted coverage plus unchecked-hash means no
cache-miss write can target them.
- coverage is the perf contract: the bake FAILS if any parseable module
lacks a pyc (empirically 0 unparseable files ship, so compileall is
strict). Probe suite: py_compile/cache_from_source, PEP 552 flags,
multi-root read, stale-source no-rewrite, read-only cache-dir import.
- launcher: the baked marker makes configure() leave sys.pycache_prefix
UNSET — the prefix relocates reads too and would hide the baked pycs.
Payload modules read their source-adjacent cache (Python's default
multi-root lookup); plugin/user modules keep caching beside their own
sources under HERMES_HOME. Unmarked payloads keep the old redirect.
- snapshot(): the sealed payload ships without tests/website/evals/
.github/nix/docker/tests-js (~69MB, 46% of tracked bytes) and without
apps/ui-tui/web/scripts — CI prebuilds those products, and
is_bundled_payload routes sealed updates to the channel updater, so
the rebuild graph never runs in a bundle (linux_desktop_entry degrades
to the themed icon). Frontend product staging keeps the full tree.
- test_bundle_native now stages the FULL relocatable toolchain (a bare
interpreter ELF falls back to its compile-time /install prefix and
cannot create a venv), and runs on the real 3.14 for the first time
this campaign — the whole battery had been running 3.12 against the
3.14-pinned lock.
Dependency acquisition during packaging left native wheels and packager
inputs outside the pre-build cache save. Compose PM and existing providers
into a preparation phase, then require builds to consume admitted inputs.
Share native preparation with PM Bundle. Keep path-bound environments and
signing outputs separate from reusable caches. Use read-only cache tokens
for commit builds and preserve the one-command local build path.
Verify pinned tools through PM, probe PTYs under the prepared Electron,
and supply dmgbuild through a build-only PM package. Resolve bundled tool
stores from their payload manifest so relocation preserves discovery.
Validation: focused Python and JS tests, checkJs, Ruff, Windows checks,
anti-slop, cache relocation, and network-denied Linux AppImage builds.
Relocated runtime smoke passed with NixOS host libraries supplied.
Native Windows/macOS signing and live GitHub cache behavior remain untested.
A source child can resolve outside the checkout. The output guard must
protect its canonical path before the builder checks prepared inputs.
Exclude generated dist/build trees so in-tree products can still rebuild.
Keep explicit prepared inputs protected even inside generated trees.
The public buildDesktop regression test first failed with a missing-icon
error instead of an overlap error. All 7 desktop-builder tests now pass,
including real cold/warm builds under apps/desktop/build/products.
Node syntax checks and git diff --check pass. No full suite or native
packaging ran.
Run historical updater completion in a fresh interpreter so cached imports
cannot revive retired dependency installers. Share Git and ZIP completion,
carry receipt and recovery state, and preserve child exit status.
Route plugin admission, binary acquisition, desktop launch and build paths
through PM. Replace redundant helpers and tests with real worker, package,
publication and launch checks. Keep the shipped compatibility surface fixed.
Targeted Python and desktop checks pass. Native update journeys and fresh
production image qualification remain pending. This is a checkpoint before
those acceptance runs.
Competing installers and checkout-local venv assumptions bypassed PM
selection, install consent, and generation lifetimes. Route consumers
through PM and installation-bound launchers. Refresh source launchers
before obsolete Python entries can be collected.
Remove Node, browser, and CUA acquisition engines, obsolete venv-holder
handling, detached sync, and unused PM APIs. Keep historical updater
exports inert and preserve external tool ownership and native integration.
Share product freshness and prepared inputs across builders. Align plugin
admission, Docker provisioning, setup instructions, and behavioral tests.
Verified targeted Python and JavaScript tests, desktop and web typechecks,
scoped lint, real product builds, and the Docker frontend smoke test.
The missed post-setup test cleanup is included and verified.
Native Windows/macOS execution, full Rust compilation, and the complete
repository suite remain unverified. Historical compatibility requirements
were preserved and extended, not fully rescanned.