4.3 KiB
4.3 KiB
Setup
Everything runs locally. The free path (your brand, a type-only hook, a scratch voice, a scratch music bed, the synthesised sound palette) needs no account. Paid generation runs on your own Higgsfield account.
What you need
| Tool | Why | Required |
|---|---|---|
| Node.js 20 or later, npm | the brand extractor and HyperFrames (the renderer) | yes |
| ffmpeg | every audio and video step | yes |
| Python 3.10 or later and uv | the audio pipeline (voice prep, sound, mix, review) | yes |
whisper.cpp (whisper-cli) and a model |
checking and timing the spoken words | yes for real voice takes |
| torch and torchaudio | forced alignment, the precise word timing | recommended, pulled on demand by uv |
| Higgsfield CLI and account | narrator voice, music, sound kits, generated footage | optional |
| Blender 5.x | 3D glass objects | optional |
| ACE-Step 1.5 | free local music takes | optional |
| Dragonfly Plate Reverb (VST3) | the shared reverb for sound effects; a built-in reverb is used without it | optional |
macOS
brew install node ffmpeg uv whisper-cpp
npm install # installs the brand extractor's headless browser
uv venv .venv && source .venv/bin/activate
uv pip install -r requirements.txt
mkdir -p models
curl -L -o models/ggml-base.en.bin https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.en.bin
curl -L -o models/ggml-small.en.bin https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.en.bin # better on product names
The scratch voice uses macOS say, which is already installed. HyperFrames downloads its own headless Chrome on the first render (npx -y hyperframes@0.8.77 ...).
Linux
sudo apt install ffmpeg espeak-ng build-essential cmake # espeak-ng is the scratch voice
curl -LsSf https://astral.sh/uv/install.sh | sh
# Node 20+: from nodesource or nvm
git clone https://github.com/ggml-org/whisper.cpp ~/whisper.cpp
cmake -S ~/whisper.cpp -B ~/whisper.cpp/build && cmake --build ~/whisper.cpp/build -j
export PATH="$HOME/whisper.cpp/build/bin:$PATH" # provides whisper-cli
Then the same npm install, venv and model steps as on macOS. Headless Chrome needs the usual system libraries (libnss3, libatk-bridge2.0-0, libgbm1, libasound2 and friends); if a render fails to launch the browser, install those.
Environment variables
| Variable | Default | Used by |
|---|---|---|
WHISPER_MODEL |
models/ggml-base.en.bin in the repo |
voice scripts, reference study |
REVERB_VST3 |
~/Library/Audio/Plug-Ins/VST3/DragonflyPlateReverb.vst3 |
sfx_forge.py |
ACE_STEP_ROOT |
~/ACE-Step-1.5 |
ace_score.py |
BLENDER_GPU |
METAL (CUDA, OPTIX or NONE elsewhere) |
blender_glass_object.py |
PYTHON |
python3 |
study_reference.sh |
Optional tools
- Forced alignment: nothing to install by hand.
uv run -q --with torch --with torchaudio --with soundfile python skills/launch-voice/scripts/force_align.py LINE.wav "the words"pulls torch on first use and caches the aligner model (about 1.2 GB) in~/.cache/torch. - Higgsfield:
npm install -g @higgsfield/cli, thenhiggsfield auth loginwith your own account. Check your balance withhiggsfield account status. For deeper Higgsfield know-how, add their skills:npx skills add higgsfield-ai/skills. - HyperFrames skills:
npx skills add heygen-com/hyperframesgives your agent the full composition contract, motion catalog and CLI docs. - Blender: install 5.x from blender.org (macOS:
brew install --cask blender). - ACE-Step 1.5: follow its README to install it into its own folder with its own venv, then set
ACE_STEP_ROOT. Models download on first run (several GB). - Dragonfly Reverb: free and open source (GPL); install the VST3 plug-in from its website. Without it the sound palette uses pedalboard's built-in reverb.
- Local voice clone:
skills/launch-voice/scripts/clone_local.pypulls Chatterbox throughuvon first run (a few GB).
Check the install
node -v && ffmpeg -version | head -1 && whisper-cli --help | head -1
source .venv/bin/activate && python3 -c "import numpy, scipy, soundfile, pedalboard, pyloudnorm, librosa; print('audio ok')"
python3 skills/launch-sound/scripts/sfx_forge.py palette /tmp/palette --key A # 13 sounds, free