caelum0x f97f271aec fix: drop verify:local script pointing at a file never committed
package.json referenced test-fixtures/run-local-verification.mjs, which
was used to produce the committed verification-run artifacts but was
never added to the repo -- `npm run verify:local` failed with
"Cannot find module" for anyone cloning the repo. The README already
documents the real manual commands used for verification, so the
script entry was dead weight; removed rather than recreated to avoid
claiming a reproduction harness that wasn't actually re-verified here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-11-17 15:03:18 +03:00

scribase-migrate

One command to move a team onto Scribase — the one account for Postgres, auth, storage and email — from Supabase, Neon, Clerk and Resend.

npx scribase-migrate supabase --to <org>/<project> --create-project \
  --source-ref <ref> --source-url <connection-string> --source-service-key <key>

npx scribase-migrate neon --to <org>/<project> \
  --source-url <connection-string> --migrate-auth-users

npx scribase-migrate clerk --to <org>/<project> --secret-key <sk_...>

npx scribase-migrate resend --api-key <re_...> --out resend-export.json

Run npx scribase-migrate --help for the full flag reference.

It needs a Scribase access token the same way scribase itself does — run npx scribase login once, or export SCRIBASE_ACCESS_TOKEN.

What each source actually moves

Source Postgres schema+data Auth users / passwords Storage Notes
Supabase yes yes (hash preserved) yes one real API call, Scribase's own POST /v1/imports
Neon yes yes, if the source has its own auth.users table — Neon has no storage/auth of its own; branches land in branch_<name> schemas
Clerk — yes, but a new one-time password (Clerk never exposes a portable hash) — every migrated user is flagged password_reset_required
Resend — — — exports sending domains (DNS records) + broadcasts as "templates" to a JSON bundle; Scribase has no write endpoint for either yet, so this is import-ready, not auto-applied

Why the design looks like this (real findings from building it)

This was built by composing Scribase's own already-shipped APIs — the scribase npm SDK (edge-repos/scribase-cli) and the GoTrue admin API every self-hosted Supabase migration tool already uses — not by reinventing a dump format. Three real constraints, found by actually running this against a live Scribase project while building it, shaped the final design:

Why not pg_restore straight into Scribase. environments.connection() returns a real Postgres URI, but its hostname is on Scribase's internal network only (the SDK already documents it SERVER-SIDE ONLY) — it does not resolve from an external machine. Unlike Supabase or Neon, Scribase has no public Postgres TCP endpoint today. The Neon/generic-Postgres source instead builds plain SQL (pg_dump --schema-only / --data-only --inserts) and sends it through POST .../migrations — the same HTTPS path scribase migrate itself uses, and the only one that actually reaches a Scribase environment from outside its network.

Why every run resends the whole supabase/migrations/ folder. That endpoint is not additive. Verified live: submitting one new migration without the version(s) already recorded answers 409 migration_conflict ("the database records migration X but it is absent from the set on disk"), and re-submitting an already-applied version with even one different byte (we hit this with a stray trailing newline) answers a different 409 ("an already-applied migration must never be edited", with the two checksums). So scribase-migrate neon writes into supabase/migrations/ using the exact same naming convention npx scribase migrate reads, and always reads the whole folder back and resends it. One practical upshot: commit that folder to the app's repo, the same as you would for scribase migrate on its own — it is the one source of truth the next run (or a teammate's) needs.

Why RLS is generated, never skipped or blanket-opened. Scribase's migration lint BLOCKs any new public table with neither RLS enabled nor a policy — verified live, and a pg_dump --schema-only of a typical Supabase/Neon app has neither (Neon has no RLS concept at all; many Supabase apps relied on the service key and never wrote policies). Writing a blanket using (true) policy to satisfy the lint would quietly make every row readable by every signed-in user — worse than the block it papers over. lib/rls.mjs instead infers an owner column (a direct foreign key to auth.users, or one hop through another table that has one) and writes a real owner policy; a table it cannot prove an owner for gets RLS enabled with a using (false) policy — locked to the service-role key (which bypasses RLS) until someone writes a real policy, never silently public.

Why Clerk users get a new password, not a migrated hash. Clerk's Backend API has no endpoint that returns a portable password hash on any plan — this is a platform limit, not a corner this tool cut. Each migrated user is created via the destination environment's GoTrue admin API (POST /auth/v1/admin/users, the same endpoint any self-hosted Supabase migration script uses) with a random one-time password and app_metadata.password_reset_required: true, and is returned in needsPasswordReset so the caller can send a real password-reset email.

A sharp edge if you hand-write auth.users rows yourself: GoTrue's Go driver scans several varchar columns (confirmation_token, recovery_token, email_change*, phone_change*, reauthentication_token) as a plain string, not nullable — leaving them at Postgres's own NULL default makes every later login 500 with converting NULL to string is unsupported. dumpAuthUsersSql forces them to '', which is what GoTrue itself would have written for an account with no pending confirmation/recovery flow.

A sharp edge in pg_dump itself: pg_dump's "can dump an older server" guarantee does not cover every statement it emits. A 17.x pg_dump against a 16.x source produced ALTER TABLE ... ALTER COLUMN ... SET (pg_attver=…) — a PG17-only column-storage attribute Postgres 16 rejects outright. pgDumpBinaryFor() probes the source's major version and prefers an exact-match Homebrew keg (postgresql@<major>) over whatever pg_dump is on PATH, warning when it has to fall back.

Real verification (not a test file — an actual live run)

This was verified against a real local Supabase-shaped Postgres and a real, live Scribase project, not mocked:

  1. A real Postgres 16 Docker container seeded with a GoTrue-shaped auth.users table (two users, one password hashed with real bcryptjs), public.profiles (owner = auth.users.id) and public.notes (owner one hop through profiles) — test-fixtures/seed-source.sql.
  2. npx scribase-migrate neon --to ereboslab/scribase-migrate-verify-internal/production --source-url ... --migrate-auth-users --schema public ran for real against a live Scribase org (ereboslab), through the real scribase SDK with a real scribase login token.
  3. The schema step generated RLS automatically (notes via the profiles→auth.uid() join rule, profiles directly) and passed Scribase's real migration lint with 0 blocking findings.
  4. Row counts verified with a real HTTP call (client.database.query, not a local connection) against the live environment: public.profiles 2/2, public.notes 3/3, both match: true.
  5. Login verified for real: POST /auth/v1/token?grant_type=password against the live environment's real api_url — the correct password returns 200 with a real access_token; a wrong password returns 400 Invalid login credentials. The password hash moved byte-for-byte and still verifies.

Full before/after artifacts: test-fixtures/verification-run/.

Not verified live (both require a provider account/test credentials this environment doesn't have — logged honestly rather than faked): the supabase source's real one-call POST /v1/imports (its own request shape was confirmed against the real API with a dry-run, which correctly rejected a fake project ref with 400 invalid_project_ref, but needs a real reachable Supabase project — Scribase's control plane connects to the source, so it must be a real, network-reachable Postgres, not a local Docker container, to run the full one-call migration this source wraps); the clerk and resend sources (logic-verified against a local mock of each provider's documented API shape — pagination, the GoTrue admin-create request body, the domains/broadcasts export — but not run against a real Clerk or Resend account).

Repo layout

  • bin/scribase-migrate.mjs — the CLI
  • lib/client.mjs, lib/auth.mjs — the scribase SDK + the same token lookup scribase itself uses (keychain / SCRIBASE_ACCESS_TOKEN / login)
  • lib/project.mjs — idempotent --create-project
  • lib/postgres.mjs — pg_dump wrapping, version matching, the GoTrue-safe auth.users export
  • lib/rls.mjs — the RLS-inference heuristic
  • lib/migrationFiles.mjs — the supabase/migrations/ read-everything, resend-everything helper
  • lib/sources/{supabase,neon,clerk,resend}.mjs — one module per source
  • lib/login.mjs — the real password-login proof
  • docs/migrate-page.md — the scribase.com/migrate docs-page snippet
  • test-fixtures/ — the real local-verification seed + its output

scribase.com/migrate

docs/migrate-page.md is a ready-to-paste docs page for scribase.com/migrate (the scribase checkout's owner adds it to apps/docs and deploys — not done by this repo).

v0.1.1 Latest
2026-11-17 17:10:39 +00:00
Languages
JavaScript 100%