# Rust Git Forge Product and Architecture Plan Last revised: 23 August 2026 Tracking issue: `videogen-zyo` ## Decision record This is a production open-source systems product, not GitGram and not a hackathon implementation. The neutral working name is **Rust Git Forge** until brand, crate, and package-name checks are complete. The product has two layers: 1. A library-first Rust implementation of Git repositories and the Git wire protocol, compatible with existing Git repositories and clients. 2. A self-hostable collaboration forge built on that implementation, including repositories, organizations, permissions, issues, pull requests, review, protected branches, webhooks, releases, and audit logs. GitGram remains a mostly TypeScript social product. It turns repositories into Instagram-like accounts with posts, reels, stories, media publishing, followers, and feeds. GitGram does not own object storage, refs, packfiles, the Git protocol, pull-request truth, or repository permissions. It integrates with GitHub and, later, Rust Git Forge through public authenticated APIs and webhooks. AITimeMachine captures and restores AI coding workflows. It may attach its checkpoints to Git commits, but it does not implement or replace Git. ## Initial customer and promise The initial user is an engineer or small team that needs a trustworthy local Git implementation with readable internals and strong compatibility evidence. The first production promise is narrower than a complete GitHub replacement: users can inspect, create, verify, exchange, and recover standard Git repositories without corrupting data. The forge becomes a supported product only after the core passes interoperability, crash-recovery, parser-fuzzing, and concurrent-ref tests. ## Compatibility scope The core implementation must cover: - loose blobs, trees, commits, annotated tags, object verification, and object traversal; - SHA-1 repository compatibility with collision detection, plus an explicit path to Git's SHA-256 repository format; - refs, symbolic refs, packed refs, reflogs, atomic compare-and-swap updates, and lock recovery; - index formats, worktrees, ignore rules, status, add, restore, and commit; - revision parsing, graph walks, merge bases, diffs, rename heuristics, and conflict representation; - pack indexes, packfiles, delta resolution, reachability, negotiation, and quarantine of received objects; - smart protocol v2 over HTTP and SSH for clone, fetch, and push; - three-way merge, rebase, cherry-pick, revert, and deterministic conflict handling. Submodules, sparse checkout, partial clone, large-file storage, replace refs, notes, and advanced maintenance enter the compatibility matrix explicitly. None is silently claimed before its test suite passes. ## Rust workspace architecture ```text crates/ git-object object IDs, parsing, validation, canonical serialization git-odb loose and packed object databases, quarantine, alternates git-refs refs, reflogs, transactions, locks git-index index formats and extensions git-worktree checkout, status, ignore rules, filesystem safety git-revision revision syntax, graph traversal, reachability git-diff text and binary diff, rename detection git-merge merge bases, strategies, conflicts, sequencer git-pack pack/index parsing, deltas, negotiation inputs git-protocol protocol v2, upload-pack, receive-pack transports git-cli compatibility-oriented command-line interface forge-domain users, organizations, repositories, issues, reviews forge-store metadata persistence and repository coordination forge-http browser/API and smart-HTTP server forge-ssh SSH command gateway and key authorization forge-jobs durable maintenance, hooks, webhooks, notifications ``` Shared crates cannot depend on the forge UI or a specific database. Parsing and storage APIs expose bounded streams rather than assuming objects or packs fit in memory. Platform-specific filesystem behavior stays behind explicit adapters and platform tests. ## Forge architecture The first forge is a single-node, self-hosted service: - repositories and packfiles on a durable local filesystem; - PostgreSQL for users, organizations, permissions, issues, reviews, jobs, and immutable audit events; - an SSH service for Git transport and an HTTP service for API, browser, and smart HTTP; - a transactional job outbox for webhooks, email, repository maintenance, and indexing; - filesystem snapshots plus PostgreSQL backups with a tested coordinated restore procedure. SQLite may support desktop or single-user mode, but it does not imply safe multi-node repository coordination. Multi-node storage, geo replication, packages, runners, Actions-compatible CI, and large-file storage require later designs with separate threat and cost reviews. Cloudflare Workers are not the Git execution or storage plane. Large pack streams, SSH, repository filesystem semantics, and long-running maintenance need a local process or conventional server. Cloudflare may serve static project documentation and, after measurement, proxy ordinary browser/API traffic. R2 is only a future cold-object or backup option under the shared account budget. ## Security contract Repository data is adversarial input. The product must defend against: - malformed object, index, pack, bitmap, commit-graph, and protocol data; - path traversal, case-folding collisions, NTFS/HFS hazards, unsafe symlinks, and checkout outside the worktree; - decompression bombs, delta chains, integer overflow, unbounded graph walks, resource exhaustion, and archive bombs; - SHA-1 collision attacks and object-ID confusion between hash formats; - ref races, lock theft, non-fast-forward policy bypass, hook injection, and partially received pushes; - cross-tenant authorization errors, leaked deploy keys or tokens, forged webhooks, and unsafe HTML or Markdown rendering. Incoming pushes remain quarantined until object verification, connectivity, authorization, policy, and size checks pass. Ref changes use atomic transactions. The forge supports scoped tokens, SSH key fingerprints, phishing-resistant operator MFA, protected branches, signed commit and tag verification, secret redaction, security logs, and auditable administrative actions. ## Verification strategy Production claims require retained evidence from: - differential tests that run equivalent operations through Rust Git Forge and the reference Git CLI, then compare repository state and command semantics; - interoperability matrices across supported Git versions, operating systems, filesystem behaviors, HTTP servers, and SSH clients; - property tests for serialization, graph traversal, deltas, refs, and index round trips; - coverage-guided fuzzing for every untrusted parser and protocol state machine; - Git project fixtures where licensing and harness boundaries permit; - fault injection around writes, fsync, locks, process death, partial network reads, interrupted clone/fetch/push, and coordinated restore; - concurrency tests for ref updates, simultaneous pushes, garbage collection, and read/write isolation; - independent security review before exposing public repository hosting. Every compatibility claim is tied to a named test. A failing repository fixture remains a release blocker or a documented unsupported capability. ## Release sequence ### Stage 0: specification and harness Publish the compatibility matrix, threat model, repository fixtures, differential harness, fuzz targets, performance budgets, and corruption-recovery rules. ### Stage 1: read-only repository core Ship object inspection, `cat-file`, `ls-tree`, log traversal, refs, pack reads, and `fsck`. Opening hostile repositories cannot write outside the repository or consume unbounded resources. ### Stage 2: local authoring Ship init, status, add, diff, commit, branch, checkout/restore, tag, reflog, and garbage-safe object writes. Pass crash tests for every state-changing command. ### Stage 3: exchange Ship protocol v2 clone, fetch, and push over HTTP and SSH. Prove quarantine, atomic refs, authentication, authorization, interrupted transfer recovery, and interop with the reference Git client and server. ### Stage 4: history editing Ship merge, rebase, cherry-pick, revert, conflict continuation/abort, and recovery. Preserve user work during failures and match documented Git behavior. ### Stage 5: single-node forge Ship accounts, organizations, repository permissions, issues, pull requests, review, branch protection, webhooks, releases, operator tools, backups, restore, export, deletion, abuse handling, and upgrade/rollback procedures. ### Stage 6: public production Onboard outside repositories only after security review, capacity tests, restore drills, support procedures, privacy/terms, observability, incident response, and an upgrade from the previous release all pass. Multi-node and CI features remain separate approvals. ## Non-goals for the first public release - replacing every Git command at once; - claiming GitHub parity; - running arbitrary CI jobs; - hosting unbounded public binary releases or package registries; - coupling repository correctness to GitGram feeds; - storing primary repositories in browser storage, R2, or a Worker runtime; - inventing a new incompatible wire protocol before Git interoperability works. ## Pre-code gate Before implementation: 1. approve the project name, license, governance, supported platforms, and MSRV; 2. freeze the Stage 1 compatibility matrix and threat model; 3. select reference Git versions and reusable fixtures after license review; 4. decide whether existing Rust Git crates are comparison subjects, dependencies, or clean-room references; 5. define benchmark repositories and memory, CPU, disk, and network budgets; 6. approve the single-node data and backup contract without using Cloudflare as the repository execution plane.