Convert gitgram/ and aitimemachine/ from embedded git repos (which showed as opaque submodule gitlinks on GitHub) into fully-tracked first-party content. AITM now merges its 7 vendored repos as first-party engines: - memory: beads (Go task DAG) + context7 (docs SDK), unified into saved_contexts - git: vendored C git (bundled binary) + gitnexus (in-process code intelligence) - agentic: a2a (@a2a-js/sdk) + pal (AITM-managed Python subprocess) Includes the full session's product build: Gitgram web/mobile/backend features, AITM website/CLI/MCP, the unified memory layer, engine build tooling (scripts/build-engines.sh, Dockerfile, CI), docs (README, ENGINES.md, ARCHITECTURE.md), and extensive test suites. Build artifacts, node_modules, external/, and gemini-cli are gitignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
10 KiB
AITimeMachine Complete Implementation Plan
Vision
"macOS Time Machine for AI Coding" - When AI makes a mistake, go back in time and fix it.
Like when your computer gets a virus and you reset via macOS Time Machine - we do this for AI chats and code.
Current State (Updated 2026-02-02)
Phases 1-6: COMPLETE
| Component | Status | Notes |
|---|---|---|
| Time Layer | 98% | Checkpoints, CAS, diffs, restore, three-way merge all working |
| MCP Server | 98% | 83+ tools, unified server, before_action/after_action linked pairs, code intelligence tools |
| CLI | 98% | 105+ commands, natural language time travel, code intelligence (index, search, find-def, find-usages) |
| API Server | 98% | /v1/ versioning, rate limiting, OpenAI-style errors, 25+ route modules, real package search |
| VSCode Extension | 90% | Chat panel, timeline, CodeLens, function tracking, MCP integration |
| Context Layer | 85% | Messages linked to checkpoints, conversation grouping, context packets |
| Augmented Layer | 90% | CodeIndexer, symbol search, universal search, npm/PyPI package search |
What Was Completed (Phases 1-5)
Phase 1: MCP Server Unification ✅
- Unified two MCP servers (simple 40-tool + full 78-tool) into one
- CLI
aitm mcpnow launches the full server with all tools - Added
before_action/after_actionwith linked checkpoint pairs - Context layer wired: checkpoints include conversation context
- MCP Resources fixed (files array bug)
linkedCheckpointIdadded to CheckpointRow and SQLite migration
Phase 2: API Server ✅
- /v1/ versioning with backwards-compatible /api/ routes
- Rate limiting middleware (tier-based)
- OpenAI-style error handling with structured error codes
- 25+ route modules: auth, keys, sessions, audit, sync, restore, dashboard, webhooks, repositories, search, contexts, research, billing, data-sources, oracle, package-search, shared-checkpoints, sso, universal-search, web-search, usage, research-papers, export, health
- Audit middleware, request logging, monitoring with Sentry integration
- Web search with Brave/SerpAPI/Serper/DuckDuckGo providers
- Web scraping with Firecrawl/Cheerio
- Export endpoints (training JSONL, analytics JSON/CSV, secret redaction)
Phase 3: CLI ✅
- Natural language time travel:
aitm back-to "before docker changes" - Function-level file history:
aitm file-history --functions --function <name> get-file,get-file-at,get-checkpoint,checkpoint-preview,find-greenlog-command,conversations,taskscommands- Enhanced
sessions --json --search,preview --json - Dual calling convention for
restore-files(extension + CLI compatible) - All commands support
--jsonoutput for programmatic use
Phase 4: VSCode Extension ✅
- MCPClient spawns
aitm mcp(unified server) - CodeLensProvider calls CLI for function-level tracking
- All CLI commands that extension depends on are implemented
- TimelineGraphProvider, ChatPanelProvider, GhostService functional
Phase 5: Data Capture & Export ✅
- Training data export (OpenAI/Anthropic/generic JSONL formats)
- Analytics export (JSON/CSV with by-day, by-hour, by-type breakdowns)
- Secret redaction (API keys, GitHub tokens, AWS keys, JWT, passwords)
- API export endpoints with async job processing
PHASE 6: Augmented Layer - COMPLETE
Goal: Real code intelligence - index, search, and understand code
6.1 Code Indexer Service ✅
Created: src/services/code-indexer.ts (~560 lines)
- Regex-based parser for TypeScript/JavaScript and Python
- Extracts: functions, classes, interfaces, types, enums, methods, properties, exports, imports
- SQLite
code_symbolstable with indexes - Methods: indexProject, searchSymbols, findDefinition, findUsages, getProjectStructure, getFileSymbols, getStats
6.2 MCP Tools for Code Intelligence ✅
Added to: src/mcp/server.ts - 5 tools with case handlers
| Tool | Status |
|---|---|
tm_index_project |
✅ Scans project, extracts symbols, stores in SQLite |
tm_search_code |
✅ Exact/prefix/contains/regex search with scoring |
tm_find_definition |
✅ Finds where symbol is defined |
tm_find_usages |
✅ Finds imports and references |
tm_get_project_structure |
✅ Project overview with top-level symbols |
6.3 Enhanced Universal Search ✅
Updated: src/mcp/server.ts tm_universal_search handler
- Uses code index for symbol-level search when available
- Falls back to glob-based grep if no index
- Searches checkpoint file paths (not just names)
- Results scored and sorted by relevance
6.4 CLI Commands for Code Intelligence ✅
Added to: src/cli-v2.ts - 5 new commands
| Command | Description |
|---|---|
aitm index |
Index project source code for symbol search |
aitm search-code <query> |
Search code symbols by name |
aitm find-def <symbol> |
Find where a symbol is defined |
aitm find-usages <symbol> |
Find all files that import/reference a symbol |
aitm project-structure |
Show project overview with top-level symbols |
6.5 API Server Search Routes ✅
api-server/src/routes/search.ts- 7 endpoints, uses IndexingService + codeEmbeddings via Drizzle ORMapi-server/src/routes/universal-search.ts- 3 endpoints, multi-source search with scoringapi-server/src/services/indexing.ts- Full indexing service with embedding support
6.6 Package Search ✅
Updated: api-server/src/routes/package-search.ts - 6 endpoints, real npm/pypi integration
| Endpoint | Status |
|---|---|
GET /info |
✅ npm + PyPI registry API |
GET /files |
✅ npm via unpkg |
POST /read-lines |
✅ npm file content via unpkg |
POST /grep |
✅ Regex search across npm package files |
POST /semantic |
✅ Keyword-based symbol search in npm packages |
POST /hybrid |
✅ Combined grep + semantic for npm |
PHASE 7: GhostService AI Integration (Future)
Goal: Real AI-powered inline suggestions in VSCode
Update: extensions/vscode/src/services/GhostService.ts
Currently uses pattern-based suggestions as fallback. Wire to:
- MCP server for context-aware suggestions (knows about checkpoints, recent changes)
- VS Code Language Model API (already has framework)
- OpenAI API fallback (already has framework)
The key is passing checkpoint context to the AI so suggestions are aware of recent changes.
PHASE 8: Production Hardening (Future)
Goal: Production-ready deployment
| Item | Status | Notes |
|---|---|---|
| Sentry integration | Done | api-server/src/lib/monitoring.ts |
| Rate limiting | Done | Tier-based middleware |
| Geo-IP | Done | api-server/src/services/sessionManager.ts |
| Web search | Done | Brave/SerpAPI/Serper/DuckDuckGo |
| Three-way merge | Done | src/mcp/git-enhanced-handler.ts |
| Hybrid store local search | Done | searchLocalMessages with scoring |
| SSO | Route exists | Needs real SAML/OAuth provider |
| Billing | Route exists | Needs Polar.sh integration |
| Email webhook | Placeholder | api-server/src/routes/internal.ts:97 |
Three-Layer Architecture
┌─────────────────────────────────────────────────────────────┐
│ AUGMENTED LAYER │
│ Code Indexer | Symbol Search | Universal Search │
│ Status: Phase 6 COMPLETE │
├─────────────────────────────────────────────────────────────┤
│ CONTEXT LAYER │
│ Chat History | Linked Checkpoints | Context Packets │
│ Status: 85% COMPLETE │
├─────────────────────────────────────────────────────────────┤
│ TIME LAYER │
│ Checkpoints | CAS Blobs | Diffs | Manifests | Pack Files │
│ Status: 98% COMPLETE │
└─────────────────────────────────────────────────────────────┘
Key Files Reference
Core
src/mcp/server.ts- Full MCP server (78+ tools)src/store/sqlite_repo.ts- SQLite repository (checkpoints, events, projects)src/core/cas.ts- Content-addressable storesrc/cli-v2.ts- CLI (100+ commands)src/cli/repo-bridge.ts- Unified bridge (local + MCP)src/mcp/git-enhanced-handler.ts- Advanced diff, timeline, three-way mergesrc/context/context-packet.ts- Context packets for restore
API Server
api-server/src/index.ts- Express app with 25+ route modulesapi-server/src/routes/- All API route handlersapi-server/src/middleware/- Auth, rate limiting, errors, auditapi-server/src/services/- Web search, session manager, monitoring
Extension
extensions/vscode/src/mcp/MCPClient.ts- MCP client (spawnsaitm mcp)extensions/vscode/src/providers/- UI providers (Chat, Timeline, CodeLens, Ghost)
Success Metrics
-
Time Machine Experience ✅
- Restore to any point with
before_action/after_actionlinked pairs - Chat context restored with files via context packets
- Natural language:
aitm back-to "before docker changes"
- Restore to any point with
-
API Quality ✅
- OpenAI-style errors, /v1/ versioning, rate limiting
- 25+ route modules covering all planned endpoints
-
Data Capture ✅
- All AI sessions captured via MCP before/after actions
- Training data export in OpenAI/Anthropic JSONL formats
- Secret redaction for safe data sharing
-
Code Intelligence ✅
- Index project symbols (functions, classes, interfaces)
- Search code by symbol name (exact, prefix, contains, regex)
- Find definitions and usages
- Universal search across code + conversations + checkpoints
- CLI commands: index, search-code, find-def, find-usages, project-structure
- Package search: npm/PyPI registry integration (info, files, grep, semantic, hybrid)