897 lines
40 KiB
Python
Raw Permalink Normal View History

"""HTTP client for Scribase's public /v1 management API.
Every function here takes an already-resolved `api_key`/`api_base` and
returns a plain dict — never raises. The caller (`__init__.py`'s tool
handler) is responsible for the handler contract (always return a JSON
string, success or error alike). This module makes network calls only to
`api_base` (default `https://api.scribase.com`, overridable via the
plugin's optional `api_base_url` config key for self-hosted Scribase
clusters) and uses only the Python standard library — no third-party
dependencies.
feat: add scribase_branch_task — real copy-on-write branch create/promote/discard Fourth tool: one action='create'|'promote'|'discard' tool that forks a real copy-on-write Postgres branch for one agent task (POST /v1/environments, kind="preview", bounded TTL so it always lands in the control plane's agent-branch slot pool), three-way merges it back into its parent on action='promote' (POST .../branches/merge, reads the real applied/clean/changes/conflicts/statements fields the compiled server renders, and surfaces a real mfa_required rejection on a forced merge instead of hiding it), or deletes it outright on action='discard' (DELETE .../environments/{env}). Built from the real control-api source (http/environments.rs, http/branches.rs, adapters/branch.rs, adapters/usage.rs::AGENT_BRANCH_MAX_TTL_HOURS) and the public docs.scribase.com environments/branches pages, not guessed. Factored create_project's polling and API-keys/connection readback into shared _poll_operation/_read_environment_access helpers, reused by both tools; create_project's own behavior is unchanged (verified with a mocked HTTP layer, happy path and warning path). plugin.yaml bumped to 1.2.0, provides_tools now lists all 4 tools. README rewritten with a one-line pitch, a demo-video pointer, and a real example call + example response for every tool. HACKATHON-SUBMISSION updated with the new tool, a fresh validator run (15/15, 0 warnings), and an honest demo-coverage note for the branch-task tool including two live production responses from the real public /v1/demo/branch endpoint this tool's workflow mirrors (a transient 402, then the real 429 per-hour rate limit — both genuine, neither invented).
2026-11-05 23:48:28 +03:00
Four tools, four endpoints:
- `create_project` — `PUT .../projects/{id}` + `POST /v1/environments`,
provisions real billable resources.
- `lint_migration` — `POST .../migrations` with `"apply": false` always
hardcoded here, so it only ever runs the control plane's migration
linter (missing-RLS, unsafe-grant, lock-unsafe-DDL style rules) and never
touches the tenant database.
- `run_query` — `POST .../database/query`, a statement the control plane
itself runs inside a Postgres `READ ONLY` transaction; this client does
not add its own write guard because the server enforces it twice
(lexically, then transactionally).
feat: add scribase_branch_task — real copy-on-write branch create/promote/discard Fourth tool: one action='create'|'promote'|'discard' tool that forks a real copy-on-write Postgres branch for one agent task (POST /v1/environments, kind="preview", bounded TTL so it always lands in the control plane's agent-branch slot pool), three-way merges it back into its parent on action='promote' (POST .../branches/merge, reads the real applied/clean/changes/conflicts/statements fields the compiled server renders, and surfaces a real mfa_required rejection on a forced merge instead of hiding it), or deletes it outright on action='discard' (DELETE .../environments/{env}). Built from the real control-api source (http/environments.rs, http/branches.rs, adapters/branch.rs, adapters/usage.rs::AGENT_BRANCH_MAX_TTL_HOURS) and the public docs.scribase.com environments/branches pages, not guessed. Factored create_project's polling and API-keys/connection readback into shared _poll_operation/_read_environment_access helpers, reused by both tools; create_project's own behavior is unchanged (verified with a mocked HTTP layer, happy path and warning path). plugin.yaml bumped to 1.2.0, provides_tools now lists all 4 tools. README rewritten with a one-line pitch, a demo-video pointer, and a real example call + example response for every tool. HACKATHON-SUBMISSION updated with the new tool, a fresh validator run (15/15, 0 warnings), and an honest demo-coverage note for the branch-task tool including two live production responses from the real public /v1/demo/branch endpoint this tool's workflow mirrors (a transient 402, then the real 429 per-hour rate limit — both genuine, neither invented).
2026-11-05 23:48:28 +03:00
- `branch_task` — `POST /v1/environments` (`kind: "preview"`), `POST
.../branches/merge`, and `DELETE .../environments/{env}`: a real
copy-on-write Postgres branch an agent can create for one task, then
either promote (merge its schema back) or discard (delete it), never
touching the parent environment except through an explicit, auditable
merge.
feat: add scribase_memory_task — real persistent agent memory (store/recall/search/forget) Hermes agents had no persistent memory via this plugin: scribase_query is read-only (enforced by the control plane itself) and scribase_lint_migration hardcodes apply:false, so none of the first four tools could ever write application data, only inspect or lint it. scribase_memory_task closes that gap using a real, previously-unused endpoint read from the compiled control plane: POST .../database/execute (the write-mode SQL route the Scribase Studio SQL editor uses in "write mode", crates/control-api/src/adapters/sql_execute.rs), for store/forget; recall/search reuse the existing read-only database/query route. Creates its own table (hermes_agent_memories, RLS enabled) on first use, scoped by an optional namespace so multiple agents/users can share a project without reading each other's memories. Every value interpolated into SQL (content, metadata, scope, memory_id) is escaped via the standard Postgres string-literal escape (_sql_literal) — the model never passes raw SQL through this tool. Verified: a throwaway monkeypatched test exercises all 4 actions + error paths + an injection-style content string (confirms balanced escaping, no executable SQL smuggled through). Then ran the real, unmodified tools.py directly against a real, no-login temporary Scribase project on production api.scribase.com: store (x2) -> recall -> search -> scope-isolation check (a second, never-used scope correctly returns zero rows) -> forget -> recall again (confirms exactly the forgotten memory is gone, the other survives). Full transcript in HACKATHON-SUBMISSION-2026-11-05.md. Re-ran validate_plugin_dir() (the function `hermes plugins validate` calls) against a fresh NousResearch/hermes-agent clone: 15/15 checks pass, 0 warnings, isolation host_ready: true, declared tools matches registrations (5 tools, including scribase_memory_task). plugin.yaml bumped to 1.3.0, provides_tools now lists all 5.
2026-11-06 18:16:17 +03:00
- `memory_task` — `POST .../database/execute` (write-mode SQL, confirmed
live against a real throwaway project: this is the write-capable sibling
of `database/query` the Scribase Studio SQL editor uses in "write mode",
gated to writer-scoped callers, see `crates/control-api/src/http/
database_editor.rs` and `adapters/sql_execute.rs`) for `store`/`forget`,
and `POST .../database/query` (the same read-only route `run_query`
calls) for `recall`/`search`: gives a Hermes agent a real, persistent,
Postgres-backed memory store — not a prompt's own context window, a real
table in the user's own Scribase project that survives across sessions.
All SQL this function sends is built by this module, never passed
through from the model's arguments as raw SQL (that's what `run_query`
is for) — only `content`/`metadata`/`scope`/`memory_id` string values are
fix: replace quote-doubling SQL literal escape with hex-decode in memory_task Independently re-verified the SQL-injection defense scribase_memory_task's _sql_literal() claimed (store/recall/search/forget). Ran a real adversarial battery (stacked-query attempts, comment sequences, UNION probes, NUL bytes, a trailing backslash, scope/memory_id injection, ILIKE breakout attempts, a 24KB payload, a Unicode lookalike quote) against a real, live, no-login temporary Scribase project on production api.scribase.com (provisioned via a disposable throwaway account to work around the daily per-address cap) — every adversarial payload landed as inert literal data, no injection, no cross-scope leak, table/RLS/row-count verified intact via an independent read-only query after each pass. The injection defense itself was sound (the quote-doubling escape is also backed by the control plane forcing standard_conforming_strings=on for every session, confirmed in psql.rs), but the same live testing found a real, reproducible, non-adversarial bug: the control plane's own backslash guard (client_command_forbidden) rejects *any* literal containing a backslash character, which the old quote-doubling helper produced whenever metadata contained a non-ASCII character (json.dumps's default \uXXXX escaping) or an embedded double quote, or content itself had a literal backslash (a Windows path, a regex) — i.e. ordinary non-English or technical content, not just hostile input, always 400'd. Replaced the doubled-single-quote literal with a hex-encoded convert_from(decode('<hex>','hex'),'UTF8') expression for every interpolated value (content/metadata/scope/memory_id). A hex literal can only ever contain 0-9a-f, so it can never contain a quote, backslash, or comment sequence — this removes the backslash-guard false positive and the escaping class of risk entirely, without depending on standard_conforming_strings either way. Re-ran the full adversarial + regression battery against the fixed code on the same live project: every real-world case that previously 400'd now stores and round-trips correctly (confirmed café/Türkçe/日本語 byte-for-byte), and every adversarial payload remains safely inert.
2026-11-06 23:05:52 +03:00
interpolated, and every one goes through `_sql_literal()` first (hex-
encoded as `convert_from(decode('<hex>', 'hex'), 'UTF8')`, not a quoted-
and-escaped literal — see `_sql_literal()`'s docstring for why: live
re-verification found that the classic doubled-single-quote escape,
while not itself SQL-injectable, tripped the control plane's own
backslash guard on ordinary non-adversarial content such as non-ASCII
metadata) before it is ever placed in a statement.
"""
from __future__ import annotations
import json
feat: add scribase_memory_task — real persistent agent memory (store/recall/search/forget) Hermes agents had no persistent memory via this plugin: scribase_query is read-only (enforced by the control plane itself) and scribase_lint_migration hardcodes apply:false, so none of the first four tools could ever write application data, only inspect or lint it. scribase_memory_task closes that gap using a real, previously-unused endpoint read from the compiled control plane: POST .../database/execute (the write-mode SQL route the Scribase Studio SQL editor uses in "write mode", crates/control-api/src/adapters/sql_execute.rs), for store/forget; recall/search reuse the existing read-only database/query route. Creates its own table (hermes_agent_memories, RLS enabled) on first use, scoped by an optional namespace so multiple agents/users can share a project without reading each other's memories. Every value interpolated into SQL (content, metadata, scope, memory_id) is escaped via the standard Postgres string-literal escape (_sql_literal) — the model never passes raw SQL through this tool. Verified: a throwaway monkeypatched test exercises all 4 actions + error paths + an injection-style content string (confirms balanced escaping, no executable SQL smuggled through). Then ran the real, unmodified tools.py directly against a real, no-login temporary Scribase project on production api.scribase.com: store (x2) -> recall -> search -> scope-isolation check (a second, never-used scope correctly returns zero rows) -> forget -> recall again (confirms exactly the forgotten memory is gone, the other survives). Full transcript in HACKATHON-SUBMISSION-2026-11-05.md. Re-ran validate_plugin_dir() (the function `hermes plugins validate` calls) against a fresh NousResearch/hermes-agent clone: 15/15 checks pass, 0 warnings, isolation host_ready: true, declared tools matches registrations (5 tools, including scribase_memory_task). plugin.yaml bumped to 1.3.0, provides_tools now lists all 5.
2026-11-06 18:16:17 +03:00
import re
import time
import urllib.error
import urllib.request
import uuid
from typing import Any, Dict, Optional, Tuple
_TIMEOUT = 20 # seconds per HTTP call
_POLL_ATTEMPTS = 8
_POLL_INTERVAL = 3 # seconds
_DEFAULT_REGION = "eu-frankfurt"
_DEFAULT_POSTGRES_MAJOR = 17
_DEFAULT_ENVIRONMENT_ID = "production"
_TERMINAL_STATES = ("succeeded", "failed", "cancelled")
feat: add scribase_branch_task — real copy-on-write branch create/promote/discard Fourth tool: one action='create'|'promote'|'discard' tool that forks a real copy-on-write Postgres branch for one agent task (POST /v1/environments, kind="preview", bounded TTL so it always lands in the control plane's agent-branch slot pool), three-way merges it back into its parent on action='promote' (POST .../branches/merge, reads the real applied/clean/changes/conflicts/statements fields the compiled server renders, and surfaces a real mfa_required rejection on a forced merge instead of hiding it), or deletes it outright on action='discard' (DELETE .../environments/{env}). Built from the real control-api source (http/environments.rs, http/branches.rs, adapters/branch.rs, adapters/usage.rs::AGENT_BRANCH_MAX_TTL_HOURS) and the public docs.scribase.com environments/branches pages, not guessed. Factored create_project's polling and API-keys/connection readback into shared _poll_operation/_read_environment_access helpers, reused by both tools; create_project's own behavior is unchanged (verified with a mocked HTTP layer, happy path and warning path). plugin.yaml bumped to 1.2.0, provides_tools now lists all 4 tools. README rewritten with a one-line pitch, a demo-video pointer, and a real example call + example response for every tool. HACKATHON-SUBMISSION updated with the new tool, a fresh validator run (15/15, 0 warnings), and an honest demo-coverage note for the branch-task tool including two live production responses from the real public /v1/demo/branch endpoint this tool's workflow mirrors (a transient 402, then the real 429 per-hour rate limit — both genuine, neither invented).
2026-11-05 23:48:28 +03:00
# A branch with a TTL of this many hours or less lands in Scribase's own
# "agent branch" slot pool instead of competing with human preview
# environments for the project's regular preview quota (confirmed against
# `control-api/src/adapters/usage.rs::AGENT_BRANCH_MAX_TTL_HOURS = 24` and
# `is_agent_branch()`). `scribase_branch_task` always sends a TTL in this
# range so an agent's branches never starve a human's preview slots, and so
# a branch nobody discards still self-deletes instead of running forever.
_AGENT_BRANCH_MAX_TTL_HOURS = 24
_DEFAULT_BRANCH_TTL_HOURS = 4
_VALID_DATA_MODES = ("snapshot", "sanitized", "schema_only")
def _request(
method: str, url: str, api_key: str,
body: Optional[Dict[str, Any]] = None,
extra_headers: Optional[Dict[str, str]] = None,
) -> Tuple[int, Any]:
"""One HTTP call against the Scribase management API.
Returns ``(status, parsed_body)``. ``status`` is 0 when the request
could not be sent at all (DNS/connection failure); callers treat that
as a terminal error, never a retryable one, to avoid hanging an
unattended run (plugin-catalog rule 12).
"""
data = json.dumps(body).encode("utf-8") if body is not None else None
fix: send a User-Agent and drop the null ttl_hours that broke every real call Found both bugs for real while running the plugin's own unmodified tools.py against production api.scribase.com during the 2026-11-06 Hermes session demo (not guessed, reproduced live): - _request() never set a User-Agent, so a bare CPython urllib call (exactly what this plugin's handler sends) got hard-blocked by Cloudflare's WAF in front of api.scribase.com (403 browser_signature_banned) before reaching the control plane at all. Confirmed by replaying the identical "Python-urllib/3.x" User-Agent string via curl and getting the same block, then clearing it with a normal UA string and getting through. - create_project()'s environment-create POST always sent "ttl_hours": null for a production environment (which has no TTL); the control plane's strict request-schema validator rejects an explicit null for that optional field with 400 invalid_json, while omitting the key entirely succeeds. Confirmed via an A/B curl replay of the exact same body with vs. without the key. Verified after the fix: re-ran create_project and run_query from the fixed, unmodified tools.py (no monkeypatching) against the real temporary project created for this demo — create_project now reaches real conflict-detection logic (409, environment already exists) instead of the WAF/schema errors, and run_query returns real rows cleanly. Re-ran validate_plugin_dir() (the function `hermes plugins validate` calls) against the fixed plugin: 15/15 checks pass, 0 warnings, isolation host_ready true. version: 1.2.0 -> 1.2.1
2026-11-06 00:24:54 +03:00
headers = {
"Authorization": f"Bearer {api_key}",
"Accept": "application/json",
# Required, not cosmetic: api.scribase.com sits behind Cloudflare, and a
# bare CPython urllib request (the literal "Python-urllib/3.x" default
# User-Agent, with no header set at all otherwise) is hard-blocked by
# Cloudflare's WAF with a 403 "browser_signature_banned" before it ever
# reaches the control plane — reproduced live against production while
# building the 2026-11-06 real-session demo. Any non-default string
# clears it; this one just also identifies the client honestly.
"User-Agent": "scribase-hermes-plugin/1.2.1 (+https://github.com/caelum0x/scribase-hermes-plugin)",
}
if data is not None:
headers["Content-Type"] = "application/json"
if extra_headers:
headers.update(extra_headers)
req = urllib.request.Request(url, data=data, method=method, headers=headers)
try:
with urllib.request.urlopen(req, timeout=_TIMEOUT) as resp:
raw = resp.read().decode("utf-8", "replace")
status = resp.status
except urllib.error.HTTPError as exc:
raw = exc.read().decode("utf-8", "replace")
status = exc.code
except urllib.error.URLError as exc:
return 0, {"error": f"could not reach the Scribase API: {exc.reason}"}
if not raw:
return status, {}
try:
return status, json.loads(raw)
except json.JSONDecodeError:
return status, {"error": raw[:500]}
def _idempotency_key() -> str:
return str(uuid.uuid4())
feat: add scribase_branch_task — real copy-on-write branch create/promote/discard Fourth tool: one action='create'|'promote'|'discard' tool that forks a real copy-on-write Postgres branch for one agent task (POST /v1/environments, kind="preview", bounded TTL so it always lands in the control plane's agent-branch slot pool), three-way merges it back into its parent on action='promote' (POST .../branches/merge, reads the real applied/clean/changes/conflicts/statements fields the compiled server renders, and surfaces a real mfa_required rejection on a forced merge instead of hiding it), or deletes it outright on action='discard' (DELETE .../environments/{env}). Built from the real control-api source (http/environments.rs, http/branches.rs, adapters/branch.rs, adapters/usage.rs::AGENT_BRANCH_MAX_TTL_HOURS) and the public docs.scribase.com environments/branches pages, not guessed. Factored create_project's polling and API-keys/connection readback into shared _poll_operation/_read_environment_access helpers, reused by both tools; create_project's own behavior is unchanged (verified with a mocked HTTP layer, happy path and warning path). plugin.yaml bumped to 1.2.0, provides_tools now lists all 4 tools. README rewritten with a one-line pitch, a demo-video pointer, and a real example call + example response for every tool. HACKATHON-SUBMISSION updated with the new tool, a fresh validator run (15/15, 0 warnings), and an honest demo-coverage note for the branch-task tool including two live production responses from the real public /v1/demo/branch endpoint this tool's workflow mirrors (a transient 402, then the real 429 per-hour rate limit — both genuine, neither invented).
2026-11-05 23:48:28 +03:00
def _poll_operation(
operation_id: str, organization_id: str, *, api_key: str, api_base: str
) -> Tuple[Optional[str], Dict[str, Any]]:
"""Polls `GET /v1/operations/{id}` until it leaves pending/running, bounded
so an unattended run (cron, gateway) can never hang on one tool call
(plugin-catalog rule 12). Returns ``(final_state, last_operation_body)``;
``final_state`` is ``None`` if every poll attempt was spent without the
operation reaching a terminal state.
"""
final_state: Optional[str] = None
last_body: Dict[str, Any] = {}
for _ in range(_POLL_ATTEMPTS):
time.sleep(_POLL_INTERVAL)
status, body = _request(
"GET", f"{api_base}/v1/operations/{operation_id}", api_key,
extra_headers={"X-Scribase-Organization-ID": organization_id},
)
if status == 200:
last_body = body
final_state = body.get("state")
if final_state in _TERMINAL_STATES:
break
return final_state, last_body
def _read_environment_access(
organization_id: str, project_id: str, environment_id: str, *, api_key: str, api_base: str
) -> Dict[str, Any]:
"""Reads back a ready environment's API keys and Postgres connection URI.
Returns a dict always containing ``status`` (``"ready"`` or a
``"warning"``-carrying partial result) plus whichever of
``api_url``/``anon_key``/``publishable_key``/``service_role_key``/
``secret_key``/``connection_uri`` could be read.
"""
status, keys = _request(
"GET",
f"{api_base}/v1/organizations/{organization_id}/projects/{project_id}"
f"/environments/{environment_id}/api-keys",
api_key,
)
if status != 200:
return {
"status": "ready",
"warning": "environment is ready, but failed to read its API keys",
"detail": keys,
}
status, conn = _request(
"GET",
f"{api_base}/v1/organizations/{organization_id}/projects/{project_id}"
f"/environments/{environment_id}/connection",
api_key,
)
connection_uri = conn.get("connection_uri") if status == 200 else None
return {
"status": "ready",
"api_url": keys.get("api_url"),
"anon_key": keys.get("anon_key"),
"publishable_key": keys.get("publishable_key"),
"service_role_key": keys.get("service_role_key"),
"secret_key": keys.get("secret_key"),
"connection_uri": connection_uri,
}
def create_project(args: Dict[str, Any], *, api_key: str, api_base: str) -> Dict[str, Any]:
"""Create a Scribase project and its first environment, then return connection info.
Required args: ``organization_id``, ``project_id``. Optional:
``environment_id`` (default "production"), ``region`` (default
"eu-frankfurt", hosted Scribase's region), ``postgres_major`` (default
17, the only currently supported version).
"""
org = str(args.get("organization_id") or "").strip()
project = str(args.get("project_id") or "").strip()
environment = str(args.get("environment_id") or "").strip() or _DEFAULT_ENVIRONMENT_ID
region = str(args.get("region") or "").strip() or _DEFAULT_REGION
try:
postgres_major = int(args.get("postgres_major") or _DEFAULT_POSTGRES_MAJOR)
except (TypeError, ValueError):
return {"error": "postgres_major must be an integer (only 17 is currently supported)."}
if not org:
return {"error": "organization_id is required (the existing Scribase org this project belongs to)."}
if not project:
return {"error": "project_id is required."}
# 1. Create (or confirm) the project — idempotent PUT.
status, body = _request(
"PUT", f"{api_base}/v1/organizations/{org}/projects/{project}", api_key,
body={"region": region, "postgres_major": postgres_major},
extra_headers={"Idempotency-Key": _idempotency_key()},
)
if status == 0:
return body
if status not in (200, 201):
return {"error": "failed to create project", "status": status, "detail": body}
# 2. Create its first environment — async; returns an operation to poll.
fix: send a User-Agent and drop the null ttl_hours that broke every real call Found both bugs for real while running the plugin's own unmodified tools.py against production api.scribase.com during the 2026-11-06 Hermes session demo (not guessed, reproduced live): - _request() never set a User-Agent, so a bare CPython urllib call (exactly what this plugin's handler sends) got hard-blocked by Cloudflare's WAF in front of api.scribase.com (403 browser_signature_banned) before reaching the control plane at all. Confirmed by replaying the identical "Python-urllib/3.x" User-Agent string via curl and getting the same block, then clearing it with a normal UA string and getting through. - create_project()'s environment-create POST always sent "ttl_hours": null for a production environment (which has no TTL); the control plane's strict request-schema validator rejects an explicit null for that optional field with 400 invalid_json, while omitting the key entirely succeeds. Confirmed via an A/B curl replay of the exact same body with vs. without the key. Verified after the fix: re-ran create_project and run_query from the fixed, unmodified tools.py (no monkeypatching) against the real temporary project created for this demo — create_project now reaches real conflict-detection logic (409, environment already exists) instead of the WAF/schema errors, and run_query returns real rows cleanly. Re-ran validate_plugin_dir() (the function `hermes plugins validate` calls) against the fixed plugin: 15/15 checks pass, 0 warnings, isolation host_ready true. version: 1.2.0 -> 1.2.1
2026-11-06 00:24:54 +03:00
# `ttl_hours` is omitted entirely rather than sent as a literal JSON
# `null` — a production environment has no TTL, and the control
# plane's request-schema validator rejects an explicit null for this
# optional field with `400 invalid_json` ("request JSON does not
# match the operation schema"), confirmed live against production
# while building the 2026-11-06 real-session demo (A/B: omitting the
# key succeeds — a real `409 conflict` on an already-provisioned
# environment, or a real `202` on a genuinely new one; sending
# `"ttl_hours": null` 400s every time).
status, body = _request(
"POST", f"{api_base}/v1/environments", api_key,
body={
"organization_id": org, "project_id": project, "environment_id": environment,
"kind": "production", "data_mode": "schema_only",
fix: send a User-Agent and drop the null ttl_hours that broke every real call Found both bugs for real while running the plugin's own unmodified tools.py against production api.scribase.com during the 2026-11-06 Hermes session demo (not guessed, reproduced live): - _request() never set a User-Agent, so a bare CPython urllib call (exactly what this plugin's handler sends) got hard-blocked by Cloudflare's WAF in front of api.scribase.com (403 browser_signature_banned) before reaching the control plane at all. Confirmed by replaying the identical "Python-urllib/3.x" User-Agent string via curl and getting the same block, then clearing it with a normal UA string and getting through. - create_project()'s environment-create POST always sent "ttl_hours": null for a production environment (which has no TTL); the control plane's strict request-schema validator rejects an explicit null for that optional field with 400 invalid_json, while omitting the key entirely succeeds. Confirmed via an A/B curl replay of the exact same body with vs. without the key. Verified after the fix: re-ran create_project and run_query from the fixed, unmodified tools.py (no monkeypatching) against the real temporary project created for this demo — create_project now reaches real conflict-detection logic (409, environment already exists) instead of the WAF/schema errors, and run_query returns real rows cleanly. Re-ran validate_plugin_dir() (the function `hermes plugins validate` calls) against the fixed plugin: 15/15 checks pass, 0 warnings, isolation host_ready true. version: 1.2.0 -> 1.2.1
2026-11-06 00:24:54 +03:00
"region": region, "postgres_major": postgres_major,
},
extra_headers={"Idempotency-Key": _idempotency_key()},
)
if status == 0:
return body
if status != 202:
return {"error": "failed to create environment", "status": status, "detail": body}
operation_id = body.get("id")
# 3. Poll the operation until it leaves pending/running/waiting_retry, bounded so an
# unattended run (cron, gateway) can never hang on this tool call (rule 12).
feat: add scribase_branch_task — real copy-on-write branch create/promote/discard Fourth tool: one action='create'|'promote'|'discard' tool that forks a real copy-on-write Postgres branch for one agent task (POST /v1/environments, kind="preview", bounded TTL so it always lands in the control plane's agent-branch slot pool), three-way merges it back into its parent on action='promote' (POST .../branches/merge, reads the real applied/clean/changes/conflicts/statements fields the compiled server renders, and surfaces a real mfa_required rejection on a forced merge instead of hiding it), or deletes it outright on action='discard' (DELETE .../environments/{env}). Built from the real control-api source (http/environments.rs, http/branches.rs, adapters/branch.rs, adapters/usage.rs::AGENT_BRANCH_MAX_TTL_HOURS) and the public docs.scribase.com environments/branches pages, not guessed. Factored create_project's polling and API-keys/connection readback into shared _poll_operation/_read_environment_access helpers, reused by both tools; create_project's own behavior is unchanged (verified with a mocked HTTP layer, happy path and warning path). plugin.yaml bumped to 1.2.0, provides_tools now lists all 4 tools. README rewritten with a one-line pitch, a demo-video pointer, and a real example call + example response for every tool. HACKATHON-SUBMISSION updated with the new tool, a fresh validator run (15/15, 0 warnings), and an honest demo-coverage note for the branch-task tool including two live production responses from the real public /v1/demo/branch endpoint this tool's workflow mirrors (a transient 402, then the real 429 per-hour rate limit — both genuine, neither invented).
2026-11-05 23:48:28 +03:00
final_state, last_op_body = _poll_operation(operation_id, org, api_key=api_key, api_base=api_base)
if final_state != "succeeded":
still_going = final_state not in ("failed", "cancelled")
return {
"status": "provisioning" if still_going else "failed",
"organization_id": org, "project_id": project, "environment_id": environment,
"operation_id": operation_id,
"message": (
f"Environment is still provisioning (state={final_state!r}). Call this tool "
f"again shortly, or poll GET /v1/operations/{operation_id} directly."
if still_going else
f"Environment provisioning failed (state={final_state!r})."
),
"operation": last_op_body,
}
# 4. Fetch the new environment's API keys and connection string.
feat: add scribase_branch_task — real copy-on-write branch create/promote/discard Fourth tool: one action='create'|'promote'|'discard' tool that forks a real copy-on-write Postgres branch for one agent task (POST /v1/environments, kind="preview", bounded TTL so it always lands in the control plane's agent-branch slot pool), three-way merges it back into its parent on action='promote' (POST .../branches/merge, reads the real applied/clean/changes/conflicts/statements fields the compiled server renders, and surfaces a real mfa_required rejection on a forced merge instead of hiding it), or deletes it outright on action='discard' (DELETE .../environments/{env}). Built from the real control-api source (http/environments.rs, http/branches.rs, adapters/branch.rs, adapters/usage.rs::AGENT_BRANCH_MAX_TTL_HOURS) and the public docs.scribase.com environments/branches pages, not guessed. Factored create_project's polling and API-keys/connection readback into shared _poll_operation/_read_environment_access helpers, reused by both tools; create_project's own behavior is unchanged (verified with a mocked HTTP layer, happy path and warning path). plugin.yaml bumped to 1.2.0, provides_tools now lists all 4 tools. README rewritten with a one-line pitch, a demo-video pointer, and a real example call + example response for every tool. HACKATHON-SUBMISSION updated with the new tool, a fresh validator run (15/15, 0 warnings), and an honest demo-coverage note for the branch-task tool including two live production responses from the real public /v1/demo/branch endpoint this tool's workflow mirrors (a transient 402, then the real 429 per-hour rate limit — both genuine, neither invented).
2026-11-05 23:48:28 +03:00
access = _read_environment_access(org, project, environment, api_key=api_key, api_base=api_base)
if access.get("status") != "ready" or "warning" in access:
return access
return {
"organization_id": org,
"project_id": project,
"environment_id": environment,
"region": region,
feat: add scribase_branch_task — real copy-on-write branch create/promote/discard Fourth tool: one action='create'|'promote'|'discard' tool that forks a real copy-on-write Postgres branch for one agent task (POST /v1/environments, kind="preview", bounded TTL so it always lands in the control plane's agent-branch slot pool), three-way merges it back into its parent on action='promote' (POST .../branches/merge, reads the real applied/clean/changes/conflicts/statements fields the compiled server renders, and surfaces a real mfa_required rejection on a forced merge instead of hiding it), or deletes it outright on action='discard' (DELETE .../environments/{env}). Built from the real control-api source (http/environments.rs, http/branches.rs, adapters/branch.rs, adapters/usage.rs::AGENT_BRANCH_MAX_TTL_HOURS) and the public docs.scribase.com environments/branches pages, not guessed. Factored create_project's polling and API-keys/connection readback into shared _poll_operation/_read_environment_access helpers, reused by both tools; create_project's own behavior is unchanged (verified with a mocked HTTP layer, happy path and warning path). plugin.yaml bumped to 1.2.0, provides_tools now lists all 4 tools. README rewritten with a one-line pitch, a demo-video pointer, and a real example call + example response for every tool. HACKATHON-SUBMISSION updated with the new tool, a fresh validator run (15/15, 0 warnings), and an honest demo-coverage note for the branch-task tool including two live production responses from the real public /v1/demo/branch endpoint this tool's workflow mirrors (a transient 402, then the real 429 per-hour rate limit — both genuine, neither invented).
2026-11-05 23:48:28 +03:00
**access,
}
def lint_migration(args: Dict[str, Any], *, api_key: str, api_base: str) -> Dict[str, Any]:
"""Lint one migration's SQL against Scribase's real migration-safety rules.
Calls ``POST .../migrations`` with a single ``migration`` entry and
``"apply": false`` (hardcoded here, never read from ``args``) — this is
the control plane's own dry-run path: the migration is parsed into a
schema model and run through the blocking rules (missing RLS, GRANT to
PUBLIC, a permissive `USING (true)` policy, a write policy with no `WITH
CHECK`, a `SECURITY DEFINER` function) and the one advisory rule (a
foreign key with no supporting index), but nothing ever touches the
tenant database — the control plane only opens a database session when
``apply`` is true and the lint is clean, which this call can never
request.
"""
org = str(args.get("organization_id") or "").strip()
project = str(args.get("project_id") or "").strip()
environment = str(args.get("environment_id") or "").strip()
up = str(args.get("up") or "").strip()
down = args.get("down")
name = str(args.get("migration_name") or "").strip() or "hermes_lint_check"
if not org:
return {"error": "organization_id is required."}
if not project:
return {"error": "project_id is required."}
if not environment:
return {"error": "environment_id is required."}
if not up:
return {"error": "up is required (the migration's forward SQL to lint)."}
migration: Dict[str, Any] = {
"version": time.strftime("%Y%m%d%H%M%S", time.gmtime()),
"name": name,
"up": up,
}
if down:
migration["down"] = str(down)
status, body = _request(
"POST",
f"{api_base}/v1/organizations/{org}/projects/{project}/environments/{environment}/migrations",
api_key,
# "apply" is always False here — this tool only ever lints.
body={"migration": migration, "apply": False},
)
if status == 0:
return body
if status != 200:
return {"error": "migration lint request failed", "status": status, "detail": body}
findings = body.get("lintFindings", []) if isinstance(body, dict) else []
blocking = [f for f in findings if isinstance(f, dict) and f.get("severity") == "BLOCK"]
warnings = [f for f in findings if isinstance(f, dict) and f.get("severity") == "WARN"]
return {
"status": "blocked" if blocking else ("warnings" if warnings else "clean"),
"safe_to_apply": not blocking,
"blocking_findings": blocking,
"warning_findings": warnings,
"plan": body.get("plan") if isinstance(body, dict) else None,
}
def run_query(args: Dict[str, Any], *, api_key: str, api_base: str) -> Dict[str, Any]:
"""Run one read-only statement via ``POST .../database/query``.
The control plane itself enforces read-only in two independent layers
(a static keyword check, then a Postgres `READ ONLY` transaction with a
bounded statement timeout), so this client adds no SQL-shape validation
of its own — any write/DDL statement is rejected by the server, not
silently reinterpreted here.
"""
org = str(args.get("organization_id") or "").strip()
project = str(args.get("project_id") or "").strip()
environment = str(args.get("environment_id") or "").strip()
query = str(args.get("query") or "").strip()
if not org:
return {"error": "organization_id is required."}
if not project:
return {"error": "project_id is required."}
if not environment:
return {"error": "environment_id is required."}
if not query:
return {"error": "query is required (a single read-only SQL statement)."}
body: Dict[str, Any] = {"query": query}
limit = args.get("limit")
if limit is not None:
try:
body["limit"] = int(limit)
except (TypeError, ValueError):
return {"error": "limit must be an integer between 1 and 1000."}
status, result = _request(
"POST",
f"{api_base}/v1/organizations/{org}/projects/{project}/environments/{environment}/database/query",
api_key,
body=body,
)
if status == 0:
return result
if status != 200:
return {"error": "query failed", "status": status, "detail": result}
return result
feat: add scribase_branch_task — real copy-on-write branch create/promote/discard Fourth tool: one action='create'|'promote'|'discard' tool that forks a real copy-on-write Postgres branch for one agent task (POST /v1/environments, kind="preview", bounded TTL so it always lands in the control plane's agent-branch slot pool), three-way merges it back into its parent on action='promote' (POST .../branches/merge, reads the real applied/clean/changes/conflicts/statements fields the compiled server renders, and surfaces a real mfa_required rejection on a forced merge instead of hiding it), or deletes it outright on action='discard' (DELETE .../environments/{env}). Built from the real control-api source (http/environments.rs, http/branches.rs, adapters/branch.rs, adapters/usage.rs::AGENT_BRANCH_MAX_TTL_HOURS) and the public docs.scribase.com environments/branches pages, not guessed. Factored create_project's polling and API-keys/connection readback into shared _poll_operation/_read_environment_access helpers, reused by both tools; create_project's own behavior is unchanged (verified with a mocked HTTP layer, happy path and warning path). plugin.yaml bumped to 1.2.0, provides_tools now lists all 4 tools. README rewritten with a one-line pitch, a demo-video pointer, and a real example call + example response for every tool. HACKATHON-SUBMISSION updated with the new tool, a fresh validator run (15/15, 0 warnings), and an honest demo-coverage note for the branch-task tool including two live production responses from the real public /v1/demo/branch endpoint this tool's workflow mirrors (a transient 402, then the real 429 per-hour rate limit — both genuine, neither invented).
2026-11-05 23:48:28 +03:00
def branch_task(args: Dict[str, Any], *, api_key: str, api_base: str) -> Dict[str, Any]:
"""Dispatches one of three real, agent-native lifecycle actions on a
throwaway, real copy-on-write Postgres branch:
- ``"create"`` — cuts a new branch environment from a parent (default
``"production"``), with a bounded TTL that keeps it in Scribase's
dedicated agent-branch slot pool, then returns its connection string
and API keys, same shape as `create_project`.
- ``"promote"`` — three-way merges the branch's schema back into its
parent via ``POST .../branches/merge``. The server applies the merge
only when it is conflict-free, or when ``force`` is set and the
organization has no MFA factor (an MFA-protected organization's API
key cannot answer the step-up challenge a forced merge requires —
the server rejects it with ``mfa_required`` rather than silently
skipping the check, and this client returns that rejection verbatim).
- ``"discard"`` — deletes the branch outright via ``DELETE
.../environments/{env}``, instead of waiting for its TTL.
No action ever touches the parent environment except an applied
promote, and even that is a schema-only three-way merge the control
plane itself plans and can refuse on conflict.
"""
action = str(args.get("action") or "").strip().lower()
org = str(args.get("organization_id") or "").strip()
project = str(args.get("project_id") or "").strip()
if not org:
return {"error": "organization_id is required."}
if not project:
return {"error": "project_id is required."}
if action == "create":
return _branch_create(args, org, project, api_key=api_key, api_base=api_base)
if action == "promote":
return _branch_promote(args, org, project, api_key=api_key, api_base=api_base)
if action == "discard":
return _branch_discard(args, org, project, api_key=api_key, api_base=api_base)
return {"error": "action must be one of: create, promote, discard."}
def _branch_create(
args: Dict[str, Any], org: str, project: str, *, api_key: str, api_base: str
) -> Dict[str, Any]:
"""``create``: cuts a new ``kind: "preview"`` environment forked from
``base_environment_id`` (default ``"production"``). The control plane
captures the parent's schema as the branch's merge base automatically on
creation — no separate `POST .../branches` call is needed (confirmed by
reading `control-api/src/http/previews.rs::provision_environment()`,
which calls `branch::record_fork_point` itself whenever a preview is cut
from an explicit or implicit base). If that capture fails, the server
reports it as a top-level `branch_lineage_error` on this same response;
this client surfaces it as `branch_lineage_warning` rather than dropping
it, since a branch with no recorded merge base cannot later be promoted.
"""
base = str(args.get("base_environment_id") or "").strip() or _DEFAULT_ENVIRONMENT_ID
branch_id = str(args.get("environment_id") or "").strip() or f"agent-{uuid.uuid4().hex[:8]}"
data_mode = str(args.get("data_mode") or "").strip() or "snapshot"
region = str(args.get("region") or "").strip() or _DEFAULT_REGION
try:
postgres_major = int(args.get("postgres_major") or _DEFAULT_POSTGRES_MAJOR)
except (TypeError, ValueError):
return {"error": "postgres_major must be an integer (only 17 is currently supported)."}
try:
ttl_hours = int(args.get("ttl_hours") or _DEFAULT_BRANCH_TTL_HOURS)
except (TypeError, ValueError):
return {"error": "ttl_hours must be an integer."}
if branch_id == base:
return {"error": "a branch cannot be its own base environment."}
if data_mode not in _VALID_DATA_MODES:
return {"error": f"data_mode must be one of: {', '.join(_VALID_DATA_MODES)}."}
if not 1 <= ttl_hours <= _AGENT_BRANCH_MAX_TTL_HOURS:
return {
"error": (
f"ttl_hours must be between 1 and {_AGENT_BRANCH_MAX_TTL_HOURS} — a branch "
"this tool creates always stays in Scribase's agent-branch slot pool, never "
"the human preview pool, and this is its allowed TTL range."
)
}
status, body = _request(
"POST", f"{api_base}/v1/environments", api_key,
body={
"organization_id": org, "project_id": project, "environment_id": branch_id,
"kind": "preview", "data_mode": data_mode, "region": region,
"postgres_major": postgres_major, "ttl_hours": ttl_hours,
"base_environment_id": base,
},
extra_headers={"Idempotency-Key": _idempotency_key()},
)
if status == 0:
return body
if status not in (200, 202):
return {"error": "failed to create branch", "status": status, "detail": body}
lineage_warning = body.get("branch_lineage_error") if isinstance(body, dict) else None
operation_id = body.get("id") if isinstance(body, dict) else None
if not operation_id:
return {"error": "branch create accepted but no operation id was returned", "detail": body}
final_state, last_op_body = _poll_operation(operation_id, org, api_key=api_key, api_base=api_base)
if final_state != "succeeded":
still_going = final_state not in ("failed", "cancelled")
result = {
"status": "provisioning" if still_going else "failed",
"organization_id": org, "project_id": project,
"environment_id": branch_id, "base_environment_id": base,
"operation_id": operation_id,
"message": (
f"Branch is still provisioning (state={final_state!r}). Call this tool again "
f"shortly with action='create' and the same environment_id, or poll "
f"GET /v1/operations/{operation_id} directly."
if still_going else
f"Branch provisioning failed (state={final_state!r})."
),
"operation": last_op_body,
}
if lineage_warning:
result["branch_lineage_warning"] = lineage_warning
return result
access = _read_environment_access(org, project, branch_id, api_key=api_key, api_base=api_base)
result = {
"organization_id": org,
"project_id": project,
"environment_id": branch_id,
"base_environment_id": base,
"data_mode": data_mode,
"ttl_hours": ttl_hours,
"region": region,
**access,
"next_step": (
"When the task is done, call this tool again with action='promote' to merge "
"this branch's schema back into "
f"{base!r}, or action='discard' to delete it immediately. Otherwise it "
f"self-deletes in {ttl_hours}h."
),
}
if lineage_warning:
result["branch_lineage_warning"] = lineage_warning
return result
def _branch_promote(
args: Dict[str, Any], org: str, project: str, *, api_key: str, api_base: str
) -> Dict[str, Any]:
"""``promote``: three-way merges ``environment_id`` (the branch, "head")
into ``base_environment_id`` (default ``"production"``) via ``POST
.../branches/merge``. Reads the real response fields the control plane
renders (`merge_to_json` in `control-api/src/adapters/branch.rs`) —
``applied``/``clean``/``changes``/``conflicts``/``statements`` — rather
than the simplified ``merged`` field the docs page's example shows,
since the compiled server is what actually answers.
"""
head = str(args.get("environment_id") or "").strip()
base = str(args.get("base_environment_id") or "").strip() or _DEFAULT_ENVIRONMENT_ID
force = bool(args.get("force"))
dry_run = bool(args.get("dry_run"))
if not head:
return {"error": "environment_id is required (the branch to promote)."}
if head == base:
return {"error": "a branch cannot be promoted into itself."}
status, body = _request(
"POST",
f"{api_base}/v1/organizations/{org}/projects/{project}/branches/merge",
api_key,
body={"base": base, "head": head, "force": force, "dry_run": dry_run},
)
if status == 0:
return body
if status != 200 or not isinstance(body, dict):
# Real, common failures here: 409 branch_not_registered / merge_base_mismatch,
# 403 mfa_required (a forced merge on an MFA-protected organization), 422
# merge_rejected, 429 rate_limited. The server's detail explains which.
return {"error": "promote (merge) failed", "status": status, "detail": body}
applied = bool(body.get("applied"))
clean = bool(body.get("clean"))
conflicts = body.get("conflicts") or []
changes = body.get("changes") or []
if applied:
branch_status = "merged"
elif conflicts:
branch_status = "blocked_by_conflicts"
elif dry_run:
branch_status = "dry_run_clean"
else:
branch_status = "no_changes"
result = {
"status": branch_status,
"base_environment_id": base,
"environment_id": head,
"applied": applied,
"dry_run": dry_run,
"clean": clean,
"changes": changes,
"conflicts": conflicts,
"statements": body.get("statements") or [],
"fork_base_trusted": body.get("fork_base_trusted"),
"fork_base_origin": body.get("fork_base_origin"),
}
if body.get("warnings"):
result["warnings"] = body["warnings"]
return result
def _branch_discard(
args: Dict[str, Any], org: str, project: str, *, api_key: str, api_base: str
) -> Dict[str, Any]:
"""``discard``: deletes ``environment_id`` via ``DELETE
.../environments/{env}`` instead of waiting for its TTL. Deletion is
permanent and unconditional — there is no confirmation step, matching
the real endpoint's own contract ("Deletion is permanent.", per
`docs.scribase.com/docs/api/environments`).
"""
branch_id = str(args.get("environment_id") or "").strip()
if not branch_id:
return {"error": "environment_id is required (the branch to discard)."}
status, body = _request(
"DELETE",
f"{api_base}/v1/organizations/{org}/projects/{project}/environments/{branch_id}",
api_key,
extra_headers={"Idempotency-Key": _idempotency_key()},
)
if status == 0:
return body
if status != 202:
return {"error": "failed to discard branch", "status": status, "detail": body}
operation_id = body.get("id") if isinstance(body, dict) else None
if not operation_id:
return {"error": "branch discard accepted but no operation id was returned", "detail": body}
final_state, last_op_body = _poll_operation(operation_id, org, api_key=api_key, api_base=api_base)
if final_state == "succeeded":
return {"status": "discarded", "organization_id": org, "project_id": project, "environment_id": branch_id}
still_going = final_state not in ("failed", "cancelled")
return {
"status": "discarding" if still_going else "failed",
"organization_id": org,
"project_id": project,
"environment_id": branch_id,
"operation_id": operation_id,
"message": (
f"Branch is still being discarded (state={final_state!r}). It will finish shortly."
if still_going else
f"Branch discard failed (state={final_state!r})."
),
"operation": last_op_body,
}
feat: add scribase_memory_task — real persistent agent memory (store/recall/search/forget) Hermes agents had no persistent memory via this plugin: scribase_query is read-only (enforced by the control plane itself) and scribase_lint_migration hardcodes apply:false, so none of the first four tools could ever write application data, only inspect or lint it. scribase_memory_task closes that gap using a real, previously-unused endpoint read from the compiled control plane: POST .../database/execute (the write-mode SQL route the Scribase Studio SQL editor uses in "write mode", crates/control-api/src/adapters/sql_execute.rs), for store/forget; recall/search reuse the existing read-only database/query route. Creates its own table (hermes_agent_memories, RLS enabled) on first use, scoped by an optional namespace so multiple agents/users can share a project without reading each other's memories. Every value interpolated into SQL (content, metadata, scope, memory_id) is escaped via the standard Postgres string-literal escape (_sql_literal) — the model never passes raw SQL through this tool. Verified: a throwaway monkeypatched test exercises all 4 actions + error paths + an injection-style content string (confirms balanced escaping, no executable SQL smuggled through). Then ran the real, unmodified tools.py directly against a real, no-login temporary Scribase project on production api.scribase.com: store (x2) -> recall -> search -> scope-isolation check (a second, never-used scope correctly returns zero rows) -> forget -> recall again (confirms exactly the forgotten memory is gone, the other survives). Full transcript in HACKATHON-SUBMISSION-2026-11-05.md. Re-ran validate_plugin_dir() (the function `hermes plugins validate` calls) against a fresh NousResearch/hermes-agent clone: 15/15 checks pass, 0 warnings, isolation host_ready: true, declared tools matches registrations (5 tools, including scribase_memory_task). plugin.yaml bumped to 1.3.0, provides_tools now lists all 5.
2026-11-06 18:16:17 +03:00
# --- memory_task -----------------------------------------------------------
#
# Backs a Hermes agent's persistent memory with one dedicated table per
# project, `public.hermes_agent_memories`, created on first use and reused
# after that (idempotent — every DDL statement below is written so a second
# call never errors). `scope` lets more than one agent/user share a project
# without reading each other's memories (e.g. one scope per end user or per
# conversation); it defaults to `"default"` for a single-agent setup.
_MEMORY_TABLE = "public.hermes_agent_memories"
_MEMORY_SCOPE_MAX_LEN = 128
_MEMORY_CONTENT_MAX_LEN = 8192
_VALID_MEMORY_ACTIONS = ("store", "recall", "search", "forget")
# A Postgres UUID's canonical textual form — validated before any `forget`
# call builds a `WHERE id = '<memory_id>'` clause from it.
_UUID_RE = re.compile(
r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$"
)
def _sql_literal(value: str) -> str:
fix: replace quote-doubling SQL literal escape with hex-decode in memory_task Independently re-verified the SQL-injection defense scribase_memory_task's _sql_literal() claimed (store/recall/search/forget). Ran a real adversarial battery (stacked-query attempts, comment sequences, UNION probes, NUL bytes, a trailing backslash, scope/memory_id injection, ILIKE breakout attempts, a 24KB payload, a Unicode lookalike quote) against a real, live, no-login temporary Scribase project on production api.scribase.com (provisioned via a disposable throwaway account to work around the daily per-address cap) — every adversarial payload landed as inert literal data, no injection, no cross-scope leak, table/RLS/row-count verified intact via an independent read-only query after each pass. The injection defense itself was sound (the quote-doubling escape is also backed by the control plane forcing standard_conforming_strings=on for every session, confirmed in psql.rs), but the same live testing found a real, reproducible, non-adversarial bug: the control plane's own backslash guard (client_command_forbidden) rejects *any* literal containing a backslash character, which the old quote-doubling helper produced whenever metadata contained a non-ASCII character (json.dumps's default \uXXXX escaping) or an embedded double quote, or content itself had a literal backslash (a Windows path, a regex) — i.e. ordinary non-English or technical content, not just hostile input, always 400'd. Replaced the doubled-single-quote literal with a hex-encoded convert_from(decode('<hex>','hex'),'UTF8') expression for every interpolated value (content/metadata/scope/memory_id). A hex literal can only ever contain 0-9a-f, so it can never contain a quote, backslash, or comment sequence — this removes the backslash-guard false positive and the escaping class of risk entirely, without depending on standard_conforming_strings either way. Re-ran the full adversarial + regression battery against the fixed code on the same live project: every real-world case that previously 400'd now stores and round-trips correctly (confirmed café/Türkçe/日本語 byte-for-byte), and every adversarial payload remains safely inert.
2026-11-06 23:05:52 +03:00
"""Encodes ``value`` as a Postgres text expression via a hex-encoded
``decode(..., 'hex')``, instead of a classically single-quoted-and-
escaped string literal.
This plugin used to double every embedded single quote (the textbook
Postgres string-literal escape) and rely on the server always running
these statements with `standard_conforming_strings = on` so a trailing
backslash could never swallow the closing quote. That assumption does
hold (`control-api/src/adapters/psql.rs` forces it on for every session
before any statement runs) — but live re-verification against a real
temporary Scribase project (2026-11-06) found a *different*, real,
reproducible bug the quote-doubling approach was not defending against
at all: the control plane's own client-syntax guard
(`client_command_forbidden`, "it has a backslash outside an E'...'
string or a dollar-quoted body") rejects the entire statement outright
whenever the built literal contains a literal backslash character —
which it will for *any* ordinary, non-adversarial content: `content`
with a literal backslash (a Windows path, a regex), or any `metadata`
value with a non-ASCII character (Python's json.dumps default
ensure_ascii escapes those as a backslash-u-XXXX sequence) or an
embedded double quote (escaped as a backslash-double-quote pair).
Confirmed live: a `store` call with a metadata value containing
non-English text (accented Latin, Turkish, Japanese) 400'd every
time under the old quote-doubling helper — a real reliability gap
for ordinary non-English content, not just an adversarial one.
A hex-encoded `decode()` call sidesteps the whole class of problem: the
literal it builds can only ever contain the ASCII digits/letters
`0-9a-f` inside a plain single-quoted string with no embedded quote,
backslash, or comment sequence possible, so it can never trip the
backslash guard and never depends on `standard_conforming_strings`
either way. NUL bytes are stripped first (Postgres `text` can never
store one).
feat: add scribase_memory_task — real persistent agent memory (store/recall/search/forget) Hermes agents had no persistent memory via this plugin: scribase_query is read-only (enforced by the control plane itself) and scribase_lint_migration hardcodes apply:false, so none of the first four tools could ever write application data, only inspect or lint it. scribase_memory_task closes that gap using a real, previously-unused endpoint read from the compiled control plane: POST .../database/execute (the write-mode SQL route the Scribase Studio SQL editor uses in "write mode", crates/control-api/src/adapters/sql_execute.rs), for store/forget; recall/search reuse the existing read-only database/query route. Creates its own table (hermes_agent_memories, RLS enabled) on first use, scoped by an optional namespace so multiple agents/users can share a project without reading each other's memories. Every value interpolated into SQL (content, metadata, scope, memory_id) is escaped via the standard Postgres string-literal escape (_sql_literal) — the model never passes raw SQL through this tool. Verified: a throwaway monkeypatched test exercises all 4 actions + error paths + an injection-style content string (confirms balanced escaping, no executable SQL smuggled through). Then ran the real, unmodified tools.py directly against a real, no-login temporary Scribase project on production api.scribase.com: store (x2) -> recall -> search -> scope-isolation check (a second, never-used scope correctly returns zero rows) -> forget -> recall again (confirms exactly the forgotten memory is gone, the other survives). Full transcript in HACKATHON-SUBMISSION-2026-11-05.md. Re-ran validate_plugin_dir() (the function `hermes plugins validate` calls) against a fresh NousResearch/hermes-agent clone: 15/15 checks pass, 0 warnings, isolation host_ready: true, declared tools matches registrations (5 tools, including scribase_memory_task). plugin.yaml bumped to 1.3.0, provides_tools now lists all 5.
2026-11-06 18:16:17 +03:00
"""
cleaned = value.replace("\x00", "")
fix: replace quote-doubling SQL literal escape with hex-decode in memory_task Independently re-verified the SQL-injection defense scribase_memory_task's _sql_literal() claimed (store/recall/search/forget). Ran a real adversarial battery (stacked-query attempts, comment sequences, UNION probes, NUL bytes, a trailing backslash, scope/memory_id injection, ILIKE breakout attempts, a 24KB payload, a Unicode lookalike quote) against a real, live, no-login temporary Scribase project on production api.scribase.com (provisioned via a disposable throwaway account to work around the daily per-address cap) — every adversarial payload landed as inert literal data, no injection, no cross-scope leak, table/RLS/row-count verified intact via an independent read-only query after each pass. The injection defense itself was sound (the quote-doubling escape is also backed by the control plane forcing standard_conforming_strings=on for every session, confirmed in psql.rs), but the same live testing found a real, reproducible, non-adversarial bug: the control plane's own backslash guard (client_command_forbidden) rejects *any* literal containing a backslash character, which the old quote-doubling helper produced whenever metadata contained a non-ASCII character (json.dumps's default \uXXXX escaping) or an embedded double quote, or content itself had a literal backslash (a Windows path, a regex) — i.e. ordinary non-English or technical content, not just hostile input, always 400'd. Replaced the doubled-single-quote literal with a hex-encoded convert_from(decode('<hex>','hex'),'UTF8') expression for every interpolated value (content/metadata/scope/memory_id). A hex literal can only ever contain 0-9a-f, so it can never contain a quote, backslash, or comment sequence — this removes the backslash-guard false positive and the escaping class of risk entirely, without depending on standard_conforming_strings either way. Re-ran the full adversarial + regression battery against the fixed code on the same live project: every real-world case that previously 400'd now stores and round-trips correctly (confirmed café/Türkçe/日本語 byte-for-byte), and every adversarial payload remains safely inert.
2026-11-06 23:05:52 +03:00
hex_value = cleaned.encode("utf-8").hex()
return f"convert_from(decode('{hex_value}', 'hex'), 'UTF8')"
feat: add scribase_memory_task — real persistent agent memory (store/recall/search/forget) Hermes agents had no persistent memory via this plugin: scribase_query is read-only (enforced by the control plane itself) and scribase_lint_migration hardcodes apply:false, so none of the first four tools could ever write application data, only inspect or lint it. scribase_memory_task closes that gap using a real, previously-unused endpoint read from the compiled control plane: POST .../database/execute (the write-mode SQL route the Scribase Studio SQL editor uses in "write mode", crates/control-api/src/adapters/sql_execute.rs), for store/forget; recall/search reuse the existing read-only database/query route. Creates its own table (hermes_agent_memories, RLS enabled) on first use, scoped by an optional namespace so multiple agents/users can share a project without reading each other's memories. Every value interpolated into SQL (content, metadata, scope, memory_id) is escaped via the standard Postgres string-literal escape (_sql_literal) — the model never passes raw SQL through this tool. Verified: a throwaway monkeypatched test exercises all 4 actions + error paths + an injection-style content string (confirms balanced escaping, no executable SQL smuggled through). Then ran the real, unmodified tools.py directly against a real, no-login temporary Scribase project on production api.scribase.com: store (x2) -> recall -> search -> scope-isolation check (a second, never-used scope correctly returns zero rows) -> forget -> recall again (confirms exactly the forgotten memory is gone, the other survives). Full transcript in HACKATHON-SUBMISSION-2026-11-05.md. Re-ran validate_plugin_dir() (the function `hermes plugins validate` calls) against a fresh NousResearch/hermes-agent clone: 15/15 checks pass, 0 warnings, isolation host_ready: true, declared tools matches registrations (5 tools, including scribase_memory_task). plugin.yaml bumped to 1.3.0, provides_tools now lists all 5.
2026-11-06 18:16:17 +03:00
def _ensure_memory_table(organization_id: str, project_id: str, environment_id: str, *, api_key: str, api_base: str):
"""Idempotent DDL: creates the memory table, its scope index, and its
permissive RLS policy if (and only if) they do not already exist. Safe
to call on every `memory_task` invocation — every statement here is a
no-op on a project that already has them.
"""
ddl = (
f"create table if not exists {_MEMORY_TABLE} ("
"id uuid primary key default gen_random_uuid(), "
"scope text not null default 'default', "
"content text not null, "
"metadata jsonb not null default '{}'::jsonb, "
"created_at timestamptz not null default now()"
"); "
f"create index if not exists hermes_agent_memories_scope_idx "
f"on {_MEMORY_TABLE} (scope, created_at desc); "
f"alter table {_MEMORY_TABLE} enable row level security; "
"do $hermes_memory_policy$ "
"begin "
"if not exists ("
" select 1 from pg_policies"
" where schemaname = 'public' and tablename = 'hermes_agent_memories'"
" and policyname = 'hermes_agent_memories_service_all'"
") then "
" create policy hermes_agent_memories_service_all on "
f" {_MEMORY_TABLE} for all using (true) with check (true); "
"end if; "
"end "
"$hermes_memory_policy$;"
)
status, body = _request(
"POST",
f"{api_base}/v1/organizations/{organization_id}/projects/{project_id}"
f"/environments/{environment_id}/database/execute",
api_key,
body={"query": ddl},
)
if status == 0:
return body
if status != 200:
return {"error": "failed to prepare the memory table", "status": status, "detail": body}
return None
def memory_task(args: Dict[str, Any], *, api_key: str, api_base: str) -> Dict[str, Any]:
"""Dispatches one of four real, persistent-memory actions, all backed by
a real table (`hermes_agent_memories`) in the project's own Postgres
database — not anything held in the agent's own context window:
- ``"store"`` — inserts one memory (`content`, optional `metadata`
object, optional `scope`) via the write-mode `database/execute` route
and returns its id.
- ``"recall"`` — lists the most recent memories in a scope (optional
`limit`, default 20, max 200) via the read-only `database/query`
route, newest first.
- ``"search"`` — same as `recall`, filtered to memories whose content
contains `query` (case-insensitive substring match, `ILIKE`).
- ``"forget"`` — deletes one memory by `memory_id` (a UUID returned by
a prior `store`) via `database/execute`. Deleting by id only — there
is no "forget everything in this scope" action, so a single bad call
can never wipe a whole scope's memory.
"""
action = str(args.get("action") or "").strip().lower()
org = str(args.get("organization_id") or "").strip()
project = str(args.get("project_id") or "").strip()
environment = str(args.get("environment_id") or "").strip() or _DEFAULT_ENVIRONMENT_ID
if action not in _VALID_MEMORY_ACTIONS:
return {"error": f"action must be one of: {', '.join(_VALID_MEMORY_ACTIONS)}."}
if not org:
return {"error": "organization_id is required."}
if not project:
return {"error": "project_id is required."}
scope = str(args.get("scope") or "default").strip() or "default"
if len(scope) > _MEMORY_SCOPE_MAX_LEN:
return {"error": f"scope must be at most {_MEMORY_SCOPE_MAX_LEN} characters."}
if action == "store":
content = str(args.get("content") or "").strip()
if not content:
return {"error": "content is required for action='store'."}
if len(content) > _MEMORY_CONTENT_MAX_LEN:
return {"error": f"content must be at most {_MEMORY_CONTENT_MAX_LEN} characters."}
metadata = args.get("metadata")
if metadata is not None and not isinstance(metadata, dict):
return {"error": "metadata must be a JSON object if provided."}
prep_error = _ensure_memory_table(org, project, environment, api_key=api_key, api_base=api_base)
if prep_error is not None:
return prep_error
metadata_json = json.dumps(metadata or {})
query = (
f"insert into {_MEMORY_TABLE} (scope, content, metadata) values ("
f"{_sql_literal(scope)}, {_sql_literal(content)}, "
f"{_sql_literal(metadata_json)}::jsonb"
") returning id, created_at;"
)
status, body = _request(
"POST",
f"{api_base}/v1/organizations/{org}/projects/{project}/environments/{environment}/database/execute",
api_key,
body={"query": query},
)
if status == 0:
return body
if status != 200 or not isinstance(body, dict):
return {"error": "failed to store memory", "status": status, "detail": body}
statements = body.get("statements") or []
rows = statements[0].get("rows") if statements and isinstance(statements[0], dict) else None
row = rows[0] if rows else {}
return {
"status": "stored",
"scope": scope,
"memory_id": row.get("id"),
"created_at": row.get("created_at"),
}
if action == "forget":
memory_id = str(args.get("memory_id") or "").strip()
if not memory_id:
return {"error": "memory_id is required for action='forget' (the id a prior store returned)."}
if not _UUID_RE.match(memory_id):
return {"error": "memory_id must be a UUID, as returned by a prior action='store' call."}
query = (
f"delete from {_MEMORY_TABLE} where id = {_sql_literal(memory_id)} "
f"and scope = {_sql_literal(scope)} returning id;"
)
status, body = _request(
"POST",
f"{api_base}/v1/organizations/{org}/projects/{project}/environments/{environment}/database/execute",
api_key,
body={"query": query},
)
if status == 0:
return body
if status != 200 or not isinstance(body, dict):
return {"error": "failed to forget memory", "status": status, "detail": body}
statements = body.get("statements") or []
affected = statements[0].get("affected_rows") if statements and isinstance(statements[0], dict) else None
rows = statements[0].get("rows") if statements and isinstance(statements[0], dict) else None
found = bool(rows) or (isinstance(affected, int) and affected > 0)
return {
"status": "forgotten" if found else "not_found",
"scope": scope,
"memory_id": memory_id,
}
# action in ("recall", "search")
try:
limit = int(args.get("limit") or 20)
except (TypeError, ValueError):
return {"error": "limit must be an integer between 1 and 200."}
limit = max(1, min(limit, 200))
where = f"scope = {_sql_literal(scope)}"
if action == "search":
search_text = str(args.get("query") or "").strip()
if not search_text:
return {"error": "query is required for action='search' (substring to match, case-insensitive)."}
where += f" and content ilike {_sql_literal('%' + search_text + '%')}"
query = (
f"select id, content, metadata, created_at from {_MEMORY_TABLE} "
f"where {where} order by created_at desc limit {limit};"
)
status, body = _request(
"POST",
f"{api_base}/v1/organizations/{org}/projects/{project}/environments/{environment}/database/query",
api_key,
body={"query": query, "limit": limit},
)
if status == 0:
return body
if status == 400 and isinstance(body, dict) and body.get("code") in ("relation_does_not_exist", "undefined_table"):
# No memory has ever been stored in this project yet — the table
# from `_ensure_memory_table` doesn't exist. Not an error: there is
# nothing to recall.
return {"status": "ok", "scope": scope, "memories": [], "note": "no memories stored yet"}
if status != 200 or not isinstance(body, dict):
return {"error": f"failed to {action} memories", "status": status, "detail": body}
return {
"status": "ok",
"scope": scope,
"memories": body.get("rows") or [],
"row_count": body.get("row_count"),
"truncated": body.get("truncated"),
}