2.4 KiB
2.4 KiB
Usage: duckdb-source
Read numeric time-series data from a local DuckDB file with read-only, SELECT-only queries and bounded, streamed result fetching.
Install
pip install -e "plugins/duckdb-source[duckdb]"
The duckdb driver is optional. Importing the package never imports duckdb;
invoke imports it lazily and raises RuntimeError if it is missing.
Constructing the source
from duckdb_source.plugin import DuckDBSource
source = DuckDBSource(max_rows=1_000_000, batch_size=10_000)
Both limits must be positive.
Request
| key | type | notes |
|---|---|---|
database |
str |
path to an existing .duckdb file |
query |
str |
one SELECT/WITH statement, no ; stacking |
params |
sequence or mapping | bound query parameters (optional) |
Response
{"records": [{col: value, ...}, ...], "row_count": int, "columns": [str, ...]}
Rows are fetched in batch_size chunks and the total is capped at max_rows.
Query safety
The query is validated before execution:
- Must begin with
SELECTorWITH. - May not contain a
;(no statement stacking). - May not contain any mutating/side-effecting keyword:
insert,update,delete,create,drop,alter,copy,attach,install,load. - The connection is opened with
read_only=True.
Anything else raises ValueError. Bind values through params instead of
string interpolation.
Example
source.invoke({
"database": "/data/market.duckdb",
"query": "SELECT time, price FROM ticks WHERE symbol = ? ORDER BY time",
"params": ["AAPL"],
})
Feeding discovery
Select a time column and numeric state columns, then build a Dataset:
records = source.invoke({"database": db, "query": "SELECT time, x, y FROM obs ORDER BY time"})["records"]
time = tuple(r["time"] for r in records)
columns = {"x": tuple(r["x"] for r in records), "y": tuple(r["y"] for r in records)}
# lawsynth.dataset.Dataset.from_columns(time, columns)
Security note
filesystem.read in plugin.toml is a declaration, not a grant. A conforming
host still confines which paths the plugin may open; see
specs/plugin-protocol/permissions.md.