Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2fe8d8beda | |||
| 5c1758578f |
131
README.md
131
README.md
@ -1,106 +1,99 @@
|
||||
# neon-cold-start-probe
|
||||
|
||||
Neon (and most serverless/autoscaling Postgres, including [Scribase](https://scribase.com)'s
|
||||
own Free/Hobby tier) suspends a compute after a few idle minutes to save cost. The next
|
||||
connection has to wake it up before it can answer — slower than every connection after it. Nobody
|
||||
publishes a number for *your* database's wake-up time, and the usual advice is "it felt slow for
|
||||
a second." This measures it for real: a brand-new connection each time (never pooled — pooling is
|
||||
exactly what would hide the thing you're trying to see), timed in three parts (connect, query,
|
||||
total).
|
||||
Measure how long your Neon (or any scale-to-zero Postgres) database really takes to answer the
|
||||
first connection after it has been idle. One command, no account, no config.
|
||||
|
||||
## 30-second quickstart
|
||||
## 10-second quickstart
|
||||
|
||||
```
|
||||
npx --yes git+https://git.ereboslab.com/caelum0x/neon-cold-start-probe.git \
|
||||
"postgres://user:pass@ep-xxxx.region.aws.neon.tech/dbname?sslmode=require"
|
||||
npx --yes git+https://rithub.dev/caelum0x/neon-cold-start-probe.git "$DATABASE_URL" --repeat 5
|
||||
```
|
||||
|
||||
or clone it and run locally:
|
||||
That is the whole setup. It needs Node 18+ and a Postgres connection string.
|
||||
|
||||
With Bun:
|
||||
|
||||
```
|
||||
git clone https://git.ereboslab.com/caelum0x/neon-cold-start-probe.git
|
||||
cd neon-cold-start-probe && npm install
|
||||
node bin/neon-cold-start-probe.mjs "$DATABASE_URL"
|
||||
bunx -p git+https://rithub.dev/caelum0x/neon-cold-start-probe.git neon-cold-start-probe "$DATABASE_URL" --repeat 5
|
||||
```
|
||||
|
||||
Output:
|
||||
## What you get
|
||||
|
||||
Real output from this exact build (Postgres 16 over TLS, `sslmode=require`, 2026-11-17):
|
||||
|
||||
```
|
||||
probe 1/1: connect=931ms query=7ms total=938ms [COLD]
|
||||
probe 1/5: connect=202ms query=10ms total=212ms [warm]
|
||||
probe 2/5: connect=55ms query=10ms total=64ms [warm]
|
||||
probe 3/5: connect=39ms query=1ms total=41ms [warm]
|
||||
probe 4/5: connect=299ms query=15ms total=314ms [warm]
|
||||
probe 5/5: connect=36ms query=3ms total=39ms [warm]
|
||||
|
||||
summary: min=39ms p50=64ms p95=314ms max=314ms (0/5 probes >= 500ms, 0 failed)
|
||||
|
||||
Neon cold starts and compute limits explained, with the fix: https://scribase.com/neon-compute-limit-cold-start?utm_source=cli&utm_medium=cli-output&utm_campaign=neon-cold-start-probe
|
||||
```
|
||||
|
||||
## Seeing an actual cold start, not just a number
|
||||
Every probe opens a brand-new connection (never pooled, because a pool hides exactly the thing
|
||||
you are trying to see) and times three parts: connect, query, total. A probe at or above the
|
||||
threshold (500ms by default) is labelled `COLD`.
|
||||
|
||||
A single probe only tells you one connection's latency. To see the *cold vs warm* gap for real:
|
||||
That sample is a database that never suspends, on a busy laptop, so every probe is warm and the
|
||||
spread is machine noise. It is here to show the format, not a Neon number.
|
||||
|
||||
1. Let the project sit idle past its suspend timeout (Neon's default/free setting is 5 minutes;
|
||||
check your project's compute settings for the real value).
|
||||
2. Run a `--repeat` probe immediately after — the **first** result is the one that had to wake
|
||||
the compute; the rest, back-to-back, will be warm because the first one already woke it:
|
||||
## How to catch a real cold start
|
||||
|
||||
```
|
||||
node bin/neon-cold-start-probe.mjs "$DATABASE_URL" --repeat 5 --every-seconds 1
|
||||
probe 1/5: connect=931ms query=7ms total=938ms [COLD]
|
||||
probe 2/5: connect=14ms query=2ms total=16ms [warm]
|
||||
probe 3/5: connect=13ms query=2ms total=15ms [warm]
|
||||
probe 4/5: connect=12ms query=2ms total=14ms [warm]
|
||||
probe 5/5: connect=15ms query=2ms total=17ms [warm]
|
||||
1. Leave the database idle past its suspend timeout. On Neon Free that is
|
||||
[5 minutes and cannot be changed](https://neon.com/docs/introduction/scale-to-zero).
|
||||
2. Run the quickstart command once. Probe 1 is the one that had to wake the compute. Probes 2 to
|
||||
5 run against a compute that is already awake.
|
||||
|
||||
summary: min=14ms p50=16ms p95=938ms max=938ms (1/5 probes >= 500ms, 0 failed)
|
||||
```
|
||||
|
||||
Running `--repeat` back-to-back from a warm start (the default, no prior idle wait) will almost
|
||||
always show every probe as warm — that's expected, not a bug; it's the same compute, still awake.
|
||||
The gap between probe 1 and the rest is your cold start, measured from your machine and your
|
||||
region. Run it back to back with no idle wait and every probe is warm. That is expected.
|
||||
|
||||
## Options
|
||||
|
||||
| Flag | Default | What it does |
|
||||
|---|---|---|
|
||||
| `--url <conn-string>` | — | Also accepts a positional arg, or `DATABASE_URL` |
|
||||
| `--url <conn-string>` | none | Also accepts a positional arg, or `DATABASE_URL` |
|
||||
| `--repeat <n>` | `1` | Number of fresh-connection probes |
|
||||
| `--every-seconds <s>` | `0` | Wait between probes |
|
||||
| `--timeout <s>` | `15` | Per-probe connect timeout |
|
||||
| `--cold-threshold-ms <ms>` | `500` | Total ms a probe is labelled `COLD` above |
|
||||
| `--json` | off | Machine-readable output, including per-probe and summary (min/p50/p95/max) |
|
||||
| `--cold-threshold-ms <ms>` | `500` | Total ms at or above which a probe is labelled `COLD` |
|
||||
| `--json` | off | Per-probe results plus summary (min/p50/p95/max) on stdout |
|
||||
| `--version` | | Print the version |
|
||||
|
||||
## Where the 500ms default comes from
|
||||
Run it from a clone instead:
|
||||
|
||||
It is **not** a number Neon publishes — Neon documents that autoscaling computes suspend and
|
||||
resume on demand but does not commit to a wake-up SLA. It's the midpoint of a real measurement
|
||||
done building this tool's sibling project, a Postgres service with the same scale-to-zero
|
||||
behavior: sleep the compute via its API, then measure the first request vs a warm one, 3 runs —
|
||||
cold 0.93–1.47s, warm 0.09–0.24s. Pass your own `--cold-threshold-ms` once you have a few real
|
||||
samples from your own project; it varies by region, plan and how large the compute is.
|
||||
```
|
||||
git clone https://rithub.dev/caelum0x/neon-cold-start-probe.git
|
||||
cd neon-cold-start-probe && npm install
|
||||
node bin/neon-cold-start-probe.mjs "$DATABASE_URL" --repeat 5
|
||||
```
|
||||
|
||||
## Honest limits
|
||||
|
||||
- **This build has no real Neon test credentials** (no Neon account token in this environment,
|
||||
and the Neon MCP integration needs an interactive browser OAuth sign-in nobody here could
|
||||
complete). Everything was tested for real against a **local Postgres 16 container** — proves
|
||||
the connect/query/timing/retry/classification logic is correct, but a local container never
|
||||
suspends, so it cannot reproduce an actual Neon cold start. Full transcript, including the
|
||||
failure paths (wrong port, wrong password, missing arg):
|
||||
`test-fixtures/local-postgres-verification.md`.
|
||||
- **A single slow reading is not proof of a cold start.** Network jitter, a loaded laptop, or a
|
||||
busy database can also push a connection over the threshold — the local-Postgres transcript
|
||||
above shows this happening for real (an 819ms and a 1664ms reading against a container that
|
||||
was never suspended, just competing with other load on the machine at the time). Take more than
|
||||
one sample, and compare the *first* probe after real idle time against the rest, not any one
|
||||
number in isolation.
|
||||
- **This is a generic Postgres latency probe**, not Neon-specific code — it works against any
|
||||
Postgres connection string (including Scribase's). The Neon framing is because Neon's free-tier
|
||||
suspend behavior is the most-searched version of this pain; if you're on a different
|
||||
autoscaling Postgres, point it there instead.
|
||||
- No `--watch`/daemon mode yet — each run is a fixed number of probes, then it exits.
|
||||
- **Not yet run against a live Neon project.** Everything here was tested for real against a
|
||||
local Postgres 16 (plain and TLS), which proves the connect, timing, retry and labelling logic
|
||||
but can never suspend. The full transcript, failure paths included, is in
|
||||
[test-fixtures/local-postgres-verification.md](https://rithub.dev/caelum0x/neon-cold-start-probe/src/branch/main/test-fixtures/local-postgres-verification.md).
|
||||
- **One slow reading is not proof of a cold start.** Network jitter or a loaded machine can push
|
||||
a warm connection over the threshold. Compare the first probe after real idle time against the
|
||||
rest, and take more than one sample.
|
||||
- **The 500ms default is not a Neon number.** Neon documents that a suspended compute wakes in
|
||||
[a few hundred milliseconds](https://neon.com/docs/connect/connection-latency) and publishes no
|
||||
SLA. 500ms is a starting point; set `--cold-threshold-ms` once you have samples of your own.
|
||||
- **It is a generic Postgres probe.** It works against any connection string, Scribase's included.
|
||||
- One-time process costs (first DNS lookup, loading Node's TLS stack) are paid before probe 1, so
|
||||
they are not counted as database wake-up time.
|
||||
|
||||
## Need more?
|
||||
## If the cold start is the problem
|
||||
|
||||
If cold starts are a dealbreaker rather than a curiosity: [Scribase](https://scribase.com/neon-compute-limit-cold-start?utm_source=rithub&utm_medium=readme&utm_campaign=neon-cold-start-probe)
|
||||
is Postgres + auth + storage + email on one account. Its own Free/Hobby tier **also** scales to
|
||||
zero and pays the same kind of wake-up cost (measured: 0.93–1.47s cold vs 0.09–0.24s warm — the
|
||||
numbers this tool's default threshold is based on) — only the $25/mo Pro tier stays always-warm.
|
||||
No "no cold starts, ever" claim here for the free tier, because it wouldn't be true; the
|
||||
comparison page lays out both tiers' real numbers side by side.
|
||||
[Neon cold starts and compute limits explained, with the fix](https://scribase.com/neon-compute-limit-cold-start?utm_source=rithub&utm_medium=readme&utm_campaign=neon-cold-start-probe):
|
||||
why the first query is slow, how the Free compute allowance is counted, and what to change.
|
||||
|
||||
This tool is built by [Scribase](https://scribase.com/?utm_source=rithub&utm_medium=readme&utm_campaign=neon-cold-start-probe):
|
||||
Postgres, sign-in, storage and email on one account instead of four. Scribase also scales idle
|
||||
environments to zero, so point this probe at it too and compare the numbers yourself.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@ -1,5 +1,4 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
// neon-cold-start-probe
|
||||
//
|
||||
@ -20,13 +19,35 @@
|
||||
// you can see the first (often cold) probe against the rest (warm).
|
||||
|
||||
import pg from 'pg';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { lookup } from 'node:dns/promises';
|
||||
|
||||
const CTA_URL =
|
||||
'https://scribase.com/neon-compute-limit-cold-start?utm_source=cli&utm_medium=cli-output&utm_campaign=neon-cold-start-probe';
|
||||
const CTA_LINE = `Neon cold starts and compute limits explained, with the fix: ${CTA_URL}`;
|
||||
const VALUE_FLAGS = new Set(['--url', '--repeat', '--every-seconds', '--timeout', '--cold-threshold-ms']);
|
||||
|
||||
function version() {
|
||||
try {
|
||||
return JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8')).version;
|
||||
} catch {
|
||||
return 'unknown';
|
||||
}
|
||||
}
|
||||
|
||||
function parseArgs(argv) {
|
||||
const out = { repeat: 1, every: 0, timeout: 15, threshold: 500 };
|
||||
const out = { repeat: 1, every: 0, timeout: 15, threshold: 500, errors: [] };
|
||||
const positional = [];
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
const a = argv[i];
|
||||
const next = () => argv[++i];
|
||||
const next = () => {
|
||||
const value = argv[++i];
|
||||
if (value === undefined || (VALUE_FLAGS.has(value))) {
|
||||
out.errors.push(`${a} needs a value`);
|
||||
return undefined;
|
||||
}
|
||||
return value;
|
||||
};
|
||||
switch (a) {
|
||||
case '--url': out.url = next(); break;
|
||||
case '--repeat': out.repeat = parseInt(next(), 10); break;
|
||||
@ -35,8 +56,10 @@ function parseArgs(argv) {
|
||||
case '--cold-threshold-ms': out.threshold = parseInt(next(), 10); break;
|
||||
case '--json': out.json = true; break;
|
||||
case '--help': case '-h': out.help = true; break;
|
||||
case '--version': case '-v': out.version = true; break;
|
||||
default:
|
||||
if (!a.startsWith('-')) positional.push(a);
|
||||
if (a.startsWith('-')) out.errors.push(`unknown option ${a}`);
|
||||
else positional.push(a);
|
||||
break;
|
||||
}
|
||||
}
|
||||
@ -58,6 +81,7 @@ Options:
|
||||
--timeout <s> Per-probe connect timeout in seconds (default 15)
|
||||
--cold-threshold-ms <ms> Total ms above which a probe is labelled "cold" (default 500)
|
||||
--json Print machine-readable JSON instead of a table
|
||||
--version Print the version
|
||||
--help This help
|
||||
|
||||
What the 500ms default is based on: not a Neon-published number (Neon
|
||||
@ -73,16 +97,51 @@ Neon's default/free settings) BEFORE you run this. Back-to-back probes
|
||||
(the default) will almost always all look warm, because the first probe
|
||||
itself keeps the compute awake for the rest.
|
||||
|
||||
Comparison, if cold starts are a dealbreaker: https://scribase.com/neon-compute-limit-cold-start?utm_source=cli&utm_medium=cli&utm_campaign=neon-cold-start-probe
|
||||
(honest numbers both ways: Scribase's own Free/Hobby tier also scales to
|
||||
zero and pays the same kind of cold-start cost; only its Pro tier stays
|
||||
always-warm.)`);
|
||||
Neon cold starts and compute limits explained, with the fix:
|
||||
https://scribase.com/neon-compute-limit-cold-start?utm_source=cli&utm_medium=cli-help&utm_campaign=neon-cold-start-probe`);
|
||||
}
|
||||
|
||||
function sleep(ms) {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
// pg >= 8.16 prints an 8-line SECURITY WARNING to stderr for sslmode=require
|
||||
// (the mode every Neon connection string ships with), because it already
|
||||
// treats require/prefer/verify-ca as verify-full and plans to change that in
|
||||
// pg 9. Asking for verify-full explicitly is the exact same behavior today,
|
||||
// stays strict after pg 9, and keeps the probe output clean.
|
||||
function normalizeSslMode(connectionString) {
|
||||
return connectionString.replace(/([?&]sslmode=)(require|prefer|verify-ca)(?=&|$)/i, '$1verify-full');
|
||||
}
|
||||
|
||||
function validate(args, url) {
|
||||
if (args.errors.length) return [...args.errors];
|
||||
const errors = [];
|
||||
if (!url) {
|
||||
errors.push('a connection string is required (positional arg, --url, or DATABASE_URL)');
|
||||
} else if (!/^postgres(ql)?:\/\//i.test(url)) {
|
||||
errors.push('the connection string must start with postgres:// or postgresql://');
|
||||
}
|
||||
if (!Number.isInteger(args.repeat) || args.repeat < 1) errors.push('--repeat must be a whole number >= 1');
|
||||
if (!Number.isFinite(args.every) || args.every < 0) errors.push('--every-seconds must be a number >= 0');
|
||||
if (!Number.isInteger(args.timeout) || args.timeout < 1) errors.push('--timeout must be a whole number of seconds >= 1');
|
||||
if (!Number.isInteger(args.threshold) || args.threshold < 1) errors.push('--cold-threshold-ms must be a whole number >= 1');
|
||||
return errors;
|
||||
}
|
||||
|
||||
// One-time process costs (first DNS lookup, loading Node's TLS stack and CAs) are
|
||||
// not the database waking up, and a long-running app never pays them per
|
||||
// connection. Pay them once before probe 1 so they are not billed to it.
|
||||
async function warmProcess(connectionString) {
|
||||
try {
|
||||
const { hostname } = new URL(connectionString);
|
||||
const [tls] = await Promise.all([import('node:tls'), hostname ? lookup(hostname) : null]);
|
||||
tls.createSecureContext(); // loads the root CA store, a one-time cost
|
||||
} catch {
|
||||
// A bad host surfaces as a normal failed probe with its real error.
|
||||
}
|
||||
}
|
||||
|
||||
async function probe(connectionString, timeoutSec) {
|
||||
const client = new pg.Client({
|
||||
connectionString,
|
||||
@ -121,16 +180,20 @@ async function main() {
|
||||
help();
|
||||
return 0;
|
||||
}
|
||||
const url = args.url || process.env.DATABASE_URL || '';
|
||||
if (!url) {
|
||||
console.error('error: a connection string is required (positional arg, --url, or DATABASE_URL). Run with --help.');
|
||||
return 1;
|
||||
}
|
||||
if (args.repeat < 1) {
|
||||
console.error('error: --repeat must be >= 1.');
|
||||
if (args.version) {
|
||||
console.log(version());
|
||||
return 0;
|
||||
}
|
||||
const rawUrl = args.url || process.env.DATABASE_URL || '';
|
||||
const errors = validate(args, rawUrl);
|
||||
if (errors.length) {
|
||||
for (const message of errors) console.error(`error: ${message}.`);
|
||||
console.error('Run with --help for usage.');
|
||||
return 1;
|
||||
}
|
||||
const url = normalizeSslMode(rawUrl);
|
||||
|
||||
await warmProcess(url);
|
||||
const results = [];
|
||||
for (let i = 0; i < args.repeat; i++) {
|
||||
if (i > 0 && args.every > 0) {
|
||||
@ -173,6 +236,11 @@ async function main() {
|
||||
);
|
||||
}
|
||||
|
||||
// One plain closing line. In --json mode it goes to stderr so stdout stays
|
||||
// valid JSON for whatever is piping it.
|
||||
if (args.json) console.error(CTA_LINE);
|
||||
else console.log(`\n${CTA_LINE}`);
|
||||
|
||||
if (failed === results.length) return 1;
|
||||
return 0;
|
||||
}
|
||||
|
||||
4
package-lock.json
generated
4
package-lock.json
generated
@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "neon-cold-start-probe",
|
||||
"version": "0.1.0",
|
||||
"version": "0.1.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "neon-cold-start-probe",
|
||||
"version": "0.1.0",
|
||||
"version": "0.1.1",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"pg": "^8.13.1"
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "neon-cold-start-probe",
|
||||
"version": "0.1.0",
|
||||
"version": "0.1.1",
|
||||
"description": "Measure the real cold-start latency of a serverless Postgres endpoint (Neon and similar) from a fresh connection, not a guess.",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
|
||||
@ -74,3 +74,34 @@ probe 1/1: connect=122ms query=24ms total=146ms [warm]
|
||||
actually suspending and this tool catching the real wake-up latency on the
|
||||
next probe. Logged honestly rather than faked — see the README's
|
||||
"Honest limits" section.
|
||||
|
||||
## 2026-11-17 (v0.1.1) re-verification
|
||||
|
||||
Throwaway Postgres 16.16 (`initdb` in `/tmp`, port 55439, self-signed TLS cert). Machine load
|
||||
average was above 70 during these runs, so the absolute numbers are noisy.
|
||||
|
||||
Bugs found by running it the way the README says, all fixed in v0.1.1:
|
||||
|
||||
- `?sslmode=require` (what every Neon connection string ships with) printed an 8-line `pg`
|
||||
SECURITY WARNING before the first probe. Now normalized to `verify-full`, which is what `pg`
|
||||
already did for `require`, silently.
|
||||
- `--repeat abc` and a bare `--repeat` exited 1 with no message at all.
|
||||
- A non-URL argument (`"not a url"`) failed as `getaddrinfo ENOTFOUND base`.
|
||||
- `--version` answered "a connection string is required".
|
||||
- Unknown flags were silently ignored.
|
||||
|
||||
TLS path, CA trusted via `NODE_EXTRA_CA_CERTS`:
|
||||
|
||||
```
|
||||
$ neon-cold-start-probe "postgres://postgres@localhost:55439/postgres?sslmode=require" --repeat 5 --every-seconds 1
|
||||
probe 1/5: connect=202ms query=10ms total=212ms [warm]
|
||||
probe 2/5: connect=55ms query=10ms total=64ms [warm]
|
||||
probe 3/5: connect=39ms query=1ms total=41ms [warm]
|
||||
probe 4/5: connect=299ms query=15ms total=314ms [warm]
|
||||
probe 5/5: connect=36ms query=3ms total=39ms [warm]
|
||||
|
||||
summary: min=39ms p50=64ms p95=314ms max=314ms (0/5 probes >= 500ms, 0 failed)
|
||||
```
|
||||
|
||||
Same server, CA not trusted: `probe 1/1: FAILED (self-signed certificate; ...)`, exit 1, no
|
||||
warning noise. `psql` confirmed the session was really TLS (`pg_stat_ssl.ssl = t`).
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user