Documentation

Quick start

SpeedyDb ships on npm as @speedydb/core — the wasm retrieval engine, the on-device embedder, and durable IndexedDB persistence behind one SpeedyDbClient, with bindings for React, Vue, Svelte, Solid, and Angular. Install, copy the runtime assets into your public dir, and warm the client:

# @speedydb/core is not published publicly yet — access is restricted while the
# package settles. Authorised consumers install it from the private registry:
npm config set @speedydb:registry <registry-url>
npm config set //<registry-host>/<path>:_authToken <token>
npm i @speedydb/core
npx speedydb-copy-assets        # optional — or pass assetBase: SPEEDYDB_CDN

import { SpeedyDbClient } from "@speedydb/core";
const client = new SpeedyDbClient({ assetBase: "/speedydb/" }); // the printed base
await client.warm();            // engine up (durable-first, in-memory fallback)

From there it's warm() → ingest → loadSemantic() → search() / ask() — the developer docs cover the full package surface, the framework bindings, and self-hosting/CSP.

Prefer no build step? Vendor the files instead: the WebAssembly bundle (pkg/) plus two host modules, durable.js (IndexedDB persistence) and embedder.js (the on-device embedding model), served from the same origin. That is how this site itself runs, driving the engine API documented below directly:

import initSpeedyDb from "./pkg/speedydb.js";
import { openDurable } from "./durable.js";
import { embedQuery, embedDocuments, EMBED_MODEL_ID, EMBED_DIM } from "./embedder.js";

// 1 · boot the engine, open (or restore) a persistent database
const mod = await import("./pkg/speedydb.js");
await mod.default();
const handle = await openDurable(mod, { dbName: "my-app" });
const db = handle.db;

// 2 · ingest — keyed sources are idempotent (same key ⇒ same rows back)
const rows = db.putSourceTextKeyed("doc:guide", ["chunk one", "chunk two"]);
handle.scheduleFlush();                      // write-behind to IndexedDB

// 3 · exact search, context expansion
db.findText("chunk", 5);
db.expandText(rows[0], 1n);                  // i64 params are BigInt

// 4 · semantic search (downloads the model once, ~80 MB, cached)
db.ensureEmbeddingModel(EMBED_MODEL_ID, EMBED_DIM);
db.putEmbeddings(EMBED_MODEL_ID, Array.from(rows, Number),
                 await embedDocuments(["chunk one", "chunk two"]));
const hits = db.findSemantic(await embedQuery("what is this about?"), 5);

await handle.close();                        // final flush + lock release

Native (Rust) usage opens the same byte-identical .spdb files through the speedydb crate — a browser export imports natively and vice versa.

API reference

The complete JavaScript surface. i64 parameters are BigInt; hit objects carry row ids as plain Numbers.

Database lifecycle

Ingest

Fetch & context

Search

Files & sync

Semantic search

SpeedyDb ships with the approved on-device embedding model — nomic-ai/nomic-embed-text-v1.5 (768-dim, mean pooling, L2-normalized), running on the vendored llama.cpp WebAssembly runtime. Weights (official GGUF Q4_K_M, ~80 MB) download on first use and are cached by the browser.

MCP server — tabular work for agents

A model can read a table it can fit in its context. Everything larger is a different problem, and pasting a CSV into a prompt is not a solution to it. The MCP server (speedydb_mcp) gives an agent a fixed set of operations over files that stay on disk: it profiles, filters, aggregates, pivots, combines and cleans, and hands back a file. Nothing but the JSON a tool returns crosses the boundary.

Every operation streams. The failure mode for a file bigger than memory is a slower answer, not a dead container — which is the whole reason this exists rather than a note saying "just use pandas". Measured on an M4 Pro, two 474 MB CSVs:

Peak stays under the size of the input: the file does not have to fit. Ingesting a CSV into an .spdb dataset first makes everything after it faster and lighter — a projection reads only the columns asked for and a row count is a manifest read.

The tools

• Look: list_files, describe_file, profile_file, preview_rows.
• Reshape: transform_file (filter / derive / sort / project), combine_files, remove_outliers (mean±k·σ or Tukey's fences), pivot_table, aggregate.
• Load: ingest_file builds an .spdb dataset from an uploaded CSV, streamed.
• Finish: end_session deletes the caller's files and reports what went.

Files arrive over plain HTTP at POST /upload and leave at GET /download, not through tool calls — a 2 GB file has no business being base64'd through a model's context in either direction. The model gets the path; the client gets the bytes.

One workspace per credential

The bearer token on the request decides which files a call can see, and nothing else does. It is deliberately not a tool argument: a scope parameter is a parameter the model fills in, and a model can be talked into filling it in with somebody else's value by a document it has just read. The directory name is a hash of the token, because directory names reach logs, listings and error messages.

• Sessions end twice over. end_session is the explicit one; an idle reaper covers every agent that crashes or wanders off, which is the common case.
• Concurrency is bounded. Each operation is bounded on its own; a slot count bounds how many run at once, since twenty agents each inside a comfortable 350 MB is 7 GB. Metadata tools take no slot, so an agent can still look at its workspace while the server is busy.
• Caps refuse, they do not truncate. A pivot axis wider than its cap is refused by name; a truncated cross-tab reads exactly like a complete one.

Trying it

Transform is that server driven from the browser: take a file, ask in plain English, download the result. A small model only chooses which tool to call — it cannot reason over a 500 MB file, but it can pick "remove_outliers on amount". Every plan is labelled with who made it, so a model that is failing looks like a model that is failing.

Step 1 is a choice of environment, and it decides where your file goes:

• SpeedyDb micro server — the file is uploaded and the tools stream over it in bounded memory. This is the one that answers a file larger than RAM.
• Local — in this browser — the file is read in the tab and never uploaded; the same tools run as WebAssembly. The whole table is held in memory, so it is right for a file that fits and wrong for the case the server exists for. Every reply carries bounded: false so the transcript cannot be mistaken for the other one.

The environments share one code path rather than being two implementations, and the semantics are deliberately identical — population standard deviation, Tukey k = 1.5, σ k = 3.0, the same aggregation names. Two environments that answered differently would make the choice a choice about answers. Choosing local also runs the model on-device: planning on the server would send your column names to it, which is the one thing "local" is claiming it does not do.

Persistence model

The engine runs synchronously over in-memory segments; IndexedDB is the durability layer. On open, durable.js hydrates every persisted record; after mutations it drains the engine's dirty record keys and writes only those back in one transaction (write-behind). Everything — content rows, sidecars, semantic vectors — rides the same mechanism and survives reloads.

• Durability is advisory: a flush scheduled but not committed when the page dies is lost; that ingest simply re-runs next visit.
• Single tab: a Web Lock makes a second tab's open fail cleanly rather than corrupt via last-flush-wins.
• Byte compatibility: exported files are byte-identical to native .spdb — a browser export opens on the server and vice versa.

Best practices

• Key your sources. Use a content hash or stable document id with putSourceTextKeyed so re-ingesting is a no-op instead of a duplicate.
• Chunk to fit. Keep each chunk within contentCapacity bytes (set at creation). Split on paragraph boundaries; group CSV lines under a repeated header.
• Flush after writes. Call scheduleFlush() after mutations and flush() on pagehide; call close() when done.
• One embedding space — and the RUNTIME is part of it. Same model, dim and quantization everywhere vectors are compared, and the same embedder build. The model file alone is not enough: llama.cpp changed how it computes this model's lfm2 architecture, and the same .gguf with the same pooling and prompts then yields vectors differing by cosine 0.84. ensureEmbeddingModel(modelId, dim) compares only the name and the dimension, so it cannot catch that — if you embed anywhere other than this browser build, verify against the golden vectors first. Treat ensureEmbeddingModel(...) === false as "re-embed now".
• Backfill in batches. Drain rowsMissingEmbeddings in bounded batches so the UI stays responsive during large backfills.
• Prefer findSemantic at scale. findText is an O(rows) scan for small sets and debugging; semantic search is the retrieval path.
• Soft-delete, don't rewrite. Deactivate sources/rows; ids are never reused, so references stay valid.

Pricing

The in-browser demo on this site is free to try. SpeedyDb itself is commercially licensed software; production pricing (per-seat, embedded/OEM, and master-server tiers) is being finalized.

Licensing

SpeedyDb is proprietary software — © 2026 Griffin Pilz, all rights reserved. It is not open source and not free software: no use, copying, modification, distribution, or reverse engineering is permitted without a separate signed written agreement, and the software is provided "as is" without warranty. The full text ships as LICENSE with the software.

Third-party components