A codedb snapshot for a 39,000-file repo used to take ~380 ms to load and ~795 MB of RAM. It now loads in ~125 ms and holds ~457 MB, about 3× faster and 338 MB lighter. We did not change the snapshot format. We stopped copying.
v0.2.5824 is mostly a performance release, with two new capabilities riding on top of it: a warm CLI daemon that makes the plain codedb CLI as fast as the MCP server, and a graph mode: a resolved call graph that ranks results by how central a symbol is. This post is the engineering: what we changed, and what each change bought.
3×
faster cold load
380ms → 125ms, ~39k files
−338 MB
peak RSS
795 MB → 457 MB
13-114×
per CLI call
warm daemon vs cold reload
+15%
ranking MRR
0.819 → 0.944, call-graph
TL;DR
- 3× faster cold load. 380 ms → 125 ms, peak RSS 795 MB → 457 MB. A load-path pass: mmap, borrow, zero-copy, pre-size, parallelize.
- CLI mode. The plain CLI proxies to a per-project warm daemon over a Unix socket. 13-114× faster per call.
- Graph mode. A deterministic call graph, persisted in the snapshot. Centrality lifts ranking MRR 0.819 → 0.944 (+15%), zero recall loss.
- Faster find. SIMD Smith-Waterman (~1.8×, byte-identical) plus a ~22× fast path for compound-identifier queries.
- Cold index. RSS 4.3 GB → 1 GB, wall time ~6.5×.
What made the load faster
The snapshot is the precomputed index codedb writes to disk so a warm process starts instantly instead of re-scanning the repo. Loading it was spending most of its time re-deriving things it already had. It opened files just to statthem, read each file's content with several positional reads, re-hashed that content, and duped every symbol name and import into a fresh allocation. The fix, in one sentence: stop copying, borrow from the mmap, store what you would recompute, and parallelize what is left.
what made the load faster: each step A/B-measured
stat() vs open+stat+close
freshness check, no fd churn
one mmap vs ~4 preads / file
content section
borrow outline strings + pre-size maps
no per-string dupes
zero-copy ContentCache
−237 MB RSS
stored content hashes
−100 MB RSS, no re-hash
parallel freshness check
−43% on a 16k-file load
Cumulative on openclaw/openclaw (~39k files): load ~380ms → ~125ms, peak RSS ~795 MB → ~457 MB.
Each row is a separate change, A/B-measured on the same snapshot with interleaved warm loads, so the numbers do not overlap. A few are worth spelling out:
- Borrowed strings, not duped ones. Restored symbol names, imports, and details are now slices into the retained
OUTLINE_STATEsection of the mmap rather than ~170k (millions on a dense repo) individualdupecalls. The load's hashmaps are pre-sized so they never rehash mid-fill. ~34% faster on its own. - Zero-copy file contents. The
ContentCacheused to own a copy of every file. Its values now point straight into the retained snapshot mmap, and a singlemmapof the content section replaces ~4 positional reads per file. ~17% faster and 237 MB lighter. - Hashes we already had.The snapshot now stores each file's content hash, so the loader trusts it instead of re-hashing every byte on the way in. ~14% faster, 100 MB lighter.
- A parallel freshness check. Deciding whether the snapshot is stale means stat-ing every file. That pass is now parallel: ~2.3× on the check, ~43% off a 16k-file load. It bottoms out around four workers (a U-curve, regardless of core count), so codedb caps it there instead of fanning out to every core.
snapshot load: openclaw (~39k files)
cold load
peak RSS
None of this touched the on-disk format, so existing snapshots load faster without a re-index. We gated the whole investigation behind a CODEDB_LOAD_PROFILE phase profiler (near-zero cost when off) so every step above is a real before/after, not a guess.
CLI mode: a warm daemon
A Pro user, @ahndohun, filed a sharp audit (#518) and asked the question that drove this feature: every plain codedb <repo> find Xreloads the snapshot from scratch. Fine once, painful in a loop. “Is there a way to keep the snapshot warm across CLI calls?”
Now there is. The first query auto-spawns a per-project warm daemon bound to a Unix socket (/tmp/codedb-<uid>-<hash>.sock). Every call after proxies to it and streams the rendered output back, hitting the same warm explorer the MCP server uses, so a CLI lookup pays the MCP dispatch cost instead of a cold re-index. 13-114× faster per call in head-to-head runs, with full nav coverage: symbol, callers, deps, context, glob, ls. On any failure (no daemon, refused connect, short read) the client silently falls back to the cold in-process path, so the proxy is never a correctness risk.
Graph mode: the call graph
codedb already had the two ingredients a precise call graph needs: function symbols with line ranges, and a symbol index that maps a name to its definition sites. The missing middle step was walking call sites and resolving them. v0.2.5824 does it deterministically, with no model calls: for each function body it extracts call sites (identifier-before-(, keyword-filtered), resolves each callee through the symbol table, and accumulates a weighted in-degree centrality per file. An ambiguous name splits its weight across candidates.
That centrality folds into ranking as a multiplier: 1 + 0.15·log(1 + centrality). It is purely additive, never a filter, so a misresolved edge can never drop a real result. On the codedb query set it lifts MRR 0.819 → 0.944 (+15%), P@1 12 → 16, recall unchanged. The graph is persisted in the snapshot (a new CALL_CENTRALITY section), so it costs ~3 ms at load instead of the ~960 ms it took to rebuild on the first query. And codedb_context now uses it to show what a symbol calls, not just who calls it.
Faster find, and the rest
- Fuzzy find. A presence prefilter plus a SIMD-across-files Smith-Waterman inner loop (~1.8×, byte-identical results), and a ~22× fast path that routes a compound-identifier query straight to the symbol index and returns the definition instead of scoring every file.
- Cold index. A worker-local parallel scan cut cold-index RSS 4.3 GB → 1 GB and wall time ~6.5×; the WordIndex build is now parallel (~1.49×).
- Hardened CLI. Robust argument parsing and validation, correct exit codes, a new
codedb status, and a globally-honored--no-telemetry, so subagents and scripts can trust the exit code. - Correctness. Non-ASCII (e.g. Korean) identifiers are now indexed in outlines,
codedb_findreturns “no match” instead of a confident wrong hit, and Pythonclassis labeledclass, notstruct. All from @ahndohun's #518 audit. New: ReScript.res/.resisupport, requested by @yousafsabir.
Upgrade
Existing snapshots load faster with no re-index.
codedb update # or a fresh install: curl -fsSL https://codedb.codegraff.com/install.sh | bash
macOS (codesigned + notarized) and Linux x86_64 / arm64, with SHA-256 checksums. Release notes and the full changelog have every number.