research(nightly): coherence-adaptive quantization — negative result

[ruvnet]

Hypothesis

Given a 4,000-vector corpus (dim=32, 12 clusters, noise=0.18), when per-vector scalar
quantization bit-width is chosen by mutual-kNN coherence (a lightweight, ground-truth-free
proxy for local graph conductance/cluster-boundary detection, in the spirit of ruvector-mincut's
conductance signal) instead of a uniform 4-bit budget, then recall@10 should recover to within
1.5pp of the uniform 8-bit baseline, subject to total memory staying at or below 65% of baseline.

Result: REJECT. The hypothesis is falsified as tested — see evidence below.

Architecture

New crate ruvector-coherence-quant:

• dataset.rs — deterministic clustered corpus + held-out jittered queries + brute-force ground truth
• coherence.rs — brute-force k-NN graph + mutual-kNN coherence score
• quantize.rs — per-vector min-max scalar quantization, real bit-packed storage (two 4-bit codes/byte)
• search.rs — brute-force search over the quantized index
• bin/benchmark.rs — baseline (uniform 8-bit) vs candidate A (uniform 4-bit) vs candidate B (coherence-adaptive 4/8-bit)

flowchart LR
    A[Clustered corpus] --> B[k-NN graph, k=12]
    B --> C[Mutual-kNN coherence score]
    C --> D{coherence >= 0.5?}
    D -->|core| E[4-bit quant]
    D -->|boundary| F[8-bit quant]
    E --> G[QuantizedIndex]
    F --> G
    G --> H[recall@10 / memory / latency]

Loading

Files changed

• crates/ruvector-coherence-quant/ (new crate: lib, dataset, coherence, quantize, search, metrics, benchmark bin, 16 unit tests)
• Cargo.toml — new workspace member
• docs/research/nightly/2026-08-15-coherence-adaptive-quant/{README.md,gist.md} — full writeup
• docs/adr/ADR-305-coherence-adaptive-quant.md — decision record

Benchmark command

cargo test --release -p ruvector-coherence-quant
cargo run --release -p ruvector-coherence-quant --bin benchmark

Real benchmark results

baseline_uniform_8bit           recall=0.9887  mem=156KB  8.00 bits/dim
candidate_A_uniform_4bit        recall=0.8583  mem= 93KB  4.00 bits/dim
candidate_B_coherence_adaptive  recall=0.8770  mem=116KB  5.47 bits/dim  (63.3% core, 36.7% boundary)

Gate 1 (uniform 4-bit shows real signal, gap > 1.5pp): gap=13.03pp  PASS
Gate 2 (candidate B within 1.5pp of baseline):         gap=11.17pp  FAIL
Gate 3 (candidate B memory <= 65% of baseline):        74.7%        FAIL

Deterministic and reproducible: two independent runs produced bit-identical recall/memory values.

Acceptance result

REJECT. Mutual-kNN coherence is a real, reproducible structural signal (verified higher on
clustered vs. random data by a dedicated unit test) but does not transfer to quantization
bit-allocation on this workload — it measures local density agreement, not per-vector
quantization sensitivity. Full failure analysis in the research README.

Darwin / Flywheel / MetaHarness

npx metaharness --help and repo tooling were inspected; MetaHarness in this environment is a
project-scaffolding generator, not a live research-orchestration daemon for this repo, and
ruvector harness subcommands (Darwin/Flywheel CLI) were not resolvable here. No evolutionary
search or flywheel replay was run as a result — this PoC used a single pre-registered
baseline/candidate-A/candidate-B comparison instead, with the acceptance gates fixed before the
benchmark ran and not adjusted afterward.

Security review

No new attack surface: standalone benchmark crate, no network I/O, no external input parsing, not
wired into any production index or MCP surface.

Main limitations

• PoC scale only (N=4,000, dim=32); brute-force O(n²) k-NN graph construction.
• Only min-max scalar quantization tested, not product/binary quantization.
• Synthetic held-out queries (jittered corpus points), not real agent-memory traffic.

Production recommendation

Do not adopt. Keep uniform quantization. If the general direction (content-aware bit allocation)
is revisited, test a quantization-error-derived signal (per-dimension coordinate range) instead of
a neighbourhood-mutuality signal, and validate against the real ruvector-mincut conductance API.

---

Generated by Claude Code