Evaluating binary and ternary retrieval under a smaller memory envelope

Context

Enterprise RAG must preserve its surrounding operating system: identity, authorization, evidence, and citation lineage. Within that boundary, the open question was narrower and more useful: can a low-bit retrieval index preserve evidence quality while fitting a smaller memory envelope than a standard float32 index?

Decision

Treat retrieval-quality and local-generation-speed as separate claims that are often wrongly conflated, and refuse to publish either claim without a properly matched control.

Approach

• Built a fair float32 comparator with the same fusion and reranking configuration as the binary arm, after finding the first comparison wasn't apples-to-apples.
• Kept low-bit local model serving as a separate, explicitly-scoped evaluation track rather than folding it into the retrieval-quality result.
• Re-verified the full result set directly against raw per-query data before treating any number as a stated finding.

Outcomes

• Binary and ternary retrieval matched float32 retrieval quality on the evaluated corpora while using a substantially smaller index footprint.
• Caught and corrected an initial unverified speed claim once full end-to-end timing was included — an apparent large win became an honest, much smaller difference.
• Produced a narrow, operational recommendation instead of a general one: use low-bit retrieval when memory capacity is the binding constraint, not as a blanket latency claim.

Control boundary

This case describes an internal applied-research evaluation. It excludes proprietary corpora, production infrastructure, and unpublished manuscript text — the summary here reflects verified, public-safe conclusions only.

What this taught me

The most valuable result wasn't the compression ratio — it was catching the team's own overclaim before it shipped. An honest, narrower finding is worth more than a flashy, wrong one.