Building sim_LAR: Why Similarity Search Needs to Be Exact, Not Approximate, in the RAG Era

When an AI model searches for context, it doesn't care about the theory behind nearest-neighbor search, or the tradeoffs engineers have made between speed and accuracy over the years. It just needs the right information to show up — every time, no matter when the index was built or how many times it's been updated since.

That's a problem, because most similarity search engines were never built to guarantee that.

At TekDatum, we build AI applications that depend on finding the right information at the right time. As we developed these systems, we noticed a recurring problem: content that answered a question perfectly one day could suddenly stop appearing after the next search index update, even though the content itself hadn't changed. Nothing was technically broken — the search engine was simply doing what it was designed to do. But for AI, inconsistent search results lead to inconsistent answers. That's why we built sim_LAR (Large Augmented Retriever): a hybrid, high-dimensional search engine designed to help AI models find the right information consistently.

01Approximate Search Used to Be Good Enough. Then RAG Changed the Rules.

Traditional similarity search tools all rely on the same basic trade-off: because finding the closest matches to a query is a genuinely hard problem, engines are allowed to return approximate results in exchange for speed. For most of the field's history, that trade-off made sense — "good enough" results really were good enough.

When we reviewed our own RAG systems, we found that relevant results would sometimes vanish entirely after the index was rebuilt — not because they stopped being relevant, but because the approximation had shifted. That's no longer good enough. We need search results that are complete and consistent every time — or, at the very least, a way to measure exactly how many results we need to request before every relevant piece is guaranteed to show up.

02An Early Bet That Paid Off

Bigger embeddings perform better — but they come at a cost. In our own RAG projects, embeddings with 1,024 dimensions or more consistently outperformed smaller ones, and the strongest embedding models on the market today generally use several thousand dimensions. That extra accuracy matters: more dimensions mean the model can capture finer shades of meaning, so it's less likely to miss or confuse similar-sounding content.

But that accuracy isn't free. High-dimensional indexes are expensive to build, slow to search, and hungry for memory — the more detail an embedding captures, the more it costs to store and compare at scale. Getting the accuracy benefits of high dimensionality without that infrastructure penalty is the real problem sim_LAR had to solve.

03Introducing sim_LAR

sim_LAR is a high-dimensional search engine built from scratch — not adapted from a tool designed for smaller, embeddings-only workloads. It's built to do three things that other engines struggle to do all at once: index high-dimensional data quickly, search it quickly, and do both while using a fraction of the memory.

To measure what "good enough" means for an AI-facing search engine, we didn't reach for the usual vector database metrics. Instead, we used something we call Max Position: for every test query, we kept expanding the result set until every relevant piece of information had appeared, then recorded how far down the list the last one landed. Lower is better — it means an AI model reading the top results is guaranteed to find everything it needs without having to dig deep into the list.

04How We Ran the Benchmark

We tested sim_LAR against several business-oriented vector databases, including Qdrant — with Chroma included in select comparisons.

Datasets. We tested against two datasets. PubMedQA is a set of expert-labeled biomedical questions and answers. We combined it with several other biomedical sources (gooaq, emrqa-msquad, MASHQA-JSON, medquad, and medwiki) to build a dataset of 8.8 million entries, then measured performance against the original PubMedQA questions. HotpotQA is a harder, multi-hop dataset that requires connecting information across multiple documents, rather than just matching text.

Test setup. We indexed 8,841,823 entries and created a test set of 10,000 questions from PubMedQA, each labeled with the three or four pieces of text needed to answer it correctly. For each question, we kept expanding the results until every correct answer appeared, then recorded the Max Position.

05Results

In the PubMedQA benchmark, two of the other engines we tested never found 100% of the relevant results, no matter how many results we requested. Every other engine eventually found everything, but sim_LAR consistently got there with a lower Max Position — meaning it surfaced the full answer sooner. On the harder HotpotQA benchmark, sim_LAR was the only engine to reach 100% retrieval at all (we left Chroma's HotpotQA results out because that test run didn't complete).

Keeping a RAG index fresh means being able to rebuild or update it quickly. sim_LAR indexed the same data 32× to 99× faster than the other engines we tested.

Search speed is the whole point of a similarity engine. sim_LAR answered queries 1.26× to 8,000× faster than the alternatives, with the biggest gaps against engines that struggled most with high-dimensional queries.

High-dimensional indexes carry a lot of internal structure, and that structure has to be stored somewhere. sim_LAR used 2× to 6× less memory than the other engines during search.

The cost difference makes this concrete. Cloud Provider 1 doesn't report memory usage directly, but building the index for our 8.8-million-entry dataset cost us $900 on that platform — enough to rent roughly 1.5 TB of RAM for a month on EC2. The same index build on Cloud Provider 2 cost just $15. Multiply a gap like that across a production RAG pipeline that's constantly being updated, and index architecture stops being a technical detail — it becomes a real line item on the budget.

06What This Means in Practice

Together, the PubMedQA and HotpotQA results tell a consistent story: sim_LAR indexes faster, searches faster, uses less memory, and delivers better — often perfect — results, all at the same time. For teams running RAG in production, that adds up to lower infrastructure costs, fresher indexes, and fewer of the silent, hard-to-debug hallucinations that happen when an AI model answers with incomplete information.

Building sim_LAR taught us more than we expected about what a similarity engine built for the RAG era actually needs to do — not just search fast, but guarantee completeness, support high-dimensional data natively, and stay reliable as both people and machines query it in increasingly creative ways. We're already putting those lessons to work in our next-generation engine.