Retrieval methods and ranking
Shared by Pavel · Updated 2026-11-12
Lexical search matches query words; token-level retrieval makes finer-grained token matches; multi-retriever systems combine complementary candidate lists and rerank them.
Lexical retrieval
- BM25 scores documents using query-term frequency, weighting rarer terms more and normalising for document length. It is useful for exact names and identifiers, but word-based matching can miss synonyms such as “car” and “automobile.”
- Contextual retrieval prepends an LLM-written description of a chunk’s place in its document before indexing it with BM25 and embeddings. Anthropic reported that adding BM25 and reranking reduced failed top-20 retrievals from 5.7% to 1.9% on its own datasets.
Token-level retrieval
- ColBERT’s late interaction separately encodes the query and document into token vectors, then compares them at token level. Since document vectors can be precomputed, it can preserve fine-grained term matches without jointly encoding every query-document pair.
- A cross-encoder instead reads the query and candidate together and assigns a relevance score. It is more accurate than a bi-encoder but runs per candidate, so it is typically used to rerank a shortlist.
Multi-retriever retrieval and ranking
- Hybrid systems combine lexical matching with dense retrieval, which compares learned embeddings to find semantically similar passages; some also add graph traversal for linked concepts.
- Each retriever contributes its own ranked list. Reciprocal rank fusion (RRF) combines them by summing (1/(k+\text{rank})) for each document across lists, rewarding items ranked highly by multiple retrievers.
- A reranker then reorders the merged candidates for precision; Maximal Marginal Relevance (MMR) can balance relevance with diversity to reduce near-duplicates.