🔎 Neural Passage Search
See what changes when a neural ranker replaces, or re-orders, a lexical one.
Ask anything
Pick an example above or type a question, then compare how each ranking system answers it.
How the rankers compare
Quality metrics are computed on the held-out test queries. Latency is the median over 200 queries.
| System | MRR@10 | NDCG@10 | P@5 | Recall@100 | Median latency (ms/query) |
|---|---|---|---|---|---|
| TF-IDF | 0.2943 | 0.3854 | 0.1091 | 0.9073 | 55.13 |
| BM25 | 0.3316 | 0.423 | 0.1192 | 0.9087 | 153.5 |
| Bi-encoder | 0.4895 | 0.579 | 0.1525 | 0.982 | 7.419 |
| BM25 → Cross-encoder | 0.4016 | 0.4835 | 0.1272 | 0.7544* | 284.9 |
Best value per column highlighted. 750 test queries, MS MARCO passages. *Cross-encoder Recall@100 was computed over a run file holding only its top-10, so it is not a true Recall@100.
Quality vs. speed
Each point is one retriever. Favor the upper-left: higher MRR with lower query latency.
What changes the ranking?
Validation experiments isolate training negatives and candidate depth; query-length results use the test split.
Negative sampling
Validation MRR@10 · hard vs random training negatives
Candidate-depth trade-off
Validation MRR@10 and total latency by BM25 depth
Query-length analysis
Test MRR@10 by bucket · 250 queries per bucket
How the systems differ
| System | Idea | Strength | Weakness |
|---|---|---|---|
| TF-IDF | Weight terms by frequency and rarity, cosine similarity | Fast, transparent | No length normalisation, exact words only |
| BM25 | TF-IDF refinement with saturation and length normalisation | Strong lexical baseline | Vocabulary mismatch (synonyms, paraphrase) |
| Bi-encoder | Embed query and passage independently, compare vectors | Very fast at query time, semantic | Single vector per passage loses detail |
| BM25 → Cross-encoder | Re-score BM25's top-N by reading query and passage together | Most precise per pair | Slow; capped by BM25's recall |
Reading the result cards
- Coloured bar: the score, scaled within that column. Scores from different systems are on different scales, so compare ranks, not raw values.
- â–² / â–¼ chips: how far a passage moved relative to BM25's ranking.
- ✨ outside BM25 top-N: the system found a passage BM25's candidate list never contained. For the cross-encoder this can't happen (it only re-orders BM25 candidates), which is exactly why a dense first stage helps.
- Highlighted words: query terms, so you can see when a system retrieves without lexical overlap.
Metrics
- MRR@10: average of 1/rank of the first relevant passage in the top 10.
- NDCG@10: rewards putting relevant passages higher, with a log discount.
- Recall@100: share of relevant passages found in the top 100.