LEXICAL BASELINE
BM25
Rare terms, repetition, and passage length. Your sliders affect this method.
RETRIEVAL, MADE VISIBLE
Find the evidence.
Understand the system.
Same question. Same documents. Different retrieval decisions. Inspect the passages, change the parameters, and measure the tradeoff.
REAL PUBLIC CORPUS
12documents123chunks
No generated answers in this lab. You are inspecting the evidence a model would receive.
Corpus 033d462b226f- 01 / INGESTPublic documentsSeparate the audience first
- 02 / INDEXHeading-aware chunksKeep source + content identity
- 03 / RETRIEVELexical + denseDifferent signals, different failures
- 04 / RANKFuse + rescoreChoose the evidence window
- 05 / GROUNDEvidence or abstentionA close passage can still be wrong
01 / EXPERIMENT
Ask the corpus a question
Include live neural embeddings
Research access is required. This makes one paid query-embedding request against a prepared corpus cache. It does not train a model or generate an answer.
0 / 123 cached passage vectors. Dense retrieval is not configured for this checkout/deployment.
Sign inRun an experiment to inspect real evidence.
02 / OBSERVE
Rankings and evidence
NEURAL BASELINE
Dense cosine
Similarity between the question embedding and each passage embedding.
EXISTING ASSISTANT LOGIC
Weighted coverage
IDF-weighted matching in public mode. Live mode adds dense evidence and fusion.
EVIDENCE INSPECTOR
Select a ranked passage
Select any result above to inspect its full indexed text.
This is source text, not a model response. An archived passage needs historical context.
WHY THESE WORDS MATTER
Query term diagnostics
Common words are removed; remaining terms are normalized. IDF grows as a term becomes rarer in the corpus.
| Term | Chunks containing it | IDF |
|---|
A zero-frequency term exposes a knowledge gap. It does not prove the person lacks that experience.
03 / MEASURE
Does a change actually help?
Five answerable questions and two questions outside the corpus. Each answerable question names a target source and heading. The benchmark uses your current BM25 settings and top k; dense retrieval is not run.
| Question / expected passage | BM25 rank | Assistant rank | Grounding gate |
|---|
04 / EXPLAIN
The decisions behind the demo
Why hybrid retrieval?
Lexical retrieval preserves exact names and numbers. Embeddings can connect different wording to the same idea. Their errors differ, so combine their candidate lists, then evaluate whether the combination improves retrieval.
RRF(d) = Σ 1 / (60 + rankᵢ(d))
In this assistant, RRF selects the candidate pool. Final order uses 0.55 × cosine + 0.30 × term coverage + 0.15 × retriever agreement. This is a hand-designed rescore, not a trained reranker.
How is the corpus indexed?
The same Markdown ingester used by the homepage splits at headings and paragraph boundaries. The target ceiling is 1,400 characters, with 200-character overlap when a single paragraph must be split. Small sections may be merged; the ceiling is not a universal guarantee.
Source and heading travel with every chunk. Content-derived IDs let unchanged chunks reuse embeddings. The corpus fingerprint ties an exported experiment to the document snapshot.
Dense search here scans cached vectors exactly. An approximate vector index becomes a separate scaling decision for a larger corpus.
What do the metrics measure?
Hit@k: did the labelled target passage appear? MRR@k: how early did it appear? nDCG@k: a logarithmic discount for its position. Here there is one binary target per question.
Abstention accuracy measures rejection on the two unanswerable questions; answerable grounding rate checks the other direction. Neither proves the quality of an answer that a model generates.
This is a tiny development set, not a held-out estimate. Choose additional questions before tuning to test generalization.
Where does the agent fit?
The homepage agent gets search, full-document lookup, corpus listing, and status through MCP. It can search, inspect a source, and search again inside its model/tool loop.
This lab isolates retrieval so you can see its failures before adding generation. A similarity score is not a probability of correctness, and an abstention flag is a heuristic rather than a proof.
Where are the security boundaries?
Only public career documents enter this corpus. Internal documents and operational database rows are excluded before retrieval. The public lab performs no model calls and has no write tools.
Live query embeddings require server-side research authorization and rate limiting. A hidden button is not authorization. Model instructions are not access control.
What would make this more advanced?
A larger held-out set, a learned cross-encoder reranker, source permissions across federated data, and measured answer faithfulness are next experiments. Each should be added with a baseline and a measurable acceptance criterion.
Your interview explanation: “I isolate retrieval from generation, inspect the evidence, compare baselines, and use failures to decide which complexity is justified.”