Eval history

Every EVAL_LOG=1 run, recorded. Two metrics, kept separate: retrieval hit@k (did the gold chunk reach the generator) vs answer accuracy (was the final answer right). A green retrieval cell with a ✗ means good retrieval but a wrong answer; a red cell with a ✓ usually means the model answered from parametric memory, not the document.

View as timeline → — the runs in chronological order with the story and deltas.

embedder:ollama:nomic-embed-text ollama:qwen3-embedding:0.6b openai:Qwen/Qwen3-Embedding-8B ✕openai:jina-embeddings-v3

label:ab:chunk ab:embedder ab:planner ab:retrieval chunk:ctx-prefix deploy:final deploy:pg-chunks_v2 deploy:pg-genfix deploy:pg-judged deploy:q8-sf deploy:q8-sf-xdocfix ✕rerank:jina-rawtext rerank:jina-v2 rrf:vw1.0 rrf:vw1.2 tbl:f3-nomic tbl:family3 tbl:family3b tbl:inmem

clear filters

Runs (1/25)

When	Config	Label	git	Retrieval	Answer	n
2026-06-25 18:01	Qwen/Qwen3-Embedding-8B + deepseek-chat · k=20	deploy:q8-sf-xdocfix	16da4bb	98%	98%	45

Per-case grid

Cell colour = retrieval (■ hit, ■ miss, ■ absent). Glyph = answer (✓/✗).

config	q001	q002	q003	q004	q005	q006	q007	q008	q009	q010	q011	q012	q013	q014	q015	q016	q017	q018	q019	q020	q021	q022	q023	q024	q025	q026	q027	q028	q029	q030	q031	q032	q033	q034	q035	q036	q037	q038	q039	q040	q041	q042	q043	q044	q045
Qwen/Qwen3-Embedding-8B + deepseek-chat · k=20 16da4bb	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✗	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓