Qwen/Qwen3-Embedding-8B + deepseek-chat · k=20
run #34 · 2026-08-06 00:48 · git 4124d2c
Retrieval hit@k
95%
Answer accuracy
100%
Cases
45 (40 grounded)
Δ vs baseline
Compared to Qwen/Qwen3-Embedding-8B + deepseek-chat · k=20 (2026-07-21 14:19). A retrieval delta with the same generator isolates the embedder/retrieval change.
No per-case changes vs the baseline.
Failures in this run (2)
| Case | Question | Retrieval | Answer | Symptom |
|---|---|---|---|---|
| q024 | How much larger were Berkshire Hathaway's 2023 net earnings than its 2023 operating earnings? | FAIL | PASS | answered without the gold chunk (parametric leak?) |
| q043 | Which had the highest net income in 2023: JPMorgan Chase, Microsoft, or Costco? | FAIL | PASS | answered without the gold chunk (parametric leak?) |