Qwen/Qwen3-Embedding-8B + deepseek-chat · k=20
run #33 · 2026-07-21 14:19 · git af7d01f
Retrieval hit@k
95%
Answer accuracy
100%
Cases
45 (40 grounded)
Δ vs baseline
Compared to Qwen/Qwen3-Embedding-8B + deepseek-chat · k=20 (2026-07-21 01:04). A retrieval delta with the same generator isolates the embedder/retrieval change.
Failures in this run (2)
| Case | Question | Retrieval | Answer | Symptom |
|---|---|---|---|---|
| q024 | How much larger were Berkshire Hathaway's 2023 net earnings than its 2023 operating earnings? | FAIL | PASS | answered without the gold chunk (parametric leak?) |
| q043 | Which had the highest net income in 2023: JPMorgan Chase, Microsoft, or Costco? | FAIL | PASS | answered without the gold chunk (parametric leak?) |