Qwen/Qwen3-Embedding-8B + deepseek-chat · k=20
run #30 · 2026-07-20 11:10 · git 537727e
Retrieval hit@k
98%
Answer accuracy
98%
Cases
45 (40 grounded)
Δ vs baseline
Compared to Qwen/Qwen3-Embedding-8B + deepseek-chat · k=20 (2026-07-20 10:37). A retrieval delta with the same generator isolates the embedder/retrieval change.
- ⚠ Answer regressed (1): q024
Failures in this run (2)
| Case | Question | Retrieval | Answer | Symptom |
|---|---|---|---|---|
| q024 | How much larger were Berkshire Hathaway's 2023 net earnings than its 2023 operating earnings? | PASS | FAIL | retrieved but generated wrong (generation/arithmetic) |
| q043 | Which had the highest net income in 2023: JPMorgan Chase, Microsoft, or Costco? | FAIL | PASS | answered without the gold chunk (parametric leak?) |