jina-v3 + deepseek-chat · k=20
run #23 · 2026-06-23 10:06 · git 16da4bb · tbl:inmem
Retrieval hit@k
75%
Answer accuracy
98%
Cases
45 (40 grounded)
table-aware chunking, in-memory, full stack
Δ vs baseline
Compared to jina-v3 + deepseek-chat · k=20 (2026-06-23 09:02). A retrieval delta with the same generator isolates the embedder/retrieval change.
- ✓ Retrieval solved here (1): q029
- ⚠ Retrieval regressed (4): q006, q009, q010, q024
- ✓ Answer solved here (1): q027
- ⚠ Answer regressed (1): q017
Failures in this run (10)
| Case | Question | Retrieval | Answer | Symptom |
|---|---|---|---|---|
| q006 | What were Berkshire Hathaway's net earnings attributable to shareholders in 2023? | FAIL | PASS | answered without the gold chunk (parametric leak?) |
| q008 | What was Berkshire Hathaway's shareholders' equity at the end of 2023? | FAIL | PASS | answered without the gold chunk (parametric leak?) |
| q009 | How much did Berkshire Hathaway's shareholders' equity increase during 2023? | FAIL | PASS | answered without the gold chunk (parametric leak?) |
| q010 | What were Berkshire Hathaway's after-tax investment gains in 2023? | FAIL | PASS | answered without the gold chunk (parametric leak?) |
| q017 | What kind of transportation business does Berkshire Hathaway operate? | FAIL | FAIL | gold chunk not retrieved → answer ungrounded |
| q024 | How much larger were Berkshire Hathaway's 2023 net earnings than its 2023 operating earnings? | FAIL | PASS | answered without the gold chunk (parametric leak?) |
| q025 | What was JPMorgan Chase's total net revenue in 2023? | FAIL | PASS | answered without the gold chunk (parametric leak?) |
| q027 | What were JPMorgan Chase's total assets at the end of 2023? | FAIL | PASS | answered without the gold chunk (parametric leak?) |
| q028 | What was JPMorgan Chase's CET1 capital ratio in 2023? | FAIL | PASS | answered without the gold chunk (parametric leak?) |
| q043 | Which had the highest net income in 2023: JPMorgan Chase, Microsoft, or Costco? | FAIL | PASS | answered without the gold chunk (parametric leak?) |