← eval history

q043

Which had the highest net income in 2023: JPMorgan Chase, Microsoft, or Costco?

expected Microsoft · type freeform · grounded in jpmorgan-2023.pdf ("Net income $ 49,552") + microsoft-2023.pdf ("72,361") + costco-2023.pdf ("Net income was $6.3 billion")

Retrieved by 1/25 configs · answered correctly by 11/25.

WhenConfigRetrievalAnswerΔ
2026-06-18 16:21jina-v3 + deepseek-chat · k=20FAILFAIL
2026-06-18 17:15qwen3-embedding:0.6b + deepseek-chat · k=20FAILFAIL
2026-06-18 18:19nomic-embed-text + deepseek-chat · k=20FAILFAIL
2026-06-18 21:17qwen3-embedding:0.6b + deepseek-chat · k=20 · 800chFAILFAIL
2026-06-19 08:23jina-v3 + deepseek-chat · k=20 · 800chFAILFAIL
2026-06-21 09:56jina-v3 + deepseek-chat · k=20FAILFAIL
2026-06-21 09:58jina-v3 + deepseek-chat · k=20FAILFAIL
2026-06-21 11:16jina-v3 + deepseek-chat · k=20FAILFAIL
2026-06-21 11:18jina-v3 + deepseek-chat · k=20FAILFAIL
2026-06-21 11:21jina-v3 + deepseek-chat · k=20FAILPASS✓ improved
2026-06-22 02:27jina-v3 + deepseek-chat · k=20FAILPASS
2026-06-22 02:29jina-v3 + deepseek-chat · k=20FAILPASS
2026-06-22 03:54jina-v3 + deepseek-chat · k=20PASSFAIL✓ improved
2026-06-22 04:44jina-v3 + deepseek-chat · k=20FAILFAIL⚠ regressed
2026-06-22 05:20jina-v3 + deepseek-chat · k=20FAILPASS✓ improved
2026-06-22 06:08jina-v3 + deepseek-chat · k=20FAILPASS
2026-06-23 05:06jina-v3 + deepseek-chat · k=20FAILPASS
2026-06-23 06:04jina-v3 + deepseek-chat · k=20FAILPASS
2026-06-23 09:02jina-v3 + deepseek-chat · k=20FAILPASS
2026-06-23 10:06jina-v3 + deepseek-chat · k=20FAILPASS
2026-06-25 09:57jina-v3 + deepseek-chat · k=20FAILFAIL⚠ regressed
2026-06-25 10:51jina-v3 + deepseek-chat · k=20FAILFAIL
2026-06-25 11:58nomic-embed-text + deepseek-chat · k=20FAILPASS✓ improved
2026-06-25 17:44Qwen/Qwen3-Embedding-8B + deepseek-chat · k=20FAILFAIL⚠ regressed
2026-06-25 18:01Qwen/Qwen3-Embedding-8B + deepseek-chat · k=20FAILPASS✓ improved