Same agent. Two ways to search the web.
How much does the search and fetch stack change the answer? We ran the same Astra agent on 246 browsing tasks with Exa and DeepSeek, twice per task.
GPT-6 AstraAccuracy · reviewed answer key
Median seconds / task attempt
Exa scored higher and finished faster on both datasets. DeepSeek had lower search and fetch fees. Exa’s lower model-token estimate reverses the cost ranking when model and tool costs are combined.
The 17.2 pp gap exceeds the registered 12 pp decision threshold. Excluding 10 pilot-overlap tasks gives a similar +18.7 pp gap.
The 8.3 pp gap is below the registered 12 pp decision threshold. “Inconclusive” follows that rule; the interval still excludes zero.
| Operational metric | Exa | DeepSeek |
|---|---|---|
| Search calls | 10,568 | 14,626 |
| Fetch calls | 2,040 | 2,389 |
| Fetch failure rate | 2.7%56 failures | 24.1%576 failures |
| Reached 60-call cap | 15.0%74 attempts | 27.0%133 attempts |
| Completed / tool isolation passed | 492 / 492 | 492 / 492 |
Two aging public datasets, one agent, and a reviewed answer key. Twenty-seven task keys changed after review. The pre-run calibration record remains incomplete; the contested-answer procedure was added after 40 staged scores.
Exa supplied $1,000 in API credits. No DeepSeek relationship or other funding. Explore all 246 task outcomes, download reviewed scores, and inspect the registered protocol.
research.site