Exa Lite found 50 of 53 answers, 1 fewer than Exa Auto, in less than half the time

Exa announced Exa 3.0 on October 19, 2026. Exa 3.0 added Exa Lite, a search type at $4 per 1,000 requests. Exa Auto costs $7, and Exa Deep costs $12.

In its announcement, Exa says Lite “runs on the same index as Exa Auto”. It also says Auto “uses more compute across the retrieval stack, including candidate generation, highlights, and ranking”. So the question for an agent team is simple. Does the cheaper search type find the same answers?

I ran all 3 search types on the 53 questions from my Cloudflare Web Search API test. Lite found 50 answers, and Auto and Deep found 51 each. Lite’s median response took 746 ms, against 1,732 ms for Auto and 4,850 ms for Deep.

The test#

I ran everything from Mumbai on October 21, 2026, with my own Exa API key.

• Questions. The same 53 questions as the Cloudflare test. 40 are random questions from SimpleQA, OpenAI’s public set of short questions with known answers. 13 ask about GitHub releases published between October 1 and 10, 2026, like “Which version of Playwright was released on October 7, 2026?”
• Calls. I called Exa’s search API directly, with type set to lite, auto or deep, 10 results and highlights. Each question ran once on each search type. The order of the 3 search types was random for each question.
• Counting answers. A question counts as answered when the answer appears in the titles or highlights of the top 10 results. I checked every miss by hand, with the same rule as the Cloudflare test. For example, “May 24, 1984” counts for “24 May 1984”.

This is the request body for one question. Only the type value changed between the 3 search types.

{
  "query": "Which version of Playwright was released on October 7, 2026?",
  "type": "lite",
  "numResults": 10,
  "contents": {"highlights": true}
}

Results#

The cost that Exa reported in each response matched the price list. The 53 searches cost $0.212 on Lite, $0.371 on Auto and $0.636 on Deep.

• Lite missed 1 question that Auto and Deep answered. The question asked which music video Metal Injection ranked number 9 on a list of viral metal videos. The answer is BABYMETAL’s “Doki Doki☆Morning”. Auto and Deep returned a page with the song title, and Lite didn’t.
• All 3 search types missed the same 2 questions. One asked for a vote margin in Turkey’s 2018 presidential election, and no result contained the number. The other asked for María Fernanda Cabal’s birth date. SimpleQA gives 1964, but the top results said 1966, so it counts as a miss, as it did in the Cloudflare test.
• All 3 search types found all 13 release answers.
• The text was about the same size. The median highlight text per query was 7,030 to 9,356 characters, depending on the search type. That’s roughly 1,800 to 2,300 tokens, at about 4 characters per token, for the model that reads it.

Lite and Auto returned mostly the same pages#

Lite and Auto returned the same top result for 45 of the 53 questions. The median overlap of their URLs was 0.67 (the URLs both returned, divided by all different URLs from both). That fits Exa’s description. Both use the same index, and Auto uses more compute for steps like ranking.

Deep’s pages differed more from Auto’s. It had the same top result as Auto for 28 questions, and a median URL overlap of 0.33. It still found the same 51 answers.

I also compared Auto with my run on October 17, 2 days before Exa 3.0. Auto found the same 51 answers, and returned the same top result for 48 of the 53 questions. Exa’s announcement reports gains on benchmarks in areas like coding, finance and company research. These short questions were already answered before Exa 3.0.

What the difference means for an agent#

An agent can call search many times in one task, so the difference grows with every search. At the median times from this test, 10 searches one after another take about 7.5 seconds on Lite, 17 seconds on Auto and 49 seconds on Deep. They cost $0.04, $0.07 and $0.12.

Exa’s pricing page suggests using Lite “inside latency-sensitive agent loops”. Its API docs describe Auto as “a balanced mode recommended for most applications”. On these questions, Lite missed 1 more answer of 53 than Auto, in less than half the median time and at 57% of the price.

My questions are single fact lookups. Exa’s docs say the deep search types can take extra query variations for broader results. Questions that need several steps could show a difference that this test didn’t.

Limits of this test#

• 53 short questions. They’re fact lookups, not research tasks.
• 1 run per search type. Search results and response times can change between runs. I measured the times from Mumbai, one request at a time.
• Exa only. For a comparison with other providers on the same questions, see the Cloudflare test.

How to choose a search type#

• Run each of your questions once on each search type, because repeated queries can measure the cache instead. Then check every miss by hand.
• Compare the cost per answered question, not only the price per 1,000 requests.
• Measure the time per agent task, not only per search.
• Test Deep on the multi-step questions that your agent handles, before you pay for it on every call.

If you want the questions, the results or the scripts, email me and I’ll share them.