Cloudflare's Web Search API, tested on 53 questions: the provider you pick matters far more than the route

Cloudflare’s Web Search API gives AI agents one endpoint for 3 search providers, Ceramic.ai, Exa and Linkup. Requests run on AI Gateway, so searches show up in the same logs and billing as model calls.

That raises 2 practical questions. Does routing searches through Cloudflare change the results, the speed or the data an agent gets? And which provider should you pick? I found no independent test of either, so I ran one.

On 53 questions with known answers, Cloudflare returned the same answers as calling Exa and Linkup directly. For Ceramic, it returned longer text and found more answers. It added about 140 to 240 ms for Ceramic and Exa, and made Linkup faster from my location. The providers were far apart. Exa found 51 answers, Linkup 48, and Ceramic 26 when used the way its own docs recommend.

The test#

I ran everything from Mumbai on October 17, 2026, with my own provider API keys stored on the gateway.

• Questions. 40 random questions from SimpleQA, OpenAI’s public set of short questions with known answers. Search providers often report results on it. I added 13 recent questions from GitHub releases published between October 1 and 10, 2026. An example is “Which version of Playwright was released on October 7, 2026?” I kept only releases with exactly one correct answer.
• Routes. Each provider through Cloudflare, and the same provider called directly in the same mode. Cloudflare’s provider page (opens in a new tab) says it uses Exa’s auto search with highlights and Linkup’s fast depth, so the direct calls used those too. I also added Linkup’s standard depth, which Cloudflare doesn’t offer.
• Counting answers. A question counts as answered when the answer appears in the top 10 results’ titles or text. I checked every miss by hand, because results often write an answer differently, like “May 24, 1984” for “24 May 1984”.

This is the request I sent through Cloudflare, with provider set to exa, linkup or ceramic.

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/websearch/ \
  --request POST \
  --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{"query": "Who was appointed Lucasian Professor of Mathematics in 1828?", "provider": "ceramic", "limit": 10, "options": {"gateway": {"id": "default"}}}'

What Cloudflare changes#

The answers didn’t change. Through Cloudflare, Exa and Linkup found exactly the same answers as when I called them directly. The result lists were identical on 45 of 53 Exa questions and 52 of 53 Linkup questions.

Cloudflare drops results that have no title. All 8 Exa differences had the same cause. Exa returned 10 results, and Cloudflare returned 9, without the one whose title was empty. The dropped result was at ranks 2 to 10, and in 6 of the 8 cases it contained the answer. Another result had it too, so no answer was lost here. An agent that reads only the top 3 results could still miss it.

The 1 Linkup difference was a duplicate. Linkup returned the same LinkedIn page twice, once with Hindi characters in the URL and once with them percent-encoded, and Cloudflare kept one.

Ceramic returns more text through Cloudflare. Ceramic’s own API returns up to 3,000 characters per result by default. Through Cloudflare, the median result had 7,893 characters, close to the 8,000 maximum. The result lists also differed on 22 of 52 questions. Through Cloudflare, Ceramic found 18 answers, against 13 directly. For an agent, that’s more context, and more tokens to pay for.

Long queries fail on Ceramic, and Cloudflare doesn’t say why. Cloudflare documents a query limit of 1,024 characters. Ceramic accepts at most 50 words. One SimpleQA question had 59 words. Ceramic’s own API explained the problem, and Cloudflare returned only an error code.

Ceramic is the default provider, so an agent that writes long queries would hit this error without knowing why.

The modes are fixed. Through Cloudflare, Linkup runs only at fast depth. Linkup’s standard depth uses an LLM to plan the search. Called directly, it found 48 answers, the same number as fast. So in this test, the fixed mode lost no answers.

Speed, and a caching trap#

My first speed results said Cloudflare was faster than calling the providers directly. That was wrong, and the reason matters for anyone who benchmarks search APIs.

Each question went to both routes. Exa answered a repeated query much faster the second time, whichever route sent it. When Cloudflare asked first, it took a median of 1,870 ms, and the direct call that followed took 388 ms. When the direct call went first, the numbers flipped. Across all 53 questions, the second call was faster, whichever route sent it. The median fell from 1,804 ms to 456 ms.

A benchmark that sends the same query twice measures the provider’s cache, not the route.

So I ran a separate test with 30 new questions per route, each sent only once.

Cloudflare added about 140 ms to Ceramic and about 240 ms to Exa at the median. For Linkup, the Cloudflare route was faster, by about 620 ms, and its slowest requests were much faster. Cloudflare’s own response reports the provider’s time, and the rest of each request took a median of about 320 to 340 ms. These times are from Mumbai. From another region, the network paths differ.

The providers#

One SimpleQA question was too long for Ceramic, as shown above, and counts as a miss.

Exa and Linkup tied on SimpleQA. The difference was the recent releases. Exa found all 13, while Linkup missed Browser Use, uv and Vite, where it returned only older release pages.

Linkup also returned 16 Grokipedia pages among its 530 results. Grokipedia is an encyclopedia written by AI. For one question, it was Linkup’s top result, above Wikipedia.

Ceramic needs a different approach. Its docs (opens in a new tab) describe it as a lexical search engine that “does not infer meaning, intent, or synonyms”. They recommend that an agent rewrite each question into keyword queries of 2 to 8 words. For anything that isn’t a simple lookup, they recommend 2 or 3 variants. So I gave a local LLM, Qwen 3.5 9B, Ceramic’s own agent instructions and let it write the queries. With 3 keyword queries per question, run in parallel, Ceramic found 25 SimpleQA answers instead of 18. The slowest of the 3 queries took a median of 959 ms.

The rewriting step adds its own risk. For a question about a performance from 1978, one of the 3 queries the model wrote named a different artwork, from 1974. That query found nothing useful, and the other 2 found the answer. Ceramic’s guide says to keep every hard constraint, like the year, so check what your rewriting model sends.

Ceramic found only 1 of the 13 release versions. To check whether it simply hadn’t indexed October’s pages yet, I searched for each exact version, like “Playwright 1.64.0”. No result mentioned any of the 13 versions. But older versions, like “Rust 1.85.0” from early 2025, returned no matching pages either. Exa found that version on 8 of its 10 results.

So version-number lookups are a weak spot for Ceramic in this test, whether the release is new or old. I can’t tell from here whether the cause is how it matches numbers or which pages it indexes.

Cost per answered question tells a different story from the price list. At Cloudflare’s list prices, Exa cost about $0.0073 per answered question and Linkup about $0.0055. Ceramic with 3 queries cost about $0.0015, roughly a fifth of Exa’s, but it answered about half as many questions.

At Extract Summit 2026, Logan Harless (opens in a new tab) described the same trade-off for scraping methods. “What you care about is how much do I have to pay to get the data I want”. For search, the full cost includes the searches that return nothing useful, and the agent steps and tokens spent after them.

Setup notes#

• Billing. With no AI Gateway credits and no stored provider key, every search returns a 402 error. You can load credits, or store each provider’s own API key on the gateway.
• Stored keys. Exa had a built-in entry under Provider Keys. Ceramic and Linkup didn’t, so I added each one as a custom provider, with the slugs ceramic and linkup, and then stored its key. The docs don’t name the slugs, but these matched.
• Data retention. Cloudflare’s provider page lists zero data retention for Ceramic and Linkup, but not for Exa.
• The first call after saving a key returned the same 402 error. Every call after that worked.

How to test a search API before your agent depends on it#

• Send each query to each route only once, or run the routes in a random order. Repeated queries measure the provider’s cache.
• Use the provider the way its docs say. A keyword engine needs keyword queries.
• Include questions about the last few weeks, not only a benchmark set. On an a16z podcast (opens in a new tab), Exa’s CEO, Will Bryk, said, “There aren’t that many standard third party retrieval evals and they’ve been … bench maxed”.
• Check every miss by hand before you compare providers.
• Count what your agent receives, like empty titles and text length, not only whether results come back.
• Measure cost per answered question.

If you want the questions, the results or the scripts, email me and I’ll share them.