What Oxylabs learned from its AI parser agent

Oxylabs built an AI agent to fix its parsers, and at first its numbers looked good. It made 251 changes in under a year for $847. Then its engineers quietly stopped using it.

An Oxylabs engineering leader told the story at OxyCon 2026 (opens in a new tab), along with a second agent that did succeed. The talk gave real numbers for both projects.

The cost of “just use AI”

The talk started with a request the speaker’s team often heard from sales and product managers. They asked the team to “just use AI”. He answered with Oxylabs’ own numbers.

The company scrapes about 7 billion pages a day. Parsing only 10% of them with an LLM, at about 1 cent a page for the cheapest model, would cost about $7 million a day. At 5 cents a page, it would cost about $35 million a day with a mid-priced model.

So the question wasn’t whether to use AI. It was where.

The parser agent

The first agent handled parsing tasks, which the speaker said nobody on the team wanted to do anymore. It took a Jira ticket, changed the code in the parsing repository, and sent the change to an engineer to approve and deploy.

At first, it looked like a success. The team set an OKR for it, and in Q4, 51% of parser changes came from the agent.

When the OKR ended, the agent’s share fell to 14%. As the speaker noted, this happened “while the models were improving”.

The detailed numbers showed why.

MeasureAgentEngineers
Merge requests that reached production59%87%
Changes that passed CI tests76%91%
Median time to production7 days20 hours

In addition, engineers made 3.6 changes on average to make each agent change ready for production. The agent created more work than it saved. In the speaker’s words, “Nobody quit their jobs… They just quietly stopped calling the agent”.

In the speaker’s own review, the team had optimized for quantity, not quality. It counted merge requests and did not measure the work needed to get each one into production.

The agent that worked

The second project started with a different problem. Scraping engineers had to make fast decisions from a huge amount of data, spread across many dashboards. Oxylabs’ systems were receiving 1.3 million metrics a second, anti-bot systems changed all the time, and nobody had documented how they work.

The team built a natural-language interface to its production systems in 3 steps.

  1. Access through a command line. One CLI gave the agent access to production and monitoring systems and to the strategy engine, with limits on how many jobs it could run.
  2. Written team knowledge. The team wrote its working methods as “skills” for the agent. This also forced the team to document its methods.
  3. Guardrails. For example, the agent must not draw conclusions from too few jobs, and tests need a 95% significance level.

In one example, the speaker told the agent that a target didn’t work and asked for the reason. It checked the monitoring systems, collected sample HTML, found a block page that was missing from the site’s strategy, added it, and ran a test.

These are the results the speaker reported.

  • 500 downloads of the tool since April 2026.
  • Up to 49 daily active users in one week.
  • The median time to fix a scraping problem fell from 5 hours to 2 hours, for the problems the team could solve.

What made the second agent work

The speaker’s conclusion was that “enhancing and not replacing engineers is actually a much better strategy”. The details show 3 more specific differences.

The first agent did the work, and the second one helped engineers make decisions. The parser agent produced code that engineers then had to check and fix. The second agent worked with an engineer step by step, collecting evidence and testing fixes.

The second agent followed the team’s standards. The skills and guardrails made it work the way the team works. The speaker added that the same guardrails raised the standard for the engineers who used it.

The first project measured activity, and the second measured outcomes. Merge request counts looked good while the real cost stayed hidden. The second project reported time to fix a problem, which is the result engineers care about.

4 checks for an agent project

This talk matches a pattern across the 2026 conferences, which I covered in a longer post. Teams get the most from AI when it builds, repairs and diagnoses. They get the least when it tries to replace an engineer.

If you’re adding an agent to a scraping team, use these 4 checks from this story.

  1. Measure the full cost of each change, including review time and the fixes that follow, not only how many changes the agent makes.
  2. Monitor usage after the launch period ends. The drop was silent. The agent’s share of the work fell once no one was measuring it.
  3. Document your team’s knowledge first. An agent with your methods and limits is more useful, and the written knowledge can also help new engineers.
  4. Give the agent evidence to collect, not decisions to make. Research and testing are good tasks for an agent, and an engineer makes the final decision.

Doccraft writes research and tutorials like this for web data and AI infrastructure companies.