Why a 200 response can still be a block

A status code of 200 tells you the server answered. It doesn’t tell you the answer contains the data you asked for. Some blocked requests return a 200, and a scraper that trusts the status code saves them as if they were real pages.

This post shows the forms these hidden blocks take, and a short check that detects them.

A real example

I requested Amazon’s homepage with Python’s httpx library and a descriptive user agent. The response had a status of 200, but it was 3,781 bytes long and its only text was this.

Response text
Amazon.com Click the button below to continue shopping Continue shopping Conditions of Use Privacy Policy

A real Amazon homepage is hundreds of kilobytes. This page is a challenge page that asks a person to click a button, and it contains none of the words that usually signal a block, like “CAPTCHA” or “robot”.

The same homepage gave different results in other runs. My 1,000-site test ran a day before the request above and used a Chrome user agent. In that test, 9 Amazon stores returned a 202 challenge page, and 3 returned an empty page with a status of 200. When I repeated the Chrome user agent request for Amazon.com, it returned a 202, sometimes with an empty body and sometimes with a short challenge script. The block changes from request to request, so a scraper can’t rely on one signal to detect it.

The forms a hidden block takes

A challenge page. The page asks the visitor to prove they’re human, and it often needs JavaScript to continue. Cloudflare, AWS WAF, DataDome and HUMAN each have their own version. Most arrive with a 403 or a 202, but some arrive with a 200.

A success code that isn’t 200. A 202 means “accepted”, and a scraper may treat it as success. Amazon’s block uses it, with a page that shows no text.

A real page with less data. The Scrape.do team described this in a Reddit AMA. A response “looks successful but the content is quietly degraded”, for example “if a page returns 5 records instead of 10”.

A real page with wrong data. At ZyteCon 2026, Zyte’s data team described honeypots, where “you get a 200 but it’s the wrong data”. Some sites serve false prices or details to clients they think are bots.

A block from a scraping API. At Prague Crawl 2026, Logan Harless said that well-known unblocking services sometimes return a 200 “when really what they’re serving you is a CAPTCHA”. A 200 from a vendor needs the same check as a 200 from a site.

A check that detects them

The most reliable check asks whether the response contains the data you expected. Known block markers find the common challenge pages, and a count of the items you need detects most of the rest.

check_response.py
import httpx
from bs4 import BeautifulSoup
BLOCK_SIGNS = [
"just a moment", # Cloudflare challenge
"_cf_chl_opt", # Cloudflare challenge page
"verify that you're not a robot", # AWS WAF challenge
"captcha-delivery", # DataDome
"px-captcha", # HUMAN (PerimeterX)
]
def check(response, selector, expected):
"""Check whether a response holds the data you asked for, instead of trusting only the status code."""
if response.status_code != 200:
return f"failed: status {response.status_code}, {len(response.content)} bytes"
text = response.text.lower()
for sign in BLOCK_SIGNS:
if sign in text:
return f"blocked: challenge page ({sign!r}) with status 200"
found = len(BeautifulSoup(response.text, "html.parser").select(selector))
if found < expected:
return f"incomplete: {found} of {expected} expected items"
return f"ok: {found} items"
headers = {"User-Agent": "MyScraper/1.0 (you@example.com) python-httpx"}
pages = [
("https://news.ycombinator.com/", "tr.athing", 30),
("https://www.twitch.tv/directory", "a[data-a-target='tw-box-art-card-link']", 20),
("https://www.amazon.com/", "#nav-logo", 1),
]
for url, selector, expected in pages:
response = httpx.get(url, headers=headers, follow_redirects=True)
print(url, "->", check(response, selector, expected))
Output
https://news.ycombinator.com/ -> ok: 30 items
https://www.twitch.tv/directory -> incomplete: 0 of 20 expected items
https://www.amazon.com/ -> incomplete: 0 of 1 expected items

I ran this twice from Mumbai, on a home connection, on October 2, 2026, with httpx 0.28, and got the same result both times. All 3 pages returned a status of 200. Only Hacker News had the data.

The Twitch page isn’t a block. Its game cards load with JavaScript, so the raw HTML has none. A browser showed 31 of them. The check can’t tell you why the data is missing, but it stops you from saving an empty page as if it contained real data.

The Amazon page passed the block-marker test and failed the item count. The highlighted lines in the code do this item count. Counting the items you expect detects blocks that aren’t on any list of markers.

What to check in production

  1. Count what you need. Check for the number of items, a price, or a field you know the page always has.
  2. Monitor the response size. A product page that’s suddenly 4 KB instead of 400 KB is rarely a real page.
  3. Keep the blocked pages. Save a sample of failed responses. They show which system blocked you and how its page changes over time.
  4. Compare values over time. Honeypot data can pass every structural check. Prices that change sharply, or values that differ from a second source, should be checked by hand.
  5. Track success by data, not status. Report the percentage of requests that returned usable data. At ZyteCon 2026, Zyte’s CTO said that, in Zyte’s experience, “one in 10 requests never delivers usable data at all”.
  6. Treat a block as information. It tells you how the site handles automated traffic. Lower your request rate, check whether the site offers an API, and change your approach instead of retrying the same request.

Doccraft writes research and tutorials like this for web data and AI infrastructure companies.