ScraperAPI Review: The Fix for "My Scraper Worked for Two Days"

This one is for a specific person, so let me describe them first.

You built your own scraper. Maybe you wrote it, maybe you had Claude or Cursor write it. It worked beautifully against 50 pages. Then it started returning empty results, or CAPTCHAs, or 403s, and you realised the hard part of scraping was never the parsing.

ScraperAPI is the layer that fixes that, and it does not ask you to throw away the scraper you already have.

What it actually does

You send it a URL. It returns the HTML.

That is the whole interface. Behind that one call it rotates residential and mobile proxies, renders JavaScript in a real browser, solves CAPTCHAs, rotates user agents between desktop and mobile, retries automatically on failure, and manages sessions if you need to stay consistent across requests.

Your code stops being a scraper and becomes a parser. You keep the logic that knows what a business listing looks like on your target site, and you hand off every piece of adversarial infrastructure to someone whose full-time job is maintaining it.

That division is the reason this tool exists. Parsing is a day of work. Staying unblocked is a permanent job.

Why this fits AI-built directories in particular

There is a pattern showing up constantly right now. Someone builds a directory with Claude, Cursor or Lovable. The build goes fast, because generating a Next.js app with a listings schema is exactly what these models are good at.

Then they ask the model to write a scraper to fill it, and that goes fast too. And then it stops working, because the model wrote correct parsing code against a site that actively defends itself, and no amount of prompting fixes an IP ban.

The model was never wrong. The missing piece is infrastructure, not code.

ScraperAPI slots into that gap with a one-line change. The scraper you already have keeps its logic and routes its requests through the API instead of directly. If you are building your directory by writing code rather than clicking through a builder, this is the tool that makes your own scraper viable.

Pricing

Enterprise starts above 22,000,000 credits with custom pricing. Annual billing takes 10% off. The trial runs 7 days with 5,000 credits and no credit card.

Every plan, including Hobby, includes the full feature set: JS rendering, premium residential and mobile proxies, JSON auto-parsing, rotating proxy pools, custom headers, CAPTCHA and anti-bot handling, session support, automatic retries, unlimited bandwidth and a 99.9% uptime guarantee. You are buying volume and concurrency as you move up, not capability.

The credit model, which is where budgets go wrong

A credit is not a request. This is the single most misread thing about ScraperAPI pricing.

A plain page on an undefended site costs a small number of credits. A page that needs JS rendering costs more. A heavily defended domain costs more again, and some well-known sites carry their own specific multiplier. ScraperAPI publishes a domain cost list, and reading it before you pick a plan is twenty minutes well spent.

The practical consequence: 100,000 credits on the Hobby plan is not 100,000 pages. Depending on your target it might be 20,000. Run the trial against your actual target and divide, rather than trusting the headline number.

Where it beats the alternatives

Against Apify: you keep control. Apify hands you someone else's Actor and its output shape. ScraperAPI hands you raw HTML and you decide what it means. If your directory has an unusual schema, or your source is obscure, that control is worth having.

Against Octoparse and Browse AI: it scales and it does not involve a browser you have to babysit. 100,000 credits on the entry plan is a different order of magnitude from a no-code tool's row allowance.

Against writing your own proxy layer: this is the honest comparison, and it is not close. Residential proxies, CAPTCHA solving and retry logic are each a project. You are renting all three for $49.

Where it does not fit

You have to write code. There is no interface, no visual builder, no point-and-click. If that sentence is a problem, Octoparse is the tool you want.

There is no free tier, only a 7-day trial. Every other tool in this category has some permanent free allowance. This one does not, so plan your evaluation for that week.

Geotargeting is limited to US and EU on the two cheapest plans. If your directory covers a region outside those, you are starting at $299 rather than $49, which changes the calculation considerably.

And it does not parse for you. It returns HTML. Turning that into structured listings is still your job, which is the trade you are making for control.

Common questions

Do I need a proxy provider as well?

No. Proxies are included on every plan, which is the main thing you are paying for.

Will it get me Google Maps data?

It can fetch pages, but for Maps specifically you want a tool that already knows the output shape. Apify's Maps Actor or SerpApi will get you structured places far faster than parsing Maps HTML yourself.

Can I use it without writing code?

Partly. It is a plain HTTP endpoint, so Make, n8n and Zapier can call it. You will still need somewhere to put the parsing logic, so this remains a developer-shaped tool.

How do I stop burning credits by accident?

Cache every response while you are developing. The most common way people waste a trial is re-fetching the same ten pages while debugging their parser. Pull once, save the HTML locally, iterate against the file.

Is 5,000 trial credits enough to evaluate it?

Enough to answer the only question that matters, which is whether it gets through to your target site. Test the hardest source you have on day one.

Verdict

ScraperAPI is not a scraper. It is the part of a scraper that is genuinely hard to build, sold separately.

That makes it the wrong first purchase for most directory builders, and exactly the right one for a particular person: someone building their own thing, generating their own scraping code, and running into the wall that no amount of better code gets you past.

If you have never been blocked, you do not need this yet. The day you get blocked, it is a one-line change and $49.