The ScrapeOps Proxy API enables you to bypass most anti-bot systems that are designed to prevent developers from scraping data from websites.
The default Proxy API is able to bypass most common techniques websites use to detect and block scrapers, however, some websites use more advanced anti-scraping techniques or 3rd party anti-bot solutions like Cloudflare, Incapsula, and PerimeterX to block scraping.
These more advanced techniques often involve TLS fingerprinting, browser fingerprinting, and JS challenges, to detect and block scrapers. To bypass these more advanced techniques you will have to activate one of our in-built bypasses by adding the
bypass query parameter to your request.
|10||The generic level 1 anti-bot bypass is designed to bypass simple anti-bots.|
|30||Generic anti-bot bypass that is designed to bypass most anti-bots.|
|50||Generic anti-bot bypass that is designed to bypass the most complex anti-bot systems.|
|10||Use to bypass Cloudflare protected sites.|
|30||Use to bypass Incapsula protected sites.|
|30||Use to bypass PerimeterX protected sites.|
|30||Use to bypass DataDome protected sites.|
As always, you only get charged for successful requests. So if the bypass doesn't work then you won't get charged.
How To Use Anti-Bot Bypasses
These bypasses rely on using headless browsers to simulate real user fingerprints and behaviours to successfully scrape protected websites. As a result, they cost more than using normal HTTP requests (1 API Credit).
To keep your costs as low as possible, you should first check to see if you can get good results without any bypass activated on your request.
Then, if the performance isn't good enough for your use case you can try a anti-bot specific bypass or one of our generic bypasses. Different bypasses use different browser, proxy, and bypass configurations so if one bypass doesn't work then another one might.
Activate Anti-Bot Bypass
For example, to scrape a Cloudflare protected website, you can enable our Cloudflare bypass by adding
bypass=cloudflare to your request.