Skip to main content

Anti-Bot Bypasses

The ScrapeOps Proxy API Aggregator enables you to bypass most anti-bot systems that are designed to prevent developers from scraping data from websites.

The default Proxy API is able to bypass most common techniques websites use to detect and block scrapers, however, some websites use more advanced anti-scraping techniques or 3rd party anti-bot solutions like Cloudflare, Incapsula, and PerimeterX to block scraping.

These more advanced techniques often involve TLS fingerprinting, browser fingerprinting, and JS challenges, to detect and block scrapers. To bypass these more advanced techniques you will have to activate one of our in-built bypasses by adding the bypass query parameter to your request.

BypassAPI CreditsDescription
generic_level_110The generic level 1 anti-bot bypass is designed to bypass simple anti-bots.
generic_level_235Generic anti-bot bypass that is designed to bypass most anti-bots. On large plans the credit multiple will be increased to maintain a flat rate of $3.50 per thousand requests.
generic_level_350Generic anti-bot bypass that is designed to bypass complex anti-bot systems. On large plans the credit multiple will be increased to maintain a flat rate of $6.50 per thousand requests.
generic_level_485Generic anti-bot bypass that is designed to bypass the most extreme anti-bot systems. On large plans the credit multiple will be increased to maintain a flat rate of $8 per thousand requests.
cloudflare_level_110Use to bypass Cloudflare protected sites with low security settings enabled.
cloudflare_level_235Use to bypass Cloudflare protected sites with medium security settings enabled. On large plans the credit multiple will be increased to maintain a flat rate of $3.50 per thousand requests.
cloudflare_level_350Use to bypass Cloudflare protected sites with high security settings enabled. On large plans the credit multiple will be increased to maintain a flat rate of $4 per thousand requests.
incapsula35Use to bypass Incapsula protected sites. On large plans the credit multiple will be increased to maintain a flat rate of $3.50 per thousand requests.
perimeterx35Use to bypass PerimeterX protected sites. On large plans the credit multiple will be increased to maintain a flat rate of $3.50 per thousand requests.
datadome50Use to bypass DataDome protected sites. On large plans the credit multiple will be increased to maintain a flat rate of $4 per thousand requests.

As always, you only get charged for successful requests. So if the bypass doesn't work then you won't get charged.

How To Use Anti-Bot Bypasses

These bypasses rely on using headless browsers to simulate real user fingerprints and behaviours to successfully scrape protected websites. As a result, they cost more than using normal HTTP requests (1 API Credit).

To keep your costs as low as possible, you should first check to see if you can get good results without any bypass activated on your request.

Then, if the performance isn't good enough for your use case you can try a anti-bot specific bypass or one of our generic bypasses. Different bypasses use different browser, proxy, and bypass configurations so if one bypass doesn't work then another one might.


Activate Anti-Bot Bypass

For example, to scrape a Cloudflare protected website, you can enable our Cloudflare bypass by adding bypass=cloudflare_level_1 to your request.


curl -k "https://proxy.scrapeops.io/v1/?api_key=YOUR_API_KEY&url=http://example.com/&bypass=cloudflare_level_1"

Logging In & Anti-Bot Bypasses

We don't support logging into websites whilst also using the bypass functionality as it makes the requests much more detectable and can degrade the performance of the bypasses for other users.


Cloudflare Bypasses

Cloudflare is the most common anti-bot system being used by websites today, and bypassing it depends on which security settings the website has enabled.

To combat this, we offer 3 different Cloudflare bypasses designed to solve the Cloudflare challenges at each security level.

Security LevelBypassAPI CreditsDescription
Lowcloudflare_level_110Use to bypass Cloudflare protected sites with low security settings enabled.
Mediumcloudflare_level_235Use to bypass Cloudflare protected sites with medium security settings enabled. On large plans the credit multiple will be increased to maintain a flat rate of $3.50 per thousand requests.
Highcloudflare_level_350Use to bypass Cloudflare protected sites with high security settings enabled. On large plans the credit multiple will be increased to maintain a flat rate of $6.50 per thousand requests.