How to Integrate ScrapeOps Residential Proxies with CapSolver
CapSolver is an amazing new browser extension. With the power of AI, CapSolver automatically recognizes CAPTCHAs on your page and solves them for you. It's also incredibly cheap. To use CapSolver, you only pay $0.40 per 1,000 CAPTCHAs. Think of how often you run into CAPTCHAs. You can eliminate CAPTCHAs from your life for $6.00 per year or less!
Today, we'll go through and test it out. We'll also show you how to integrate our Residential Proxies with it. This way, you can solve your CAPTCHAs with the utmost privacy. Follow along and you'll hopefully never have to deal with CAPTCHAs again.
- Introduction
- Prerequisites
- Setting Up CapSolver With ScrapeOps Proxies
- Testing the Integration
- Configuring CapSolver for Proxy Integration
- Common Issues and Troubleshooting
- Conclusion
- More Python Web Scraping Guides
Introduction
What is CapSolver?
CAPTCHAs have been getting more and more time consuming over the years. 2captcha has been the default CAPTCHA solver for years. 2captcha depends on human workers who get paid next to nothing. While 2captcha gives us the ability to solve CAPTCHAs, it leaves a lot to be desired both morally and in its implementation.
CapSolver is a browser extension. This allows you to use it with your regular browser. When you navigate to a site with a CAPTCHA, CapSolver automatically recognizes CAPTCHAs on the page and immediately begins solving them.
No more clicking through unending pictures. No more reading obscure text. Just simple, stress free browsing.
What are ScrapeOps Residential Proxies?
A proxy acts as a middleman between your browser and the rest of the internet. When you visit a site, your browser sends a GET request to the site and then receives a response back from the site.
The browser reads the response and renders the page for you to view. With proxies, your request goes to the proxy server. Then it routes that request through a proxy and waits for a response. Once the response comes back, the proxy server sends it back to your browser.
Through this process, your browser can theoretically visit any page on the internet without actually going to that page. This is very useful because it gives you anonymity, location control, and enhanced security. The proxy server is the one interacting with any potentially dangerous site, not your browser.
Prerequisites
To follow along, you need Chrome. Since you're reading this article from inside the ScrapeOps blog, we're going to assume you've already got a web browser, and you're perfectly capable of installing Chrome if you need.
You'll also need both a CapSolver account and a ScrapeOps account (if you decide to use proxies along with the extension).
Creating a CapSolver Account
We'll get started by signing up for CapSolver.
- As you can see below, they offer 1,000 CAPTCHA solves for only $0.40. Go ahead and click Sign in.
- Next, you'll be given the option to create an account or log in with your existing one. Once you're logged in, you'll be taken to the account dashboard as you can see below. Click on Add funds.
- You get redirected to a page where you can add funds to your account. The minimum charge is $6. It looks like I won't need to solve any CAPTCHAs for a few years! Once you've added your funds you'll be good to go.
- When you're done adding your payment information, go ahead and click Submit order when you're ready.
- Make sure to copy your API key from the dashboard. You need this key to connect the browser extension to your CapSolver account. Put this key somewhere safe.
Creating a ScrapeOps Account
To use our proxies, you'll need to create an account. You can head on over to our registration page and get a free trial here.
We don't ask for much, just that you submit some basic information and complete a CAPTCHA. Hopefully, this will be the last CAPTCHA you ever solve.
Once you've registered, you'll be taken to our dashboard.
On the left hand side of the page, click Proxy Aggregator, and you should wind up with a screen that looks like the one below. You can see some basic account metrics and your usage information.
Click on Request Builder to reveal your API key. Similar to your CapSolver API key, put this key somewhere safe. We'll use this to connect the CapSolver extension to your ScrapeOps account. This is how you use your proxy connection.
Setting Up CapSolver With ScrapeOps Proxies
We've got everything we need. Now, it's time to set it all up. We'll need to download the CapSolver extension and then add our proxy connection to it. Follow the steps below to be up and running in minutes!
You can get the Chrome extension here. If you're ready, click Add to Chrome.
Next, the CapSolver extension. In the box titled API key, add your CapSolver API key.
If you scroll further down, you should see a Proxy section. Toggle it on.
Now, there are a few things we need to enter to create our proxy connection. In the IP/Host box, enter residential-proxy.scrapeops.io. Enter 8181 in the Port section. For Login, this is the username for the connection, in this case scrapeops. In the Password box, enter the ScrapeOps API key that you saved earlier.
To check if CapSolver is working, head over to this site. This is Google's sample ReCAPTCHA. You can do a sample ReCAPTCHA here. At first, your CAPTCHA will look like what you see below.
CapSolver will automatically recognize the CAPTCHA and solve it. As you can see in the image below, the CAPTCHA has been solved.
Testing the Integration
Let's give this thing a real test. Here, we'll solve some CAPTCHAs with our new tool. Since we want more intensive testing, we'll use NopeCHA. This site gives us all sorts of CAPTCHAs to try out our new favorite tool.
You can test it yourself on an AWS CAPTCHA here. You can view a video of it below.
That one was impressive, but let's give it something a little more challenging. In the next video, it solves 3 CAPTCHAs simultaneously.
As you probably noticed in the videos above, this is an incredibly powerful tool. CapSolver easily solves even the most difficult challenges.
Configuring CapSolver for Proxy Integration
We already know how to create a basic proxy connection. With ScrapeOps Residential Proxies, we actually get some really cool ways to fine-tune our connection as well.
We can use geotargeting to pick a specific location and we can also use static proxies (sticky sessions) to maintain our connection through a single IP address.
To tweak these settings, all we need to do is change our username. Follow along below, and you'll be able to tweak your connection however you want!
Geotargeting
Geotargeting gives us the ability to choose our location. We can do this by appending the country flag to our username followed by a country code. If we wish to appear in the US, our username would be scrapeops.country=us.
You can view our documentation on geotargeting here.
Here's what your username will look like in CapSolver.
Using Static Proxies
We can use a static proxy (sticky session) pretty much the same way. Instead of adding the country flag here, we're going to add the sticky_session flag. Our sticky sessions are good for 10 minute increments. Each session requires a number. You can use anything between 0 and 10000.
In the example below, we create a sticky session and give it the number, 7. Our full username is scrapeops.sticky_session=7.
Common Issues and Troubleshooting
CAPTCHAs Not Solving
If CapSolver isn't solving CAPTCHAs for you, this is a major problem. If your CAPTCHAs aren't being solved, double-check your API key. If your API key is correct, double-check your balance, you might be out of credits.
If everything is alright with your account and your API key is correct, take a look at your CAPTCHA settings. Make sure Manual Solving is turned off.
Proxy Connection Not Working
If your proxy connection isn't working, take a look at your username and your API key (in the browser extension, your API key is in the Password field).
If everything is correct, double-check your ScrapeOps account at the account dashboard. You probably need to add credits to your ScrapeOps account.
If you've tried all the above steps and your proxy isn't working, you click the Help section on the dashboard to contact us.
Conclusion
In conclusion, CapSolver is a truly amazing piece of technology. It's brand new to the tech scene and this is really going to disrupt security on the web. As you probably saw in our testing section, CapSolver was able to prove that it's human and broke through even the most stringent of CAPTCHAs we threw at it.
When you combine CapSolver with our Residential Proxies, you get all the strength of CapSolver with the bonus features of added privacy, geotargeting, and sticky sessions.
In the next few years, technologies like CapSolver will completely upend the tech world and how we spot bots.
More Python Web Scraping Guides
At ScrapeOps, we love all things web scraping. We've even written several playbooks on it.
You can take a look at our Python Web Scraping Playbook. We also have playbooks for NodeJS, Selenium, Puppeteer and Playwright.
If you want a taste of some of these articles, take a look at the links below.