Skip to main content

BrowserScan Integration

How to Integrate ScrapeOps Residential Proxies with BrowserScan

BrowserScan is an awesome tool for all sorts of browser related things. From fingerprinting, to speed testing, to OS and browser detection, they're pretty much a one-stop shop when it comes to testing your internet connection.

Today, we're going to integrate Google Chrome with ScrapeOps Residential Proxies. You're going to learn how to use our Residential Proxy Aggregator with Chrome Browser and test it using Browserscan!

Introduction​

What is a Proxy?

A proxy connection is like a go-between. Without getting too technical, browsers communicate using HTTP (Hypertext Transfer Protocol). When you visit a site with your browser, you're making an HTTP request to that site.

When using a proxy, your browser sends the request to the proxy server instead. The proxy server reads your request, and then forwards it to the site you're visiting. Once it receives a valid response, it sends that response back to your browser.

With ScrapeOps Residential Proxies, not only do you get to connect using a proxy, you get to use a connection located on an actual residential internet connection. This allows you to completely blend in with the rest of the internet while still protecting your privacy, location, and browser security.

Browserscan gives us all sorts of cool internet tools. In this tutorial specifically, we're going to use Browserscan to verify our proxy connection. When you visit Browserscan, it displays your IP address, browser type, operating system, physical location, and your ISP (internet service provider).

Prerequisites​

Our requirements here are pretty simple. To follow along, you need:

  1. Chrome Browser,
  2. A computer, and
  3. A ScrapeOps account.

Chrome actually uses the proxy settings on your host machine. If you set your computer to use a proxy connection, Chrome follows those rules and uses that proxy connection. You need a ScrapeOps account in order to authenticate your proxy connection... you're using our bandwidth, so we need to make sure that you actually have access to our services.

You're probably already reading this article with Google Chrome, but if you're not, you can download Chrome here. Follow the setup instructions for your OS.

After it's installed feel free to navigate to any site on the web and test it out. Go to Browserscan and wait for the page to load. Once it's finished loading, you should see something similar to the screenshot below. As you can see, Browserscan knows pretty much everything about us!

Browserscan: No Proxy

If you don't have one already, you need to create a ScrapeOps account. Head on over to our registration page. You'll immediately get a free trial with our Proxy Aggregator and our Residential Proxy Aggregator. Fill in the details below, and you should be good to go!

ScrapeOps: Registration

Once you've got an account, you'll get access to 1,000 free API credits with our Proxy Aggregator and 500MB of free data with our Residential Proxy Aggregator.

Make sure you to save your API key. You'll need this API key in order to sign in to the Residential Proxy.

Setting Up ScrapeOps Proxies​

Now it's time to get started setting up your proxy. This tutorial focuses on Windows, but it's doable with both Windows and Mac as well. To start, open up Chrome and click the Settings icon. It's pretty easy to find... just three dots in the upper right hand corner of the screen.

Chrome Settings Icon

After clicking it, a dropdown appears. If you go all the way to the bottom of the dropdown options, you'll see a Settings option. Click on it.

Google Chrome: Settings Dropdown

In the search bar, enter the word, "proxy." You'll see several options pop up. Click on Open your computer's proxy settings. This will bring up the proxy options for your specific system.

Google Chrome: Proxy Settings

On Windows, you should see a popup similar to the one below. In the Use a proxy server section, click on Set up. If you're on Linux, you can find instructions for Ubuntu here. You can find instructions for Mac here.

Windows: Proxy Settings

Now, you should see something similar to what you see below.

In the Proxy IP address box, enter http://residential-proxy.scrapeops.io. This is the URL for our proxy connection. In the Port box, enter 8181. This is the port we use to listen to incoming requests for the Residential Proxy.

Windows: Proxy Setup

When you open up a new instance of Chrome, you'll be prompted to sign into ScrapeOps. Your Username will be scrapeops. Remember the API key we told you to save? Enter that into the Password box.

Google Chrome: Sign into proxy

Testing the Integration​

We're going to test our integration with Browserscan. Using your newly proxied browser, head on over to Browserscan. You'll need to wait for the page to load. Once the page has loaded, you'll see something similar to what you see in the screenshot below.

Browserscan with Residential Proxy

There are a ton of things that have changed. Most obviously, our IP address. If you look a little more closely though, here are some other things that have changed.

  • IP Timezone: This was originally America/Detroit. Now, it's Asia/Dhaka.
  • Location: Our latitude and longitude are completely different.
  • Postal Code: This changed from 48185 to 9100.
  • ISP: Our service provider changed from WildOpenWest Finance LLC to Ramdia Online.
  • Proxy: Originally, this was No. It has changed to Yes. Browserscan hasn't discovered our actual location, but they have figured out that we're using a proxy. Using a proxy is not inherently bad and there's nothing wrong with this.

Configuring ScrapeOps Residential Proxy Integration​

In these next few sections, we're going to tweak our configuration settings.

Earlier, our location was displayed as Bangladesh. With ScrapeOps Residential Proxies, we can actually control our location. If you wish to show up in the US, we can route you through a server in the US. If you want to show up in the UK, or any of our other numerous countries, we can do that as well.

We'll also take a look at Static Proxies, also known as Sticky Sessions. With a normal proxy session, our proxies are not static. When you go to a new page, we route you through a new proxy and your IP address will change. With a Sticky Session, we keep you connected to the same IP address on the same machine.

These settings are actually controlled through our Username. Depending on what you want to tweak, you add a flag to the Username box. Our servers then read, interpret, and execute your browser actions using the custom flag.

Geotargeting​

As we mentioned above, we can tweak our proxy connection using specific flags. For geotargeting, we append our Username with the country flag. In the example below, we set our country to uk. Our full Username is scrapeops.country=uk. Your password is still your API key.

Google Chrome Geotargeting Sign in

Here are the test results from Browserscan,

Browserscan: Geotargeting Results

Here are some key things to pay attention to:

  • IP Time Zone: Europe/London
  • Postal Code: E8
  • ISP: Vodafone UK

Our geotargeting is working effectively. We are showing up in the UK... just like we wanted to!

You can view some of our supported locations here. We're adding new locations all the time.

If your target country isn't on the list, feel free to look up its country code and give it a try!

Using Static Proxies​

Sticky Sessions (Static Proxies) can be a must when you're using a proxy with your browser. Have you ever visited a site that made you log in again because your location changed? Sticky Sessions solve this problem. With a Sticky Session, we reuse the IP address connected to your session. This allows you to stay logged in while using the proxy connection!

To create a Sticky Session we use the sticky_session flag. You can give your session any number between 0 and 10000. Sessions remain active for up to 10 minutes. Once the 10 minute mark has been hit, we give you a new IP address, regardless of your Sticky Session.

In the example below, we set our ID to 7. Our full Username is scrapeops.sticky_session=7. Once again, we use our API key as our Password. Guard your API key with your life.

Google Chrome Sticky Sign In

We'll head on over to Browserscan and view our results. You can see them below.

Sticky Session: 1st Result

Next, we're going to refresh the page. This creates a new HTTP request, without a Sticky Session, ScrapeOps would automatically give us a new IP address.

Sticky Session: 2nd Result

As you can see above, everything is the same. Our proxy connection reused the IP address. This is incredibly useful when you're looking to stay logged in.

You can view our documentation on Sticky Sessions here.

Common Issues and Troubleshooting​

When using Chrome, if you encounter an issue with the proxy, Chrome just switches back to non-proxied mode and doesn't tell you.

Browserscan is the perfect tool to identify issues with your connection. With Browserscan, your IP address and location information are displayed. If you are encountering errors, or if you are continually prompted to sign in, Browserscan is the first place you should go in order to debug.

No Proxy Connection​

If Browserscan is detecting your actual location and IP address, the connection is not working at all. Open up your machine's proxy settings and double-check the IP address and Port.

Windows Proxy Settings

The correct connection settings are:

  • Proxy IP address: residential-proxy.scrapeops.io
  • Port: 8181

Site Can't Be Reached​

Google Chrome: Site Can't Be Reached

If your site is unreachable after creating the proxy connection, you are failing to sign in, or you are out of API bandwidth. You can check your bandwidth usage here.

ScrapeOps Dashboard

If you still have bandwidth and the proxy isn't working, you probably aren't signed in correctly. Double-check your Username and Password (API key). Something was probably entered incorrectly.

Google Chrome: Proxy Sign In

Conclusion​

You've successfully navigated the world of proxy integration with Chrome and you now know how to test your proxy using Browserscan. You can now disguise your browser location and stay logged in.

Go ahead and browse the web with anonymity, geotargeting and enhanced security. It might seem daunting, but proxy integration is actually pretty easy once you know how to do it.

Whether you're scraping the web, or you just want to watch a TV show from another country, ScrapeOps has you covered. Use your new knowledge to take control of your online presence and digital identity.

Your browsing history is your business, and now you know how to protect it.

More Python Web Scraping Guides​

Whether you're scraping the web, or just browsing anonymously, here at ScrapeOps, we've got you covered. Our blog is filled to the brim with learning materials.

If you're interested in learning more about Residential Proxies, check out the links below!