Where Do Residential & Mobile Proxies Come From?
Using residential & mobile proxies have become very popular for web scraping, botting, and automation. But where do residential & mobile proxy providers get their proxies from?
In this article, we will uncover the 5 main ways proxy providers get their residential & mobile proxies, and you can see why this business model can be insanely profitable!
- Free VPNs
- Proxy SDKs In Apps & Chrome Extensions
- Paid Bandwidth Products
- Device Proxy Farms
- Proxy Reselling
We will also explore the ethics of each method, and show you how some of the biggest proxy providers get their proxies.
Need help scraping the web?
Then check out ScrapeOps, the complete toolkit for web scraping.
Free VPNs
The original way proxy providers started getting residential & mobile proxies was by offering free VPN products to average consumers in exchange for their device being used as a proxy exit node.
Here, a user installs a VPN on their laptop or smartphone giving them to change their IP address when accessing websites. In exchange, the company providing the VPN is able to use the users device as a exit node for their proxy network.
This model was pioneered by Bright Data (then Luminati), with their sister company HolaVPN. HolaVPN offered users a free VPN in exchange for their device being used by Luminati as a proxy.
When the HolaVPN first launched, the fact that you were giving permission to for Hola/Luminati to use your device as a proxy node wasn't 100% clear so there has been some bad press about this method of acquiring residential & mobile proxies.
However, in recent years companies have become much more transparent about why they are offering free VPNs and what you are giving in exchange. In 2021, Bright Data launched their new VPN, BrightVPN, which clearly says why their VPN is free to use.
This business model can be insanely profitable as often times, the proxy provider pays nothing to the user for the proxy traffic but charges developers $2-40 per GB of traffic they send through the proxy network.
Proxy SDKs In Apps & Chrome Extensions
One of the leaser known ways for proxy providers to acquire large residential & mobile proxy networks is through the use of App & Chrome extension developers embedding proxy SDKs into their products.
Here, instead of an App or Chrome extension developer having to monetise their free product with ads, they can integrate the SDK of one of the large proxy providers into their product and get paid for every monthly active user they have.
Two of the most popular include the Bright Data SDK and Infatica SDK.
Once embedded, these SDKs allow end users to opt-in to having ad free access to the App or Chrome Extension in exchange for the proxy provider being able send traffic through their device for the purposes of web scraping, ad verification or website testing.
This can be good for developers and end users because:
- Opt-In: End users can explicitedly opt-in to having an ad free experience in exchange for their bandwidth being used.
- Better UX: End users have a better user experience if they aren't being interupted by ads.
- Predictable Revenue: The developers of these Apps and Chrome extensions get predictable revenue of $1,000 - $50,000 per month.
However, it is the proxy providers that really win big.
On average, proxy providers will pay the owners of these Apps & Chrome extensions $0.05 for every MAU (monthly active user) the App has.
However, these proxy providers charge their customers between $2 to $15 per GB of traffic sent through their residential proxy networks, or $10 to $40 per GB for traffic sent through their mobile proxy networks.
Given that the proxy provider can easily send 10-20 GB of traffic through each App or Chrome Extension their SDK has been installed in each month, the proxy provider can be easily making $100-500 for every MAU but only paying $0.05 to send the traffic.
The profitability of this model can be crazy, with a proxy provider able to make $10 million in revenue from their SDK being installed into a single App.
However, this method can have questionable ethics as the end user likely won't know their device is being used as a proxy endpoint unless they explicitedly agreed to it.
Paid Bandwidth Products
A more recent way for proxy providers to build residential and mobile proxy networks has been to create paid bandwidth products.
These products directly pay users for allowing a proxy provider to route traffic through the devices IP address. This approach to building a proxy network is often a fairer and more ethical approach than either free VPNs or embedded proxy SDKs as:
- Fair Compensation: Users of these products are paid based on the amount of bandwidth the proxy provider uses on their device. Not a very low flat rate like with the embedded proxy SDKs approach.
- Explicit Opt-In: Users are explicitedly told what their devices will be used for and they have to personally download and run these products in the background on their devices.
Oxylab's Honeygain, and PacketStream are the two largest options in the market.
- Honeygain: Has a more opaque model with credits being convertable to cash. However, it roughly works out as $0.30 per GB of bandwidth shared.
- PacketStream: Offers users $0.10 per GB of bandwidth their device provides in a Peer-to-Peer bandwidth sharing program.
Not to be left behind, Bight Data has recently have their own paid bandwidth product EarnApp.
EarnApp is offering between $0.10 to $1.20 per GB of bandwidth shared. With rates varying based on your location and the type of device (Windows, macOS, Linux, Pi).
It is more expensive for proxy providers to acquire proxy bandwidth this way, however, as they can be selling bandwidth to their end users from between $2 to $40 per GB they have plently margin.
Device Proxy Farms
In the last year or two there have been a number of smaller proxy providers popping up specializing in mobile proxies.
In most cases they don't have the resources to build their own free VPNs, proxy SDKs, or paid bandwidth programs so they either buy their mobile proxies from another provider or build a device farm.
Here, the proxy provider buys hundreds of cheap mobile devices (Android, Apple, etc.), buys data plans for each device, connects them to a power source and use sometime like DeviceFarmer/stf to control them.
They then can route their proxy traffic through these devices using the IP addresses their 4G carrier assigns to them.
Since the requests are coming from real devices, it is extremely difficult & risky for websites & anti-bot solutions to detect and ban these devices as they could be inadvertantly be banning hundreds if not thousands of real users who are using the same 4G carrier.
This method can be very effective, but oftentimes it can be a bit unreliably when being done at scale as it is messy to manage this many devices.
Other options include emulating Android devices with Android emulators so the provider doesn't need to manage real devices. This is simpler, but also makes detecting them easier for websites too.
Companies, like Proxidize have taken this a step further and sell pre-packaged LTE USB modems, USB hubs, servers & software so that you build your own mobile proxy network at home for as little as $499.
Proxy Reselling
The last but one of the most common methods proxy providers get their residential & mobile proxies is by simply buying them from a provider with their own residential or mobile proxy networks.
Behind the scenes the proxy market is very interconnected, with proxy providers buying and selling access to their proxy networks with each other. So even if you switch to another proxy provider, you might be using the exact same proxy network under the hood.
This method is extremely common, especially for providers who just offer residential & mobile proxies as an add-on to their existing service.
Many smaller proxy providers or the smart proxy APIs don't actually own their own residential & mobile proxy networks they instead by plans from the big providers like Bright Data, Oxylabs, Infatica and Geonode.
More Web Scraping Articles
This was a deep dive into how proxy providers acquire their residential & mobile proxies.
If you would like to learn more about proxies or web scraping in general, then be sure to check out The Web Scraping Playbook, or check out one of our other in-depth guides: