ScrapeOps Quickstart Guide: Proxy APIs, Monitoring & Scheduling

Welcome to the ScrapeOps Documentation pages. Here you will find info on how to integrate and use our products.

ScrapeOps Proxy API Aggregator is an easy to use proxy that gives you access to the best performing Proxy APIs via a single endpoint. We take care of finding the best proxies, so you can focus on the data.

To use the ScrapeOps Proxy API Aggregator, you first need an API key which you can get by signing up for a free account here.

Getting Started

To make requests you need send the URL you want to scrape to the ScrapeOps Proxy API endpoint https://proxy.scrapeops.io/v1/ by adding your API Key and URL to the request using the api_key and url query parameter:

import requests

response = requests.get(
    'https://proxy.scrapeops.io/v1/',
    params={
        'api_key': 'YOUR_API_KEY',
        'url': 'http://httpbin.org/anything'
    }
)
print(response.text)  # Returns the HTML content

With the ScrapeOps Proxy API Aggregator you are only charged for successful requests (200 and 404 status codes).

To learn how to use the ScrapeOps Proxy API Aggregator and customise it to your requirement then check out the QuickStart Guide.

ScrapeOps Residential Proxy Aggregator is an easy to use proxy that gives you access to the best performing Residential Proxy providers via a single proxy port. We take care of finding the best proxies, so you can focus on the data.

To use the ScrapeOps Residential Proxy Aggregator, you first need an API key which you can get by signing up for a free account here.

Getting Started

To make requests you need send the URL you want to scrape to set their proxy port to the ScrapeOps Residential Proxy Port http://scrapeops:YOUR_API_KEY@residential-proxy.scrapeops.io:8181

The username for the proxy is scrapeops and the password is your API key.

import requests

proxies = {
    'http': 'http://scrapeops:YOUR_API_KEY@residential-proxy.scrapeops.io:8181',
    'https': 'http://scrapeops:YOUR_API_KEY@residential-proxy.scrapeops.io:8181'
}

response = requests.get('https://httpbin.org/ip', proxies=proxies)
print(response.text)

Proxy

residential-proxy.scrapeops.io

Port

8181

Username

scrapeops

Password

YOUR_API_KEY

With the ScrapeOps Residential Proxy Aggregator you are charged for bandwidth consumed.

To learn how to use the ScrapeOps Residential Proxy Aggregator and customise it to your requirement then check out the QuickStart Guide.

ScrapeOps Monitoring is a monitoring tool purpose built for web scraping. With a simple 30 seconds install of one of our SDKs, your scraper's performance & error stats will be automatically aggregated and shipped to your ScrapeOps dashboard.

Features & Functionality

Getting Started

To use ScrapeOps you first need to create a free account and get your free API_KEY.

Currently ScrapeOps integrates with both Python Requests & Python Scrapy scrapers:

1. Python Requests Integration
2. Python Scrapy Integration

More ScrapeOps Monitoring integrations are on the way.

ScrapeOps Server Manager & Job Scheduler is a easy to use server integration that enables you to deploy, manage and schedule your scrapers from the ScrapeOps dashboard.

There are two options to integrate ScrapeOps with your servers:

1. Via SSH (Recommended)
2. Via Scrapyd Server HTTP Endpoints (Only Applicable to Python Scrapy)

Features & Functionality

To learn how to setup the integrate ScrapeOps with your servers with this guide.

ScrapeOps n8n Node is a powerful integration that brings all of ScrapeOps' web scraping capabilities into the n8n workflow automation platform. Perfect for no-code builders and developers who want to integrate web scraping into their automation workflows.

Features & Functionality

Getting Started

1. Install the node from the n8n community nodes
2. Get your API key from ScrapeOps
3. Configure credentials in n8n
4. Start building powerful scraping workflows

ScrapeOps MCP Server exposes the ScrapeOps Proxy API to MCP-compatible IDEs (Cursor, Claude Desktop, VS Code, Windsurf). It gives AI agents first-class scraping and browsing tools with proxy support, anti-bot bypass, JavaScript rendering, screenshots, and LLM-powered extraction.

Features & Functionality

Getting Started

Run the MCP server with npx, then configure your IDE (Cursor, Claude Desktop, VS Code, or Windsurf) to connect to the MCP server:

env SCRAPEOPS_API_KEY=YOUR_API_KEY npx -y @scrapeops/mcp

{
  "mcpServers": {
    "@scrapeops/mcp": {
      "command": "npx",
      "args": ["-y", "@scrapeops/mcp"],
      "env": { "SCRAPEOPS_API_KEY": "YOUR_API_KEY" }
    }
  }
}

ScrapeOps AI Scraper Builder automatically generates production-ready web scrapers from any e-commerce product page URL using AI. Simply provide URLs, choose your language, and get a working scraper that outputs structured JSON data.

Supported Page Types

Getting Started

To use the AI Scraper Builder, you first need to create a free account and get your free API_KEY. Then navigate to the AI Assistant → Scraper Generator in the dashboard:

1. Enter up to 5 URLs from the same website
2. Choose your language (Python or Node.js) and library
3. Click Generate and get production-ready scraper code

ScrapeOps Parser API takes raw HTML pages and parses them into structured JSON format. Simply send us the HTML page for supported websites, and we'll extract the data for you.

Supported Parsers

Getting Started

To use the Parser API, you first need to create a free account and get your free API_KEY. Send HTML to the Parser API endpoint and receive structured JSON:

import requests

# First get the HTML from the target page
html = requests.get('https://www.amazon.com/dp/B08WM3LMJF').text

# Send HTML to the Parser API
response = requests.post(
    'https://parser.scrapeops.io/v2/amazon?api_key=YOUR_API_KEY',
    data={'html': html}
)
print(response.json())

ScrapeOps Data APIs are a collection of APIs that allow you to access structured JSON data from popular websites using dedicated endpoints. No need to build or maintain your own parsers—just call the API and get the data.

Available APIs

Getting Started

To use the Data APIs, you first need to create a free account and get your free API_KEY. Query the API by passing in the product ID or URL:

import requests

response = requests.get(
    'https://proxy.scrapeops.io/v1/structured-data/amazon/product',
    params={
        'api_key': 'YOUR_API_KEY',
        'asin': 'B0BNLTS1T3',
        'country': 'us'
    }
)
data = response.json()
print(data['title'])   # "Apple AirPods Pro (2nd Generation)"
print(data['price'])   # "$249.00"