Skip to main content

Server Management & Scheduling Overview

The following is an overview of how to integrate ScrapeOps with your servers so that you can deploy, manage and schedule your scrapers from the ScrapeOps dashboard.

There are two options to integrate ScrapeOps with your servers:

  1. Via SSH (Recommended)
  2. Via Scrapyd Server HTTP Endpoints (Only Applicable to Python Scrapy)

💻 Demo

Login to the Demo Account and navigate to the Scheduler or Server Pages.


⭐ Scrapyd Server Integrtion

The Scrapyd server integration allows you to:

  • 🔗 Integrate With ScrapyD Servers
  • ⏰ Schedule Periodic Jobs
  • 💯 All Scrapyd JSON API Supported
  • 🔐 Secure Your ScrapyD with BasicAuth, HTTPS or Whitelisted IPs

To learn how to setup the Scrapyd integration follow this guide.