Server Management & Scheduling Overview
The following is an overview of how to integrate ScrapeOps with your servers so that you can deploy, manage and schedule your scrapers from the ScrapeOps dashboard.
There are two options to integrate ScrapeOps with your servers:
- Via SSH (Recommended)
- Via Scrapyd Server HTTP Endpoints (Only Applicable to Python Scrapy)
💻 Demo
Login to the Demo Account and navigate to the Scheduler or Server Pages.
⭐ SSH Server Integration
The SSH server integration allows you to deploy, schedule & run any type of scraper (Python, NodeJs, Scrapy, etc) on a server that is accessible via SSH. Which includes most major server & VM providers (Digital Ocean, Vultr, AWS, etc.).
The ScrapeOps server manager allows you to:
- 🔗 Integrate With Any SSH Capably Server
- 🕷 Deploy scrapers directly from GitHub to your servers.
- ⏰ Schedule Periodic Jobs
To learn how to setup the SSH server integration follow this guides:
Get $100 Free Credits with Vultr using this link, and $100 Free Credits with Digital Ocean using this link.
⭐ Scrapyd Server Integration
The Scrapyd server integration allows you to:
- 🔗 Integrate With ScrapyD Servers
- ⏰ Schedule Periodic Jobs
- 💯 All Scrapyd JSON API Supported
- 🔐 Secure Your ScrapyD with BasicAuth, HTTPS or Whitelisted IPs
To learn how to setup the Scrapyd integration follow this guide.