Skip to main content
freeCodeCamp Scrapy Beginners Course Logo

The freeCodeCamp
Python Scrapy
Beginners Course

The Complete 12-Part Scrapy Beginners Course

Scrapy Community:
Scrapy Reddit Community
Scrapy Discord Community
Scrapy Twitter

The freeCodeCamp Scrapy Beginners Course is a complete Scrapy beginners course that will teach you everything you need to learn to start scraping websites at scale using Python Scrapy. Including:

  • Creating your first Scrapy spider
  • Crawling through websites & scraping data from each page
  • Cleaning data with Items & Item Pipelines
  • Saving data to CSV files, MySQL & Postgres databases
  • Using fake user-agents & headers to avoid getting blocked
  • Using proxies to scale up your web scraping without getting banned
  • Deploying your scraper to the cloud & scheduling it to run periodically

In Part 1: Scrapy & Course Overview we go through what is Scrapy and what you will learn in this course. Including:

freeCodeCamp Scrapy Beginners Course Part 1 - Overview

In Part 2: Setting Up Environment & Scrapy we go through how to setup your Python environment along with installing Scrapy. We walk through:

freeCodeCamp Scrapy Beginners Course Part 2 - Setting Up Scrapy

In Part 3: Creating Scrapy Project we go through how create a Scrapy project and explain all of its components. We walk through:

freeCodeCamp Scrapy Beginners Course Part 3: Creating Scrapy Project

In Part 5: Crawling With Scrapy we go through how to create a more advanced Scrapy spider that will crawl the entire BooksToScrape.com website and scrape the data from each individual book page. We walk through:

freeCodeCamp Scrapy Beginners Course Part 5: Crawling With Scrapy

In Part 6: Cleaning Data With Item Pipelines we go through how to use Scrapy Items & Item Pipelines to structure and clean your scraped data as scraped data can be very messy and unstructured. We walk through:

freeCodeCamp Scrapy Beginners Course Part 6: Cleaning Data With Item Pipelines

In Part 7: Saving Data To Files & Databases we go through how to save our scraped data to CSV files and MySQL & Postgres databases. We walk through:

freeCodeCamp Scrapy Beginners Course Part 7: Saving Data To Files & Databases

In Part 9: Rotating Proxies & Proxy APIs we go through how you can use rotating proxy pools to hide your IP address and scrape at scale without getting blocked. We walk through:

freeCodeCamp Scrapy Beginners Course Part 9: Rotating Proxies & Proxy APIs

In Part 10: Deploying & Scheduling Spiders With Scrapyd we go through how you can deploy and run your spiders in the cloud with Scrapyd. We walk through:

freeCodeCamp Scrapy Beginners Course Part 10: Deploying & Scheduling Spiders With Scrapyd

In Part 12: Deploying & Scheduling Spiders With Scrapy Cloud we go through how you can deploy, schedule and run your spiders on any server with Scrapy Cloud. We walk through:

freeCodeCamp Scrapy Beginners Course Part 12: Deploying & Scheduling Spiders With Scrapy Cloud

In Part 13: Conclusion & Next Steps we summarize what we learned during this course and outline some of the next steps you can take to advance your knowledge of Scrapy even further. We walk through:

freeCodeCamp Scrapy Beginners Course Part 13: Conclusion & Next Steps