Skip to main content
Selenium Web Scraping Playbook - Selenium Logo

The
Selenium
Web Scraping Playbook

Your guide to becoming a Selenium Web Scraping Pro!

Web Scraping Community:
Web Scraping Reddit Community
Web Scraping Discord Community

Stay Up To Date

Get notified of the latest Selenium news, guides, extensions, and scrapers as soon as they are released. Also, suggest new guides and extensions for the Selenium Playbook then signup here.

Web Scraping For Beginners Series

Part 1: How To Build Your First Python Scraper

In Part 1 of the series, we go over the basics of how to build a scraper with Python using Python Selenium.

Part 2: Cleaning Dirty Data & Dealing With Edge Cases

In Part 2 of the series, we're going to show you how to make your scraper more robust and reliable.

Part 3: Storing Data in AWS S3, MySQL & Postgres DBs

In Part 3 of the series, we'll explore several different ways we can store the data and talk about their pros, and cons and in which situations you would use them.

Part 4: Managing Retries & Concurrency

In Part 4 of the series, we make our scraper more robust and scalable by handling failed requests and using concurrency.

Intro to Selenium

Intro Web Scraping With Selenium

In this guide we show you how to use Python Selenium to render, automate and scrape Javascript heavy websites.

How to Scrape With Selenium Series

How to Scrape Google Search With Selenium

In this guide, we'll take you through how to scrape Google search results using Selenium

How to Scrape Reddit With Selenium

In this guide, we'll take you through how to scrape Reddit using Selenium

How to Scrape Amazon With Selenium

In this guide, we'll take you through how to scrape Amazon using Selenium

Comparison of Selenium To Other Web Scraping Libraries

Python Selenium VS Python Requests Compared

Should you use Python Selenium or Python Requests? In this guide we compare both options and tell you when you should use each.

Python Selenium VS Python Scrapy Compared

Should you use Python Selenium or Python Scrapy? In this guide we compare both options and tell you when you should use each.

Python Selenium vs NodeJS Playwright Compared

Should you use Python Selenium or NodeJS Playwright? In this guide we compare both options and tell you when you should use each.

Extracting & Parsing Data

Python Selenium: Finding Elements by CSS Selectors

In this comprehensive guide, we will demystify the art of locating HTML elements using CSS Selectors with Selenium.

Python Selenium: How To Find Elements by XPath

XPath is an extremely powerful tool for locating elements. In this guide we will deep dive into using XPath to find elements in Selenium.

Python Selenium: How to Capture Background XHR Requests

The web apps rely on background XHR requests to load content. In this article, we'll explore why you need to capture them, how to capture them with Selenium.

Python Selenium: Waiting For Page or Element To Load

Managing the timing differences between script execution and the varying speeds at which web pages load is crucial.In this Selenium guide, we focus on the nuanced strategies of waiting for pages or elements to load.

Python Selenium: How To Take Screenshots

Screenshots play a crucial role in Selenium automation for various reasons, serving beyond mere documentation of web journeys. In this comprehensive guide, we unveil the art of taking screenshots with Selenium.

Navigating & Pagination

Python Selenium: How To Scroll The Page

The ability to navigate through a page, especially one with lazy-loaded or infinite-scrolling content, is crucial for comprehensive data extraction and interaction.

Python Selenium: Submitting Forms with Selenium

In this Selenium guide, we focus on the fundamental concepts, practical examples, and best practices for navigating and interacting with web forms.

Managing Cookies & Sessions

Python Selenium: Managing Cookies

Learn how to handle cookies with Selenium. Get, set, edit and delete cookies from your Selenium scraper.

Proxies & User-Agents

Using Proxies With Python Selenium

In this guide, we walk through how to use proxies with Python Selenium.

Python Selenium: Using Fake User Agents

In this guide, we'll explore the fake user agents, their implementation in Selenium scripts and provide practical insights to elevate your web scraping endeavors.

Customizing Selenium

Disable Images Loading In Python Selenium

In this guide, we walk through how to disable Python Selenium from loading images to speed up your scrapers and reduce bandwidth usage.

Python Selenium: How to Block Images and Resources

Blocking unnecessary resources, such as images, can enhance test performance by speeding up page loading and minimizing data usage.

Python Selenium: Downloading a File

In this guide, we will walk you through the process of downloading a file using Selenium.

Python Selenium: How To Make Selenium Undetectable

In this guide, we'll dive into various methods to make your Selenium applications undetectable.

Fortified Selenium Browsers

Selenium Undetected Chromedriver: Bypass Anti-Bots With Ease

Selenium Stealth is a powerful tool designed to augment Selenium's capabilities for web scraping by adding an extra layer of anonymity.

How To Use Selenium Stealth For Web Scraping

In this guide, we walk through how to use proxies with Python Selenium.

Bypassing Anti-Bots

Python Selenium: How to Bypass Datadome with Selenium

In this Selenium guide, we explore the challenges posed by DataDome and delve into a practical solution using Selenium to bypass its protections.

Python Selenium: How to Bypass PerimeterX with Selenium

In this Selenium guide, we explore the challenges posed by PerimeterX and delve into a practical solution using Selenium to bypass its protections.

Python Selenium: How to Bypass Cloudflare with Selenium

In this Selenium guide, we explore the challenges posed by Cloudflare and delve into a practical solution using Selenium to bypass its protections.

Selenium Environments

Python Selenium: Run Selenium Using a Jupyter Notebook

In this article, we will embark on a journey to explore the capabilities and applications of Jupyter Notebook using Selenium.

Selenium Extensions

Selenium Wire Guide: Extending Python Selenium

In this guide, we walk through Selenium Wire a extension for Python Selenium that gives you more control over requests & responses.

Selenium Errors

Selenium Error - ChromeDriver Executable Needs to Be in PATH

In this article, we'll delve into the causes of ChromeDriver Executable Needs to Be in PATH error and explore effective solutions to resolve it.

Selenium Error - Geckodriver Executable Needs to Be in PATH

In this article, we'll delve into the causes of Geckodriver Executable Needs to Be in PATH error and explore effective solutions to resolve it.

Need an easy way to monitor your scrapers?

Sign up for a free ScrapeOps account today.