The
Selenium
Web Scraping Playbook

In Part 1 of the series, we go over the basics of how to build a scraper with Python using Python Selenium.

In Part 2 of the series, we're going to show you how to make your scraper more robust and reliable.

In Part 3 of the series, we'll explore several different ways we can store the data and talk about their pros, and cons and in which situations you would use them.

In Part 4 of the series, we make our scraper more robust and scalable by handling failed requests and using concurrency.

In this guide we show you how to use Python Selenium to render, automate and scrape Javascript heavy websites.

In this guide, we'll take you through how to scrape Google search results using Selenium

In this guide, we'll take you through how to scrape Reddit using Selenium

In this guide, we'll take you through how to scrape Amazon using Selenium

In this guide, we'll take you through how to scrape Trustpilot using Selenium

In this guide, we'll take you through how to scrape G2 using Selenium

In this guide, we'll take you through how to scrape Pinterest using Selenium

In this guide, we'll take you through how to scrape Airbnb using Selenium

In this guide, we'll take you through how to scrape Etsy using Selenium

In this guide, we'll take you through how to scrape Indeed using Selenium

In this guide, we'll take you through how to scrape Leboncoin using Selenium

In this guide, we'll take you through how to scrape Linkedin Jobs using Selenium

In this guide, we'll take you through how to scrape Linkedin Profiles using Selenium

In this guide, we'll take you through how to scrape Quora using Selenium

In this guide, we'll take you through how to scrape Redfin using Selenium

In this guide, we'll take you through how to scrape SimilarWeb using Selenium

In this guide, we'll take you through how to scrape Walmart using Selenium

In this guide, we'll take you through how to scrape Yelp using Selenium

In this guide, we'll take you through how to scrape Zillow using Selenium

Should you use Python Selenium or Python Requests? In this guide we compare both options and tell you when you should use each.

Should you use Python Selenium or Python Scrapy? In this guide we compare both options and tell you when you should use each.

Should you use Python Selenium or NodeJS Playwright? In this guide we compare both options and tell you when you should use each.

In this comprehensive guide, we will demystify the art of locating HTML elements using CSS Selectors with Selenium.

XPath is an extremely powerful tool for locating elements. In this guide we will deep dive into using XPath to find elements in Selenium.

The web apps rely on background XHR requests to load content. In this article, we'll explore why you need to capture them, how to capture them with Selenium.

Managing the timing differences between script execution and the varying speeds at which web pages load is crucial.In this Selenium guide, we focus on the nuanced strategies of waiting for pages or elements to load.

Screenshots play a crucial role in Selenium automation for various reasons, serving beyond mere documentation of web journeys. In this comprehensive guide, we unveil the art of taking screenshots with Selenium.

The ability to navigate through a page, especially one with lazy-loaded or infinite-scrolling content, is crucial for comprehensive data extraction and interaction.

In this Selenium guide, we focus on the fundamental concepts, practical examples, and best practices for navigating and interacting with web forms.

Learn how to handle cookies with Selenium. Get, set, edit and delete cookies from your Selenium scraper.

In this guide, we walk through how to use proxies with Python Selenium.

In this guide, we'll explore the fake user agents, their implementation in Selenium scripts and provide practical insights to elevate your web scraping endeavors.

In this guide, we walk through how to disable Python Selenium from loading images to speed up your scrapers and reduce bandwidth usage.

Blocking unnecessary resources, such as images, can enhance test performance by speeding up page loading and minimizing data usage.

In this guide, we will walk you through the process of downloading a file using Selenium.

In this guide, we'll dive into various methods to make your Selenium applications undetectable.

Selenium Stealth is a powerful tool designed to augment Selenium's capabilities for web scraping by adding an extra layer of anonymity.

In this guide, we walk through how to use proxies with Python Selenium.

In this guide, we’ll explore how to overcome this limitation by making adjustments and applying patches to render Selenium undetectable to bot detectors.

In this Selenium guide, we explore the challenges posed by DataDome and delve into a practical solution using Selenium to bypass its protections.

In this Selenium guide, we explore the challenges posed by PerimeterX and delve into a practical solution using Selenium to bypass its protections.

In this Selenium guide, we explore the challenges posed by Cloudflare and delve into a practical solution using Selenium to bypass its protections.

In this article, we will embark on a journey to explore the capabilities and applications of Jupyter Notebook using Selenium.

In this guide, we walk through Selenium Wire a extension for Python Selenium that gives you more control over requests & responses.

In this article, we'll delve into the causes of ChromeDriver Executable Needs to Be in PATH error and explore effective solutions to resolve it.

In this article, we'll delve into the causes of Geckodriver Executable Needs to Be in PATH error and explore effective solutions to resolve it.