ScrapeOps
ScrapeOps Playbook · 29 guidesSelenium Web Scraping Guides and Tutorials
Selenium guides for finding elements, pagination, cookies, proxies, user-agents, and making a browser harder to detect while scraping.
Stay Up To Date
Get notified of the latest Selenium guides as soon as they are released. Suggest new guides for the playbook, then sign up here.
Guides by topic
Browse the Playbook
01
Intro to Selenium
1 guide
02
Comparison of Selenium To Other Web Scraping Libraries
3 guides
Python Selenium VS Python Requests ComparedShould you use Python Selenium or Python Requests? In this guide we compare both options and tell you when you should use each.→Python Selenium VS Python Scrapy ComparedShould you use Python Selenium or Python Scrapy? In this guide we compare both options and tell you when you should use each.→ Python Selenium vs NodeJS Playwright ComparedShould you use Python Selenium or NodeJS Playwright? In this guide we compare both options and tell you when you should use each.→
03
Extracting & Parsing Data
5 guides
Python Selenium: Finding Elements by CSS SelectorsIn this comprehensive guide, we will demystify the art of locating HTML elements using CSS Selectors with Selenium.→Python Selenium: How To Find Elements by XPathXPath is an extremely powerful tool for locating elements. In this guide we will deep dive into using XPath to find elements in Selenium.→Python Selenium: How to Capture Background XHR RequestsThe web apps rely on background XHR requests to load content. In this article, we'll explore why you need to capture them, how to capture them with Selenium.→Python Selenium: Waiting For Page or Element To LoadManaging the timing differences between script execution and the varying speeds at which web pages load is crucial.In this Selenium guide, we focus on the nuanced strategies of waiting for pages or elements to load.→Python Selenium: How To Take ScreenshotsScreenshots play a crucial role in Selenium automation for various reasons, serving beyond mere documentation of web journeys. In this comprehensive guide, we unveil the art of taking screenshots with Selenium.→
06
Proxies & User-Agents
2 guides
Using Proxies With Python SeleniumIn this guide, we walk through how to use proxies with Python Selenium.→Python Selenium: Using Fake User AgentsIn this guide, we'll explore the fake user agents, their implementation in Selenium scripts and provide practical insights to elevate your web scraping endeavors.→
07
Customizing Selenium
4 guides
Disable Images Loading In Python SeleniumIn this guide, we walk through how to disable Python Selenium from loading images to speed up your scrapers and reduce bandwidth usage.→Python Selenium: How to Block Images and ResourcesBlocking unnecessary resources, such as images, can enhance test performance by speeding up page loading and minimizing data usage.→Python Selenium: Downloading a FileIn this guide, we will walk you through the process of downloading a file using Selenium.→Python Selenium: How To Make Selenium UndetectableIn this guide, we'll dive into various methods to make your Selenium applications undetectable.→
08
Fortified Selenium Browsers
2 guides
Selenium Undetected Chromedriver: Bypass Anti-Bots With EaseSelenium Stealth is a powerful tool designed to augment Selenium's capabilities for web scraping by adding an extra layer of anonymity.→How To Use Selenium Stealth For Web ScrapingIn this guide, we walk through how to use proxies with Python Selenium.→
09
Avoiding Bans & Anti-Bots With Puppeteer
2 guides
How To Make Selenium UndetectableIn this guide, we’ll explore how to overcome this limitation by making adjustments and applying patches to render Selenium undetectable to bot detectors. →How to Scrape Linkedin Profiles With SeleniumIn this guide, we'll take you through how to scrape Linkedin Profiles using Selenium→
10
Bypassing Anti-Bots
3 guides
Python Selenium: How to Bypass Datadome with SeleniumIn this Selenium guide, we explore the challenges posed by DataDome and delve into a practical solution using Selenium to bypass its protections.→Python Selenium: How to Bypass PerimeterX with SeleniumIn this Selenium guide, we explore the challenges posed by PerimeterX and delve into a practical solution using Selenium to bypass its protections.→Python Selenium: How to Bypass Cloudflare with SeleniumIn this Selenium guide, we explore the challenges posed by Cloudflare and delve into a practical solution using Selenium to bypass its protections.→
11
Selenium Environments
1 guide
12
Selenium Extensions
1 guide
13
Selenium Errors
2 guides
Selenium Error - ChromeDriver Executable Needs to Be in PATHIn this article, we'll delve into the causes of ChromeDriver Executable Needs to Be in PATH error and explore effective solutions to resolve it.→Selenium Error - Geckodriver Executable Needs to Be in PATHIn this article, we'll delve into the causes of Geckodriver Executable Needs to Be in PATH error and explore effective solutions to resolve it.→
Need an easy way to monitor your scrapers?
Sign up for a free ScrapeOps account today.