The Best Web Scraping Communities Every Web Scraper Should Know About
Web scraping is such a challenging and ever changing field that it is very important to have some sort of support system.
With many popular web scraping libraries/frameworks and websites constantly trying to block you, having a community of experienced web scrapers to guide you dramatically increases your chances of success in this field.
You just need to know where to find them.
In this guide we're going to share with you the best web scraping communities so you can stay on top of the latest trends and get your questions answered when you run into trouble.
- Scraping Enthusiasts Discord Server
- r/WebScraping Subreddit
- r/Scrapy & Scrapy Discord
- r/Selenium Subreddit
- r/Puppeteer Subreddit
- Scraping In Prod Discord Server
- Oxylabs Discord Server
Need help scraping the web?
Then check out ScrapeOps, the complete toolkit for web scraping.
Scraping Enthusiasts Discord Server
If you could only be part of one web scraping community then being part of the Scraping Enthusiasts Discord server is a must.
Scraping Enthusiasts has a large and active discord server that is ram packed with great web scraping resources and experienced web scrapers who can help you get the data you need.
What makes Scraping Enthusiasts really stand out versus other web scraping communities is the communities experience in bypassing antibot countermeasures. The server includes a number of the lead maintainers from the puppeteer-extra team who develop the puppeteer-extra-plugin-stealth, and number of other experienced antibot reverse engineers.
And it shows. Not only is it an active server who are willing to answer your scraping questions about how to use specific web scraping stacks, they have community bots that:
- Detect which antibot software does a website use.
- Detect hidden API endpoints on a website.
- Monitors the most common antibots for updates.
- Monitors the most common headless browsers and drivers for updates.
If you are really into web scraping, and have to deal with a lot of heavily protected websites then joining Scraping Enthusiasts is a must.
Join The Scraping Enthusiasts Discord Server
r/WebScraping Subreddit
The best web scraping subreddit is r/WebScraping, the largest subreddit dedicated to everything web scraping. With over 9,000 members and a pretty active community it is a great place to keep up to date on the latest news in web scraping and get help if you run into trouble with a particular website.
You can find resources/advice on anything from:
- How to scrape specific websites?
- How to bypass specific anti-bots?
- What are the best web scraping proxy providers?
- How to use particular web scraping libraries?
If you can't find a answer to your question then, just ask your own and someone in the community will help you out.
r/Scrapy Subreddit & Scrapy Discord Server
When it comes to web scraping then Python Scrapy is king. So it should be no surprise to include both the r/Scrapy subreddit and the Scrapy Discord Server on this list.
Both r/Scrapy and the Scrapy Discord Server are the offical community centers for the Scrapy framework. Making them best place to share cool articles, spiders, Scrapy extensions and whatnots.
A number of the original creators and current maintainers of Scrapy are actively daily on the subreddit so if you have a Scrapy specific issue then the Scrapy subreddit is a great place to go.
The moderators are very open to members asking for help, advice and code reviews so it is a great place to go if you need any help with:
- Getting started with Scrapy
- Using some of Scrapy's less well-known functionality
- Customising Scrapy for your specific use cases.
- Picking Scrapy extensions or web scraping stacks.
r/Selenium Subreddit
Whilst, not dedicated to web scraping, given how popular Selenium is amongst the web scraping community as a headless browser the r/Selenium subreddit is a great community to be a part of if you use Selenium as part of your web scraping stack.
r/Selenium is a large and active subreddit, dedicated to helping you to use Selenium in your automation and web scraping stacks.
In this subreddit, you can get help with Selenium specific setup, configuration and scaling issues from a experienced and helpful community.
r/Puppeteer Subreddit
Like the Puppeteer subreddit, r/Puppeteer is a small but extremely helpful subreddit dedicated to helping you setup, configure and use Puppeteer in your automation and web scraping stacks.
The r/Puppeteer community are great for helping you with topics like:
- Getting started with Puppeteer.
- Debugging Puppeteer specific issues.
- How to deploy your Puppeteer headless browsers to production.
- Efficient scaling strategies for your Puppeteer headless browsers.
Scraping In Prod Discord Server
Finally we have the Scraping In Prod discord server. Although not as active as Scraping Enthusiasts, checking out Scraping In Prod is well worth it.
The Scraping In Prod server is a great place to go if you have general web scraping questions, need help with a specific website or anti-bot.
They mainly focus on web scraping with Python and NodeJs, but you can also ask questions for other languages too.
Join The Scraping In Prod Discord Server
Oxylabs Discord Community
The official Oxylabs Discord server is a rapidly growing hub for all things web scraping that you don't want to miss out on.
Run by seasoned web scraping experts from Oxylabs, it's a place to connect with like-minded peers and gain professional know-how to push your scraping projects through blocks to success. Whether you're facing technical challenges or want to sharpen your skills, an always-active network of enthusiasts will help you out.
It's not only a community, it's an ecosystem that supports anyone engaged in public data gathering. Here's what it offers:
- Q&A sessions with Oxylabs scraping experts.
- Live webinars sharing the latest industry insights.
- Exclusive discounts for Oxylabs products.
- Quick access to the newest Oxylabs updates.
- Swift and efficient customer support.
Join the Oxylabs Discord Server
More Options
If you would like to try out more web scraping communities then check out our list of the best web scraping discord communities or our list of the best web scraping subreddits.
If you would like to learn more about web scraping in general, then be sure to check out The Web Scraping Playbook. Or check out some our other popular articles like: