Project Name	Stars	Repos Using This	Packages Using This	Most Recent Commit	Total Releases	Latest Release	Open Issues	License	Language
Scrapy	49,918	4,185	445	3 months ago	96	September 18, 2023	692	bsd-3-clause	Python
Scrapy, a fast high-level web crawling & scraping framework for Python.
Crawlab	10,521			4 months ago	1	March 03, 2019	58	bsd-3-clause	Go
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Awesome Crawler	5,859			5 months ago			27	mit
A collection of awesome web crawler,spider in different languages
Webscraping From 0 To Hero	1,305			10 months ago
The web scraping open project repository aims to share knowledge and experiences about web scraping with Python
Django Dynamic Scraper	1,069	26		2 years ago	65	June 25, 2021	38	bsd-3-clause	Python
Creating Scrapy scrapers via the Django admin interface
Faster Than Requests	1,061		4	3 months ago	18	February 24, 2021		mit	Nim
Faster requests on Python 3
Scrapy Fake Useragent	654	50	5	8 months ago	12	August 15, 2020	5	mit	Python
Random User-Agent middleware based on fake-useragent
Phpscraper	486			5 months ago	34	June 18, 2023	20	gpl-3.0	PHP
A universal web-util for PHP.
Scrapple	452	1		5 years ago	10	September 24, 2016	4	mit	Python
A framework for creating semi-automatic web content extractors
City Scrapers	315			3 months ago			54	mit	Python
Scrape, standardize and share public meetings from local government websites

Alternatives To Goscrapy

Select To Compare

Scrapy ⭐ 49,918

Scrapy, a fast high-level web crawling & scraping framework for Python.

dependent packages 445total releases 96most recent commit 3 months ago

Crawlab ⭐ 10,521

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

total releases 1most recent commit 4 months ago

Awesome Crawler ⭐ 5,859

A collection of awesome web crawler,spider in different languages

most recent commit 5 months ago

Webscraping From 0 To Hero ⭐ 1,305

The web scraping open project repository aims to share knowledge and experiences about web scraping with Python

most recent commit 10 months ago

Django Dynamic Scraper ⭐ 1,069

Creating Scrapy scrapers via the Django admin interface

total releases 65most recent commit 2 years ago

Faster Than Requests ⭐ 1,061

Faster requests on Python 3

dependent packages 4total releases 18most recent commit 3 months ago

Scrapy Fake Useragent ⭐ 654

Random User-Agent middleware based on fake-useragent

dependent packages 5total releases 12most recent commit 8 months ago

Phpscraper ⭐ 486

A universal web-util for PHP.

total releases 34most recent commit 5 months ago

Scrapple ⭐ 452

A framework for creating semi-automatic web content extractors

total releases 10most recent commit 5 years ago

City Scrapers ⭐ 315

Scrape, standardize and share public meetings from local government websites

most recent commit 3 months ago

Suggest An Alternative To goscrapy

Alternative Project Comparisons

Goscrapy vs Scrapy

Goscrapy vs Crawlab

Goscrapy vs Awesome Crawler

Goscrapy vs Webscraping From 0 To Hero

Goscrapy vs Django Dynamic Scraper

Goscrapy vs Faster Than Requests

Goscrapy vs Scrapy Fake Useragent

Goscrapy vs Phpscraper

Goscrapy vs Scrapple

Goscrapy vs City Scrapers

Popular Scrapy Projects

Learn_python3_spider ⭐ 14,425

python爬虫教程系列、从0到1学习python爬虫，包括浏览器抓包，手机APP抓包，如 fiddler、mitmproxy，各种爬虫涉及的模块的使用，如：requests、beautifu 爬虫加密逆向破解，JS爬虫逆向，分布式爬虫，爬虫项目实战实例等

total releases 2latest release August 07, 2019most recent commit 4 months ago

Cs Book ⭐ 11,024

计算机类常用电子书整理，并且附带下载链接，包括Java，Python，Linux，Go，C，C++，

most recent commit 9 months ago

Portia ⭐ 8,982

Visual scraping for Scrapy

dependent packages 2total releases 26latest release May 25, 2015most recent commit 8 months ago

Pythonspidernotes ⭐ 6,664

Python入门网络爬虫之精华版

most recent commit 3 years ago

Wechatsogou ⭐ 5,777

基于搜狗微信搜索的微信公众号爬虫接口

total releases 25latest release April 10, 2019most recent commit 6 months ago

Popular Web Crawler Projects

Huginn ⭐ 41,465

Create agents that monitor and act on your behalf. Your agents are standing by!

dependent packages 52total releases 8latest release September 22, 2017most recent commit 22 days ago

Changedetection.io ⭐ 13,943

The best and simplest free open source website change detection, website watcher, restock monitor and notification service. Restock Monitor, change detection. Designed for simplicity - Simply monitor which websites had a text change for free. Free Open source web page change detection, Website defacement monitoring, Price change notification

most recent commit 3 months ago

Crawlee ⭐ 12,158

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

dependent packages 42total releases 747latest release December 10, 2023most recent commit 3 days ago

Spider Flow ⭐ 8,075

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

most recent commit a year ago

Katana ⭐ 7,995

A next-generation crawling and spidering framework.

dependent packages 1total releases 8latest release September 14, 2023most recent commit 3 months ago

Popular Data Processing Categories