Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Scrapy Open Source Projects

Open source projects categorized as Scrapy
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96most recent commit a year agopypi Scrapy} Downloads
Learn_python3_spider ⭐ 14,425
python爬虫教程系列、从0到1学习python爬虫，包括浏览器抓包，手机APP抓包，如 fiddler、mitmproxy，各种爬虫涉及的模块的使用，如：requests、beautifu 爬虫加密逆向破解，JS爬虫逆向，分布式爬虫，爬虫项目实战实例等
total releases 2most recent commit a year ago
Cs Book ⭐ 11,024
计算机类常用电子书整理，并且附带下载链接，包括Java，Python，Linux，Go，C，C++，
most recent commit 2 years ago
Crawlab ⭐ 10,521
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
total releases 1most recent commit a year ago
Portia ⭐ 8,982
Visual scraping for Scrapy
dependent packages 2total releases 26most recent commit 2 years agopypi slybot} Downloads
Pythonspidernotes ⭐ 7,028
Python入门网络爬虫之精华版
most recent commit 4 years ago
Wechatsogou ⭐ 5,967
基于搜狗微信搜索的微信公众号爬虫接口
total releases 25most recent commit a year ago
Scrapy Redis ⭐ 5,546
Redis-based components for Scrapy.
dependent packages 21total releases 18most recent commit 9 months agopypi scrapy-redis} Downloads
Haipproxy ⭐ 5,384
💖 High available distributed ip proxy pool, powerd by Scrapy and Redis
total releases 7most recent commit 2 years ago
Splash ⭐ 3,860
Lightweight, scriptable browser as a service with an HTTP API
dependent packages 1total releases 30most recent commit 2 years agopypi splash} Downloads
Ecommercecrawlers ⭐ 3,724
实战🐍多种网站、电商数据爬虫🕷。包含🕸：淘宝商品、微信公众号、大众点评、企查查、招聘网站、闲鱼
most recent commit 2 years ago
Weibospider ⭐ 3,294
持续维护的新浪微博采集工具🚀🚀🚀
most recent commit a year ago
Distribute_crawler ⭐ 3,176
使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用re
most recent commit 8 years ago
Gerapy ⭐ 3,144
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
total releases 49most recent commit a year ago
Python3 Spider ⭐ 3,064
Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️
most recent commit a year ago
Scrapyd ⭐ 2,766
A service daemon to run Scrapy spiders
dependent packages 15total releases 11most recent commit a year agopypi scrapyd} Downloads
Webporter ⭐ 2,601
基于 webmagic 的 Java 爬虫应用
total releases 1most recent commit 3 years ago
Python 100 Days ⭐ 2,502
出处：https://github.com/jackfrued/Python-100-Days.gi
most recent commit 7 years ago
Feapder ⭐ 2,333
🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、
dependent packages 3total releases 173most recent commit a year agopypi feapder} Downloads
Image Downloader ⭐ 2,029
Download images from Google, Bing, Baidu. 谷歌、百度、必应图片下载.
most recent commit a year ago
Python Doc ⭐ 1,862
translate python documents to Chinese for convenient reference 简而言之，这里用来存放那些Python文档君们，并且尽力将其翻译成中文~~
most recent commit a year ago
Scrapely ⭐ 1,668
A pure-python HTML screen-scraping library
dependent packages 2total releases 13most recent commit 5 years agopypi scrapely} Downloads
Python Crawler ⭐ 1,576
从头开始 系统化的 学习如何写Python爬虫。 Python版本 3.6
most recent commit 3 years ago
The 10 Most Depended On Scrapy Open Source Projects
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96latest release September 18, 2023most recent commit a year agopypi Scrapy} Downloads
Scrapy Redis ⭐ 5,546
Redis-based components for Scrapy.
dependent packages 21total releases 18latest release July 26, 2022most recent commit 9 months agopypi scrapy-redis} Downloads
Scrapyd ⭐ 2,766
A service daemon to run Scrapy spiders
dependent packages 15total releases 11latest release September 25, 2023most recent commit a year agopypi scrapyd} Downloads
Node Scrapy ⭐ 114
Simple, lightweight and expressive web scraping with Node.js
dependent packages 11total releases 11latest release August 24, 2020most recent commit 5 years agonpm node-scrapy} Downloads
Icrawler ⭐ 792
A multi-thread crawler framework with many builtin image crawlers provided.
dependent packages 5total releases 42latest release June 30, 2023most recent commit a year agopypi icrawler} Downloads
Flake8 Scrapy ⭐ 18
A Flake8 plugin to catch common issues on Scrapy spiders
dependent packages 4total releases 1latest release October 10, 2018most recent commit 3 years agopypi flake8-scrapy} Downloads
Web Poet ⭐ 85
Web scraping Page Objects core library
dependent packages 4total releases 23latest release November 21, 2023most recent commit a year agopypi web-poet} Downloads
Scrapy Poet ⭐ 108
Page Object pattern for Scrapy
dependent packages 3total releases 23latest release November 02, 2023most recent commit a year agopypi scrapy-poet} Downloads
Scrapy Selenium ⭐ 842
Scrapy middleware to handle javascript pages using selenium
dependent packages 2total releases 6latest release January 24, 2019most recent commit a year agopypi scrapy-selenium} Downloads
The 10 Latest Releases In Scrapy Open Source Projects
Web Poet ⭐ 85
Web scraping Page Objects core library
dependent packages 4total releases 23latest release November 21, 2023most recent commit a year agopypi web-poet} Downloads
Scrapy Poet ⭐ 108
Page Object pattern for Scrapy
dependent packages 3total releases 23latest release November 02, 2023most recent commit a year agopypi scrapy-poet} Downloads
Webcomix ⭐ 26
Webcomic downloader
total releases 33latest release October 29, 2023most recent commit a year ago
Aio Scrapy ⭐ 43
Implement scrapy with asyncio
total releases 33latest release October 20, 2023most recent commit a year ago
Scrapyd ⭐ 2,766
A service daemon to run Scrapy spiders
dependent packages 15total releases 11latest release September 25, 2023most recent commit a year agopypi scrapyd} Downloads
Goscrapy ⭐ 34
GoScrapy: Harnessing Go's power for efficient web scraping, inspired by Python's Scrapy framework.
total releases 26latest release September 20, 2023most recent commit a year ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Kafka
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2025 Awesome Open Source.  All rights reserved.