Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Scrapy Open Source Projects

Open source projects categorized as Python Scrapy
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96most recent commit 3 months agopypi Scrapy} Downloads
Learn_python3_spider ⭐ 14,425
python爬虫教程系列、从0到1学习python爬虫，包括浏览器抓包，手机APP抓包，如 fiddler、mitmproxy，各种爬虫涉及的模块的使用，如：requests、beautifu 爬虫加密逆向破解，JS爬虫逆向，分布式爬虫，爬虫项目实战实例等
total releases 2most recent commit 3 months ago
Cs Book ⭐ 11,024
计算机类常用电子书整理，并且附带下载链接，包括Java，Python，Linux，Go，C，C++，
most recent commit 9 months ago
Portia ⭐ 8,982
Visual scraping for Scrapy
dependent packages 2total releases 26most recent commit 7 months agopypi slybot} Downloads
Pythonspidernotes ⭐ 6,609
Python入门网络爬虫之精华版
most recent commit 3 years ago
Wechatsogou ⭐ 5,753
基于搜狗微信搜索的微信公众号爬虫接口
total releases 25most recent commit 5 months ago
Splash ⭐ 3,860
Lightweight, scriptable browser as a service with an HTTP API
dependent packages 1total releases 30most recent commit 9 months agopypi splash} Downloads
Ecommercecrawlers ⭐ 3,724
实战🐍多种网站、电商数据爬虫🕷。包含🕸：淘宝商品、微信公众号、大众点评、企查查、招聘网站、闲鱼
most recent commit a year ago
Weibospider ⭐ 3,294
持续维护的新浪微博采集工具🚀🚀🚀
most recent commit 4 months ago
Distribute_crawler ⭐ 3,176
使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用re
most recent commit 7 years ago
Gerapy ⭐ 3,144
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
total releases 49most recent commit 3 months ago
Scrapydweb ⭐ 2,839
Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO 👉
total releases 18most recent commit 6 months ago
Scrapyd ⭐ 2,766
A service daemon to run Scrapy spiders
dependent packages 15total releases 11most recent commit 3 months agopypi scrapyd} Downloads
Python3 Spider ⭐ 2,582
Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️
most recent commit 6 months ago
Scrapy Examples ⭐ 2,550
Multifarious Scrapy examples. Spiders for alexa / amazon / douban / douyu / github / linkedin etc.
most recent commit 6 years ago
Python 100 Days ⭐ 2,502
出处：https://github.com/jackfrued/Python-100-Days.gi
most recent commit 6 years ago
Feapder ⭐ 2,333
🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、
dependent packages 3total releases 173most recent commit 3 months agopypi feapder} Downloads
Image Downloader ⭐ 2,029
Download images from Google, Bing, Baidu. 谷歌、百度、必应图片下载.
most recent commit 3 months ago
Python Doc ⭐ 1,862
translate python documents to Chinese for convenient reference 简而言之，这里用来存放那些Python文档君们，并且尽力将其翻译成中文~~
most recent commit 3 months ago
Python Crawler ⭐ 1,576
从头开始 系统化的 学习如何写Python爬虫。 Python版本 3.6
most recent commit 2 years ago
Scrapy Proxies ⭐ 1,376
Random proxy middleware for Scrapy
total releases 17most recent commit 5 years ago
Webscraping From 0 To Hero ⭐ 1,305
The web scraping open project repository aims to share knowledge and experiences about web scraping with Python
most recent commit 9 months ago
Weibo Search ⭐ 1,277
获取微博搜索结果信息，搜索即可以是微博关键词搜索，也可以是微博话题搜索
most recent commit 6 months ago
The 10 Most Depended On Python Scrapy Open Source Projects
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96latest release September 18, 2023most recent commit 3 months agopypi Scrapy} Downloads
Random_user_agent ⭐ 79
A package to get list of user agents based on filters such as operating system, software name etc..
dependent packages 30total releases 9latest release December 26, 2018most recent commit 10 months agopypi random-user-agent} Downloads
Itemadapter ⭐ 55
Common interface for data container classes
dependent packages 26total releases 17latest release August 02, 2022most recent commit 6 months agopypi itemadapter} Downloads
Scrapyd ⭐ 2,766
A service daemon to run Scrapy spiders
dependent packages 15total releases 11latest release September 25, 2023most recent commit 3 months agopypi scrapyd} Downloads
Anti Useragent ⭐ 34
fake pc or app browser useragent, anti useragent, and other awesome tools
dependent packages 8total releases 13latest release September 20, 2022most recent commit 2 years agopypi anti-useragent} Downloads
Scrapy Fake Useragent ⭐ 654
Random User-Agent middleware based on fake-useragent
dependent packages 5total releases 12latest release August 15, 2020most recent commit 7 months agopypi scrapy-fake-useragent} Downloads
Advertools ⭐ 975
advertools - online marketing productivity and analysis tools
dependent packages 5total releases 54latest release August 21, 2023most recent commit 3 months agopypi advertools} Downloads
Icrawler ⭐ 792
A multi-thread crawler framework with many builtin image crawlers provided.
dependent packages 5total releases 42latest release June 30, 2023most recent commit 3 months agopypi icrawler} Downloads
Faster Than Requests ⭐ 1,061
Faster requests on Python 3
dependent packages 4total releases 18latest release February 24, 2021most recent commit 3 months agopypi faster-than-requests} Downloads
Web Poet ⭐ 85
Web scraping Page Objects core library
dependent packages 4total releases 23latest release November 21, 2023most recent commit 3 months agopypi web-poet} Downloads
The 10 Latest Releases In Python Scrapy Open Source Projects
Web Poet ⭐ 85
Web scraping Page Objects core library
dependent packages 4total releases 23latest release November 21, 2023most recent commit 3 months agopypi web-poet} Downloads
Scrapy Poet ⭐ 108
Page Object pattern for Scrapy
dependent packages 3total releases 23latest release November 02, 2023most recent commit 3 months agopypi scrapy-poet} Downloads
Webcomix ⭐ 26
Webcomic downloader
total releases 33latest release October 29, 2023most recent commit 4 months ago
Aio Scrapy ⭐ 43
Implement scrapy with asyncio
total releases 33latest release October 20, 2023most recent commit 3 months ago
Scrapy Zyte Api ⭐ 30
Zyte API integration for Scrapy
dependent packages 1total releases 23latest release October 19, 2023most recent commit 3 months agopypi scrapy-zyte-api} Downloads
Scrapy Playwright ⭐ 747
🎭 Playwright integration for Scrapy
dependent packages 2total releases 32latest release October 19, 2023most recent commit 3 months agopypi scrapy-playwright} Downloads
Scrapyd ⭐ 2,766
A service daemon to run Scrapy spiders
dependent packages 15total releases 11latest release September 25, 2023most recent commit 3 months agopypi scrapyd} Downloads
Popular Programming Languages Categories
Python
Javascript
Java
Typescript
C Plus Plus
Php
Golang
C
C Sharp
Shell
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.