Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Scrapy Open Source Projects

Open source projects categorized as Python Scrapy
Livetv_mining ⭐ 190
直播网站数据采集
most recent commit 7 years ago
Scrapyz ⭐ 188
"Scrape Easy" - an extension of the Scrapy framework.
total releases 6most recent commit 8 years ago
Pigat ⭐ 187
pigat ( Passive Intelligence Gathering Aggregation Tool ) 被动信息收集聚合工具
most recent commit 2 years ago
Zi5book ⭐ 183
book.zi5.me全站kindle电子书籍爬取，按照作者书籍名分类，每本书有mobi和equb两
most recent commit 5 years ago
Scrapy Samples ⭐ 183
Scrapy examples crawling Craigslist
most recent commit 8 years ago
Aadhaarsearchengine ⭐ 179
Find Aadhaar cards thanks to Google
most recent commit 5 years ago
China_stock_announcement ⭐ 173
该项目通过scrapy爬虫从巨潮网络的服务器获取中国股市的公告
most recent commit 4 years ago
Jobspiders ⭐ 171
scrapy框架爬取51job(scrapy.Spider)，智联招聘(扒接口)，拉勾网(Crawl
most recent commit a year ago
Scrapy_example ⭐ 170
This repository store some example to learn scrapy better
most recent commit 4 years ago
Scrapy Random Useragent ⭐ 170
Scrapy Middleware to set a random User-Agent for every Request.
total releases 2most recent commit 5 years ago
Qqmusicspider ⭐ 168
基于Scrapy的QQ音乐爬虫(QQ Music Spider)，爬取歌曲信息、歌词、精彩评论等，并且分享了QQ音乐中排名前6400名的内地和港台歌手
most recent commit 2 years ago
Maria Quiteria ⭐ 168
Backend para coleta e disponibilização dos dados 📜
most recent commit 7 months ago
Movie_rating_prediction ⭐ 168
Predict movie's IMDB rating
most recent commit 5 years ago
Wenshu_spider ⭐ 166
🌈Wenshu_Spider-Scrapy框架爬取中国裁判文书网案件数据(2019-1-9最新版)
most recent commit 5 years ago
Scrapy Dynamic Configurable ⭐ 160
A dynamic configurable news crawler based Scrapy
most recent commit 7 years ago
Weibo_scrapy ⭐ 154
WEIBO_SCRAPY is a Multi-Threading SINA WEIBO data extraction Framework in Python.
most recent commit 7 years ago
Fp Server ⭐ 154
Free proxy server, continuously crawling and providing proxies, based on Tornado and Scrapy. 免费代理服务器，基于Tornado和Scrapy，在本地搭建属于自己的代理池
most recent commit 2 years ago
Pythoncrawler Scrapy Mysql File Template ⭐ 153
scrapy爬虫框架模板，将数据保存到Mysql数据库或者文件中。
most recent commit 7 years ago
Airbnb Scraper ⭐ 153
Airbnb Scraper: Advanced Airbnb Search using Scrapy
most recent commit 2 years ago
Hncrawl ⭐ 150
A scrapy-based Hacker News crawler.
most recent commit 11 years ago
Scrapy_demo ⭐ 150
all kinds of scrapy demo
most recent commit a year ago
Django Covid19 ⭐ 150
实时接口获取中国各个城市、省份、国家的新型冠状肺炎（新冠肺炎 / 2019-nCoV / Covid-19）。疫情数据以及整体统计详情，新增美国各州统计、每日疫情数据 API。爬虫实时追踪新冠疫情变化，数据来自丁香园和 covidtracking.com。数据大屏示例：http://ncov.leafcoder.cn/ 项目文档：http://ncov.leafcoder.cn/docs/
total releases 8most recent commit a year ago
Zhihuspider ⭐ 149
知乎分布式爬虫（Scrapy、Redis）
most recent commit 6 years ago
The 10 Latest Releases In Python Scrapy Open Source Projects
Scrapy Sqs Exporter ⭐ 6
Scrapy extension for outputting scraped items to an Amazon SQS instance
total releases 6latest release June 11, 2018most recent commit a year ago
Scrapy Httpcache ⭐ 8
A scrapy middleware to save http cache to MongoDB
total releases 5latest release May 09, 2018most recent commit 2 years ago
Scrapy Statsd ⭐ 7
Publish Scrapy stats to statsd daemon
total releases 2latest release April 14, 2018most recent commit 6 years ago
Portia2code ⭐ 39
total releases 14latest release March 27, 2018most recent commit 6 years ago
Tweetscraper ⭐ 698
TweetScraper is a simple crawler/spider for Twitter Search without using API
total releases 8latest release March 07, 2018most recent commit 3 years ago
Scrapy Cloudflare Middleware ⭐ 32
A Scrapy middleware to bypass the CloudFlare's anti-bot protection
total releases 2latest release February 10, 2018most recent commit 5 years ago
Scrapy Pipeline Mongodb ⭐ 7
A pipeline of saving item into MongoDB
total releases 5latest release December 23, 2017most recent commit 5 years ago
Scrapy Proxies ⭐ 1,376
Random proxy middleware for Scrapy
total releases 17latest release December 21, 2017most recent commit 5 years ago
Popular Programming Languages Categories
Python
Javascript
Java
Typescript
C Plus Plus
Php
Golang
C
C Sharp
Shell
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.