Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Spider Scrapy Open Source Projects

Open source projects categorized as Python Spider Scrapy
Hk0weather ⭐ 49
Web scraper project to collect the useful Hong Kong weather data from HKO website
most recent commit 8 months ago
Scrapy Boilerplate ⭐ 49
Small set of utilities to simplify writing Scrapy spiders.
total releases 3most recent commit 9 years ago
Scrapy Pyppeteer ⭐ 48
Use pyppeteer from a Scrapy spider
most recent commit 4 years ago
Github Trending ⭐ 47
GitHub trending repositories and developers APIs for real time, powered by crawlers | 通过爬虫获取 GitHub 热门项目和开发者的实时 API
most recent commit 2 months ago
Scrapy Do ⭐ 46
A daemon for scheduling Scrapy spiders
most recent commit 3 years ago
Zhihu_spider ⭐ 46
Scrapy the Zhihu content and user social network information
most recent commit 10 years ago
Scrapy Puppeteer ⭐ 45
Library that helps use puppeteer in scrapy.
dependent packages 1total releases 10most recent commit 4 months agopypi scrapy-puppeteer-client} Downloads
Devsearch ⭐ 45
A web search engine built with Python which uses TF-IDF and PageRank to sort search results.
most recent commit 3 years ago
Scrapybook 2nd Edition ⭐ 45
Scrapy Book 2nd Edition Code http://scrapybook.com/
most recent commit 2 years ago
Reddit ⭐ 44
Scrapy (Python Framework) Example using reddit.com
most recent commit 6 years ago
Aio Scrapy ⭐ 43
Implement scrapy with asyncio
total releases 33most recent commit 4 months ago
T66y Spider ⭐ 43
crawler for t66y
most recent commit 7 years ago
Warcmiddleware ⭐ 42
WarcMiddleware lets users seamlessly download a mirror copy of a website when running a web crawl with the Python web crawler Scrapy.
most recent commit 6 years ago
Scrapy Autounit ⭐ 42
Automatic unit test generation for Scrapy.
total releases 32most recent commit 3 years ago
Baikespider ⭐ 42
BaiKe Spider
most recent commit 3 years ago
Scrapy Admin ⭐ 40
A django admin site for scrapy
most recent commit 7 years ago
Scrapy Distributed ⭐ 40
A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.
total releases 8most recent commit a year ago
Scrapy Magicfields ⭐ 40
Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
total releases 2most recent commit 6 years ago
Heartsong ⭐ 40
A simple spider power by scrapy, aimed to crawl forums power by discuz .
most recent commit 7 years ago
Scrapy Autoextract ⭐ 39
Zyte Automatic Extraction integration for Scrapy
total releases 11most recent commit 2 years ago
Financial Market Data Analysis ⭐ 39
Real-Time Financial Market Data Processing and Prediction application
most recent commit 2 years ago
Jd_spider ⭐ 39
👍 京东爬虫（大量注释，对刚入门爬虫者极度友好）
most recent commit 5 years ago
Portia2code ⭐ 39
total releases 14most recent commit 6 years ago
Popular Programming Languages Categories
Python
Javascript
Java
Typescript
C Plus Plus
Php
Golang
C
C Sharp
Shell
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.