Scrapy Kafka Export
Scrapy extension which writes crawled items to Kafka
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Movie_recommend | 1,441 | 5 years ago | 10 | mit | Java | |||||
| 基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统 | ||||||||||
| Scrapy Cluster | 1,137 |  | 18 | 2 | 6 months ago | 15 | December 23, 2020 | 17 | mit | Python |
| This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster. | ||||||||||
| Spiderman | 498 | a year ago | 3 | mit | Python | |||||
| 基于 scrapy-redis 的通用分布式爬虫框架 | ||||||||||
| Sourcecodeofbook | 255 | 2 years ago | 4 | mit | Python | |||||
| 《Python爬虫开发 从入门到实战》配套源代码。 | ||||||||||
| Scrapy_demo | 150 | a year ago | 2 | Python | ||||||
| all kinds of scrapy demo | ||||||||||
| Scrapy Kafka | 63 | 6 years ago | 1 | August 14, 2015 | 1 | apache-2.0 | Python | |||
| Kafka-based components for Scrapy | ||||||||||
| Scrapy Distributed | 40 | a year ago | 8 | February 20, 2021 | 10 | Python | ||||
| A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy. | ||||||||||
| Scrapy Kafka Redis | 35 | 3 years ago | 5 | July 24, 2018 | apache-2.0 | Python | ||||
| Distributed crawling/scraping, Kafka And Redis based components for Scrapy | ||||||||||
| Blog | 34 | 5 months ago | 30 | |||||||
| my tech blog, using markdown | ||||||||||
| Blog | 24 | 2 years ago | ||||||||
| 博客 | ||||||||||
Alternatives To Scrapy Kafka ExportSelect To Compare
Alternative Project Comparisons
Popular Kafka Projects
Popular Scrapy Projects
Popular Data Processing Categories
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Kafka
Ssl
Scrapy