Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Jlitespider | 151 | 7 years ago | 3 | February 12, 2017 | 2 | apache-2.0 | Java | |||
A lite distributed Java spider framework :-) | ||||||||||
Scrapy Rabbitmq | 76 | 1 | 2 years ago | 2 | July 02, 2015 | 3 | mit | Python | ||
A RabbitMQ Scheduler for Scrapy | ||||||||||
Distributed Multi User Scrapy System With A Web Ui | 71 | 6 years ago | 5 | Python | ||||||
Django based application that allows creating, deploying and running Scrapy spiders in a distributed manner | ||||||||||
Scrapy Distributed | 40 | a year ago | 8 | February 20, 2021 | 10 | Python | ||||
A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy. | ||||||||||
Scrapy Boilerplate | 32 | a year ago | 16 | mit | Python | |||||
Scrapy project boilerplate done right | ||||||||||
Spider_docker | 30 | 8 years ago | 1 | |||||||
为爬虫引用创建container,包括的模块:scrapy, mongo, celery, rabbitmq | ||||||||||
Saushengine | 14 | 10 years ago | Ruby | |||||||
Rb Spider | 14 | 6 years ago | 1 | mit | Ruby | |||||
基于 RabbitMQ 中间件的爬虫的 Ruby 实现 [Developing] | ||||||||||
Scrapy Mq Redis | 10 | 8 years ago | gpl-3.0 | Python | ||||||
A RabbitMQ/Redis tool for Scrapy | ||||||||||
Aragog | 9 | 2 | 2 years ago | 9 | December 15, 2021 | mit | TypeScript | |||
Distributed web scraping framework |