Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Webspider | 356 | 8 months ago | 9 | mit | Python | |||||
A website of IT position data & analysis, helps you to get a better understanding of the requirements and trends of the IT job market | ||||||||||
Celerystalk | 294 | 4 years ago | 24 | mit | Python | |||||
An asynchronous enumeration & vulnerability scanner. Run all the tools on all the hosts. | ||||||||||
Videospider | 178 | 4 years ago | 2 | mit | Python | |||||
抓取豆瓣,bilibili等中的电视剧、电影、动漫演员等信息 | ||||||||||
Spiders | 152 | 4 years ago | 1 | Python | ||||||
各种大小爬虫集合 | ||||||||||
Scrapyscript | 92 | 2 years ago | 10 | December 11, 2021 | 6 | mit | Python | |||
Run a Scrapy spider programmatically from a script or a Celery task - no project required. | ||||||||||
Ark | 58 | 8 years ago | Python | |||||||
分布式扫描框架 | ||||||||||
Spider_docker | 30 | 8 years ago | 1 | |||||||
为爬虫引用创建container,包括的模块:scrapy, mongo, celery, rabbitmq | ||||||||||
Deadpool | 22 | 4 years ago | Python | |||||||
该项目是一个使用celery作为主体框架的爬虫应用,能够灵活的添加爬虫任务,并且同时运行多站点的爬虫工作,所有组件都能够原生支持规模并发和分布式,加上celery原生的分布式调用,实现大规模并发。 | ||||||||||
Djangoscraper | 17 | 14 years ago | Python | |||||||
Django Scrapy App | ||||||||||
Antitools | 17 | 4 years ago | 1 | Python | ||||||
antitools |