Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Icrawler | 792 | 11 | 5 | a year ago | 42 | June 30, 2023 | 30 | mit | Python | |
A multi-thread crawler framework with many builtin image crawlers provided. | ||||||||||
Zhihu Spider | 719 | 10 years ago | 1 | mit | Python | |||||
A web spider for zhihu.com | ||||||||||
Qqzonemood | 499 | 2 years ago | mit | Python | ||||||
QQZone mood spider and analysis. QQ空间多线程爬虫和数据挖掘。提供线上服务,扫码登陆即可自动爬取和分析数据,还有网易云年度报告风格的数据展示;使用docker-compose打包程序,方便部署;额外提供QQ空间抽奖小程序。 | ||||||||||
Tieba_spider | 298 | 3 years ago | 5 | Python | ||||||
百度贴吧爬虫(基于scrapy和mysql) | ||||||||||
Itsy | 168 | 6 | 10 years ago | 3 | October 05, 2015 | 5 | Clojure | |||
A threaded web-spider written in Clojure | ||||||||||
Arachnid | 38 | 6 | 11 years ago | 12 | January 17, 2014 | 1 | Ruby | |||
Extremely fast and efficient Ruby domain spider | ||||||||||
Scrapy Flask | 34 | 7 years ago | 1 | Python | ||||||
Execute Scrapy spiders in a Flask web application | ||||||||||
Detectorist Scraper | 19 | 9 years ago | 1 | Python | ||||||
A scrapy spider to extract post, thread, and user information from a vBulletin forum to a MongoDB database. | ||||||||||
Wspider | 15 | 2 years ago | gpl-3.0 | Shell | ||||||
Probably one of the fastest crawler/spider tool around, mirror any website faster than you thought was possible | ||||||||||
Bitcointalk Ann | 10 | 6 years ago | 1 | HTML | ||||||
Scrapes 1000+ thread into a single html document for better reading and analysis |