Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Icrawler | 792 | 11 | 5 | 4 months ago | 42 | June 30, 2023 | 30 | mit | Python | |
A multi-thread crawler framework with many builtin image crawlers provided. | ||||||||||
Fkdownloader | 225 | 4 years ago | 33 | June 11, 2020 | 1 | mit | Objective-C | |||
👍🏻📥Maybe the best file downloader. | ||||||||||
Scrapy Monitor | 99 | 7 years ago | 1 | mit | Python | |||||
scrapy-monitor,实现爬虫可视化,监控实时状态 | ||||||||||
Android Apps Crawler | 88 | 8 years ago | Python | |||||||
An extensible crawler for downloading Android applications in third-party markets. | ||||||||||
Nscrapy | 82 | 2 years ago | 12 | apache-2.0 | C# | |||||
NScrapy is a .net core corss platform Distributed Spider Framework which provide an easy way to write your own Spider | ||||||||||
Dotnetcrawler | 63 | 4 years ago | C# | |||||||
DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c | ||||||||||
Recruitment | 57 | 7 years ago | 4 | JavaScript | ||||||
A project crawling online recruitment websites getting offer information | ||||||||||
Scrapy Pyppeteer | 48 | 4 years ago | 1 | Python | ||||||
Use pyppeteer from a Scrapy spider | ||||||||||
Scrapy Autoextract | 39 | 1 | 2 years ago | 11 | August 05, 2021 | 6 | bsd-3-clause | Python | ||
Zyte Automatic Extraction integration for Scrapy | ||||||||||
Met_book_downloader | 30 | 6 years ago | Python | |||||||