Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Crawlab | 10,521 | 3 months ago | 1 | March 03, 2019 | 58 | bsd-3-clause | Go | |||
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架 | ||||||||||
Browsertrix Crawler | 470 | 3 months ago | 91 | agpl-3.0 | JavaScript | |||||
Run a high-fidelity browser-based crawler in a single Docker container | ||||||||||
Morph | 454 | 2 years ago | 351 | agpl-3.0 | Ruby | |||||
Take the hassle out of web scraping | ||||||||||
Spidy | 287 | 2 years ago | 12 | January 25, 2018 | 11 | gpl-3.0 | Python | |||
The simple, easy to use command line web crawler. | ||||||||||
Zimit | 209 | 3 months ago | 31 | gpl-3.0 | Python | |||||
Make a ZIM file from any Web site and surf offline! | ||||||||||
Portia Dashboard | 190 | 6 years ago | 6 | other | Python | |||||
portia-dashboard is a visual web crawler based on scrapinghub/portia | ||||||||||
Gotor | 150 | 5 months ago | 3 | November 10, 2022 | 3 | gpl-3.0 | Go | |||
This program provides efficient web scraping services for Tor and non-Tor sites. The program has both a CLI and REST API. | ||||||||||
Estela | 142 | 3 months ago | 10 | mit | TypeScript | |||||
estela, an elastic web scraping cluster 🕸 | ||||||||||
Splashr | 88 | 4 years ago | 13 | other | R | |||||
:sweat_drops: Tools to Work with the 'Splash' JavaScript Rendering Service in R | ||||||||||
Scrapper | 83 | 3 months ago | apache-2.0 | JavaScript | ||||||
Web scraper with a simple REST API living in Docker and using a Headless browser and Readability.js for parsing. |