Crawler Benchmark
A Reference Framework for the Automated Exploration of Web Applications. Provides some general web features to let you test crawlers in a well defined environment.
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Crawlab | 10,521 | 3 months ago | 1 | March 03, 2019 | 58 | bsd-3-clause | Go | |||
| Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架 | ||||||||||
| Proxypool | 5,154 | 4 months ago | 40 | mit | Python | |||||
| An Efficient ProxyPool with Getter, Tester and Server | ||||||||||
| Gerapy | 3,144 | 8 | 3 months ago | 49 | July 19, 2023 | 60 | mit | Python | ||
| Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js | ||||||||||
| Packtpub Crawler | 701 | 4 years ago | 15 | mit | Python | |||||
| Download your daily free Packt Publishing eBook https://www.packtpub.com/packt/offers/free-learning | ||||||||||
| Newcrawler | 583 | 5 months ago | 25 | JavaScript | ||||||
| Free Web Scraping Tool with Java | ||||||||||
| Crawljax | 493 | 28 | 6 | 10 months ago | 18 | June 01, 2023 | 67 | apache-2.0 | Java | |
| Crawljax | ||||||||||
| Browsertrix Crawler | 470 | 3 months ago | 91 | agpl-3.0 | JavaScript | |||||
| Run a high-fidelity browser-based crawler in a single Docker container | ||||||||||
| Scrapybook | 378 | 6 years ago | 9 | Python | ||||||
| Scrapy Book Code | ||||||||||
| Spidy | 287 | 2 years ago | 12 | January 25, 2018 | 11 | gpl-3.0 | Python | |||
| The simple, easy to use command line web crawler. | ||||||||||
| Go Movies | 232 | 5 months ago | 2 | mit | Go | |||||
| golang spider Crawler 爬虫 电影 | ||||||||||
Alternatives To Crawler BenchmarkSelect To Compare
Alternative Project Comparisons
Popular Crawler Projects
Popular Docker Projects
Popular Data Processing Categories
Related Searches
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Css
Docker
Flask
Docker Image
Crawler