Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Autocrawler | 1,454 | 6 months ago | 11 | apache-2.0 | Python | |||||
Google, Naver multiprocess image web crawler (Selenium) | ||||||||||
Icrawler | 792 | 11 | 5 | 4 months ago | 42 | June 30, 2023 | 30 | mit | Python | |
A multi-thread crawler framework with many builtin image crawlers provided. | ||||||||||
Tumblr_crawler | 258 | 6 years ago | 2 | gpl-3.0 | Python | |||||
This is a Multi-thread crawler for Tumblr. | ||||||||||
Drill | 256 | a year ago | 20 | gpl-2.0 | C# | |||||
Search files without indexing, but clever crawling | ||||||||||
4chan Downloader | 251 | 5 months ago | 2 | mit | Python | |||||
Python3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation | ||||||||||
Pornhubbot | 144 | 2 years ago | 5 | mit | Python | |||||
基于Python3的pornhub网站爬虫 | ||||||||||
Pylinkvalidator | 85 | 1 | 5 years ago | 3 | August 18, 2015 | 18 | other | Python | ||
pylinkvalidator is a standalone and pure python link validator and crawler that traverses a web site and reports errors (e.g., 500 and 404 errors) encountered. | ||||||||||
Tumblr Crawler Cli | 74 | 5 years ago | 3 | mit | Python | |||||
Tumblr Download Tool with High Speed and Customization. 高性能&高定制化的Tumblr下载工具。 | ||||||||||
Link Checker | 74 | 4 years ago | 26 | mit | JavaScript | |||||
A Ruby gem for checking the links in a web site. Can either scan files or crawl pages. Multi-threaded, with red/green colored output, support for SSL, and support for following redirects. Works great with Octopress, Jekyll, or any collection of static HTML files. With 100% RSpec coverage. | ||||||||||
Pronhubspider | 50 | 7 months ago | 2 | Python | ||||||
pornhubをクロールしているWebHubBotプロジェクトの模倣、効率が遅すぎる、方法を探しています |