Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Crawler Detect | 1,842 | 477 | 110 | 5 months ago | 156 | July 21, 2023 | 6 | mit | PHP | |
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent | ||||||||||
Crawlergo_x_xray | 915 | 2 years ago | 3 | Python | ||||||
360/0Kee-Team/crawlergo动态爬虫结合长亭XRAY扫描器的被动扫描功能 | ||||||||||
Awesome Scrapy | 450 | a year ago | 2 | |||||||
A curated list of awesome packages, articles, and other cool resources from the Scrapy community. | ||||||||||
Phpfetcher | 281 | 7 years ago | 10 | HTML | ||||||
A PHP web crawler framework | ||||||||||
Nginx Http User Agent | 155 | 5 years ago | 1 | C | ||||||
A nginx module to match browsers and crawlers | ||||||||||
Urlbuster | 115 | 3 years ago | 8 | January 30, 2021 | 1 | mit | Python | |||
Powerful mutable web directory fuzzer to bruteforce existing and/or hidden files or directories. | ||||||||||
Crawler_detect | 106 | 5 months ago | 23 | September 05, 2021 | 4 | mit | Ruby | |||
Ruby gem to detect bots and crawlers via the user agent | ||||||||||
Crawlerpack | 99 | 51 | 7 years ago | 9 | December 10, 2016 | apache-2.0 | Java | |||
Java 網路資料爬蟲包 | ||||||||||
Bots Zoo | 90 | 5 months ago | mit | JavaScript | ||||||
Es6 Crawler Detect | 88 | 4 | 6 months ago | 15 | January 06, 2022 | 1 | mit | JavaScript | ||
:spider: This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent. |