Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Webrobots | 24 | 4,142 | 9 | 9 months ago | 16 | January 04, 2016 | 2 | bsd-2-clause | Ruby | |
A Ruby library to help write robots.txt compliant web robots | ||||||||||
Waterplus_map_tools | 12 | a year ago | gpl-2.0 | C++ | ||||||
Map tools for the Waterplus Robots | ||||||||||
Robots | 6 | 1 | 3 years ago | 6 | February 03, 2021 | 8 | bsd-3-clause | Java | ||
Support for Robots Exclusion Protocol, including parsing and matching against robot.txt directives. | ||||||||||
Robot_rules | 6 | 15 years ago | 1 | August 11, 2014 | mit | Ruby | ||||
A tool to determine if the robots.txt would prevent a given user agent from making a request to a given URI. |