Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Ruby Web Crawler Open Source Projects

Open source projects categorized as Ruby Web Crawler
Spidr ⭐ 775
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
dependent packages 16total releases 26most recent commit 3 months agorubygems spidr} Downloads
Morph ⭐ 454
Take the hassle out of web scraping
most recent commit 2 years ago
Archivebot ⭐ 328
ArchiveBot, an IRC bot for archiving websites
most recent commit 5 months ago
Awesomewebscraping ⭐ 189
List of libraries, tools and APIs for web scraping and data processing.
most recent commit 7 months ago
Nokolexbor ⭐ 149
High-performance HTML5 parser for Ruby based on Lexbor, with support for both CSS selectors and XPath.
dependent packages 3total releases 20most recent commit 3 months agorubygems nokolexbor} Downloads
Direct_web_spider ⭐ 143
A direct web spider framworks for Ruby
most recent commit 5 years ago
Polipus ⭐ 95
Polipus: distributed and scalable web-crawler framework
dependent packages 3total releases 20most recent commit 9 years agorubygems polipus} Downloads
Hawker ⭐ 70
The Hawker gem is a web scraper which allows you to pull the basic information for given social media profile URL
most recent commit 6 years ago
Youtube Audio ⭐ 66
extract videos from youtube in audio format using webscraping techniques 🎶
total releases 3most recent commit 4 years ago
Faexport ⭐ 62
The API for Furaffinity you wish existed
most recent commit 7 months ago
Clauneck ⭐ 57
A tool for scraping emails, social media accounts, and much more information from websites using Google Search Results.
total releases 2most recent commit 9 months ago
Slurp ⭐ 54
One repo to rule them all !!?!?!! 🤓 😎
most recent commit 3 months ago
Ronin Web ⭐ 40
ronin-web is a collection of useful web helper methods and commands.
dependent packages 3total releases 16most recent commit 4 months agorubygems ronin-web} Downloads
Grell ⭐ 36
Web crawler with a Ruby API
total releases 21most recent commit 3 years ago
Linkextractor ⭐ 31
A Docker tutorial using a link extraction application example
most recent commit 3 years ago
Boilerpipe Ruby ⭐ 30
Pure ruby implementation of the Boilerpipe content extraction algorithm tuned for online articles
total releases 11most recent commit 3 years ago
Danger Mention ⭐ 29
Danger plugin to automatically mention potential reviewers on pull requests
total releases 6most recent commit 7 years ago
Deviantart Gallery Downloader ⭐ 25
fetch deviantart's images using mechanize
most recent commit 2 years ago
Igscraperkit ⭐ 23
Create dynamic web scraper in Objective-C or Ruby!
total releases 5most recent commit 9 years ago
Wsoc ⭐ 23
The Web Spider Obstacle Course
dependent packages 1total releases 5most recent commit 6 years agorubygems wsoc} Downloads
Klepto ⭐ 21
A mean little DSL'd poltergeist (capybara) based web crawler that stuffs data into your Rails app.
total releases 44most recent commit 11 years ago
Halfstaff ⭐ 20
🇺🇸 Is the US flag at half-staff?
most recent commit 6 years ago
Web_scrapers_ruby ⭐ 16
Samples of Web Scraper using Mechanize, Delayed Job, Parallel, Mongo, etc.
most recent commit 13 years ago
The 6 Most Depended On Ruby Web Crawler Open Source Projects
Ronin Web ⭐ 40
ronin-web is a collection of useful web helper methods and commands.
dependent packages 3total releases 16latest release April 04, 2023most recent commit 4 months agorubygems ronin-web} Downloads
Polipus ⭐ 95
Polipus: distributed and scalable web-crawler framework
dependent packages 3total releases 20latest release July 17, 2015most recent commit 9 years agorubygems polipus} Downloads
Nokolexbor ⭐ 149
High-performance HTML5 parser for Ruby based on Lexbor, with support for both CSS selectors and XPath.
dependent packages 3total releases 20latest release October 03, 2023most recent commit 3 months agorubygems nokolexbor} Downloads
Wgit ⭐ 10
Wgit allows you to crawl and extract the data you want from the web
dependent packages 1total releases 34latest release November 26, 2021most recent commit 3 months agorubygems wgit} Downloads
Wsoc ⭐ 23
The Web Spider Obstacle Course
dependent packages 1total releases 5latest release December 09, 2011most recent commit 6 years agorubygems wsoc} Downloads
The 10 Latest Releases In Ruby Web Crawler Open Source Projects
Nokolexbor ⭐ 149
High-performance HTML5 parser for Ruby based on Lexbor, with support for both CSS selectors and XPath.
dependent packages 3total releases 20latest release October 03, 2023most recent commit 3 months agorubygems nokolexbor} Downloads
Socials_regex ⭐ 8
🪡 Social account detection and extraction in ruby, e.g. for crawling/scraping.
total releases 1latest release July 02, 2023most recent commit 7 months ago
Ronin Web ⭐ 40
ronin-web is a collection of useful web helper methods and commands.
dependent packages 3total releases 16latest release April 04, 2023most recent commit 4 months agorubygems ronin-web} Downloads
Wgit ⭐ 10
Wgit allows you to crawl and extract the data you want from the web
dependent packages 1total releases 34latest release November 26, 2021most recent commit 3 months agorubygems wgit} Downloads
Grell ⭐ 36
Web crawler with a Ruby API
total releases 21latest release February 17, 2021most recent commit 3 years ago
Mtg Db ⭐ 8
Ruby gem containing structured data for all Magic: The Gathering cards
total releases 112latest release January 22, 2021most recent commit 3 years ago
Arachnid2 ⭐ 8
a simple, fast web-crawler written in Ruby using Watir or Typhoeus
total releases 17latest release July 15, 2020most recent commit 4 years ago
Popular Programming Languages Categories
Python
Javascript
Java
Typescript
C Plus Plus
Php
Golang
C
C Sharp
Shell
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.