Awesome Open Source

Programming Languages

Search results for ruby scraper

135 search results found

Mechanize ⭐ 4,338

Mechanize is a ruby library that makes automated web interaction easy.

Wombat ⭐ 1,297

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

World_cup_json ⭐ 937

Rails backend for a scraper that outputs World Cup data as JSON

Kimuraframework ⭐ 874

Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Linkedin Scraper ⭐ 512

Scrapes the public profile of the linkedin page

Facebook_data_analyzer ⭐ 507

Analyze facebook copy of your data with ruby language. Download zip file from facebook and get info about friends ranking by message, vocabulary, contacts, friends added statistics and more

Take the hassle out of web scraping

Video_info ⭐ 407

Get video info from Dailymotion, Vimeo, Wistia, and YouTube URLs.

Market_bot ⭐ 361

Google Play Android App store scraper

Job Hunter ⭐ 298

Job scraper and applier for indeed.com

Webinspector ⭐ 286

Ruby gem to inspect completely a web page. It scrapes a given URL, and returns you its meta, links, images more.

Harvestman ⭐ 278

Quick and dirty web crawling.

Docs.konghq.com ⭐ 193

🦍 Source code for docs.konghq.com website.

A loose framework for crawling and scraping web sites.

Instagram Crawler ⭐ 157

Crawl instagram photos, posts and videos for download.

Nooverviewavailable.com ⭐ 155

A survey of Apple developer documentation.

LOOKING FOR A MAINTAINER

Nibbler ⭐ 142

A cute HTML scraper / data extraction tool in under 70 lines of code

Html2rss ⭐ 106

📰 Build RSS 2.0 feeds from websites (and JSON APIs) with a few CSS selectors.

An open API for the University of Maryland

Craigslist_scraper ⭐ 73

A gem to scrape and analyze craigslist data

Html2rss Web ⭐ 72

🕸 Generates and delivers RSS feeds via HTTP. Docker image available! Create your own feeds or get started quickly with the included configs.

Rubium is a lightweight alternative to Selenium/Capybara/Watir if you need to perform some operations (like web scraping) using Headless Chromium and Ruby

Bankjob is a command-line ruby program for scraping online banking sites and producing statements in OFX (Open Fincancial Exchange) or CSV (Comma Separated Values) formats.

Property_web_scraper ⭐ 61

Web based UI to make scraping data from real estate websites super simple

Conformist ⭐ 60

Bend CSVs to your will with declarative schemas.

Journal Scrapers ⭐ 54

Journal scraper definitions for the ContentMine framework

Watchdog ⭐ 53

IF (接口/网页有变化) THEN (提醒你)

Domain-specific language for extracting structured data from HTML documents

Yasuri (鑢) is easy web scraping library.

Nokogiri Ruby Tutorial ⭐ 41

Data Scraping And More With Ruby, Nokogiri, Sinatra and Heroku Tutorial - sample code

Ruby Instagram Scraper ⭐ 40

А simple module for requests to Instagram without an API key.

Ronin Web ⭐ 40

ronin-web is a collection of useful web helper methods and commands.

Formless ⭐ 40

Completely transparent, unobtrusive form populator for web applications and content scrapers.

🗡 Maxwell makes web scraping more simpler and faster with Ruby.

Gscraper ⭐ 33

A web-scraping interface to various Google Services.

Monkeyshines ⭐ 31

A simple, lightweight scraper for huffing API or feed data (rather than a page-by-page wander)

Espn Scraper ⭐ 30

a simple gem for scraping teams and scores from ESPN

Spider is a Web spidering library for Ruby. It handles the robots.txt, scraping, collecting, and looping so that you can just handle the data.

German Gov Domains ⭐ 28

An incomplete listing of german government domains

URL metadata API for scraping titles, descriptions, images, and videos from URL's.

Fb_scrape ⭐ 27

A simple scraper for Facebook Groups.

Worm_scraper ⭐ 26

(Broken, Not maintained) A Craigslist listings scraper.

Scrapifier ⭐ 24

Simple way to extract meta information from URIs.

Igscraperkit ⭐ 23

Create dynamic web scraper in Objective-C or Ruby!

Image_scraper ⭐ 22

simple utility that pulls image URLs from a web page

Bnet_scraper ⭐ 22

A Nokogiri-based scraper of Battle.net profiles. Currently this only includes Starcraft2.

Scrape Framework ⭐ 22

A web / screen scraper framework written in ruby that assumes scrapes should be configured, not programmed. No data is safe with the Scrape Framework on the loose!

Unsong_scraper ⭐ 21

Unsong for kindle, based on https://github.com/rhelsing/worm_scraper

Disease Info ⭐ 21

Repo for disease info; a health information data scrapper. Raw data from the scrapper's API is displayed in JSON -

Html2rss Configs ⭐ 21

📇 A growing collection of html2rss feed configs. Generate configs with the handy generator.

Twfy_local_parser ⭐ 21

Local scraper parser to scrape and parse local authority data

Berlin_corona_cases ⭐ 20

Scraper for the official dashboard with current Corona case numbers, traffic light indicators ("Corona-Ampel") and vaccination situation for Berlin.

Kimurai is a modern web scraping framework written in Ruby which works out of box with headless chromium/firefox, phantomjs, or simple HTTP requests and allows to scrape and interact with javascript rendered websites

Utilityscripts ⭐ 19

Scripts for managing scrapers

Video_scraper ⭐ 18

Web scraping library for video sharing sites.

Mangascrap ⭐ 18

MangaScrap is a Ruby based scrapper that generates a local website with the downloaded mangas

Wowaudit ⭐ 18

WoW Audit Spreadsheet

Currency scraper for the Dominican Republic

Adwords_scraper ⭐ 18

scraping google adwords ads

Vulnscrape ⭐ 18

xss vulnerability scraper

Advanced Web Scraping ⭐ 17

Web_scrapers_ruby ⭐ 16

Samples of Web Scraper using Mechanize, Delayed Job, Parallel, Mongo, etc.

Freelance Job Scraper ⭐ 16

This is a freelance job scraper built using Ruby by following OOP. It takes keywords from the user input, scrapes 3 job sites and exports them to 3 separate CSV files.

Uk_planning_scraper ⭐ 16

A Ruby gem to get planning applications data from UK council websites.

Simply awesome web scraping with Nokogiri

Virginia Tech Hacks ⭐ 16

A Blacksburg Transit CLI, A Drop/Add Script to register you for a class when someone drops it, A timetable scraper, and more..

Pagescan ⭐ 15

PageScan is a web content scraper for web-based malware analysis. It assist analyst by detecting and listing any redirection, iframe, javascript, and links found inside the web page.

Google Seo Allintitle Scraper ⭐ 15

Google SEO scraper for "allintitle:keyword" queries.

Python Tutorials ⭐ 15

Learning notes of python

Spidey Mongo ⭐ 15

Implements a MongoDB back-end for Spidey (https://github.com/joeyAghion/spidey), a framework for crawling and scraping web sites.

Hsbcscraper ⭐ 14

Web scraper for downloading statements from HSBC Business Banking

Ruby Web Scraper Telegram Bot

Proxycrawl Ruby ⭐ 14

ProxyCrawl API ruby gem for scraping and crawling

Amz_reviews ⭐ 14

Amazon review scraper - download as CSV with auto saving and full pagination

Tadeurubycapstone ⭐ 14

Back-end project. The purpose here was to create a web scraper, which retrieves someone's Github repositories info, by searching their username.

Web_scraper_ruby ⭐ 13

The program is implementing a scraper for search results of a job offers website. This scraper helps you to search by keywords on https://www.indeed.com webpage, to scrape the results and to display them in the terminal, or export it into a CSV or a Text files.

Irasutoya ⭐ 12

👩 Ruby library for いらすとや

Web_scraper ⭐ 12

An application designed to scrap the web and retrieve information from movie websites. Built with Ruby.

Covid19br Pub ⭐ 12

Projeto de monitoramento de publicações oficiais relacionadas a COVID-19 no Brasil.

Scraperwiki Ruby ⭐ 12

ScraperWiki Ruby library for scraping and saving data

Securities ⭐ 11

Financial information scraper gem.

Airport_scraper ⭐ 11

Code for scraping airports from freeform text

Gmail Scraper Gem ⭐ 10

(Fast) Gmail's scraper to extract conversations and emails

Tvdb Scraper ⭐ 10

Scrapes TV metadata from thetvdb

Fail-fast web scraping

Kindle The Economist ⭐ 10

Creates Kindle compatible mobi files of the print edition by scraping it from the economist website

Quora Scraper ⭐ 10

Quora Question Scraper - Find & Export relevant Questions 10x faster

Simple Scraper ⭐ 10

A fairly simple gem that will help you simplify the parsing of web pages.

Web Scraper Ruby Capstone Project ⭐ 10

Webrat Scraper ⭐ 9

A web scraper using Webrat::MechanizeSession - does acceptance-based web scraping relying on the great webrat behavior description language.

When you need those jobs hypersonic 🚀 scrape 🔪

Scrapeybara ⭐ 9

A web scraping tool based on Capybara. This project is deprecated.

Tess_scrapers ⭐ 9

TeSS HTML page scrapers in Ruby looking for training resources and events metadata.

Twitter Scraper ⭐ 9

A scraper that uses the twitter API to download pertinent data to text files

ruby wrapper/scraper for last.fm

Yahoo Stock ⭐ 9

A Yahoo! stocks scraper

A scraper for capturing flight arrival/departure information for Heathrow. Will hopefully feed a twitter bot or something

Related Searches

Ruby Command Line (35,999)

Javascript Ruby (6,657)

Ruby Plugin (6,573)

Ruby Chef (4,661)

Ruby Testing (4,020)

Ruby Sinatra (3,377)

Ruby Rspec (3,278)

Ruby Activerecord (3,234)

Ruby Heroku (2,928)

Ruby Vagrant (2,919)

1-100 of 135 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.