The Top 23 Scrape Open Source Projects

Open source projects categorized as Scrape

Categories > Scrape

Scraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username

most recent commit 3 months ago

Portia Dashboard ⭐ 190

portia-dashboard is a visual web crawler based on scrapinghub/portia

most recent commit 6 years ago

Scrapyz ⭐ 188

"Scrape Easy" - an extension of the Scrapy framework.

total releases 6most recent commit 8 years ago

Scrapy Samples ⭐ 183

Scrapy examples crawling Craigslist

most recent commit 8 years ago

Instagram Follower Scraper ⭐ 181

Scrapes all the data of followers of any instagram account

most recent commit 10 months ago

Cloudflare Exporter ⭐ 175

Prometheus CloudFlare Exporter

most recent commit a year ago

Link Preview Generator ⭐ 171

Get preview data (a title, description, image, domain name) from a url. Library uses puppeteer headless browser to scrape the web site.

dependent packages 6total releases 10most recent commit 2 years ago

Demeter ⭐ 162

Demeter is a tool for scraping the calibre web ui

total releases 5most recent commit 3 years ago

Instagram Screen Scrape ⭐ 162

scrape public instagram data w/out API access

dependent packages 7total releases 5most recent commit 7 years ago

Atp World Tour Tennis Data ⭐ 162

Using Python to scrape ATP World Tour tennis data

most recent commit 8 months ago

Prometheus To Cloudwatch ⭐ 159

Utility for scraping Prometheus metrics from a Prometheus client endpoint and publishing them to CloudWatch

total releases 2most recent commit 4 months ago

Worm Scraper ⭐ 158

Scrapes the web serial Worm and its sequel Ward into an eBook format

dependent packages 1total releases 29most recent commit 2 years ago

Nginx_exporter ⭐ 153

Simple server that scrapes nginx stats and exports them via HTTP for Prometheus consumption

most recent commit 2 years ago

Media Scraper ⭐ 150

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

most recent commit 4 years ago

Laravel Scavenger ⭐ 150

The most integrated web scraper package for Laravel.

total releases 24most recent commit 3 months ago

Hncrawl ⭐ 150

A scrapy-based Hacker News crawler.

most recent commit 11 years ago

Scrapy Webdriver ⭐ 147

most recent commit 8 years ago

Inputscanner ⭐ 147

most recent commit 6 years ago

Sqrape ⭐ 145

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

total releases 1most recent commit 7 years ago

Ralger ⭐ 145

ralger makes it easy to scrape a website. Built on the shoulders of titans: rvest, xml2.

total releases 9most recent commit a year ago

Google Search ⭐ 142

scrape google search results

dependent packages 7total releases 10most recent commit 9 months ago

News Fetch ⭐ 140

A Python Package which helps to scrape all news details from any news websites

total releases 28most recent commit 9 months ago

Telegram Channel Scraper ⭐ 137

Scrape messages from Telegram in a NodeJS cli program

most recent commit 2 years ago

The 10 Most Depended On Scrape Open Source Projects

Extract Css Core ⭐ 35

Extract all CSS from a given url, both server side and client side rendered.

dependent packages 2total releases 12latest release May 03, 2022most recent commit a year ago

Fbnix ⭐ 11

✂ Agent library for scraping Facebook groups and pages in Node.js

dependent packages 2total releases 5latest release June 14, 2020most recent commit 2 years ago

Tableau Scraping ⭐ 79

Tableau scraper python library. R and Python scripts to scrape data from Tableau viz

dependent packages 2total releases 40latest release November 10, 2021most recent commit 2 years ago

Cep Scraper ⭐ 12

A Javascript client to search Brazilian postcodes

dependent packages 2total releases 6latest release March 02, 2015most recent commit 8 years ago

Daily Poem ⭐ 11

:book: Read a new poem every day! (from http://poems.com)

dependent packages 2total releases 10latest release April 24, 2017most recent commit 7 years ago

Amazon Product Api ⭐ 457

Amazon Scraper. Scrape products from the amazon search result or reviews from the specific product

dependent packages 2total releases 87latest release November 13, 2022most recent commit a year ago

Hltv Live Games ⭐ 15

Get live games on HLTV and their matchid.

dependent packages 2total releases 2latest release December 22, 2016most recent commit 7 years ago

X Ray Scraper ⭐ 5

Scraper next gen forked from x-ray (2.3.2)

dependent packages 2total releases 7latest release June 06, 2020most recent commit 3 years ago

Node Scraper ⭐ 6

Scraping websites made easy! A minimalistic yet powerful tool for collecting data from websites.

dependent packages 2total releases 4latest release January 03, 2019most recent commit 5 years ago

Upton ⭐ 1,615

A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)

dependent packages 1total releases 20latest release December 26, 2018most recent commit 5 years ago

The 10 Latest Releases In Scrape Open Source Projects

Wikilake ⭐ 8

Scrape lake metadata tables from Wikipedia 🏞

dependent packages 1total releases 6latest release October 05, 2021most recent commit a year ago

Imgur Scraper ⭐ 23

Retrieve years of imgur.com's data without any authentication.

total releases 15latest release September 30, 2021most recent commit 3 years ago

Pyscrapers ⭐ 23

project to produce various useful scrapers

total releases 62latest release September 28, 2021most recent commit 3 months ago

Emq_exporter ⭐ 29

Simple server that scrapes EMQ metrics and exporters them via HTTP for Prometheus consumption

total releases 15latest release September 16, 2021most recent commit 3 years ago

Prometheus Ecs Discovery ⭐ 243

A Prometheus discoverer that scrapes Amazon ECS and a generates file SD configuration file.

total releases 3latest release September 06, 2021most recent commit 3 months ago

Goq ⭐ 220

A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library

total releases 2latest release September 02, 2021most recent commit 3 years ago

Pastepwn ⭐ 109

Python framework to scrape Pastebin pastes and analyze them

total releases 22latest release August 15, 2021most recent commit 10 months ago

Tiktok Scraper ⭐ 3,554

TikTok Scraper. Download video posts, collect user/trend/hashtag/music feed metadata, sign URL and etc.

dependent packages 18total releases 91latest release August 14, 2021most recent commit a year ago

Seo Scraper ⭐ 22

Scrape SEO elements or whatever you need with this scraper built in Node.js

total releases 6latest release August 07, 2021most recent commit 2 years ago

Yahoo_fin ⭐ 251

Scrape stock price history from new (Spring 2017) Yahoo Finance layout

dependent packages 15total releases 20latest release July 10, 2021most recent commit 6 months ago