Awesome Open Source

Programming Languages

Search results for scraper scrape

466 search results found

Portia ⭐ 8,982

Visual scraping for Scrapy

Autoscraper ⭐ 5,159

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Scraperjs ⭐ 3,575

A complete and versatile web scraper.

Tiktok Scraper ⭐ 3,554

TikTok Scraper. Download video posts, collect user/trend/hashtag/music feed metadata, sign URL and etc.

Facebook Page Post Scraper ⭐ 2,014

Data scraper for Facebook Pages, and also code accompanying the blog post How to Scrape Data From Facebook Page Posts for Statistical Analysis

Facebook Scraper ⭐ 1,936

Scrape Facebook public pages without an API key

Twitterscraper ⭐ 1,852

Scrape Twitter for Tweets

Scrapely ⭐ 1,668

A pure-python HTML screen-scraping library

Upton ⭐ 1,615

A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)

Linkedin_scraper ⭐ 1,534

A library that scrapes Linkedin for user data

Jobfunnel ⭐ 1,533

Scrape job websites into a single spreadsheet with no duplicates.

Scrape ⭐ 1,464

A simple, higher level interface for Go web scraping.

How To Prevent Scraping ⭐ 1,417

The ultimate guide on preventing Website Scraping

Loklak_scraper_js ⭐ 1,094

Scrapers for loklak in javascript

Redditdownloader ⭐ 1,045

Scrapes Reddit to download media of your choice.

Pjscrape ⭐ 1,003

A web-scraping framework written in Javascript, using PhantomJS and jQuery

Mlscraper ⭐ 935

🤖 Scrape data from HTML websites automatically by just providing examples

Twitter Scraper ⭐ 769

Scrape the Twitter frontend API without authentication with Golang.

A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers, user info, images...

Skrape.it ⭐ 714

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

Robintrack ⭐ 679

Scrapes the Robinhood API to retrieve + store popularity and price data.

Voyager ⭐ 618

crawl and scrape web pages in rust

Javmoviescraper ⭐ 618

Scrape XBMC and Kodi movie metadeta and automatically rename files for Japanese Adult Videos (JAV), American Adult DVDs, and American Adult Webcontent

Universal Reddit Scraper - A comprehensive Reddit scraping command-line tool.

Imagescraper ⭐ 572

✂️ High performance, multi-threaded image scraper

Instascrape ⭐ 554

Powerful and flexible Instagram scraping library for Python, providing easy-to-use and expressive tools for accessing data programmatically

Communityscrapers ⭐ 545

This is a public repository containing scrapers created by the Stash Community.

Linkedin Scraper ⭐ 512

Scrapes the public profile of the linkedin page

Se Scraper ⭐ 477

Javascript scraping module based on puppeteer for many different search engines...

Openwebtext ⭐ 463

Open clone of OpenAI's unreleased WebText dataset scraper. This version uses pushshift.io files instead of the API for speed.

Instagram Scraper ⭐ 367

Instagram Scraper. Scrape useful data/posts from instagram users, hashtag and locations pages. Comments and people who liked specific posts and soon more. No login or API keys are required

Micro Open Graph ⭐ 362

A tiny Node.js microservice to scrape open graph data with joy.

Scrape Linkedin Selenium ⭐ 353

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

City Scrapers ⭐ 315

Scrape, standardize and share public meetings from local government websites

Advanced python library to scrap Twitter (tweets, users) from unofficial API

Python web scraping framework

Elixir Scrape ⭐ 300

Scrape any website, article or RSS/Atom Feed with ease!

Webinspector ⭐ 286

Ruby gem to inspect completely a web page. It scrapes a given URL, and returns you its meta, links, images more.

Python Automation Scripts ⭐ 264

Simple yet powerful automation stuffs.

Proxy Scraper ⭐ 260

scrape proxies from more than 5 different sources and check which ones are still alive

Scraper ⭐ 254

Simple web scraping for Google Chrome.

Mwoffliner ⭐ 245

Mediawiki scraper: all your wiki articles in one highly compressed ZIM file

Rcrawler ⭐ 240

An R web crawler and scraper

Spatula ⭐ 233

A modern Python library for writing maintainable web scrapers.

Google Images Download ⭐ 231

Google/Bing Images Web Downloader

Nudecrawler ⭐ 231

Crawl telegra.ph searching for nudes!

Youtube Comment Scraper ⭐ 229

A web client that scrapes YouTube comments

Proxyscrape ⭐ 224

Python library for retrieving free proxies (HTTP, HTTPS, SOCKS4, SOCKS5).

Amazon Scraper ⭐ 219

A simple web scraper to extract Product Data and Pricing from Amazon

Wayback Machine Scraper ⭐ 219

A command-line utility and Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.

Images Scraper ⭐ 218

Simple and fast scraper for Google

Skraper ⭐ 217

Kotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, Coub, Vimeo, IFunny, VK, Odnoklassniki, Pikabu)

Transistor ⭐ 209

Transistor, a Python web scraping framework for intelligent use cases.

Ha Multiscrape ⭐ 198

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.

Scrape a website efficiently, block by block, page by page. Based on cheerio and curl.

Humanoid ⭐ 191

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

Letterboxd_recommendations ⭐ 190

Scraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username

Scrapyz ⭐ 188

"Scrape Easy" - an extension of the Scrapy framework.

Instagram Follower Scraper ⭐ 181

Scrapes all the data of followers of any instagram account

Instagram Screen Scrape ⭐ 162

scrape public instagram data w/out API access

Demeter ⭐ 162

Demeter is a tool for scraping the calibre web ui

Atp World Tour Tennis Data ⭐ 162

Using Python to scrape ATP World Tour tennis data

Media Scraper ⭐ 150

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

Laravel Scavenger ⭐ 150

The most integrated web scraper package for Laravel.

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

News Fetch ⭐ 140

A Python Package which helps to scrape all news details from any news websites

Telegram Channel Scraper ⭐ 137

Scrape messages from Telegram in a NodeJS cli program

a command-line web scraping tool

Hockey Scraper ⭐ 134

Python Package for scraping NHL Play-by-Play and Shift data

Grawler ⭐ 128

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.

Node Metainspector ⭐ 124

Node npm for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, an array with all the links, all the images in it, etc. Inspired by the metainspector Ruby gem

Ao3scraper ⭐ 123

A Python scraper for getting fan fiction content and metadata from Archive of Our Own.

Gsoc Organisation Scraper ⭐ 120

Scrape GSoC organisations using a single script.

Trackback_scraper ⭐ 116

Instagram Python Scraper ⭐ 116

A instagram scraper wrote in python. Similar to instagram-php-scraper.Usages are in example.py. Enjoy it!

Linkedinscraper ⭐ 112

Scrapes public information off of LinkedIn

Jimutmap ⭐ 112

API to get enormous amount of high resolution satellite images from satellites.pro quickly through multi-threading! create map your own map dataset. Bringing data to Humans.

Giggity ⭐ 110

Wraps github api for openly available information about an organization, user, or repo

Pastepwn ⭐ 109

Python framework to scrape Pastebin pastes and analyze them

Html2rss ⭐ 106

📰 Build RSS 2.0 feeds from websites (and JSON APIs) with a few CSS selectors.

Scraper ⭐ 104

Web scraper for scraping, tracking and visualizing prices of products on various websites.

Gesetze Tools ⭐ 103

Scripts to maintain German law git repository

Reactriot2017 Dotamania ⭐ 101

🌐 Web scraping made easy with the visual 🗺 mind map editor to JSON

Web Scraper ⭐ 97

Perl web scraping toolkit

Vsco Scraper ⭐ 96

Easily allows for scraping a VSCO

Scrapemark ⭐ 94

Super-convenient web scraping in Python

Scraperboard ⭐ 94

Golang library to easily scrape websites based on simple XML declarations

Daily Scraper ⭐ 93

Fetches information about every webpage 🤖

Open-source Ethereum data scraper and indexer developed by Alethio

Scrape/Parse Pastebin using GO and expression grammar (PEG)

Facebook Post Scraper ⭐ 89

Facebook Post Scraper 🕵️🖱️

Glassdoor Review Scraper ⭐ 88

Scrape reviews from Glassdoor

Funda Scraper ⭐ 88

Scraper of the Dutch real estate website www.funda.nl, implemented in Python with Scrapy

🕷️ The PHP SERP Spider - A search engine scraper

Metadata Scraper ⭐ 86

🏷️ A JavaScript library for scraping/parsing metadata from a web page.

Instagramy ⭐ 86

Python Package for Instagram Users, Posts, and Hashtag data.

Twitterscraper ⭐ 86

Scrape a User's Twitter data! Bypass the 3,200 tweet API limit for a User!

Linkedin Jobs Scraper ⭐ 82

Scrape LinkedIn job postings using Selenium WebDriver with python bindings

Dashboard Metrics Scraper ⭐ 80

Container to scrape, store, and retrieve a window of time from the Metrics Server.

Ca Property Tax ⭐ 78

CA property tax visualization

Related Searches

Python Scraper (5,696)

Python Scrape (2,235)

Javascript Scraper (2,047)

Scraper Web Crawler (1,528)

Javascript Scrape (921)

Scraper Crawler (904)

Html Scraper (757)

Scraper Webscraper (643)

Scraper Scrapy (575)

Php Scraper (448)

1-100 of 466 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.