Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for scraper webscraper
scraper
x
webscraper
x
57 search results found
Crawlee
⭐
12,059
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Awesome Crawler
⭐
5,859
A collection of awesome web crawler,spider in different languages
Autoscraper
⭐
5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Node Osmosis
⭐
4,083
Web scraper for NodeJS
Soup
⭐
2,074
Web Scraper in Go, similar to BeautifulSoup
Tomorrow
⭐
1,463
Magic decorator syntax for asynchronous code in Python
Stealth
⭐
923
🚀 Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy
Spidr
⭐
775
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Xidel
⭐
611
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
Scrapers
⭐
511
A list of scrapers from around the web.
Phpscraper
⭐
486
A universal web-util for PHP.
Scrapple
⭐
452
A framework for creating semi-automatic web content extractors
Pulsarrpa
⭐
413
Automate webpages at scale, scrape web data completely and accurately with high performance, distributed RPA.
Basketball_reference_web_scraper
⭐
382
NBA Stats API via Basketball Reference
Scrape Linkedin Selenium
⭐
353
`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
Hquery.php
⭐
345
An extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.
Crawler
⭐
285
Library for Rapid (Web) Crawler and Scraper Development
Gopa
⭐
281
[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Web Scraping
⭐
276
Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
Ant
⭐
271
A web crawler for Go
Rcrawler
⭐
240
An R web crawler and scraper
Summarizer
⭐
236
A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.
Getsy
⭐
234
A simple browser/client-side web scraper.
Amazon Scraper
⭐
219
A simple web scraper to extract Product Data and Pricing from Amazon
Daath Ai Parser
⭐
184
Daath AI Parser is an open-source application that uses OpenAI to parse visible text of HTML elements.
Antch
⭐
177
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
Goscrape
⭐
172
Web scraper that can create an offline readable version of a website
Screenslicer
⭐
159
Automatic, zero-config web scraping -- written in Java, has no dependency on Java EE or app servers, and the web scraper has a restful/JSON API. Currently unmaintained.
Facebook_page_scraper
⭐
150
Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
Google News Scraper
⭐
144
Lightweight scraper for Google News
Not Your Average Web Crawler
⭐
130
A web crawler (for bug hunting) that gathers more than you can imagine.
Evine
⭐
117
Interactive CLI Web Crawler
Html Metadata
⭐
115
MetaData html scraper and parser for Node.js (supports Promises and callback style)
Raspagem De Dados Para Iniciantes
⭐
115
Raspagem de dados para iniciante usando Scrapy e outras libs básicas
Gflare Tk
⭐
110
Open-Source Python Based SEO Web Crawler
Seleniumcrawler
⭐
105
An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
Node Web Crawler
⭐
104
A web scraper with a web user interface which shows scraping stats in realtime. Uses Node.JS, jQuery, socket.io and Express.
Brutescrape
⭐
95
A web scraper for generating password files based on plain text found
Bing Ip2hosts
⭐
91
bingip2hosts is a Bing.com web scraper that discovers websites by IP address
Animeez
⭐
88
AnimeEZ - An Anime Streaming website without any ads for free (Demo - https://animeez.live) BTW ITS MADE IN HTML
Tacocat
⭐
86
A platform displaying the latest software engineer job information to entry-level new graduates
Crabler
⭐
85
Web Crawler for Crabs
Tatooine
⭐
78
A powerful scraper for JavaScript Developers.
Rotten Tomatoes Dataset
⭐
77
Scraping Rotten Tomatoes then Sentiment Analysis with Logit Written Manually
Goodreadsscraper
⭐
76
Scrape data from Goodreads using Scrapy and Selenium 📚
Pymarketcap
⭐
74
Python3 API wrapper and web scraper for https://coinmarketcap.com
Web_scraper
⭐
73
A very basic web scraper implementation to scrap html elements from a web page.
Shutterscrape
⭐
73
Speedy, lightweight web scrapper for Shutterstock.
Spotifyscraper
⭐
72
Spotify Scraper to extract all the information from spotify, download mp3 with cover of the song
Autoscrape Py
⭐
70
An automated, programming-free web scraper for interactive sites
Top Github Scraper
⭐
67
Scape top GitHub repositories and users based on keywords
Amazon_scraper
⭐
64
Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt
Dotnetcrawler
⭐
63
DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-w
Kenpompy
⭐
61
A simple yet comprehensive web scraper for kenpom.com.
Scraping Tripadvisor With Python 2020
⭐
58
Python implementation of web scraping of TripAdvisor with Selenium in a new 2019 website
Linkedin Email Extractor
⭐
57
A node web scraper to extract your linkedin connection emails
Searchifyx
⭐
56
Fast flashcard searcher study tool
Nodejs Web Scraper
⭐
54
Scraperx
⭐
53
Library for scraping websites or apis at any scale
Scalawebscraper
⭐
48
Scala Webscraper
Trscraper
⭐
47
TRScraper, doğal dil işleme uygulamalarında kullanılmak amacıyla geliştirilmiş, Türkçe içerik girilen büyük platformlarda metin madenciliği yapma imkanı sunan bir uygulamadır.
Scrapy Craigslist
⭐
47
Web Scraping Craigslist's Engineering Jobs in NY with Scrapy
Bookingscraper
⭐
47
🌎 🏨 Scrape Booking.com 🏨 🌎
Lead Generation
⭐
46
Python script, which empowers people with no programming background to generate robust leads on a mass scale. This repo will be compiled of various versatile techniques used in lead generation.
Uoft Scrapers
⭐
44
Public web scraping scripts for the University of Toronto.
Yellowpages Scraper
⭐
43
Yellowpages.com Web Scraper written in Python and LXML to extract business details available based on a particular category and location.
Jason The Miner
⭐
40
⛏ A versatile Web scraper for Node.js
Scrapemate
⭐
39
Golang Crawling and scraping framework
Chafed
⭐
37
Web scraper for Scala
Webscraper
⭐
35
iOS library for web scraping
Public Roadmap
⭐
34
Public roadmap for SerpApi, LLC (https://serpapi.com)
Cobweb Lnx
⭐
34
CobWeb is a Python library for web scraping. The library consists of two classes: Spider and Scraper.
Jsonscraper
⭐
33
JSON configurable concurrent scraper
Animal Crossing Scraper
⭐
33
Web scraper for Animal Crossing - New Horizons data using bs4
Linkedin Client
⭐
32
Web scraper for grabing data from Linkedin profiles or company pages (personal project)
Gcf Packs
⭐
30
Library packs for google cloud functions
Nodejs Web Scraper Cookbook
⭐
30
📝 Resources for web scraping with node.js
Stormscraper
⭐
29
A Storm based web crawler with Cassandra backend
Linkedin Web Scraper
⭐
28
Python Web Scraper for LinkedIn to collect and store company data (e.g. name, description, industry, etc.) into .xls file
Professional Javascript
⭐
28
Fast-track your web development career using the powerful features of advanced JavaScript
React Node Web Scraper
⭐
28
Final Year project, scraping data of e-commerce stores and display in ReactJS app.
Email Report
⭐
27
A modular template for scraping data from the web to send yourself scheduled email reports
Webscraper Ios
⭐
27
a Web Scraping Utility with embed UIwebView context engine.
Craigslistscraper
⭐
27
Simple webscraper for Craigslist.
Hipposcraper
⭐
26
A Linux terminal tool for parsing and scraping Holberton project pages to automate repetitive tasks.
Abrade
⭐
26
A fast Web API scraper written in C++ and built on Boost ASIO
Mimo Crawler
⭐
24
A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.
Glassdoor Interview Scraper
⭐
24
Web scraper for Glassdoor interview review data
Igscraperkit
⭐
23
Create dynamic web scraper in Objective-C or Ruby!
Otakudesu Scraper
⭐
23
unofficial otakudesu.cam rest api
Twitterscraper4j
⭐
22
a java library which scrapes twitter to fetch publicly available info
Assessor Scraper
⭐
22
A project to scrape the assessor's website and make the data accessible for advanced queries
Zillow_scraper
⭐
21
Repo for Zillow Web scraper
Scrapemeagain
⭐
21
Yet another Python web scraping application
Hepsiburada Review Scraper
⭐
20
Hepsiburada review/comment and rating scraper. Turkish text dataset creator for data science and NLP projects. 📜
Society Email Scrape
⭐
20
Scrapes Every Email Address of Every Society in Every University
Scrapegen
⭐
20
A simple python tool that generates a requests/bs4 based web scraper
Tagalog Dictionary Scraper
⭐
19
Builds a Tagalog dictionary by collecting Tagalog words from tagalog.pinoydictionary.com
Scrapy Azuresearch Crawler Samples
⭐
19
Scrapy as a Web Crawler for Azure Search Samples
Words Scraper
⭐
19
Selenium based web scraper to generate passwords list
Related Searches
Python Scraper (3,513)
Scraper Scrape (2,048)
Web Crawler Webscraper (1,659)
Scraper Web Crawler (1,528)
Javascript Scraper (1,440)
Python Webscraper (1,022)
Scraper Crawler (922)
Html Scraper (757)
1-57 of 57 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.