Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for scraper scraping websites
scraper
x
scraping-websites
x
70 search results found
Ferret
⭐
5,540
Declarative web scraping
Crawly
⭐
790
Crawly, a high-level web crawling & scraping framework for Elixir.
Edu Mail Generator
⭐
707
Generate Free Edu Mail(s) within minutes
Phpscraper
⭐
486
A universal web-util for PHP.
Linkedin Profile Scraper Api
⭐
404
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON.
Dataflowkit
⭐
394
Extract structured data from web sites. Web sites scraping.
Tinking
⭐
337
🧶 Extract data from any website without code, just clicks.
Crawler
⭐
285
Library for Rapid (Web) Crawler and Scraper Development
Web Scraping
⭐
281
Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup
Python Automation Scripts
⭐
264
Simple yet powerful automation stuffs.
Requests Html
⭐
207
Pythonic HTML Parsing for Humans™
Email Extractor
⭐
134
The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
Spyscrap
⭐
124
CLI and GUI for OSINT. Are you very exhibited on the Internet? Check it! Twitter, Tinder, Facebook, Google, Yandex, BOE. It uses facial recognition to provide more accurate results.
Nintendeals
⭐
124
Library with a set of tools for scraping information about Nintendo games and its prices across all regions (NA, EU and JP).
Cloudflaresolverre
⭐
117
Cloudflare Javascript & reCaptcha challenge (I'm Under Attack Mode or IUAM) solving / bypass .NET Standard library.
Leetcode Questions Scraper
⭐
117
Scrape Algorithm Questions from leetcode and generate html and epub file
Scraply
⭐
114
Scraply a simple dom scraper to fetch information from any html based website
Html2rss
⭐
106
📰 Build RSS 2.0 feeds from websites (and JSON APIs) with a few CSS selectors.
Seleniumcrawler
⭐
105
An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
The Youtube Scraper
⭐
101
Download YouTube video description and video comments without using the YouTube API.
Anitop
⭐
67
Anitop is an unofficial simple API from https://anitrendz.net/ site
Amazon_scraper
⭐
64
Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt
Webreaper
⭐
59
Web scraper, crawler and parser in C#. Designed as simple, declarative and scalable web scraping solution.
Outscraper Python
⭐
58
The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.
Proxycrawl Python
⭐
57
ProxyCrawl Python library for scraping and crawling
Social Scraper
⭐
56
Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt
Leetcode
⭐
52
At present contains scraped data from around 1500 problems present on the site. More to follow....
Dart Scraper
⭐
51
한국 금융감독원에서 운영하는 다트(Dart) 시스템을 이용한 기업 재무제표 추출 프로그램
Instagram To Discord
⭐
49
Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2021!
Beautifulsoup Tutorial
⭐
45
✨ 🍜 Scrape webpage metadata using BeautifulSoup.
Crawling Projects
⭐
43
Web scraping and automation using python
Teslaservicemanualscraper
⭐
42
This script will download the Tesla Service Manual onto a local doc folder for offline access.
Proxy_web_crawler
⭐
39
Automates the process of repeatedly searching for a website via scraped proxy IP and search keywords
Readability Cli
⭐
35
A CLI for Mozilla Readability. Get clean, uncluttered, ready-to-read HTML from any webpage!
Botvid 19
⭐
34
Messenger Bot that scrapes for COVID-19 data and periodically updates subscribers via Facebook Messages. Created using Python/Flask, MYSQL, HTML, Heroku
Imdb Scraper
⭐
32
🎬 An attempt at the most complete IMDb API
Fulldom Server
⭐
31
Proxy-like server that will show you the DOM of a page after JS runs
Onlyfans Scrapper
⭐
29
Script to download media, post and more from creators on OnlyFans
Gopher Parse Sitemap
⭐
28
A high effective golang library for parsing big-sized sitemaps and avoiding high memory usage. The sitemap parser was written on golang without external dependencies.
Linkedin Web Scraper
⭐
28
Python Web Scraper for LinkedIn to collect and store company data (e.g. name, description, industry, etc.) into .xls file
React Node Web Scraper
⭐
28
Final Year project, scraping data of e-commerce stores and display in ReactJS app.
Video Crawler
⭐
26
Crawl websites for videos from Youtube, Vimeo, Soundcloud, etc
Sample Web Scraping With Electron
⭐
26
Sample project for web scraping with Electron
Instagram Downloader
⭐
24
Instagram user's photos and videos downloader. Download all media files from any username. Working 2021!
Reason Rust Scraper
⭐
21
🦀 Scraping & crawling websites using Rust, and ReasonML
Metafetch
⭐
21
NodeJS package that fetches a given URL's title, description, images, links etc.
Tradetheevent
⭐
20
Implementation of "Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading." In Findings of ACL2021
Scrapman
⭐
20
Retrieve real (with Javascript executed) HTML code from an URL, ultra fast and supports multiple parallel loading of webs
Proxycrawl Node
⭐
20
ProxyCrawl Node library for scraping and crawling
Proxycrawl Php
⭐
19
ProxyCrawl PHP library for scraping and crawling websites
Indonesia News Scraper
⭐
18
A news scraper for nodejs that help to scrap news from Indonesian news portal.
Document Dl
⭐
18
Command line program to download documents from web portals
Sinaweiboscraper
⭐
18
Web Scraper for Sina Weibo Search by Keywords
Torchestrator
⭐
18
Spin up Tor containers and proxy HTTP requests via these tor instances.
Multporn
⭐
16
A python library used to scrape and download from Multporn.net
Fbspider
⭐
16
Scraping Facebook information
Anisync
⭐
16
Mapping sites to AniList and back.
Ecommerce Scraping Api
⭐
16
eCommerce Scraping API code examples for Python, PHP and Node.js
Lazyscraper
⭐
15
Lazy helper tool to make easier scraping with simple tasks
Ryuanime
⭐
15
A free anime streaming , using the jkanime content by scraping the jkanime website.
Firstcyclingapi
⭐
15
An unofficial Python API wrapper for firstcycling.com
Outscraper Php
⭐
15
The library provides convenient access to the Outscraper API from applications written in the PHP language. Allows using Outscraper's services from your code.
Web Scraping Api
⭐
15
Web Scraping API code examples for Python, PHP and Node.js
Scavenger
⭐
14
Scrape and take screenshots of dynamic and static webpages
Spb Unofficial Wrapper
⭐
14
Unofficial NodeJS wrapper for the ScrapingBee API
Kick Off Web Scraping Python Selenium Beautifulsoup
⭐
13
A tutorial-based introduction to web scraping with Python.
Olx_scraper
⭐
13
📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.
Gochanges
⭐
13
**[ARCHIVED]** website changes tracker 🔍
Dataset Indian Companies
⭐
12
Web Scraping "List of companies in India" from AmbitionBox Website using Python and Beautiful Soup
Worker
⭐
12
Containerized Ferret worker
Competitive Coding Problem Classifier And Recommender
⭐
11
Competitive Coding Problem Classifier and Problem Recommendation
Puregym Tracker
⭐
11
A simple shell script written to scrape gym activity data from the PureGym website members' area.
Telegram Scraper Api
⭐
11
An easy-to-use PHP api for Getting contents of Telegram's public channels and groups Messages without needing to join them.
Fbnix
⭐
11
✂ Agent library for scraping Facebook groups and pages in Node.js
Flashscorescraping
⭐
10
🤖 Web Scrape to Flashcore
Alibaba_scraper
⭐
10
Alibaba scraper with using of rotating proxies and headless Chrome from ScrapingAnt
Cotoami Scraper
⭐
10
Cotoami Scraper is a Chrome Extension that scrapes web pages to generate inputs for Cotoami knowledge-bases.
Iscrape
⭐
10
A basic web scrapper for Instagram
Croncert Config
⭐
9
configuration and github actions for concertcloud.live (fka croncert.ch), a website that shows you concerts in various cities
Justdial Scrapper
⭐
9
A 100% working Justdial scrapper, Just enter the url and it'll extract business info from it
Linkedin Scraper
⭐
9
Selenium based LinkedIn profile data scraper
Auto Registrar
⭐
9
A CLI script to check/register courses from KFUPM Banner 9.
Scrapyteer
⭐
9
Web crawling & scraping framework for Node.js on top of headless Chrome browser
Results_fetcher
⭐
9
Facilitates retrieval of large amounts of data from GITAM's results platform
Amazonbestsellerbooksscraper
⭐
8
This program scrapes Amazon to get book data (the ISBN, title, author, price, etc.) of the top 100 Best Selling books in EVERY category and every subcategory in both US and International domains. Download EXE from here:
Duckduckgo
⭐
8
A simple DuckDuckGo URL scraper.
Nhkore
⭐
8
🇯🇵📰🗻 NHK News Web (Easy) word frequency (core list) scraper for Japanese language learners.
Stock Informer
⭐
8
📈 A small and simple DOM detection changer for when you're in desperate need of a new Nvidia graphics card or anything else that tickles your pickle.
Dcurooms
⭐
8
A command line tool used to display room information and book and request rooms.
Scrawler
⭐
7
Declarative, scriptable web robot (crawler) and scrapper
Instagram Scraping
⭐
7
Python script to scrap deskgram (Deskgram is a desktop version of Instagram)
Crawling Book
⭐
7
🧾🔍 끝내주는 크롤링&메크로 스크립트를 작성하는 방법 (with Python)
Oloviz
⭐
7
🍟 Scrape your Foodpanda and Deliveroo orders
Sih2022 Interface
⭐
7
An interface for viewing different problem statements of smart india hackathon 2022 with basic summary built with ❤️ using streamlit
Engine
⭐
7
Owl Engine – The Open Source Web Scraper Engine
A2oj Online Judge Category Wise Problems List
⭐
6
Firstly, I like to thank "Ahmed Aly" for making this awesome site. This site is very helpful for the programmer. Recently I came to know that, this site will be no more. So I want to collect the problem set for future uses.
Bing Scraper
⭐
6
The bingscraper is python3 package which extracts the text and images content on search engine 'bing.com'
Producthunt Scraper
⭐
6
Producthunt.com famous website scraper script. Scrap all offers and save in spreadsheet excel file.
Xvlib
⭐
6
A Python package for scrapping at xvideos.com easily.
Faqs Automatic Scraper
⭐
6
Automatic Scraping project for extracting FAQ and Help center articles
Related Searches
Python Scraper (5,478)
Javascript Scraper (2,056)
Scraper Scrape (2,048)
Scraper Web Crawler (1,364)
Scraper Webscraper (1,362)
Scraper Crawler (853)
Html Scraper (757)
Jupyter Notebook Scraper (740)
1-70 of 70 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.