Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for javascript web crawler
javascript
x
web-crawler
x
44 search results found
Crawlee
โญย
12,106
CrawleeโA web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Node Osmosis
โญย
4,083
Web scraper for NodeJS
Browser Fingerprinting
โญย
3,353
Analysis of Bot Protection systems with available countermeasures ๐ฟ. How to defeat anti-bot system ๐ป and get around browser fingerprinting scripts ๐ต๏ธโโ๏ธ when scraping the web?
Thal
โญย
2,268
Getting started with Puppeteer and Chrome Headless for Web Scraping
100projectsofcode
โญย
1,293
A list of practical knowledge-building projects.
Stealth
โญย
923
๐ Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy
User Agents
โญย
871
A JavaScript library for generating random user agents with data that's updated daily.
Rpa
โญย
858
UI.Vision: Open-Source RPA Software (formerly Kantu) - Modern Robotic Process Automation with Selenium IDE++
Coolqlcool
โญย
626
Nextjs server to query websites with GraphQL
Browsertrix Crawler
โญย
470
Run a high-fidelity browser-based crawler in a single Docker container
Archivebot
โญย
328
ArchiveBot, an IRC bot for archiving websites
Supercrawler
โญย
324
A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.
Youtube Projects
โญย
272
This repository contains all the code I use in my YouTube tutorials.
Netflix Clone
โญย
245
Netflix like full-stack application with SPA client and backend implemented in service oriented architecture
Nofasel
โญย
239
A streaming app with no ADs.
Imdb Api
โญย
193
Serverless IMDB API powered by Cloudflare Worker
Awesomewebscraping
โญย
189
List of libraries, tools and APIs for web scraping and data processing.
Spider Less
โญย
186
Web spider as a service, spider on serverless
Helena
โญย
165
A Chrome extension for writing custom web scraping programs and web automation programs. Just demonstrate how to collect the first row of data, then let the extension write the program for collecting all rows.
Animos
โญย
154
Animos - Clean and minimal Anime-streaming desktop application without any ads.
Imghash
โญย
152
Perceptual image hashing for Node.js
Web Scraper Chrome Extension
โญย
149
Web data extraction tool implemented as chrome extension
Google News Scraper
โญย
144
Lightweight scraper for Google News
Amazon Scraper
โญย
140
Free Trial Amazon Scraper API for extracting search, product, offer listing, reviews, question and answers, best sellers and sellers data.
Actor Page Analyzer
โญย
136
Apify actor that opens a web page in headless Chrome and analyzes the HTML and JavaScript objects, looks for schema.org microdata and JSON-LD metadata, analyzes AJAX requests, etc.
Clicknium Docs
โญย
127
A next-generation GUI automation framework for Web and Desktop Application Testing and Automation.
Gpt4v Scraper
โญย
126
AI agent that can SEE ๐๏ธ, control, navigate, & do stuff for you on your browser.
Algolisted
โญย
118
Algolisted is an AI-powered nonprofit analytics firm dedicated to assisting computer science students in preparing for placements and internships. Our services include tracking and analytics across various platforms and topics.
Educative.io_scraper
โญย
117
Educative.io Course Downloader developed using Python and Selenium. Refer Readme.md for setup instructions.
Html Metadata
โญย
115
MetaData html scraper and parser for Node.js (supports Promises and callback style)
Node Web Crawler
โญย
104
A web scraper with a web user interface which shows scraping stats in realtime. Uses Node.JS, jQuery, socket.io and Express.
Haikei
โญย
102
HaiKei is an anime streaming website that uses the consumet API
Actor Scraper
โญย
93
House of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.
Anmeldung Berlin
โญย
91
This app will find and book any service.berlin.de appointment that can be booked online.
Torrents Api
โญย
90
Torrent Api โจ
Animeez
โญย
88
AnimeEZ - An Anime Streaming website without any ads for free (Demo - https://animeez.live) BTW ITS MADE IN HTML
Tacocat
โญย
86
A platform displaying the latest software engineer job information to entry-level new graduates
Raise
โญย
83
A simple (and unofficial) GitHub Trending client that lives in your menubar.
Scrapper
โญย
83
Web scraper with a simple REST API living in Docker and using a Headless browser and Readability.js for parsing.
Node Search Engine
โญย
79
Sample search engine with web crawler, built on Node.js + CouchDB + Limestone
Tatooine
โญย
78
A powerful scraper for JavaScript Developers.
Catalyst
โญย
71
A VS code Extension to accelerate the process of solving problems on Codeforces.
Amazon_scraper
โญย
64
Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt
Decapitated
โญย
64
Headless 'Chrome' Orchestration in R
Simplestorm
โญย
62
Simple Storm-like distributed application implementation
Node Js Functionalities
โญย
58
This repository contains very useful restful API's and functionalities in node-js containing many important tutorial code for mastering node-js, all tutorials have been published on medium.com, tutorials link is given below
Siteshooter
โญย
58
๐ท Automate full website screenshots and PDF generation with multiple viewport support.
Linkedin Email Extractor
โญย
57
A node web scraper to extract your linkedin connection emails
Nba Search
โญย
56
flask application designed to explore NBA data ๐
Nodejs Web Scraper
โญย
54
Slurp
โญย
54
One repo to rule them all !!?!?!! ๐ค ๐
Google Search Results Nodejs
โญย
53
SerpApi client library for Node.js. Previously: Google Search Results Node.js.
Kampus Scraper
โญย
50
Scraper & GraphQL API untuk data Perguruan Tinggi di Indonesia berdasarkan dari website Kementrian RISTEKDIKTI.
Maps To Lead
โญย
48
Esse projeto tem como objetivo obter leads em formato JSON e enviar para um webhook
Nightmare Heroku
โญย
48
๐ฑ a setup for nightmarejs on heroku
Keeper Core Api
โญย
44
Nunux Keeper core API
Hawk
โญย
43
Blazingly fast web crawler for mapping and updating data
Jason The Miner
โญย
40
โ A versatile Web scraper for Node.js
X Ray Crawler
โญย
39
Friendly web crawler for x-ray
Animeflv
โญย
34
Animeflv is a custom API that has the entire catalog of the animeflv.net website. You can enjoy all the content with subtitles in Spanish and the latest in the world of anime for free.
Linkedin Client
โญย
32
Web scraper for grabing data from Linkedin profiles or company pages (personal project)
Htmlunit
โญย
32
๐ธ๐งฐโ๏ธTools to Scrape Dynamic Web Content via the 'HtmlUnit' Java Library
Scraping Dynamic Javascript Ajax Websites With Beautifulsoup
โญย
29
A guide on how to scrape JavaScript rendered websites with Python and BeautifulSoup.
Professional Javascript
โญย
28
Fast-track your web development career using the powerful features of advanced JavaScript
Grailer
โญย
28
web scraping tool for grailed.com
Python
โญย
28
covers python basic to advance topics, practice questions, logical problems in python, web development using html, css, bootstrap, jquery, DOM, Django ๐๐. ๐ฅ ๐
React Node Web Scraper
โญย
28
Final Year project, scraping data of e-commerce stores and display in ReactJS app.
Heroku Casper Node
โญย
27
A sample Heroku application that cascades NodeJS on CasperJS, suitable for web scraping and automation.
Puppeteer Service
โญย
27
๐ Run headless Chrome (aka Puppeteer) as a service.
Conactivity
โญย
25
A tool built with Puppeteer that parses the LinkedIn profiles of a company's employees and returns the list of active employees.
Gsoc Data Analyser
โญย
24
Simple search for organisations participating/participated in the GSoC
Mimo Crawler
โญย
24
A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.
Nian Crawler
โญย
23
A web crawler for api.nian.so
Eztrackr
โญย
23
v3 of Eztrackr's Chrome extension. Designed to ease your job hunt by adding your jobs in an organized Trello board โจ
Node Web Crawler
โญย
21
Web Crawler in Node.js
Codepen Puppeteer
โญย
21
Use Puppeteer to download pens from Codepen.io as single html pages
Browser Automation Api
โญย
20
Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other things like capture a screenshot, generate pdf, extract content or execute custom Puppeteer, Playwright functions.
Covid19 Api
โญย
20
Covid19 Data API (JSON) - LIVE
Webbot
โญย
19
Browser extension to simulate browsing behaviour in search engines.
Sourcegrade
โญย
18
A web scraper for www.gradesource.com built with MongoDB, Express.js, Node.js, React and Redux
Pipe2time.ir
โญย
18
Web Crawler for Time.ir to Retrive JSON File, jalali, qamari, miladi JSON Calendar API.
Ptt Crawler
โญย
18
ptt-crawler is a web crawler module designed to scarpe data from Ptt.
Json Web Crawler
โญย
17
Use JSON to list all elements (with css 3 and jquery selector) that you want to crawl.
Aniscrape
โญย
17
Web scraping tool to download videos from TรผrkAnimeTV and AnimeciX
Crawler
โญย
17
Web Crawler created with Node.js and Puppeteer
Fanfiction Manager
โญย
17
Provides extreme flexibility with the help of Rule system to power-users* in downloading, tracking-status & reading novels from ANY site they want.
Js Crawler
โญย
16
A short and simple python crawler, that uses Webkit and executes Javascript
Anisync
โญย
16
Mapping sites to AniList and back.
Workers Tutorial
โญย
15
This repository holds the code for a tutorial that teaches how to build to build a web-crawler using Node Workers.
Web Scraping Api
โญย
15
Web Scraping API code examples for Python, PHP and Node.js
Scraper
โญย
15
All In One API to easily scrape data from any website, without worrying about captchas and bot detection mecanisms.
Puppeteer Daily Menu Scraper
โญย
14
Scraping daily menus of restaurants from downtown Budapest with Puppeteer.
Actor Content Checker
โญย
14
You can use this act to monitor any page's content and get a notification when content changes.
Webscraper
โญย
13
Node Krawler
โญย
13
Fast and lightweight web crawler with built-in cheerio, xml and json parser.
Bdo Scraper
โญย
13
A scraper for BDDatabase.net.
Webmiddle
โญย
13
Node.js framework for modular web scraping and data extraction
Dse Cse Market Update
โญย
13
Dhaka Stock Exchange & Chittagong Stock Exchange Share Market Real Time Update Data
Song Playlist Downloader Nodejs
โญย
12
Download Song Or Your Own Playlist
Animad
โญย
12
An anime streaming website.
Related Searches
Javascript Reactjs (121,118)
Javascript Html (48,022)
Javascript Css (46,402)
Javascript Deployment (32,768)
Javascript Express (23,082)
Javascript Command Line (22,145)
Javascript Testing (21,652)
Javascript Redux (21,545)
Javascript Vue (19,769)
Javascript Jquery (18,285)
1-44 of 44 search results
Privacy
ย |ย
About
ย |ย
Terms
ย |ย
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source.ย All rights reserved.