The Top 23 Javascript Webscraper Open Source Projects

Open source projects categorized as Javascript Webscraper

Categories > Programming Languages > Javascript

Categories > Webscraper

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

dependent packages 42total releases 747most recent commit 2 days ago

Node Osmosis ⭐ 4,083

Web scraper for NodeJS

dependent packages 58total releases 27most recent commit 9 months ago

100projectsofcode ⭐ 1,293

A list of practical knowledge-building projects.

most recent commit 3 years ago

Stealth ⭐ 923

🚀 Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy

most recent commit 7 months ago

Browsertrix Crawler ⭐ 470

Run a high-fidelity browser-based crawler in a single Docker container

most recent commit 3 months ago

Archivebot ⭐ 328

ArchiveBot, an IRC bot for archiving websites

most recent commit 5 months ago

Supercrawler ⭐ 324

A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

dependent packages 3total releases 45most recent commit 2 years ago

Google News Scraper ⭐ 144

Lightweight scraper for Google News

dependent packages 2total releases 12most recent commit 4 months ago

Html Metadata ⭐ 115

MetaData html scraper and parser for Node.js (supports Promises and callback style)

dependent packages 26total releases 20most recent commit 4 years ago

Node Web Crawler ⭐ 104

A web scraper with a web user interface which shows scraping stats in realtime. Uses Node.JS, jQuery, socket.io and Express.

most recent commit 10 years ago

Haikei ⭐ 102

HaiKei is an anime streaming website that uses the consumet API

most recent commit 4 months ago

Gdpr Cli ⭐ 88

A command line tool for checking your website for GDPR compliance.

dependent packages 2total releases 4most recent commit 5 years ago

Animeez ⭐ 88

AnimeEZ - An Anime Streaming website without any ads for free (Demo - https://animeez.live) BTW ITS MADE IN HTML

most recent commit a year ago

Tacocat ⭐ 86

A platform displaying the latest software engineer job information to entry-level new graduates

most recent commit 2 years ago

Node Search Engine ⭐ 79

Sample search engine with web crawler, built on Node.js + CouchDB + Limestone

most recent commit 13 years ago

Tatooine ⭐ 78

A powerful scraper for JavaScript Developers.

dependent packages 1total releases 84most recent commit a year ago

Amazon_scraper ⭐ 64

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

dependent packages 2total releases 13most recent commit 3 months ago

npm @scrapingant/amazon-proxy-scraper} Downloads

Simplestorm ⭐ 62

Simple Storm-like distributed application implementation

dependent packages 1total releases 6most recent commit 10 years ago

Siteshooter ⭐ 58

📷 Automate full website screenshots and PDF generation with multiple viewport support.

dependent packages 1total releases 73most recent commit 5 years ago

Linkedin Email Extractor ⭐ 57

A node web scraper to extract your linkedin connection emails

most recent commit 4 years ago

Nodejs Web Scraper ⭐ 54

dependent packages 5total releases 81most recent commit a year ago

Keeper Core Api ⭐ 44

Nunux Keeper core API

most recent commit 4 years ago

Hawk ⭐ 43

Blazingly fast web crawler for mapping and updating data

dependent packages 4total releases 2most recent commit 7 years ago

The 10 Most Depended On Javascript Webscraper Open Source Projects

Node Osmosis ⭐ 4,083

Web scraper for NodeJS

dependent packages 58total releases 27latest release March 01, 2019most recent commit 9 months ago

Crawlee ⭐ 12,059

dependent packages 42total releases 747latest release December 10, 2023most recent commit 2 days ago

Html Metadata ⭐ 115

MetaData html scraper and parser for Node.js (supports Promises and callback style)

dependent packages 26total releases 20latest release November 20, 2019most recent commit 4 years ago

X Ray Crawler ⭐ 39

Friendly web crawler for x-ray

dependent packages 19total releases 7latest release August 26, 2020most recent commit 3 years ago

Node Krawler ⭐ 13

Fast and lightweight web crawler with built-in cheerio, xml and json parser.

dependent packages 7total releases 10latest release June 23, 2014most recent commit 10 years ago

Nodejs Web Scraper ⭐ 54

dependent packages 5total releases 81latest release January 07, 2023most recent commit a year ago

Device Detective ⭐ 8

A Node.js module which determines whether a user agent is a phone, tablet, desktop, text browser, or search crawler.

dependent packages 4total releases 6latest release December 19, 2015most recent commit 8 years ago

Hawk ⭐ 43

Blazingly fast web crawler for mapping and updating data

dependent packages 4total releases 2latest release April 07, 2016most recent commit 7 years ago

Painlesscrawler ⭐ 10

(WIP) A painless Node.js web crawler that simply works

dependent packages 4total releases 3latest release August 05, 2015most recent commit 8 years ago

Kirinuki Core ⭐ 12

Kirinuki is a library that convert any html to JSON using CSS selectors.

dependent packages 3total releases 14latest release March 06, 2019most recent commit 10 months ago

The 10 Latest Releases In Javascript Webscraper Open Source Projects

Crawlee ⭐ 12,059

dependent packages 42total releases 747latest release December 10, 2023most recent commit 2 days ago

Google News Scraper ⭐ 144

Lightweight scraper for Google News

dependent packages 2total releases 12latest release December 06, 2023most recent commit 4 months ago

Tt Mobile ⭐ 6

PWA and web scraper for swiss table tennis results

dependent packages 1total releases 11latest release August 03, 2023most recent commit 4 months ago

Tatooine ⭐ 78

A powerful scraper for JavaScript Developers.

dependent packages 1total releases 84latest release March 27, 2023most recent commit a year ago

Nodejs Web Scraper ⭐ 54

dependent packages 5total releases 81latest release January 07, 2023most recent commit a year ago

Amazon_scraper ⭐ 64

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

dependent packages 2total releases 13latest release June 10, 2022most recent commit 3 months ago

Ptt Crawler ⭐ 18

ptt-crawler is a web crawler module designed to scarpe data from Ptt.

total releases 12latest release January 29, 2022most recent commit a year ago

Json Web Crawler ⭐ 17

Use JSON to list all elements (with css 3 and jquery selector) that you want to crawl.

total releases 19latest release December 22, 2021most recent commit 2 years ago

Yolo Scraper ⭐ 5

A simple way to structure your web scraper.

dependent packages 3total releases 6latest release February 20, 2021most recent commit 3 years ago

X Ray Crawler ⭐ 39

Friendly web crawler for x-ray

dependent packages 19total releases 7latest release August 26, 2020most recent commit 3 years ago

Popular Programming Languages Categories