The Top 23 Javascript Scraper Crawler Open Source Projects

Open source projects categorized as Javascript Scraper Crawler

Categories > Data Processing > Crawler

Categories > Programming Languages > Javascript

Categories > Data Processing > Scraper

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

dependent packages 42total releases 747most recent commit 6 days ago

Headless Chrome Crawler ⭐ 5,051

Distributed crawler powered by Headless Chrome

dependent packages 12total releases 21most recent commit 3 years ago

Browser Fingerprinting ⭐ 3,353

Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️‍♂️ when scraping the web?

most recent commit a year ago

Google Play Scraper ⭐ 2,108

Node.js scraper to get data from Google Play

dependent packages 29total releases 86most recent commit 5 months ago

Geziyor ⭐ 1,892

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

dependent packages 11total releases 6most recent commit a year ago

Article Extractor ⭐ 1,297

To extract main article from given URL with Node.js

dependent packages 10total releases 156most recent commit 3 months ago

Newcrawler ⭐ 583

Free Web Scraping Tool with Java

most recent commit 5 months ago

Nodejs Stuff ⭐ 484

Node.js libs I want to keep in mind.

most recent commit 4 years ago

Webster ⭐ 465

a reliable high-level web crawling & scraping framework for Node.js.

dependent packages 2total releases 42most recent commit 5 months ago

Youtube Projects ⭐ 272

This repository contains all the code I use in my YouTube tutorials.

most recent commit 3 years ago

Aliexpress Product Scraper ⭐ 152

Get Aliexpress product details as a json response including feedbacks, variants, shipping info, description, images, etc.,

most recent commit 4 months ago

Site Audit Seo ⭐ 151

Web service and CLI tool for SEO site audit: crawl site, lighthouse all pages, view public reports in browser. Also output to console, json, csv, xlsx, Google Drive.

dependent packages 1total releases 30most recent commit 3 months ago

Google News Scraper ⭐ 144

Lightweight scraper for Google News

dependent packages 2total releases 12most recent commit 4 months ago

Node Web Crawler ⭐ 104

A web scraper with a web user interface which shows scraping stats in realtime. Uses Node.JS, jQuery, socket.io and Express.

most recent commit 10 years ago

Actor Scraper ⭐ 93

House of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.

dependent packages 2total releases 12most recent commit a year ago

Bots Zoo ⭐ 90

most recent commit 5 months ago

Tacocat ⭐ 86

A platform displaying the latest software engineer job information to entry-level new graduates

most recent commit 2 years ago

Aliexscrape ⭐ 84

Get Aliexpress product details in JSON

dependent packages 2total releases 12most recent commit a year ago

Scrapper ⭐ 83

Web scraper with a simple REST API living in Docker and using a Headless browser and Readability.js for parsing.

most recent commit 3 months ago

Local Api Examples ⭐ 64

Useful and easy to understand examples written in Node.js and .NET Core about web scraping and automated browsing with Kameleo Client

most recent commit 9 months ago

Timbr_v1 ⭐ 50

A web service that turns an arbitrary web page into structural JSON data and easy-to-use APIs with just a few clicks

most recent commit 6 years ago

Browser As A Service ⭐ 43

A web browser 🌎 hosted as a service, to render your JavaScript web pages as HTML

most recent commit a year ago

Jason The Miner ⭐ 40

⛏ A versatile Web scraper for Node.js

dependent packages 3total releases 13most recent commit 2 years ago

The 10 Most Depended On Javascript Scraper Crawler Open Source Projects

Crawlee ⭐ 11,957

dependent packages 42total releases 747latest release December 10, 2023most recent commit 6 days ago

Google Play Scraper ⭐ 2,108

Node.js scraper to get data from Google Play

dependent packages 29total releases 86latest release August 09, 2023most recent commit 5 months ago

Extension ⭐ 32

web scraping extension

dependent packages 13total releases 7latest release September 26, 2020most recent commit a year ago

npm get-set-fetch-extension-commons} Downloads

Headless Chrome Crawler ⭐ 5,051

Distributed crawler powered by Headless Chrome

dependent packages 12total releases 21latest release June 11, 2018most recent commit 3 years ago

Geziyor ⭐ 1,892

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

dependent packages 11total releases 6latest release April 19, 2021most recent commit a year ago

Article Extractor ⭐ 1,297

To extract main article from given URL with Node.js

dependent packages 10total releases 156latest release December 01, 2022most recent commit 3 months ago

Crawlkit ⭐ 23

A crawler based on Phantom. Allows discovery of dynamic content and supports custom scrapers.

dependent packages 5total releases 34latest release May 23, 2016most recent commit 7 years ago

Web Master ⭐ 5

Web mastering tools for my personal services

dependent packages 4total releases 17latest release November 01, 2019most recent commit 4 years ago

npm @web-master/node-web-scraper} Downloads

Jason The Miner ⭐ 40

⛏ A versatile Web scraper for Node.js

dependent packages 3total releases 13latest release June 01, 2020most recent commit 2 years ago

Proxycrawl Node ⭐ 20

ProxyCrawl Node library for scraping and crawling

dependent packages 3total releases 20latest release March 25, 2022most recent commit 9 months ago

The 10 Latest Releases In Javascript Scraper Crawler Open Source Projects

Crawlee ⭐ 11,957

dependent packages 42total releases 747latest release December 10, 2023most recent commit 6 days ago

Google News Scraper ⭐ 144

Lightweight scraper for Google News

dependent packages 2total releases 12latest release December 06, 2023most recent commit 4 months ago

Webster ⭐ 465

a reliable high-level web crawling & scraping framework for Node.js.

dependent packages 2total releases 42latest release November 09, 2023most recent commit 5 months ago

Socialblade Com Api ⭐ 9

Unofficial APIs for socialblade.com website.

dependent packages 1total releases 15latest release September 26, 2023most recent commit 4 months ago

Google Play Scraper ⭐ 2,108

Node.js scraper to get data from Google Play

dependent packages 29total releases 86latest release August 09, 2023most recent commit 5 months ago

Saffron ⭐ 11

A fairly intuitive & powerful framework that enables you to collect & save articles and news from all over the web.

dependent packages 2total releases 102latest release February 07, 2023most recent commit 4 months ago

Article Extractor ⭐ 1,297

To extract main article from given URL with Node.js

dependent packages 10total releases 156latest release December 01, 2022most recent commit 3 months ago

Penjabarberita ⭐ 16

Extract the article list from its raw news HTML

dependent packages 2total releases 7latest release July 15, 2022most recent commit a year ago

Proxycrawl Node ⭐ 20

ProxyCrawl Node library for scraping and crawling

dependent packages 3total releases 20latest release March 25, 2022most recent commit 9 months ago

Ptt Crawler ⭐ 18

ptt-crawler is a web crawler module designed to scarpe data from Ptt.

total releases 12latest release January 29, 2022most recent commit a year ago

Popular Data Processing Categories