Awesome Open Source

Programming Languages

Search results for javascript webscraper

12 search results found

Crawlee ⭐ 12,106

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Node Osmosis ⭐ 4,083

Web scraper for NodeJS

100projectsofcode ⭐ 1,293

A list of practical knowledge-building projects.

Stealth ⭐ 923

🚀 Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy

Browsertrix Crawler ⭐ 470

Run a high-fidelity browser-based crawler in a single Docker container

Archivebot ⭐ 328

ArchiveBot, an IRC bot for archiving websites

Supercrawler ⭐ 324

A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

Google News Scraper ⭐ 144

Lightweight scraper for Google News

Html Metadata ⭐ 115

MetaData html scraper and parser for Node.js (supports Promises and callback style)

Node Web Crawler ⭐ 104

A web scraper with a web user interface which shows scraping stats in realtime. Uses Node.JS, jQuery, socket.io and Express.

HaiKei is an anime streaming website that uses the consumet API

Gdpr Cli ⭐ 88

A command line tool for checking your website for GDPR compliance.

AnimeEZ - An Anime Streaming website without any ads for free (Demo - https://animeez.live) BTW ITS MADE IN HTML

A platform displaying the latest software engineer job information to entry-level new graduates

Node Search Engine ⭐ 79

Sample search engine with web crawler, built on Node.js + CouchDB + Limestone

Tatooine ⭐ 78

A powerful scraper for JavaScript Developers.

Amazon_scraper ⭐ 64

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

Simplestorm ⭐ 62

Simple Storm-like distributed application implementation

Siteshooter ⭐ 58

📷 Automate full website screenshots and PDF generation with multiple viewport support.

Linkedin Email Extractor ⭐ 57

A node web scraper to extract your linkedin connection emails

Nodejs Web Scraper ⭐ 54

Keeper Core Api ⭐ 44

Nunux Keeper core API

Blazingly fast web crawler for mapping and updating data

Jason The Miner ⭐ 40

⛏ A versatile Web scraper for Node.js

Makenews ⭐ 40

MakeNews is for journalists and newsrooms. It helps you track news from web and social media in real-time.

X Ray Crawler ⭐ 39

Friendly web crawler for x-ray

Linkedin Client ⭐ 32

Web scraper for grabing data from Linkedin profiles or company pages (personal project)

React Node Web Scraper ⭐ 28

Final Year project, scraping data of e-commerce stores and display in ReactJS app.

Professional Javascript ⭐ 28

Fast-track your web development career using the powerful features of advanced JavaScript

Puppeteer Service ⭐ 27

🎠 Run headless Chrome (aka Puppeteer) as a service.

Mimo Crawler ⭐ 24

A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.

Nian Crawler ⭐ 23

A web crawler for api.nian.so

Node Web Crawler ⭐ 21

Web Crawler in Node.js

Ptt Crawler ⭐ 18

ptt-crawler is a web crawler module designed to scarpe data from Ptt.

Pipe2time.ir ⭐ 18

Web Crawler for Time.ir to Retrive JSON File, jalali, qamari, miladi JSON Calendar API.

Sourcegrade ⭐ 18

A web scraper for www.gradesource.com built with MongoDB, Express.js, Node.js, React and Redux

Fanfiction Manager ⭐ 17

Provides extreme flexibility with the help of Rule system to power-users* in downloading, tracking-status & reading novels from ANY site they want.

Json Web Crawler ⭐ 17

Use JSON to list all elements (with css 3 and jquery selector) that you want to crawl.

Web Crawler created with Node.js and Puppeteer

Js Crawler ⭐ 16

A short and simple python crawler, that uses Webkit and executes Javascript

Workers Tutorial ⭐ 15

This repository holds the code for a tutorial that teaches how to build to build a web-crawler using Node Workers.

Web Scraping Api ⭐ 15

Web Scraping API code examples for Python, PHP and Node.js

Node Krawler ⭐ 13

Fast and lightweight web crawler with built-in cheerio, xml and json parser.

Bdo Scraper ⭐ 13

A scraper for BDDatabase.net.

Webscraper ⭐ 13

Kirinuki Core ⭐ 12

Kirinuki is a library that convert any html to JSON using CSS selectors.

Web Crawler ⭐ 12

Simple web crawler built using Node.js

Meteor Crawler ⭐ 12

A simple web-crawler.

A simple web scraper built around the JavaFX WebEngine

Crawlerr ⭐ 11

A simple and fully customizable web crawler/spider for Node.js with server-side DOM. Comes with elegant and hell-simple APIs.

Webcrawler ⭐ 11

A focused web crawler based on Playwright, RMQ, Kafka and Flink.

Hwcollection ⭐ 11

A project to create a HOT WHEELS COLLECTION

web crawler just for links

Moodle Scrape ⭐ 11

Easily scrape data from Moodle LMS sites

Goodreads Quote Scraper ⭐ 10

Nightmarejs based web scraper that writes quotes from a chosen author to json file

Wsu Accessibility Collector ⭐ 10

Scans and collects accessibility data for a given set of URLs

Painlesscrawler ⭐ 10

(WIP) A painless Node.js web crawler that simply works

Resumewords ⭐ 10

Resume-Words helps tailor your resume to individual job postings using AI and NLP

When you need those jobs hypersonic 🚀 scrape 🔪

Scraper Tutorial ⭐ 9

Web scraper built with Node.js

Device Detective ⭐ 8

A Node.js module which determines whether a user agent is a phone, tablet, desktop, text browser, or search crawler.

Scraper Api ⭐ 8

An HTML to JSON API webscraper for ResearchGate, adaptable for other sites

Serverless Link Preview ⭐ 8

A serverless, scalable service to get website description and preview deployed on AWS Lambda.

Unicrawler ⭐ 8

Web crawler in Node.js

Actor Legacy Phantomjs Crawler ⭐ 7

The actor implements the legacy Apify Crawler product. It uses PhantomJS headless browser to recursively crawl websites and extract data from them using a piece of JavaScript code.

Web Crawler which can config rules to collect information and generation e-book files

Ajfabriqnode ⭐ 6

A Distributed Application Framework for NodeJs

Ncdaily Opensource ⭐ 6

Fullstack Flask notices app 📢. Used by 700+ students.

Websitecontactharvester ⭐ 6

Crawl websites for contact information. Extract email, phone, facebook, twitter.

Tt Mobile ⭐ 6

PWA and web scraper for swiss table tennis results

Roblox Scraper ⭐ 6

node.js web scraper for roblox.com

Crawler Web Nodejs ⭐ 6

Web Crawler written in nodeJS and MongoDB.

Metascrape ⭐ 6

Node web scraper that extracts metadata static & javascript websites using HTML, Open Graph, Twitter metadata

US Presidential Actions feed bot. Not an official tool of the US Government.

Web scraper for EZTV.ag | This is a public mirror of the Popcon Time Community Edition repository. All pull requests are ignored. This repository may recieve forced updates.

Web Crawler ⭐ 5

Web crawler to return static assets of all reachable URLs from a web page

Prohardver Push Notifier ⭐ 5

Web scraper and push notifier for new messages on the Prohardver! forums (www.prohardver.hu).

Hurricane ⭐ 5

Alexa Skill for providing hurricane updates based on data that comes from the National Hurricane Center.

Site Mapper ⭐ 5

A minimal web crawler to generate visual site maps

Shopasource ⭐ 5

Easiest way to find best lowest price products online.

Job Scraper ⭐ 5

Basic web scraper listing new jobs built with Gatsby and AWS Lambda systems

Spiderman ⭐ 5

Minimalistic web crawler for Node.js

Webscraper ⭐ 5

webcrawler made from Beautiful soup

Yolo Scraper ⭐ 5

A simple way to structure your web scraper.

Nodejs Web Scraping ⭐ 5

Scraping the Web With Node.js

Open Source Crawler ⭐ 5

Web crawler finding open source GitHub repositories, parsing README files, and scanning for typo/security issues.

Related Searches

Javascript Reactjs (121,118)

Javascript Css (46,402)

Javascript Deployment (32,768)

Javascript Express (27,853)

Javascript Html (24,117)

Javascript Command Line (22,145)

Javascript Testing (21,652)

Javascript Redux (21,545)

Javascript Vue (19,769)

Javascript Mongodb (18,598)

1-12 of 12 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.