Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for javascript webspider
javascript
x
webspider
x
0 search results found
Crawlee
⭐
12,158
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Browsertrix Crawler
⭐
470
Run a high-fidelity browser-based crawler in a single Docker container
Archivebot
⭐
328
ArchiveBot, an IRC bot for archiving websites
Supercrawler
⭐
324
A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.
Spider Less
⭐
186
Web spider as a service, spider on serverless
Google News Scraper
⭐
144
Lightweight scraper for Google News
Node Web Crawler
⭐
104
A web scraper with a web user interface which shows scraping stats in realtime. Uses Node.JS, jQuery, socket.io and Express.
Tacocat
⭐
86
A platform displaying the latest software engineer job information to entry-level new graduates
Node Search Engine
⭐
79
Sample search engine with web crawler, built on Node.js + CouchDB + Limestone
Amazon_scraper
⭐
64
Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt
Simplestorm
⭐
62
Simple Storm-like distributed application implementation
Siteshooter
⭐
58
📷 Automate full website screenshots and PDF generation with multiple viewport support.
Hawk
⭐
43
Blazingly fast web crawler for mapping and updating data
X Ray Crawler
⭐
39
Friendly web crawler for x-ray
Puppeteer Service
⭐
27
🎠 Run headless Chrome (aka Puppeteer) as a service.
Mimo Crawler
⭐
24
A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.
Nian Crawler
⭐
23
A web crawler for api.nian.so
Node Web Crawler
⭐
21
Web Crawler in Node.js
Pipe2time.ir
⭐
18
Web Crawler for Time.ir to Retrive JSON File, jalali, qamari, miladi JSON Calendar API.
Ptt Crawler
⭐
18
ptt-crawler is a web crawler module designed to scarpe data from Ptt.
Json Web Crawler
⭐
17
Use JSON to list all elements (with css 3 and jquery selector) that you want to crawl.
Crawler
⭐
17
Web Crawler created with Node.js and Puppeteer
Js Crawler
⭐
16
A short and simple python crawler, that uses Webkit and executes Javascript
Workers Tutorial
⭐
15
This repository holds the code for a tutorial that teaches how to build to build a web-crawler using Node Workers.
Node Krawler
⭐
13
Fast and lightweight web crawler with built-in cheerio, xml and json parser.
Web Crawler
⭐
12
Simple web crawler built using Node.js
Meteor Crawler
⭐
12
A simple web-crawler.
Hwcollection
⭐
11
A project to create a HOT WHEELS COLLECTION
Slinky
⭐
11
web crawler just for links
Crawlerr
⭐
11
A simple and fully customizable web crawler/spider for Node.js with server-side DOM. Comes with elegant and hell-simple APIs.
Webcrawler
⭐
11
A focused web crawler based on Playwright, RMQ, Kafka and Flink.
Floodesh
⭐
11
Floodesh is a distributed web spider written with Nodejs.
Wsu Accessibility Collector
⭐
10
Scans and collects accessibility data for a given set of URLs
Painlesscrawler
⭐
10
(WIP) A painless Node.js web crawler that simply works
Scrape
⭐
9
When you need those jobs hypersonic 🚀 scrape 🔪
Web Spider
⭐
8
这是一个用superagent + phantomjs 写的一个小爬虫,尽量简单。
Device Detective
⭐
8
A Node.js module which determines whether a user agent is a phone, tablet, desktop, text browser, or search crawler.
Unicrawler
⭐
8
Web crawler in Node.js
Node Ebk
⭐
7
Web Crawler which can config rules to collect information and generation e-book files
Espider
⭐
7
A web spider based on electron
Actor Legacy Phantomjs Crawler
⭐
7
The actor implements the legacy Apify Crawler product. It uses PhantomJS headless browser to recursively crawl websites and extract data from them using a piece of JavaScript code.
Websitecontactharvester
⭐
6
Crawl websites for contact information. Extract email, phone, facebook, twitter.
Ajfabriqnode
⭐
6
A Distributed Application Framework for NodeJs
Crawler Web Nodejs
⭐
6
Web Crawler written in nodeJS and MongoDB.
Site Mapper
⭐
5
A minimal web crawler to generate visual site maps
Open Source Crawler
⭐
5
Web crawler finding open source GitHub repositories, parsing README files, and scanning for typo/security issues.
Spiderman
⭐
5
Minimalistic web crawler for Node.js
Web Crawler
⭐
5
Web crawler to return static assets of all reachable URLs from a web page
1-0 of 0 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.