Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Web Webspider Open Source Projects

Open source projects categorized as Web Webspider
Categories > Webspider
Awesome Crawler ⭐ 5,859
A collection of awesome web crawler,spider in different languages
most recent commit 4 months ago
Hakrawler ⭐ 4,120
Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
total releases 11most recent commit 3 months ago
Webcollector ⭐ 2,974
WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
dependent packages 1total releases 22most recent commit a year ago
Gospider ⭐ 2,190
Gospider - Fast web spider written in Go
dependent packages 1total releases 17most recent commit 5 months ago
Spider ⭐ 907
A configurable web spider with a easy-to-use web console
most recent commit 6 years ago
Spidr ⭐ 775
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
dependent packages 16total releases 26most recent commit 3 months agorubygems spidr} Downloads
Spidy ⭐ 287
The simple, easy to use command line web crawler.
total releases 12most recent commit 2 years ago
Selenops ⭐ 215
A Swift Web Crawler 🕷
most recent commit 3 years ago
Antch ⭐ 177
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
dependent packages 1total releases 1most recent commit 4 years ago
Crawlbox ⭐ 112
Easy way to brute-force web directory.
most recent commit 5 years ago
Node Web Crawler ⭐ 104
A web scraper with a web user interface which shows scraping stats in realtime. Uses Node.JS, jQuery, socket.io and Express.
most recent commit 10 years ago
Crawl Anywhere ⭐ 98
Crawl-Anywhere - Web Crawler and document processing pipeline with Solr integration.
most recent commit 7 years ago
Crabler ⭐ 85
Web Crawler for Crabs
dependent packages 1total releases 9most recent commit 2 years agocargo crabler_derive} Downloads
Davedavefind ⭐ 71
A simple search engine based on the web crawler developed in Udacity's CS101 course.
most recent commit 11 years ago
Spiderq ⭐ 50
web spider
most recent commit 11 years ago
Pysearch ⭐ 48
Web crawler and Search engine in Python.
most recent commit 8 years ago
Webcrawler ⭐ 47
Just a simple web crawler which return crawled links as IObservable using reactive extension and async await.
total releases 10most recent commit 5 years ago
Webcollector Python ⭐ 47
WebCollector-Python is an open source web crawler framework based on Python.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
total releases 4most recent commit 5 years ago
Hawk ⭐ 43
Blazingly fast web crawler for mapping and updating data
dependent packages 4total releases 2most recent commit 7 years agonpm hawk-webcrawler} Downloads
Ajax_crawler ⭐ 41
A flexible web crawler based on Scrapy for fetching most of Ajax or other various types of web pages. Easy to use: To customize a new web crawler-You just need to write a config file and run.
most recent commit 8 years ago
Creepy ⭐ 40
Dead simple web crawler for Python
total releases 4most recent commit 4 years ago
Maman ⭐ 40
Rust Web Crawler saving pages on Redis
total releases 16most recent commit 3 years ago
Web Miner ⭐ 28
Crawls sites, to find new content and scrap it
most recent commit 3 years ago
The 8 Most Depended On Web Webspider Open Source Projects
Hawk ⭐ 43
Blazingly fast web crawler for mapping and updating data
dependent packages 4total releases 2latest release April 07, 2016most recent commit 7 years agonpm hawk-webcrawler} Downloads
Spiderman ⭐ 5
Minimalistic web crawler for Node.js
dependent packages 2total releases 32latest release July 19, 2020most recent commit 2 years agonpm @albert-team/spiderman} Downloads
Crabler ⭐ 85
Web Crawler for Crabs
dependent packages 1total releases 9latest release January 08, 2022most recent commit 2 years agocargo crabler_derive} Downloads
Gospider ⭐ 2,190
Gospider - Fast web spider written in Go
dependent packages 1total releases 17latest release December 25, 2021most recent commit 5 months ago
Wsoc ⭐ 23
The Web Spider Obstacle Course
dependent packages 1total releases 5latest release December 09, 2011most recent commit 6 years agorubygems wsoc} Downloads
Antch ⭐ 177
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
dependent packages 1total releases 1latest release December 25, 2017most recent commit 4 years ago
The 10 Latest Releases In Web Webspider Open Source Projects
Pagemunch Php ⭐ 6
A PHP library for the PageMunch web crawler API
latest release December 10, 2023most recent commit 7 years ago
Crabler ⭐ 85
Web Crawler for Crabs
dependent packages 1total releases 9latest release January 08, 2022most recent commit 2 years agocargo crabler_derive} Downloads
Gospider ⭐ 2,190
Gospider - Fast web spider written in Go
dependent packages 1total releases 17latest release December 25, 2021most recent commit 5 months ago
Biu ⭐ 5
biubiubiu~~ I'm a tiny web crawler framework
total releases 13latest release August 18, 2020most recent commit 4 years ago
Spiderman ⭐ 5
Minimalistic web crawler for Node.js
dependent packages 2total releases 32latest release July 19, 2020most recent commit 2 years agonpm @albert-team/spiderman} Downloads
Spiderman ⭐ 16
your friendly neighborhood web crawler
total releases 2latest release March 22, 2020most recent commit 2 years ago
Popular Networking Categories
Server
Network
Http
Proxy
Port
Email
Websocket
Hosts
Router
Upload
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.