Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Java Web Crawler Open Source Projects

Open source projects categorized as Java Web Crawler
Spider Flow ⭐ 8,075
新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。
most recent commit 10 months ago
Webcollector ⭐ 2,974
WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
dependent packages 1total releases 22most recent commit a year ago
Nutch ⭐ 2,742
Apache Nutch is an extensible and scalable web crawler
dependent packages 1total releases 26most recent commit 3 months ago
Gecco ⭐ 2,403
Easy to use lightweight web crawler（易用的轻量化网络爬虫）
dependent packages 4total releases 28most recent commit 2 years ago
100projectsofcode ⭐ 1,293
A list of practical knowledge-building projects.
most recent commit 3 years ago
Spider ⭐ 907
A configurable web spider with a easy-to-use web console
most recent commit 6 years ago
Storm Crawler ⭐ 834
A scalable, mature and versatile web crawler based on Apache Storm
dependent packages 10total releases 36most recent commit 3 months ago
Sparkler ⭐ 401
Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
most recent commit a year ago
News Crawl ⭐ 229
News crawling with StormCrawler - stores content as WARC
most recent commit 4 months ago
Crawler Commons ⭐ 217
A set of reusable Java components that implement functionality common to any web crawler
dependent packages 7total releases 10most recent commit 5 months ago
Collector Http ⭐ 162
Norconex Web Crawler (or spider) is a flexible web crawler for collecting, parsing, and manipulating data from the Internet (or Intranet) to various data repositories such as search engines.
total releases 25most recent commit 6 months ago
Screenslicer ⭐ 159
Automatic, zero-config web scraping -- written in Java, has no dependency on Java EE or app servers, and the web scraper has a restful/JSON API. Currently unmaintained.
most recent commit 7 years ago
Save For Offline ⭐ 118
Android app for saving webpages for offline reading.
most recent commit 3 years ago
Animewatcher ⭐ 62
The goal of this project/app is to let the user watch anime without ads. It uses Jsoup to extract data from the website and Exoplayer to show videos.
most recent commit 3 years ago
Project Tauro ⭐ 48
A Router WiFi key recovery/cracking tool with a twist.
most recent commit 5 years ago
Android Web Scraper ⭐ 45
Android Web Scraper is a simple library for android web automation. You can perform web task in background to fetch website data programmatically.
most recent commit 3 years ago
Wpcrawler ⭐ 43
a web crawler for single WordPress site
most recent commit 10 years ago
Flink Crawler ⭐ 35
Continuous scalable web crawler built on top of Flink and crawler-commons
most recent commit 5 years ago
Google Search Results Java ⭐ 30
Google Search Results JAVA API via SerpApi
most recent commit 4 months ago
Crawler4j ⭐ 30
Open Source Simple Web Crawler for Java. Simple Flexible And Lightweight
most recent commit 11 years ago
Stormscraper ⭐ 29
A Storm based web crawler with Cassandra backend
most recent commit 10 years ago
Jfitbit ⭐ 26
A web scraper to download intraday Fitbit data (previously) unavailable in the official API
most recent commit 7 years ago
Charles ⭐ 25
Java web crawling library
most recent commit 6 years ago
The 5 Most Depended On Java Web Crawler Open Source Projects
Storm Crawler ⭐ 834
A scalable, mature and versatile web crawler based on Apache Storm
dependent packages 10total releases 36latest release October 25, 2023most recent commit 3 months ago
Crawler Commons ⭐ 217
A set of reusable Java components that implement functionality common to any web crawler
dependent packages 7total releases 10latest release July 13, 2023most recent commit 5 months ago
Gecco ⭐ 2,403
Easy to use lightweight web crawler（易用的轻量化网络爬虫）
dependent packages 4total releases 28latest release July 04, 2020most recent commit 2 years ago
Nutch ⭐ 2,742
Apache Nutch is an extensible and scalable web crawler
dependent packages 1total releases 26latest release August 22, 2022most recent commit 3 months ago
The 8 Latest Releases In Java Web Crawler Open Source Projects
Storm Crawler ⭐ 834
A scalable, mature and versatile web crawler based on Apache Storm
dependent packages 10total releases 36latest release October 25, 2023most recent commit 3 months ago
Crawler Commons ⭐ 217
A set of reusable Java components that implement functionality common to any web crawler
dependent packages 7total releases 10latest release July 13, 2023most recent commit 5 months ago
Nutch ⭐ 2,742
Apache Nutch is an extensible and scalable web crawler
dependent packages 1total releases 26latest release August 22, 2022most recent commit 3 months ago
Gecco ⭐ 2,403
Easy to use lightweight web crawler（易用的轻量化网络爬虫）
dependent packages 4total releases 28latest release July 04, 2020most recent commit 2 years ago
Gospy ⭐ 7
🕷 a flexible web crawler framework
total releases 2latest release May 19, 2017most recent commit 6 years ago
Popular Programming Languages Categories
Python
Javascript
Java
Typescript
C Plus Plus
Php
Golang
C
C Sharp
Shell
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.