Awesome Open Source

Programming Languages

Search results for java web crawler

26 search results found

Spider Flow ⭐ 8,075

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

Webcollector ⭐ 2,974

WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.

Nutch ⭐ 2,742

Apache Nutch is an extensible and scalable web crawler

Gecco ⭐ 2,403

Easy to use lightweight web crawler（易用的轻量化网络爬虫）

100projectsofcode ⭐ 1,293

A list of practical knowledge-building projects.

A configurable web spider with a easy-to-use web console

Storm Crawler ⭐ 834

A scalable, mature and versatile web crawler based on Apache Storm

Sparkler ⭐ 401

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

News Crawl ⭐ 229

News crawling with StormCrawler - stores content as WARC

Crawler Commons ⭐ 217

A set of reusable Java components that implement functionality common to any web crawler

Collector Http ⭐ 162

Norconex Web Crawler (or spider) is a flexible web crawler for collecting, parsing, and manipulating data from the Internet (or Intranet) to various data repositories such as search engines.

Screenslicer ⭐ 159

Automatic, zero-config web scraping -- written in Java, has no dependency on Java EE or app servers, and the web scraper has a restful/JSON API. Currently unmaintained.

Save For Offline ⭐ 118

Android app for saving webpages for offline reading.

Animewatcher ⭐ 62

The goal of this project/app is to let the user watch anime without ads. It uses Jsoup to extract data from the website and Exoplayer to show videos.

Project Tauro ⭐ 48

A Router WiFi key recovery/cracking tool with a twist.

Android Web Scraper ⭐ 45

Android Web Scraper is a simple library for android web automation. You can perform web task in background to fetch website data programmatically.

Wpcrawler ⭐ 43

a web crawler for single WordPress site

Flink Crawler ⭐ 35

Continuous scalable web crawler built on top of Flink and crawler-commons

Google Search Results Java ⭐ 30

Google Search Results JAVA API via SerpApi

Crawler4j ⭐ 30

Open Source Simple Web Crawler for Java. Simple Flexible And Lightweight

Stormscraper ⭐ 29

A Storm based web crawler with Cassandra backend

A web scraper to download intraday Fitbit data (previously) unavailable in the official API

Java web crawling library

Distributed, asynchronous web crawler

Wcofun.com_downloader ⭐ 17

A java & Kotlin cartoon and anime downloader for https://www.wcofun.com/

Gumtree Scraper ⭐ 16

Are you looking for an apartment on gumtree.pl? You will like this a lot.

Spring Boot Web Scraper ⭐ 16

Simple web scrapping app made using Spring Boot + Thymeleaf + Jsoup + Java 8 Lambdas & Streams

Koronawirus Api ⭐ 14

API do śledzenia statystyk o koronawirusie w Polsce

Robots.txt ⭐ 13

🤖 robots.txt as a service. Crawls robots.txt files, downloads and parses them to check rules through an API

Android App ⭐ 12

The must-have app for an Amritian.

A simple web scraper built around the JavaFX WebEngine

Motogp Api ⭐ 11

MotoGP Api: Library that reads the results of the MotoGP, 500cc, Moto2 and Moto3 classes from a website.

Hwcollection ⭐ 11

A project to create a HOT WHEELS COLLECTION

Bobik_java_sdk ⭐ 11

Web scraping in Java using remote bots

Ghost Login ⭐ 11

Specifically designed to solve the web crawler when collecting Internet web data who need to login the web-site by useing some Simulated ways.

Web Scraping ⭐ 11

Code samples of web scraping using Java.

Semantic Web Scraper ⭐ 10

Codechef Crawler ⭐ 9

[deprecated] A web crawler that can download all successful submissions of a user in Codechef. Just provide the username.

Googleplayscraper ⭐ 9

Simple web scraper to get app information from Google Play

Chronicrawl ⭐ 8

Experimental continouous web crawler for web archiving

Invesdwin Webproxy ⭐ 8

A parallel download manager for web scraping that supports proxy servers

Cis555 Project ⭐ 7

The final project for CIS555 at the University of Pennsylvania.

Vertx Crawler ⭐ 7

Web Crawler based on Vert.x

Songdownloader ⭐ 7

Code to download .mp3 files off the web

🕷 a flexible web crawler framework

Kankorrobot ⭐ 7

Afghanistan Kankor Results Robot

Born2crawl ⭐ 7

A highly performant and versatile crawling engine, designed with scalability and extensibility in mind.

Drill Html Tools ⭐ 6

Apache Drill UDFs for retrieving and working with HTML text

Infk Uebungsscraper Java ⭐ 6

get all your exercises with one click

Price_comparison_app ⭐ 6

An android application used to compare prices of online products

Ar Shopping ⭐ 6

AR-based shopping application for Google Glass Enterprise Edition 2.

Jsoupwebscrap ⭐ 6

Aplikasi Android untuk memenuhi kebutuhan asupan perwibuan berupa streaming anime dan baca manga berbahasa Indonesia yang insya Allah up to date as long as website yang jadi sumber dari konten aplikasi ini gak down atau gak menghilang dari dunia yang fana ini.

Java Deepdive Final ⭐ 5

This is our final project for our "Java Deep Dive" course, containing technologies such as Webscraping, postgres, JPA, Threading

Postdenuncia ⭐ 5

Projeto de software para cidadãos denunciarem problemas urbanos.

Webcrawler ⭐ 5

A Recursive Web crawler built with Java 8, reactive streams, async queues and AWS DynamoDB.

Book searching application

Duckduckgo ⭐ 5

An unofficial Duckduckgo.com API with performance and simplicity in mind

Social Web Spider ⭐ 5

CS473 Project 2

Related Searches

Java Spring (21,350)

Java Spring Boot (11,982)

Java Video Game (8,093)

Java Gradle (8,072)

Java Docker (6,180)

Java Database (6,015)

Java Mysql (5,954)

Java Sdk (5,864)

Java Rest (4,956)

Java Algorithms (4,705)

1-26 of 26 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.