Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Elasticsearch Crawler Open Source Projects

Open source projects categorized as Elasticsearch Crawler
Categories > Data Processing > Crawler
Categories > Data Storage > Elasticsearch
Wooyun_public ⭐ 3,701
This repo is archived. Thanks for wooyun! 乌云公开漏洞、知识库爬虫和搜索 crawl and search for wooyun.org public bug(vulnerability) and drops
most recent commit 5 years ago
Spring Boot Quick ⭐ 2,282
🌿 基于springboot的快速学习示例,整合自己遇到的开源框架,如：rabbitmq(延迟队列)、K
most recent commit 6 months ago
News Please ⭐ 1,821
news-please - an integrated web crawler and information extractor for news that just works
dependent packages 4total releases 121most recent commit 4 months agopypi news-please} Downloads
Diskover Community ⭐ 1,391
Diskover Community Edition - Open source file indexer, file search engine and data management and analytics powered by Elasticsearch
most recent commit 23 days ago
Fscrawler ⭐ 1,279
Elasticsearch File System Crawler (FS Crawler)
dependent packages 1total releases 5most recent commit 3 months ago
Fess ⭐ 943
Fess is very powerful and easily deployable Enterprise Search Server.
dependent packages 26total releases 148most recent commit 3 months ago
Ipfs Search ⭐ 779
Search engine for the Interplanetary Filesystem.
dependent packages 1total releases 19most recent commit a year ago
Monocle ⭐ 326
Monocle helps teams and individual to better organize daily duties and to detect anomalies in the way changes are produced and reviewed.
most recent commit 3 months ago
Freshonions Torscraper ⭐ 313
Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion
most recent commit 4 years ago
Gopa ⭐ 281
[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
total releases 6most recent commit 3 years ago
Elasticsearch River Web ⭐ 232
Web Crawler for Elasticsearch
total releases 10most recent commit 4 years ago
News Crawl ⭐ 229
News crawling with StormCrawler - stores content as WARC
most recent commit 4 months ago
Weixin_crawler ⭐ 209
高效微信公众号历史文章和阅读数据爬虫powered by scrapy
most recent commit 5 years ago
Od Database ⭐ 113
Distributed crawler, database and web frontend for public directories indexing
most recent commit 4 years ago
Bathyscaphe ⭐ 83
Fast, highly configurable, cloud native dark web crawler.
total releases 16most recent commit 10 months ago
Movie Elasticsearch ⭐ 76
使用 SpringBoot2.0+ElasticSearch 实现的开源电影搜索引擎
most recent commit 2 years ago
Dodder ⭐ 71
A distributed DHT crawler that sniffs torrents from BitTorrent network
most recent commit 3 years ago
Docker Diskover ⭐ 66
A Docker container for the Diskover space mapping application
most recent commit 3 months ago
Harvester ⭐ 59
Web crawling and document processing through a usable interface.
most recent commit 7 years ago
Fishfishjump ⭐ 57
Fish Fish Jump is a solution in the python that simply and basic for search engines. 🐟 🐟 🐟
total releases 10most recent commit 6 years ago
Gdht ⭐ 48
A distributed self-host DHT torrent search suite
most recent commit 4 years ago
Anilist Crawler ⭐ 43
Crawl data from anilist API and store in MariaDB.
most recent commit 3 months ago
Go Crawler Distributed ⭐ 39
分布式爬虫项目，本项目支持个性化定制页面解析器二次开发，项目整体采用微服务架构，通过消息队列实现消息 gorm, goquery, easyjson, viper, amqp, zap, go-micro，并通过Docker实现容器化部署，中间爬虫节点支持水平拓展。
most recent commit 6 months ago
The 7 Most Depended On Elasticsearch Crawler Open Source Projects
Fess ⭐ 943
Fess is very powerful and easily deployable Enterprise Search Server.
dependent packages 26total releases 148latest release October 19, 2023most recent commit 3 months ago
News Please ⭐ 1,821
news-please - an integrated web crawler and information extractor for news that just works
dependent packages 4total releases 121latest release August 30, 2023most recent commit 4 months agopypi news-please} Downloads
Bot Marvin ⭐ 9
Highly scalable crawler with best features.
dependent packages 2total releases 6latest release July 21, 2016most recent commit 8 years agonpm bot-marvin} Downloads
Ipfs Search ⭐ 779
Search engine for the Interplanetary Filesystem.
dependent packages 1total releases 19latest release April 20, 2021most recent commit a year ago
Microcrawler Js ⭐ 11
Scrapping made easy...
dependent packages 1total releases 35latest release September 03, 2016most recent commit 8 years agonpm microcrawler} Downloads
Crawling Framework ⭐ 21
Easily crawl news portals or blog sites using Storm Crawler.
dependent packages 1total releases 1latest release September 19, 2017most recent commit 3 years ago
Fscrawler ⭐ 1,279
Elasticsearch File System Crawler (FS Crawler)
dependent packages 1total releases 5latest release January 10, 2022most recent commit 3 months ago
The 10 Latest Releases In Elasticsearch Crawler Open Source Projects
I_love_indexes ⭐ 7
latest release November 26, 2023most recent commit 9 years ago
Fess ⭐ 943
Fess is very powerful and easily deployable Enterprise Search Server.
dependent packages 26total releases 148latest release October 19, 2023most recent commit 3 months ago
News Please ⭐ 1,821
news-please - an integrated web crawler and information extractor for news that just works
dependent packages 4total releases 121latest release August 30, 2023most recent commit 4 months agopypi news-please} Downloads
Fscrawler ⭐ 1,279
Elasticsearch File System Crawler (FS Crawler)
dependent packages 1total releases 5latest release January 10, 2022most recent commit 3 months ago
Gopa ⭐ 281
[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
total releases 6latest release May 19, 2021most recent commit 3 years ago
Gumo ⭐ 21
A crawler that extracts data from a dynamic webpage. Written in node js.
total releases 6latest release April 26, 2021most recent commit 2 years ago
Ipfs Search ⭐ 779
Search engine for the Interplanetary Filesystem.
dependent packages 1total releases 19latest release April 20, 2021most recent commit a year ago
Bathyscaphe ⭐ 83
Fast, highly configurable, cloud native dark web crawler.
total releases 16latest release March 05, 2021most recent commit 10 months ago
Trollhunter ⭐ 27
Twitter Troll & Fake News Hunter - Crawls news websites and twitter to identify fake news
total releases 15latest release March 27, 2020most recent commit 3 years ago
Torture ⭐ 13
FTP search based on Go! and ElasticSearch for the 31. Chaos Communication Congress
total releases 1latest release December 27, 2018most recent commit 5 years ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.