Awesome Open Source

Programming Languages

Search results for java hadoop

800 search results found

Pignlproc ⭐ 160

Apache Pig utilities to build training corpora for machine learning / NLP out of public Wikipedia and DBpedia dumps.

Simple Yarn App ⭐ 156

Simple YARN application

Warcbase ⭐ 154

Warcbase is an open-source platform for managing analyzing web archives

Logparser ⭐ 153

Easy parsing of Apache HTTPD and NGINX access logs with Java, Hadoop, Hive, Flink, Beam, Storm, Drill, ...

Spatialhadoop2 ⭐ 148

The second generation of SpatialHadoop that ships as an extension

Csdn Code ⭐ 138

停止维护 -->移步 https://github.com/vbay/tutorials

Bigdata Learning ⭐ 136

大数据学习记录

Logvision ⭐ 136

分布式实时日志分析与入侵检测系统

Hadoop Framework Examples ⭐ 135

An implementation of a real-world map-reduce workflow in each major framework.

Ephemeral Hadoop clusters using Google Compute Platform

Hadoop Hdfs Fsimage Exporter ⭐ 131

Exports Hadoop HDFS content statistics to Prometheus

Hdfs Shell ⭐ 129

HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS

Mirror of Apache Tajo

Hadoopdemo ⭐ 128

Hadoop简单应用案例，包括MapReduce、单词统计、HDFS基本操作、web日志分析、Zoo

HIPI: Hadoop Image Processing Interface

hRaven collects run time data and statistics from MapReduce jobs in an easily queryable format

Ctenopharyngodon Idella ⭐ 125

Hadoop, MapReduce Distributed Crawling of Data Information from All Chinese Universities. （Hadoop,mapreduce分布式爬取掌上高考的所有中国大学数据）

Smart Storage Management for Big Data, a comprehensive hot/cold data optimized solution

Shuttle ⭐ 123

Shuttle：High Available, High Performance Remote Shuffle Service

Sequenceiq Samples ⭐ 119

SequenceIQ Hadoop examples

Hbaseclient ⭐ 118

HBase客户端数据管理软件

Spark Terasort ⭐ 116

Hackathon ⭐ 114

Library and resources for hack/reduce Hackathon events

Hopsworks ⭐ 114

HopsWorks - Hadoop for Humans

Xichuan_note ⭐ 114

xichuan的学习总结笔记,覆盖了java、spring、java其他常用框架,以及大数据相关组件

Asakusafw ⭐ 113

Asakusa Framework

Spark Streaming监控平台，支持任务部署与告警、自启动

Mpich2 Yarn ⭐ 112

Running MPICH2 on Yarn

Avro Hadoop Starter ⭐ 111

Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.

The Apache Gora open source framework provides an in-memory data model and persistence for big data.

Jetstream ⭐ 110

Jetstream is a streaming processing framework

Mirror of Apache DataFu

Calcite Avatica Go ⭐ 110

Mirror of Apache Calcite - Avatica Go SQL Driver

Dynamometer ⭐ 110

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

Hprof2flamegraph ⭐ 108

Flame Graph visualization for Java (HPROF, Honest-profiler)

Nnanalytics ⭐ 106

NameNodeAnalytics is a self-help utility for scouting and maintaining the namespace of an HDFS instance.

Logisland ⭐ 106

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

Kafka Connect Fs ⭐ 106

Kafka Connect FileSystem Connector

Introtohadoopandmr__udacity_course ⭐ 103

🐘 Source code for assignments of Udacity course "Introduction to Hadoop and MapReduce"

Metronome ⭐ 103

Suite of parallel iterative algorithms built on top of Iterative Reduce

Play Videos In Hdfs ⭐ 102

This project realizes playing videos storing in HDFS(Hadoop) in the web page online.在线播放HDFS中视频文件

Please use the luke bundled with lucene! This repo is archived and frozen now.

Big Data ETL and Utilities for Hadoop Map Reduce, Spark and Storm

S3committer ⭐ 99

Hadoop output committers for S3

Socialite ⭐ 99

SociaLite: query language for large-scale graph analysis and data mining

AntsDB is a low latency, high concurrency, MySQL compliant SQL layer for HBase

kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)

基于wifi抓取信息的大数据查询分析系统

Mongo Spark ⭐ 93

Example application on how to use mongo-hadoop connector with Spark

My Tutorial ⭐ 93

我想构建形成自己的知识的体系，工作职位是大数据，所以主要还是以大数据为主，从主流框架Hadoop，S 大数据开发是很繁琐的，正确的运行环境是成功的第一步，所以我尽量从搭建，部署，开发整个流程都做出来，单

Mirror of Apache REEF

Hadoop 2.7.1 Windows 64 Binaries ⭐ 92

Pre-compiled, unofficial Win64 Binaries for Hadoop 2.7.1

Flink Sql Benchmark ⭐ 92

Graphbuilder ⭐ 90

The GraphBuilder library provides functions to construct large scale graphs. It is implemented on Apache Hadoop.

Halyard is an extremely horizontally scalable Triplestore with support for Named Graphs, designed for integration of extremely large Semantic Data Models, and for storage and SPARQL 1.1 querying of the whole Linked Data universe snapshots.

Mirror of Linkedin's Camus

Kafka Hadoop Loader ⭐ 84

Hadoop Job for schemaless incremental loading of messages from Kafka topics onto hdfs with configurable output partitioning. 🚫

Pig on Apache Spark

Hadoop Data Integration with various databases, ftp servers, salesforce. Incremental update, dedup, append, merge your data on Hadoop.

Management and automation platform for Stateful Distributed Systems

Wikihadoop ⭐ 84

Stream-based InputFormat for processing the compressed XML dumps of Wikipedia with Hadoop

Machine Learning for Cascading

提供基于spring-cloud系列整合的依赖jar包，再加入了分布式锁，分布式事务，接口文档，多数

Templates ⭐ 82

DevOps Templates for Kubernetes, AWS, GCP, Terraform, Docker, Packer, Jenkins, CircleCI, GitHub Actions, Lambda, AWS CodeBuild, GCP Cloud Build, Vagrant, Puppet, Python, Bash, Go, Perl, Java, Scala, Groovy, Maven, SBT, Gradle, Make, Jenkinsfile, Makefile, Dockerfile, docker-compose.yml, Vagrantfile, M4 etc...

Trino Storage ⭐ 79

Storage connector for Trino

A bunch of utility classes for Java, Hadoop, HBase, Pig, etc.

Mirror of Apache Chukwa

Trident Lambda Splout ⭐ 78

A toy example of a "Lambda architecture" using Storm's Trident as real-time layer and Splout SQL as batch layer.

Common metadata layer for Hadoop's Map Reduce, Pig, and Hive

Practical Machine Learning ⭐ 77

Docker Spark ⭐ 77

🚢 Docker image for Apache Spark

Platform Extension Framework: Federated Query Engine

Source code that accompanies the book "Hadoop in Practice, Second Edition".

Hadoop Cos ⭐ 75

hadoop-cos（CosN文件系统）为Apache Hadoop、Spark以及Tez等大数据计算框架集成提供支持，可以像访问HDFS一样读写存储在腾讯 Storage

Kangaroo ⭐ 75

Hadoop utilities for Kafka, S3, and more

Hdfs2cass ⭐ 75

Hadoop mapreduce job to bulk load data into Cassandra

Euphoria ⭐ 74

Euphoria is an open source Java API for creating unified big-data processing flows. It provides an engine independent programming model which can express both batch and stream transformations.

open source search platform based on Lucene, Solr, HBase

本项目已废弃，笔记收藏整理参考：

Kafka_spark_hbase_demo ⭐ 72

kafka spark hbase 日志统计

Hbase Orm ⭐ 72

A production-grade HBase ORM library that makes accessing HBase clean, fast and fun (Can also be used as Bigtable ORM)

The Apache Ignite Book ⭐ 72

All code samples, scripts and more in-depth examples for The Apache Ignite Book. Include Apache Ignite 2.6 or above

An iterative computing framework for both Hadoop MapReduce and Hadoop YARN.

Hadoop Map Reduce Patterns ⭐ 71

Hadoop Map-Reduce Design Patterns

Searchanalytics Bigdata ⭐ 71

Customer Product search clicks analytics using big data Hadoop, Hive, Oozie, ElasticSearch, Akka, Spring Data

Coderreviewnotes ⭐ 70

程序员的百科全书

Cascading Dbmigrate ⭐ 69

Tool to help users migrate large relational databases into Hadoop clusters.

Jumbune, an open source BigData APM & Data Quality Management Platform for Data Clouds. Enterprise feature offering is available at http://jumbune.com. More details of open source offering are at,

Hive_test ⭐ 68

Unit test framework for hive and hive-service

Glusterfs Hadoop ⭐ 68

GlusterFS plugin for Hadoop HCFS

Datamingproject ⭐ 67

大数据平台相关代码（ES/Hive/Hadoop/hdfs/hbase）

Hadoop Bam ⭐ 66

Hadoop-BAM is a Java library for the manipulation of files in common bioinformatics formats using the Hadoop MapReduce framework

Hadoop Java Example ⭐ 66

A very simple example of using Hadoop's MapReduce functionality in Java.

Splittablegzip ⭐ 66

Splittable Gzip codec for Hadoop

Sqlwindowing ⭐ 65

SQL Windowing Functions for Hadoop

Hdfs File Slurper ⭐ 64

Utility to easily copy files into HDFS

Hive Io Experimental ⭐ 62

Hive I/O Library

Hive Funnel Udf ⭐ 61

Hive UDFs for funnel analysis

Examples ⭐ 61

A repository for the code examples of my blog

Related Searches

Java Spring (21,350)

Java Jar (7,924)

Java Testing (7,163)

Java Database (6,015)

Java Mysql (5,954)

Javascript Java (5,468)

Java Algorithms (4,705)

Java Apache (4,283)

Java Cloud Computing (4,240)

Java Json (3,692)

101-200 of 800 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.