Awesome Open Source

Programming Languages

Search results for java hdfs

197 search results found

Bigdata Notes ⭐ 14,872

大数据入门指南 ⭐

Mycat Server ⭐ 9,431

God Of Bigdata ⭐ 8,483

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive.

Xlearning ⭐ 1,729

Hopsworks ⭐ 1,041

Hopsworks - Data-Intensive AI platform with a Feature Store

Addax ⭐ 1,034

Addax is a versatile open-source ETL tool that can seamlessly transfer data between various RDBMS and NoSQL databases, making it an ideal solution for data migration.

Mirror of Apache Sqoop

Kafka Connect Hdfs ⭐ 473

Kafka Connect HDFS connector

Storm Yarn ⭐ 419

Storm-yarn enables Storm clusters to be deployed into machines managed by Hadoop YARN.

Zdh_web ⭐ 379

大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批

Bigdata ⭐ 358

💎🔥大数据学习笔记

Cloudeon ⭐ 345

CloudEon uses Kubernetes to install and deploy open-source big data components, enabling the containerized operation of an open-source big data platform. This allows you to reduce your focus on underlying resource management and maintenance.

Hops Hadoop is a distribution of Apache Hadoop with distributed metadata.

Bigdata File Viewer ⭐ 269

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

Graph Analytics Engine

Terrapin ⭐ 168

Serving system for batch generated data sets

Dcos Commons ⭐ 162

DC/OS SDK is a collection of tools, libraries, and documentation for easy integration of technologies such as Kafka, Cassandra, HDFS, Spark, and TensorFlow with DC/OS.

Bigdata In Practice ⭐ 154

大数据实践项目 Hadoop、Spark、Kafka、Hbase、Flink.....

Lambda Arch ⭐ 151

A full big data pipeline (Lambda Architecture) with Spark, Kafka, HDFS and Cassandra.

Distributed Graph Analytics ⭐ 135

Distributed Graph Analytics (DGA) is a compendium of graph analytics written for Bulk-Synchronous-Parallel (BSP) processing frameworks such as Giraph and GraphX. The analytics included are High Betweenness Set Extraction, Weakly Connected Components, Page Rank, Leaf Compression, and Louvain Modularity.

Hdfs Shell ⭐ 129

HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS

Hadoopdemo ⭐ 128

Hadoop简单应用案例，包括MapReduce、单词统计、HDFS基本操作、web日志分析、Zoo

Smart Storage Management for Big Data, a comprehensive hot/cold data optimized solution

Cache File System optimized for columnar formats and object stores

Dynamometer ⭐ 110

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

Play Videos In Hdfs ⭐ 102

This project realizes playing videos storing in HDFS(Hadoop) in the web page online.在线播放HDFS中视频文件

基于wifi抓取信息的大数据查询分析系统

My Tutorial ⭐ 93

我想构建形成自己的知识的体系，工作职位是大数据，所以主要还是以大数据为主，从主流框架Hadoop，S 大数据开发是很繁琐的，正确的运行环境是成功的第一步，所以我尽量从搭建，部署，开发整个流程都做出来，单

Mirror of Linkedin's Camus

Trino Storage ⭐ 79

Storage connector for Trino

Mirror of Apache Chukwa

Platform Extension Framework: Federated Query Engine

Datamingproject ⭐ 67

大数据平台相关代码（ES/Hive/Hadoop/hdfs/hbase）

Hdfs File Slurper ⭐ 64

Utility to easily copy files into HDFS

Storm Hdfs ⭐ 61

Storm components for interacting with HDFS file systems

Hdfs Netdisc ⭐ 61

基于Hadoop的分布式云存储系统 🌴

A data layout optimization framework for wide tables stored on HDFS. See rainbow's webpage

A testing framework for Presto

Cloud Note ⭐ 59

基于分布式的云笔记（参考某道云笔记），数据存储在redis与hbase中

Elasticsearch Hdfs ⭐ 56

Hadoop Plugin for ElasticSearch

Shapefile ⭐ 56

Java library to read point and polygon shape files

Flume Canal Source ⭐ 54

Flume NG Canal source

Mapreduce Demo ⭐ 53

Hadoop，MapReduce编程学习练手实例

Arabesque ⭐ 50

Scalable Graph Mining

Hdfs Metadata ⭐ 48

Tool for gathering blocks and replicas meta data from HDFS. It also builds a heat map showing how replicas are distributed along disks and nodes.

Hadoop Unit ⭐ 45

Hadoop-Unit is a project which allow testing projects which need hadoop ecosysteme like kafka, solr, hdfs, hive, hbase, ...

Hadoop FSImage Analyzer (HFSA)

Dk Sqoop Plus ⭐ 42

本项目在原生sqoop 的基础上进行更多实用功能的扩展。

This repository contains open-source sample applications for IBM Streams.

Garmadon ⭐ 39

Java event logs collector for hadoop and frameworks

Hbase Meta Repair ⭐ 39

Repair hbase metadata table from hdfs.

Data Polygamy ⭐ 38

Data Polygamy is a topology-based framework that allows users to query for statistically significant relationships between spatio-temporal data sets.

Csds Material ⭐ 38

Course material for the Computer Systems for Data Science class at Columbia

Xxhadoop ⭐ 37

Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !

Combining datasets with MapReduce on NFL play by play data.

基于hdfs spark的视频非结构化数据计算

Hadoop Cli ⭐ 31

HADOOP-CLI is an interactive command line shell that makes interacting with the Hadoop Distribted Filesystem (HDFS) simpler and more intuitive than the standard command-line tools that come with Hadoop. If you're familiar with OS X, Linux, or even Windows terminal/console-based applications, then you are likely familiar with features such as tab completion, command history, and ANSI formatting.

Arvo2parquet ⭐ 30

Example program that writes Parquet formatted data to plain files (i.e., not Hadoop hdfs); Parquet is a columnar storage format.

Hive Solr ⭐ 30

使用Hive读写solr

Bigfatlm ⭐ 30

Hadoop MapReduce training of modified Kneser-Ney smoothed language models

Mongoreduce ⭐ 29

Hadoop Input and Ouput formats for MongoDB

Learning Spark ⭐ 29

Tidy up Spark and Hadoop tutorials.

Mapreduce ⭐ 29

清华大数据作业MapReduce处理几百个G的JSON数据

Fourinone ⭐ 29

Bigdatas ⭐ 27

this is a db-hdfs tools used to transfer big database datas to hadoop hdfs like sqoop,but bboss bigdata tool is very nice monitor and event drivered model,and high perfermance,support Distributed executor tasks Ability.

Hive Tools ⭐ 27

Sparkproject ⭐ 26

Using Apache Spark in an ArcMap Toolbox

Kafka Parquet Writer ⭐ 26

This project provides a compenent that reads logs from Kafka and writes it as parquet file on HDFS.

Hdfs Cli ⭐ 26

Interactive shell for interacting with Hadoop HDFS. Supports multiple HDFS hosts, command line history and tab completion.

Example Java Read And Write From Hdfs ⭐ 25

Example for Saagie Wiki - Read and write to HDFS with Java

Datatear ⭐ 25

Split into data blocks,In this format, efficient reading can be realized,Avoid unnecessary data reading operations.

Heterogeneity-incorporating Workflow ApplicationMaster for YARN

Csv To Orc ⭐ 24

Convert a CSV fle to ORCFile

Cobol To Hive ⭐ 24

Serde for Cobol Layout to Hive table

Scaling Hdfs Namenode ⭐ 23

NEW: see http://www.hops.io/. OLD: This work aims to re-engineer the Hadoop Distributed File System (HDFS) so that it can be 1) highly available, and 2) horizontally scalable. This is achieved by replacing the central master server with a distributed real-time database (in our implementation, MySQL Cluster).

Architect Java ⭐ 23

java后端架构师技术图谱

Neo4j Graphx ⭐ 23

Similar to Shifu - Neo4j-GraphX extends Neo4j graph database to process big data graph algorithms with HDFS and Apache Spark on a scalable data set

Volume Balancer ⭐ 23

DataNode Volumes Rebalancing tool for Apache Hadoop HDFS (HDFS-1312)

Navigator Sdk ⭐ 23

Bigdata Tutorial ⭐ 22

A testing framework for Trino

Document_search_engine_architecture ⭐ 22

📄🚀 Unleash a powerful Document Search Engine with Apache NiFi for lightning-fast, comprehensive text indexing and search.

⛺ blog by Vuepress

Storm Kafka Examples ⭐ 21

storm kafka hdfs examples

Bigdata_platform ⭐ 20

大数据建模分析平台

Spark Notes ⭐ 18

Note anything during writing spark or scala

HA, fault-tolerant, non-intrusive INotify for Hadoop HDFS

Data Pipeline Project ⭐ 18

Data pipeline project

Mapreduce Knn ⭐ 17

(java) K nearest neighbour implementation for Hadoop MapReduce

Pyramidio ⭐ 17

Image pyramid reader and writer

Hadoop Etl Udfs ⭐ 17

The Hadoop ETL UDFs are the main way to load data from Hadoop into EXASOL

Httpfs Client ⭐ 17

httpfs java client, read & write hdfs filesystem with the webhdfs REST HTTP API

Kube2hadoop ⭐ 17

Secure HDFS Access from Kubernetes

Big Data Application Firewall

Aerospike Hadoop ⭐ 16

Aerospike Hadoop Connector

Datax Src ⭐ 16

DataX 是异构数据广泛使用的离线数据同步工具/平台，实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase 等各种异构数据源之间高效的数据同步功能。

Hdfschecksumforlocalfile ⭐ 16

This program / jar creates checksum, with same algorithm that hadoop uses to create on hdfs files. So integrity of file can be verified on local and hadoop system. Can also, be used to check if file exist based on checksum, before uploading and cluttering hdfs with duplicate files.

Eshadoop ⭐ 15

Packt Publication - "Elasticsearch for Hadoop" Book Source Code

Minispark ⭐ 15

Java implementation of a mini Spark-like framework named MiniSpark that can run on top of a HDFS cluster. MiniSpark supports operators including Map, FlatMap, MapPair, Reduce, ReduceByKey, Collect, Count, Parallelize, Join and Filter.

Related Searches

Java Docker (6,180)

Java Database (6,015)

Java Mysql (5,954)

Java Sdk (5,864)

Java Apache (4,283)

Java Json (3,692)

Java Command Line (3,457)

Python Java (3,215)

Java Jdbc (2,549)

Java Hadoop (2,117)

1-100 of 197 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.