Awesome Open Source

Programming Languages

Search results for hadoop hdfs

404 search results found

Bigdata Notes ⭐ 14,872

大数据入门指南 ⭐

God Of Bigdata ⭐ 8,483

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive.

Tensorflowonspark ⭐ 3,851

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

The flexibility of Python with the scale and performance of modern SQL.

Docker Hadoop ⭐ 1,955

Apache Hadoop docker image

Xlearning ⭐ 1,729

Poseidon ⭐ 1,543

A search engine which can hold 100 trillion lines of log data.

Bigdata Interview ⭐ 1,397

🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop

A native go client for HDFS

Bigdata Growth ⭐ 1,256

大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。

Addax ⭐ 1,034

Addax is a versatile open-source ETL tool that can seamlessly transfer data between various RDBMS and NoSQL databases, making it an ideal solution for data migration.

Snakebite ⭐ 854

A pure python HDFS client

Mirror of Apache Sqoop

Devops Python Tools ⭐ 709

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Kafka Connect Hdfs ⭐ 473

Kafka Connect HDFS connector

Storm Yarn ⭐ 419

Storm-yarn enables Storm clusters to be deployed into machines managed by Hadoop YARN.

Bigdata ⭐ 358

💎🔥大数据学习笔记

Cloudeon ⭐ 345

CloudEon uses Kubernetes to install and deploy open-source big data components, enabling the containerized operation of an open-source big data platform. This allows you to reduce your focus on underlying resource management and maintenance.

Hops Hadoop is a distribution of Apache Hadoop with distributed metadata.

Graph Analytics Engine

Hadoop Mini Clusters ⭐ 251

hadoop-mini-clusters provides an easy way to test Hadoop projects directly in your IDE

Hadoopy ⭐ 244

Python MapReduce library written in Cython. Visit us in #hadoopy on freenode. See the link below for documentation and tutorials.

Bigdata_docker ⭐ 226

Big Data Ecosystem Docker

Hadoop Attack Library ⭐ 200

A collection of pentest tools and resources targeting Hadoop environments

Wifiprobeanalysis ⭐ 189

基于WIFI探针的商业大数据分析技术

Magpie contains a number of scripts for running Big Data software in HPC environments, including Hadoop and Spark. There is support for Lustre, Slurm, Moab, Torque. LSF, Flux, and more.

Terrapin ⭐ 168

Serving system for batch generated data sets

Juicy Bigdata ⭐ 162

🎉🎉🐳 Datawhale大数据处理导论教程 | 大数据技术方向的开篇课程🎉🎉

Ipython Spark Docker ⭐ 151

Hsuntzu ⭐ 134

HDFS compress tar zip snappy gzip uncompress untar codec hadoop spark

Hadoop Hdfs Fsimage Exporter ⭐ 131

Exports Hadoop HDFS content statistics to Prometheus

Hdfs_fdw ⭐ 131

PostgreSQL foreign data wrapper for HDFS

Hdfs Shell ⭐ 129

HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS

Hadoopdemo ⭐ 128

Hadoop简单应用案例，包括MapReduce、单词统计、HDFS基本操作、web日志分析、Zoo

A tool and library for easily deploying applications on Apache YARN

Smart Storage Management for Big Data, a comprehensive hot/cold data optimized solution

A Go client binding for Hadoop HDFS using WebHDFS.

Vagrant Hadoop Spark Cluster ⭐ 121

Vagrant project to spin up a cluster of 4 32-bit CentOS6.5 Linux virtual machines with Hadoop v2.6.0 and Spark v1.1.1

Kylin Docker ⭐ 116

This repository trackes the code and files for building docker image with Apache Kylin.

Mpich2 Yarn ⭐ 112

Running MPICH2 on Yarn

Construct and run Hadoop MapReduce programs in Haskell

Dynamometer ⭐ 110

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

Kafka Connect Fs ⭐ 106

Kafka Connect FileSystem Connector

Nnanalytics ⭐ 106

NameNodeAnalytics is a self-help utility for scouting and maintaining the namespace of an HDFS instance.

Play Videos In Hdfs ⭐ 102

This project realizes playing videos storing in HDFS(Hadoop) in the web page online.在线播放HDFS中视频文件

Spark With Python ⭐ 98

Fundamentals of Spark with Python (using PySpark), code examples

Ros_hadoop ⭐ 98

Hadoop splittable InputFormat for ROS. Process rosbag with Hadoop Spark and other HDFS compatible systems.

基于wifi抓取信息的大数据查询分析系统

My Tutorial ⭐ 93

我想构建形成自己的知识的体系，工作职位是大数据，所以主要还是以大数据为主，从主流框架Hadoop，S 大数据开发是很繁琐的，正确的运行环境是成功的第一步，所以我尽量从搭建，部署，开发整个流程都做出来，单

Correlation Approximation ⭐ 90

Spark implementation of the Google Correlate algorithm to quickly find highly correlated vectors in huge datasets

Focusbigdata ⭐ 89

【大数据成神之路学习路径+面经+简历】

Python HDFS client

Devops Perl Tools ⭐ 88

25+ DevOps CLI Tools - Anonymizer, SQL ReCaser (MySQL, PostgreSQL, AWS Redshift, Snowflake, Apache Drill, Hive, Impala, Cassandra CQL, Microsoft SQL Server, Oracle, Couchbase N1QL, Dockerfiles), Hadoop HDFS & Hive tools, Solr/SolrCloud CLI, Nginx stats & HTTP(S) URL watchers for load-balanced web farms, Linux tools etc.

Mirror of Linkedin's Camus

Kafka Hadoop Loader ⭐ 84

Hadoop Job for schemaless incremental loading of messages from Kafka topics onto hdfs with configurable output partitioning. 🚫

Hadoop_cookbook ⭐ 80

Cookbook to install Hadoop 2.0+ using Chef

Trino Storage ⭐ 79

Storage connector for Trino

Vagrant Hadoop 2.4.1 Spark 1.0.1 ⭐ 79

Vagrant project to spin up a cluster virtual machines with Hadoop v2.4.1 and Spark v1.0.1

Mirror of Apache Chukwa

Docker Hadoop ⭐ 76

Dockerfile for running Hadoop on Ubuntu

Platform Extension Framework: Federated Query Engine

Euphoria ⭐ 74

Euphoria is an open source Java API for creating unified big-data processing flows. It provides an engine independent programming model which can express both batch and stream transformations.

Sparkplugins ⭐ 70

Code and examples of how to write and deploy Apache Spark Plugins with Spark 3.x. Spark plugins allow runnig custom code on the executors as they are initialized. This also allows extending the Spark metrics systems with user-provided monitoring probes.

Scala Hadoop ⭐ 70

Using Hadoop with Scala

Datamingproject ⭐ 67

大数据平台相关代码（ES/Hive/Hadoop/hdfs/hbase）

Python Hdfs ⭐ 67

HDFS client for Python

Hdfs File Slurper ⭐ 64

Utility to easily copy files into HDFS

Hdfs Netdisc ⭐ 61

基于Hadoop的分布式云存储系统 🌴

Storm Hdfs ⭐ 61

Storm components for interacting with HDFS file systems

Chef Bach ⭐ 60

Chef recipes for Bloomberg's deployment of Hadoop and related components

Textgrounder ⭐ 60

A system for connecting language to space and time.

Mylearningnotes ⭐ 58

Because its never late to start taking notes and 'public' it...

Platys Modern Data Platform ⭐ 58

Support for generating modern platforms dynamically with services such as Kafka, Spark, Streamsets, HDFS, ....

Fluent Plugin Webhdfs ⭐ 58

Hadoop WebHDFS output plugin for Fluentd

Docker Hadoop ⭐ 57

Docker image for main Apache Hadoop components (Yarn/Hdfs)

Jruby On Hadoop ⭐ 56

Using Hadoop by Ruby script, supported by JRuby. Not Hadoop streaming.

Elasticsearch Hdfs ⭐ 56

Hadoop Plugin for ElasticSearch

Geodocker ⭐ 55

Central repository for the GeoDocker project

Bigdataparty ⭐ 54

大数据组件 All-in-One 的 Dockerfile

Clickhouse Hdfs Loader ⭐ 54

loading hdfs data to clickhouse

Ganapati ⭐ 53

Ruby interface to Hadoop's HDFS via Thrift

Mapreduce Demo ⭐ 53

Hadoop，MapReduce编程学习练手实例

Docker Hadoop ⭐ 51

A Docker container with a full Hadoop cluster setup with Spark and Zeppelin

Teraslice ⭐ 50

Scalable data processing pipelines in JavaScript

Node Hdfs ⭐ 50

Access to Hadoop's HDFS via libhdfs and JNI

Arabesque ⭐ 50

Scalable Graph Mining

Solr Map Reduce Example ⭐ 50

A project meant to be a reference to get started building indexes for Solr with Hadoop's map-reduce.

Hadoop Papyrus ⭐ 48

Hadoop MapReduce DSL framework by Ruby. Changed from hadoop-rubydsl.

Jsr203 Hadoop ⭐ 47

A Java NIO file system provider for HDFS

Hadoop Unit ⭐ 45

Hadoop-Unit is a project which allow testing projects which need hadoop ecosysteme like kafka, solr, hdfs, hive, hbase, ...

Flume Kafka Storm ⭐ 45

大数据实时计算的基础框架

Logstash Hdfs ⭐ 45

Logstash HDFS plugin

Implementation of "A Parallel Spatial Co-location Mining Algorithm Based on MapReduce" paper

Hadoop FSImage Analyzer (HFSA)

Sparkoscope ⭐ 43

Enabling Spark Optimization through Cross-stack Monitoring and Visualization

Neo4j Dbpedia Importer ⭐ 43

DBpedia.org RDF to CSV for import into Neo4j

gomrjob - a Go Framework for Hadoop Map Reduce Jobs

Hadoop HDFS and Yarn client

Related Searches

Java Hadoop (2,117)

Spark Hadoop (1,188)

Hadoop Mapreduce (851)

Shell Hadoop (766)

Python Hadoop (761)

Java Hdfs (752)

Hadoop Hive (703)

Spark Hdfs (573)

Apache Hadoop (514)

Scala Hadoop (479)

1-100 of 404 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.