Awesome Open Source
Awesome Open Source
Combined Topics
hdfs x
Advertising 📦 10
All Projects
Application Programming Interfaces 📦 124
Applications 📦 192
Artificial Intelligence 📦 78
Blockchain 📦 73
Build Tools 📦 113
Cloud Computing 📦 80
Code Quality 📦 28
Collaboration 📦 32
Command Line Interface 📦 49
Community 📦 83
Companies 📦 60
Compilers 📦 63
Computer Science 📦 80
Configuration Management 📦 42
Content Management 📦 175
Control Flow 📦 213
Data Formats 📦 78
Data Processing 📦 276
Data Storage 📦 135
Economics 📦 64
Frameworks 📦 215
Games 📦 129
Graphics 📦 110
Hardware 📦 152
Integrated Development Environments 📦 49
Learning Resources 📦 166
Legal 📦 29
Libraries 📦 129
Lists Of Projects 📦 22
Machine Learning 📦 347
Mapping 📦 64
Marketing 📦 15
Mathematics 📦 55
Media 📦 239
Messaging 📦 98
Networking 📦 315
Operating Systems 📦 89
Operations 📦 121
Package Managers 📦 55
Programming Languages 📦 245
Runtime Environments 📦 100
Science 📦 42
Security 📦 396
Social Media 📦 27
Software Architecture 📦 72
Software Development 📦 72
Software Performance 📦 58
Software Quality 📦 133
Text Editors 📦 49
Text Processing 📦 136
User Interface 📦 330
User Interface Components 📦 514
Version Control 📦 30
Virtualization 📦 71
Web Browsers 📦 42
Web Servers 📦 26
Web User Interface 📦 210

The Top 41 Hdfs Open Source Projects

Seaweedfs ⭐ 11,797
SeaweedFS is a distributed storage system for blobs, objects, files, and data warehouse, to store and serve billions of files fast! Blob store has O(1) disk seek, local tiering, cloud tiering. Filer supports cross-cluster active-active replication, Kubernetes, POSIX, S3 API, encryption, Erasure Coding for warm storage, FUSE mount, Hadoop, WebDAV.
Bigdata Notes ⭐ 9,413
大数据入门指南 ⭐️
God Of Bigdata ⭐ 4,384
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
Juicefs ⭐ 3,095
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
Smart_open ⭐ 2,004
Utils for streaming large files (S3, HDFS, gzip, bz2...)
Ibis ⭐ 1,526
A pandas-like deferred expression system, with first-class SQL support
Tiledb ⭐ 1,089
The Universal Storage Engine
Hdfs ⭐ 1,001
A native go client for HDFS
Bigdata Interview ⭐ 908
🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
Snakebite ⭐ 828
A pure python HDFS client
Sparta ⭐ 516
Real Time Analytics and Data Pipelines based on Spark Streaming
Bigdata ⭐ 495
💎🔥大数据学习笔记
Devops Python Tools ⭐ 430
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
Kafka Connect Hdfs ⭐ 403
Kafka Connect HDFS connector
Kafka Connect Ui ⭐ 395
Web tool for Kafka Connect |
Divolte Collector ⭐ 265
Divolte Collector
Storagetapper ⭐ 236
StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service
Hdfs ⭐ 210
API and command line interface for HDFS
Bigdata_docker ⭐ 177
Big Data Ecosystem Docker
Dcos Commons ⭐ 163
DC/OS SDK is a collection of tools, libraries, and documentation for easy integration of technologies such as Kafka, Cassandra, HDFS, Spark, and TensorFlow with DC/OS.
Spark With Python ⭐ 160
Fundamentals of Spark with Python (using PySpark), code examples
Wradlib ⭐ 145
weather radar data processing - python package
Hsuntzu ⭐ 135
HDFS compress tar zip snappy gzip uncompress untar codec hadoop spark
Elasticctr ⭐ 125
ElasticCTR，即飞桨弹性计算推荐系统，是基于Kubernetes的企业级推荐系统开源解决方案。该方案融合了百度业务场景下持续打磨的高精度CTR模型、飞桨开源框架的大规模分布式训练能力、工业级稀疏参数弹性调度服务，帮助用户在Kubernetes环境中一键完成推荐系统部署，具备高性能、工业级部署、端到端体验的特点，并且作为开源套件，满足二次深度开发的需求。
Dynamometer ⭐ 122
A tool for scale and performance testing of HDFS with a specific focus on the NameNode.
Repository ⭐ 119
个人学习知识库涉及到数据仓库建模、实时计算、大数据、Java、算法等。
Hdfs Shell ⭐ 118
HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS
Bigdata File Viewer ⭐ 98
A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.
Wifi ⭐ 93
基于wifi抓取信息的大数据查询分析系统
Camus ⭐ 81
Mirror of Linkedin's Camus
Tiledb Py ⭐ 81
Python interface to the TileDB storage manager
Cloud Note ⭐ 74
基于分布式的云笔记（参考某道云笔记），数据存储在redis与hbase中
Big Data Engineering Coursera Yandex ⭐ 72
Big Data for Data Engineers Coursera Specialization from Yandex
Rumble ⭐ 58
⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more
Flume Canal Source ⭐ 57
Flume NG Canal source
Learning Spark ⭐ 37
零基础学习spark，大数据学习
Jsr203 Hadoop ⭐ 35
A Java NIO file system provider for HDFS
Pucket ⭐ 29
Bucketing and partitioning system for Parquet
Cluster Pack ⭐ 25
A library on top of either pex or conda-pack to make your Python code easily available on a cluster
Yandex Big Data Engineering ⭐ 17
Hadoop For Geoevent ⭐ 5
ArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.
1-41 of 41 projects
Advertising 📦 10
All Projects
Application Programming Interfaces 📦 124
Applications 📦 192
Artificial Intelligence 📦 78
Blockchain 📦 73
Build Tools 📦 113
Cloud Computing 📦 80
Code Quality 📦 28
Collaboration 📦 32
Command Line Interface 📦 49
Community 📦 83
Companies 📦 60
Compilers 📦 63
Computer Science 📦 80
Configuration Management 📦 42
Content Management 📦 175
Control Flow 📦 213
Data Formats 📦 78
Data Processing 📦 276
Data Storage 📦 135
Economics 📦 64
Frameworks 📦 215
Games 📦 129
Graphics 📦 110
Hardware 📦 152
Integrated Development Environments 📦 49
Learning Resources 📦 166
Legal 📦 29
Libraries 📦 129
Lists Of Projects 📦 22
Machine Learning 📦 347
Mapping 📦 64
Marketing 📦 15
Mathematics 📦 55
Media 📦 239
Messaging 📦 98
Networking 📦 315
Operating Systems 📦 89
Operations 📦 121
Package Managers 📦 55
Programming Languages 📦 245
Runtime Environments 📦 100
Science 📦 42
Security 📦 396
Social Media 📦 27
Software Architecture 📦 72
Software Development 📦 72
Software Performance 📦 58
Software Quality 📦 133
Text Editors 📦 49
Text Processing 📦 136
User Interface 📦 330
User Interface Components 📦 514
Version Control 📦 30
Virtualization 📦 71
Web Browsers 📦 42
Web Servers 📦 26
Web User Interface 📦 210