Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 S3 Hdfs Open Source Projects

Open source projects categorized as S3 Hdfs
Seaweedfs ⭐ 20,994
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
dependent packages 2total releases 296most recent commit 7 days ago
Ceph ⭐ 12,859
Ceph is a distributed object, block, and file storage platform
dependent packages 1total releases 1most recent commit 3 months ago
Juicefs ⭐ 9,252
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
dependent packages 1total releases 136most recent commit 3 months ago
Smart_open ⭐ 3,065
Utils for streaming large files (S3, HDFS, gzip, bz2...)
most recent commit a month ago
Tiledb ⭐ 1,700
The Universal Storage Engine
dependent packages 6total releases 87most recent commit 3 months ago
Kafka Connect Ui ⭐ 494
Web tool for Kafka Connect |
most recent commit 4 months ago
Bigdata File Viewer ⭐ 269
A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.
most recent commit 6 months ago
Storagetapper ⭐ 269
StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service
total releases 4most recent commit 2 years ago
Pysparkling ⭐ 253
A pure Python implementation of Apache Spark's RDD and DStream interfaces.
dependent packages 1total releases 69most recent commit a year agopypi pysparkling} Downloads
Rumble ⭐ 194
⛈️ RumbleDB 1.21.0 "Hawthorn blossom" 🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more
total releases 4most recent commit a year ago
Tiledb Py ⭐ 167
Python interface to the TileDB storage engine
dependent packages 23total releases 125most recent commit 3 months agopypi tiledb} Downloads
Elbencho ⭐ 125
A distributed storage benchmark for file systems, object stores & block devices with support for GPUs
most recent commit 3 months ago
Kafka Connect Fs ⭐ 106
Kafka Connect FileSystem Connector
most recent commit a year ago
Megfile ⭐ 99
Megvii FILE Library - Working with Files in Python same as the standard library
dependent packages 2total releases 68most recent commit 3 months agopypi megfile} Downloads
Tiledb R ⭐ 96
R interface to TileDB: The Modern Database
total releases 33most recent commit 3 months ago
Pxf ⭐ 76
Platform Extension Framework: Federated Query Engine
most recent commit 3 months ago
Helyim ⭐ 76
seaweedfs implemented in pure Rust
most recent commit 3 months ago
Monix Connect ⭐ 60
A set of connectors for Monix. 🔛
dependent packages 1total releases 14most recent commit a year ago
Uio ⭐ 59
a Clojure library for accessing HDFS, S3, SFTP and other file systems via a single API
total releases 2most recent commit 5 years ago
Cluster Pack ⭐ 44
A library on top of either pex or conda-pack to make your Python code easily available on a cluster
dependent packages 4total releases 39most recent commit 4 months agopypi cluster-pack} Downloads
Tiledb Go ⭐ 43
Go Interface to the TileDB storage manager
dependent packages 1total releases 111most recent commit 3 months ago
Qscamel ⭐ 38
qscamel is a command line tool to migrate data between different endpoint efficiently.
total releases 25most recent commit 3 months ago
Etl Light ⭐ 38
A light Kafka to HDFS/S3 ETL library based on Apache Spark
most recent commit 7 years ago
The 10 Most Depended On S3 Hdfs Open Source Projects
Tiledb Py ⭐ 167
Python interface to the TileDB storage engine
dependent packages 23total releases 125latest release November 29, 2023most recent commit 3 months agopypi tiledb} Downloads
Tiledb ⭐ 1,700
The Universal Storage Engine
dependent packages 6total releases 87latest release November 05, 2022most recent commit 3 months ago
Cluster Pack ⭐ 44
A library on top of either pex or conda-pack to make your Python code easily available on a cluster
dependent packages 4total releases 39latest release November 07, 2023most recent commit 4 months agopypi cluster-pack} Downloads
Stream Loader ⭐ 30
Components for building stream loaders from Kafka to arbitrary storages
dependent packages 4total releases 24latest release August 30, 2023most recent commit 8 months ago
Megfile ⭐ 99
Megvii FILE Library - Working with Files in Python same as the standard library
dependent packages 2total releases 68latest release November 27, 2023most recent commit 3 months agopypi megfile} Downloads
Pysparkling ⭐ 253
A pure Python implementation of Apache Spark's RDD and DStream interfaces.
dependent packages 1total releases 69latest release November 13, 2022most recent commit a year agopypi pysparkling} Downloads
Tiledb Go ⭐ 43
Go Interface to the TileDB storage manager
dependent packages 1total releases 111latest release November 27, 2023most recent commit 3 months ago
Ceph ⭐ 12,859
Ceph is a distributed object, block, and file storage platform
dependent packages 1total releases 1latest release August 26, 2014most recent commit 3 months ago
Juicefs ⭐ 9,252
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
dependent packages 1total releases 136latest release November 28, 2023most recent commit 3 months ago
The 10 Latest Releases In S3 Hdfs Open Source Projects
Tiledb Py ⭐ 167
Python interface to the TileDB storage engine
dependent packages 23total releases 125latest release November 29, 2023most recent commit 3 months agopypi tiledb} Downloads
Juicefs ⭐ 9,252
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
dependent packages 1total releases 136latest release November 28, 2023most recent commit 3 months ago
Tiledb Go ⭐ 43
Go Interface to the TileDB storage manager
dependent packages 1total releases 111latest release November 27, 2023most recent commit 3 months ago
Megfile ⭐ 99
Megvii FILE Library - Working with Files in Python same as the standard library
dependent packages 2total releases 68latest release November 27, 2023most recent commit 3 months agopypi megfile} Downloads
Cluster Pack ⭐ 44
A library on top of either pex or conda-pack to make your Python code easily available on a cluster
dependent packages 4total releases 39latest release November 07, 2023most recent commit 4 months agopypi cluster-pack} Downloads
Tiledb R ⭐ 96
R interface to TileDB: The Modern Database
total releases 33latest release September 27, 2023most recent commit 3 months ago
Stream Loader ⭐ 30
Components for building stream loaders from Kafka to arbitrary storages
dependent packages 4total releases 24latest release August 30, 2023most recent commit 8 months ago
Pysparkling ⭐ 253
A pure Python implementation of Apache Spark's RDD and DStream interfaces.
dependent packages 1total releases 69latest release November 13, 2022most recent commit a year agopypi pysparkling} Downloads
Monix Connect ⭐ 60
A set of connectors for Monix. 🔛
dependent packages 1total releases 14latest release November 07, 2022most recent commit a year ago
Tiledb ⭐ 1,700
The Universal Storage Engine
dependent packages 6total releases 87latest release November 05, 2022most recent commit 3 months ago
Popular Data Storage Categories
Database
Mysql
Redis
Mongodb
Postgresql
Firebase
Elasticsearch
Backup
Sqlite
Archive
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.