Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Big Data Open Source Projects

Open source projects categorized as Big Data
Categories > Data Processing > Big Data
Awesome Scalability ⭐ 48,317
The Patterns of Scalable, Reliable, and Performant Large-Scale Systems
most recent commit 16 days ago
Spark ⭐ 36,830
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 903total releases 46most recent commit 12 hours ago
Clickhouse ⭐ 30,982
ClickHouse® is a free analytics DBMS for big data
total releases 699most recent commit 13 hours ago
Data Science Ipython Notebooks ⭐ 25,242
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
most recent commit 3 months ago
Flink ⭐ 22,017
Apache Flink
dependent packages 38total releases 11most recent commit 13 hours ago
Tdengine ⭐ 21,774
TDengine is an open source, high-performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, Industrial IoT and DevOps.
dependent packages 2total releases 12most recent commit 19 hours agonpm td2.0-connector} Downloads
Shardingsphere ⭐ 18,817
Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
dependent packages 7total releases 7most recent commit 13 hours ago
Gun ⭐ 17,345
An open source cybersecurity protocol for syncing decentralized graph data.
dependent packages 153total releases 236most recent commit a month agonpm gun} Downloads
Presto ⭐ 15,099
The official home of the Presto distributed SQL query engine for big data
dependent packages 11total releases 286most recent commit 16 hours ago
Bigdata Notes ⭐ 14,410
大数据入门指南 ⭐️
most recent commit 15 days ago
Questdb ⭐ 12,559
An open source time-series database for fast ingest and SQL queries
dependent packages 2total releases 57most recent commit 13 hours ago
Predictionio ⭐ 12,553
PredictionIO, a machine learning server for developers and ML engineers.
dependent packages 8total releases 5most recent commit 3 years ago
Awesome Bigdata ⭐ 12,343
A curated list of awesome big data frameworks, ressources and other awesomeness.
most recent commit 2 months ago
Cookbook ⭐ 11,769
The Data Engineering Cookbook
most recent commit 6 months ago
Cmak ⭐ 11,441
CMAK is a tool for managing Apache Kafka clusters
most recent commit 2 months ago
Nebula ⭐ 9,486
A distributed, fast open-source graph database featuring horizontal scalability and high availability
dependent packages 1total releases 1most recent commit 4 days ago
Juicefs ⭐ 8,668
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
dependent packages 1total releases 129most recent commit 2 days ago
Trino ⭐ 8,568
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
dependent packages 18total releases 67most recent commit 13 hours ago
God Of Bigdata ⭐ 8,483
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive.
most recent commit 2 months ago
Cython ⭐ 8,402
The most widely used Python to C compiler
dependent packages 2,234total releases 129most recent commit 17 hours agopypi Cython} Downloads
Vaex ⭐ 7,985
Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
dependent packages 26total releases 69most recent commit a month agopypi vaex} Downloads
Catboost ⭐ 7,368
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.
dependent packages 6total releases 60most recent commit 15 hours ago
Beam ⭐ 7,157
Apache Beam is a unified programming model for Batch and Streaming data processing.
dependent packages 13total releases 557most recent commit 13 hours ago
The 10 Most Depended On Big Data Open Source Projects
Fastjson2 ⭐ 3,033
🚄 FASTJSON2 is a Java JSON library with excellent performance.
dependent packages 2,744total releases 328latest release July 29, 2023most recent commit 2 days ago
Cython ⭐ 8,402
The most widely used Python to C compiler
dependent packages 2,234total releases 129latest release July 17, 2023most recent commit 17 hours agopypi Cython} Downloads
Avro ⭐ 2,581
Apache Avro is a data serialization system.
dependent packages 1,320total releases 149latest release June 26, 2023most recent commit 2 days ago
Spark ⭐ 36,830
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 903total releases 46latest release May 09, 2021most recent commit 12 hours ago
Parquet Mr ⭐ 2,162
Apache Parquet
dependent packages 170total releases 15latest release May 12, 2023most recent commit 14 hours ago
Gun ⭐ 17,345
An open source cybersecurity protocol for syncing decentralized graph data.
dependent packages 153total releases 236latest release December 23, 2022most recent commit a month agonpm gun} Downloads
Calcite ⭐ 4,039
Apache Calcite
dependent packages 114total releases 1,713latest release July 21, 2023most recent commit 13 hours ago
Vue Virtual Scroll List ⭐ 4,049
⚡️A vue component support big amount data list with high render performance and efficient.
dependent packages 108total releases 83latest release May 29, 2023most recent commit 3 months agonpm vue-virtual-scroll-list} Downloads
Vespa ⭐ 4,743
The open big data serving engine. https://vespa.ai
dependent packages 58total releases 682latest release August 02, 2023most recent commit 13 hours ago
The 10 Latest Releases In Big Data Open Source Projects
Openmetadata ⭐ 2,870
Open Standard for Metadata. A Single place to Discover, Collaborate and Get your data right.
dependent packages 2total releases 20latest release August 07, 2023most recent commit 13 hours ago
Pachyderm ⭐ 5,979
Data-Centric Pipelines and Data Versioning
dependent packages 1total releases 504latest release August 04, 2023most recent commit 19 hours ago
Vespa ⭐ 4,743
The open big data serving engine. https://vespa.ai
dependent packages 58total releases 682latest release August 02, 2023most recent commit 13 hours ago
Fastjson2 ⭐ 3,033
🚄 FASTJSON2 is a Java JSON library with excellent performance.
dependent packages 2,744total releases 328latest release July 29, 2023most recent commit 2 days ago
Arrow Datafusion ⭐ 4,044
Apache Arrow DataFusion SQL Query Engine
dependent packages 48total releases 78latest release July 25, 2023most recent commit 12 hours agocargo datafusion} Downloads
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Classification
Data Science
Scraper
Transaction
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2023 Awesome Open Source.  All rights reserved.