Awesome Open Source

Programming Languages

The Top 23 Spark Hadoop Open Source Projects

Open source projects categorized as Spark Hadoop

Categories > Data Processing > Hadoop

Categories > Data Processing > Spark

Spark ⭐ 37,661

Apache Spark - A unified analytics engine for large-scale data processing

dependent packages 939total releases 46most recent commit 3 months ago

Data Science Ipython Notebooks ⭐ 25,668

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

most recent commit 6 months ago

Bigdata Notes ⭐ 14,872

大数据入门指南 ⭐

most recent commit 4 months ago

Deeplearning4j ⭐ 13,397

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.

dependent packages 119total releases 54most recent commit 25 days ago

Cookbook ⭐ 12,557

The Data Engineering Cookbook

most recent commit 3 months ago

Doris ⭐ 11,243

Apache Doris is an easy-to-use, high performance and unified analytics database.

total releases 8most recent commit 11 days ago

It_book ⭐ 8,543

本项目收藏这些年来看过或者听过的一些不错的常用的上千本书籍，没准你想找的书就在这里呢，包含了互联网行

most recent commit 2 years ago

God Of Bigdata ⭐ 8,483

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive.

most recent commit 8 months ago

H2o 3 ⭐ 6,618

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

dependent packages 33total releases 49most recent commit 3 months ago

Alluxio ⭐ 6,612

Alluxio, data orchestration for analytics and machine learning in the cloud

dependent packages 53total releases 73most recent commit a month ago

Bigdl ⭐ 4,728

Accelerate LLM with low-bit (FP4 / INT4 / FP8 / INT8) optimizations using bigdl-llm

dependent packages 10total releases 16most recent commit 3 months ago

Tensorflowonspark ⭐ 3,851

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

total releases 32most recent commit 9 months ago

The flexibility of Python with the scale and performance of modern SQL.

dependent packages 29total releases 68most recent commit 3 months ago

pypi ibis-framework} Downloads

Dataspherestudio ⭐ 2,860

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

dependent packages 39total releases 7most recent commit 3 months ago

Bigdataguide ⭐ 2,355

大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料

most recent commit 4 months ago

Szt Bigdata ⭐ 2,055

深圳地铁大数据客流分析系统🚇🚄🌟

most recent commit 3 months ago

Elasticsearch Hadoop ⭐ 1,914

🐘 Elasticsearch real-time search and analytics natively integrated with Hadoop

dependent packages 15total releases 295most recent commit 3 months ago

Kyuubi ⭐ 1,849

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

dependent packages 24total releases 11most recent commit 3 months ago

Docker Spark ⭐ 1,783

Apache Spark docker image

most recent commit 2 years ago

Gaffer ⭐ 1,724

A large-scale entity and relation database supporting aggregation of properties

dependent packages 31total releases 101most recent commit 3 months ago

Movie_recommend ⭐ 1,441

基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统

most recent commit 5 years ago

Carbondata ⭐ 1,401

High performance data store solution

total releases 1most recent commit 4 months ago

Bigdata Interview ⭐ 1,397

🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop

most recent commit 3 years ago

The 10 Most Depended On Spark Hadoop Open Source Projects

Spark ⭐ 37,661

Apache Spark - A unified analytics engine for large-scale data processing

dependent packages 939total releases 46latest release May 09, 2021most recent commit 3 months ago

Deeplearning4j ⭐ 13,397

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.

dependent packages 119total releases 54latest release August 10, 2022most recent commit 25 days ago

An open source framework for building data analytic applications.

dependent packages 56total releases 23latest release September 01, 2023most recent commit 3 months ago

Alluxio ⭐ 6,612

Alluxio, data orchestration for analytics and machine learning in the cloud

dependent packages 53total releases 73latest release November 29, 2023most recent commit a month ago

Dataspherestudio ⭐ 2,860

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

dependent packages 39total releases 7latest release August 07, 2023most recent commit 3 months ago

Logisland ⭐ 106

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

dependent packages 34total releases 12latest release January 24, 2023most recent commit a year ago

H2o 3 ⭐ 6,618

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

dependent packages 33total releases 49latest release August 09, 2023most recent commit 3 months ago

Gaffer ⭐ 1,724

A large-scale entity and relation database supporting aggregation of properties

dependent packages 31total releases 101latest release November 14, 2023most recent commit 3 months ago

The flexibility of Python with the scale and performance of modern SQL.

dependent packages 29total releases 68latest release December 10, 2023most recent commit 3 months ago

pypi ibis-framework} Downloads

Kyuubi ⭐ 1,849

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

dependent packages 24total releases 11latest release November 03, 2023most recent commit 3 months ago

The 10 Latest Releases In Spark Hadoop Open Source Projects

The flexibility of Python with the scale and performance of modern SQL.

dependent packages 29total releases 68latest release December 10, 2023most recent commit 3 months ago

pypi ibis-framework} Downloads

Alluxio ⭐ 6,612

Alluxio, data orchestration for analytics and machine learning in the cloud

dependent packages 53total releases 73latest release November 29, 2023most recent commit a month ago

Flintrock ⭐ 627

A command-line tool for launching Apache Spark clusters.

total releases 14latest release November 27, 2023most recent commit 5 months ago

Gaffer ⭐ 1,724

A large-scale entity and relation database supporting aggregation of properties

dependent packages 31total releases 101latest release November 14, 2023most recent commit 3 months ago

Elasticsearch Hadoop ⭐ 1,914

:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop

dependent packages 15total releases 295latest release November 11, 2023most recent commit 3 months ago

Kyuubi ⭐ 1,849

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

dependent packages 24total releases 11latest release November 03, 2023most recent commit 3 months ago

Smart Data Lake ⭐ 87

Smart Automation Tool for building modern Data Lakes and Data Pipelines

dependent packages 8total releases 26latest release October 25, 2023most recent commit 3 months ago

Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.

dependent packages 24total releases 65latest release October 16, 2023most recent commit 4 months ago

This repository is ReadOnly now. please go to https://github.com/apache/incubator-iotdb

dependent packages 14total releases 26latest release October 11, 2023most recent commit 5 years ago

Doris ⭐ 11,243

Apache Doris is an easy-to-use, high performance and unified analytics database.

total releases 8latest release September 27, 2023most recent commit 11 days ago

Popular Data Processing Categories

Jupyter Notebook

Categories

Application Programming Interfaces

Artificial Intelligence

Cloud Computing

Command Line Interface

Computer Science

Configuration Management

Content Management

Data Processing

Integrated Development Environments

Learning Resources

Lists Of Projects

Machine Learning

Operating Systems

Package Managers

Programming Languages

Runtime Environments

Software Architecture

Software Development

Software Performance

Software Quality

Text Processing

User Interface Components

Version Control

Web User Interface

Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source. All rights reserved.