Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Apache Spark Open Source Projects

Open source projects categorized as Apache Spark
Categories > Data Processing > Apache Spark
Spark ⭐ 35,945
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 882total releases 46most recent commit 14 hours ago
Mlflow ⭐ 14,550
Open source platform for the machine learning lifecycle
dependent packages 252total releases 62most recent commit 13 hours agopypi mlflow} Downloads
Sparkinternals ⭐ 4,665
Notes talking about the design and implementation of Apache Spark
most recent commit 2 years ago
Synapseml ⭐ 4,295
Simple and Distributed Machine Learning
dependent packages 1total releases 5most recent commit 2 days agopypi synapseml} Downloads
Hudi ⭐ 4,261
Upserts, Deletes And Incremental Processing on Big Data.
dependent packages 26total releases 13most recent commit 14 hours ago
Bigdl ⭐ 4,223
Fast, distributed, secure AI for Big Data
dependent packages 10total releases 16most recent commit 16 hours ago
Lakefs ⭐ 3,434
lakeFS - Data version control for your data lake | Git for data
dependent packages 1total releases 62most recent commit 14 hours agopypi lakefs-client} Downloads
Coolplayspark ⭐ 3,397
酷玩 Spark: Spark 源代码解析、Spark 类库等
most recent commit a year ago
Spark Nlp ⭐ 3,278
State of the Art Natural Language Processing
dependent packages 2total releases 90most recent commit a day ago
Koalas ⭐ 3,228
Koalas: pandas API on Apache Spark
dependent packages 12total releases 47most recent commit 6 months agopypi koalas} Downloads
Spark Notebook ⭐ 3,138
Interactive and Reactive Data Science using Scala and Spark.
most recent commit 25 days ago
Deequ ⭐ 2,817
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
dependent packages 4total releases 31most recent commit 3 days ago
Analytics Zoo ⭐ 2,565
Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray
dependent packages 3total releases 302most recent commit 2 months agopypi bigdl-orca} Downloads
Spark On K8s Operator ⭐ 2,357
Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
dependent packages 21total releases 18most recent commit 2 days ago
Ballista ⭐ 2,244
Distributed compute platform implemented in Rust, and powered by Apache Arrow.
dependent packages 13total releases 4most recent commit 2 years ago
Transmogrifai ⭐ 2,099
TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning
dependent packages 3total releases 9most recent commit a year ago
Vega ⭐ 1,904
A new arguably faster implementation of Apache Spark from scratch in Rust
most recent commit 2 years ago
Spark ⭐ 1,878
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
dependent packages 16total releases 20most recent commit 4 months agonuget Microsoft.Spark} Downloads
Oryx ⭐ 1,796
Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
dependent packages 2total releases 14most recent commit 2 years ago
Feathr ⭐ 1,788
Feathr – A scalable, unified data and AI engineering platform for enterprise
most recent commit 4 days ago
Docker Spark ⭐ 1,783
Apache Spark docker image
most recent commit a year ago
Awesome Spark ⭐ 1,461
A curated list of awesome Apache Spark packages and resources.
most recent commit 2 months ago
Dr Elephant ⭐ 1,301
Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark
most recent commit 2 months ago
The 10 Most Depended On Apache Spark Open Source Projects
Spark ⭐ 35,945
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 882total releases 46latest release May 09, 2021most recent commit 14 hours ago
Mlflow ⭐ 14,550
Open source platform for the machine learning lifecycle
dependent packages 252total releases 62latest release June 29, 2022most recent commit 13 hours agopypi mlflow} Downloads
Sparklyr ⭐ 903
R interface for Apache Spark
dependent packages 26total releases 46latest release January 08, 2022most recent commit 3 months ago
Hudi ⭐ 4,261
Upserts, Deletes And Incremental Processing on Big Data.
dependent packages 26total releases 13latest release August 16, 2022most recent commit 14 hours ago
Spark On K8s Operator ⭐ 2,357
Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
dependent packages 21total releases 18latest release April 24, 2021most recent commit 2 days ago
Parquet Dotnet ⭐ 323
Fully managed Apache Parquet implementation
dependent packages 20total releases 217latest release December 21, 2022most recent commit 19 days agonuget Parquet.Net} Downloads
Spark ⭐ 1,878
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
dependent packages 16total releases 20latest release June 01, 2022most recent commit 4 months agonuget Microsoft.Spark} Downloads
Ballista ⭐ 2,244
Distributed compute platform implemented in Rust, and powered by Apache Arrow.
dependent packages 13total releases 4latest release May 10, 2020most recent commit 2 years ago
Koalas ⭐ 3,228
Koalas: pandas API on Apache Spark
dependent packages 12total releases 47latest release October 19, 2021most recent commit 6 months agopypi koalas} Downloads
Bigdl ⭐ 4,223
Fast, distributed, secure AI for Big Data
dependent packages 10total releases 16latest release April 19, 2021most recent commit 16 hours ago
The 10 Latest Releases In Apache Spark Open Source Projects
Parquet Dotnet ⭐ 323
Fully managed Apache Parquet implementation
dependent packages 20total releases 217latest release December 21, 2022most recent commit 19 days agonuget Parquet.Net} Downloads
Hudi ⭐ 4,261
Upserts, Deletes And Incremental Processing on Big Data.
dependent packages 26total releases 13latest release August 16, 2022most recent commit 14 hours ago
Analytics Zoo ⭐ 2,565
Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray
dependent packages 3total releases 302latest release July 07, 2022most recent commit 2 months agopypi bigdl-orca} Downloads
Mlflow ⭐ 14,550
Open source platform for the machine learning lifecycle
dependent packages 252total releases 62latest release June 29, 2022most recent commit 13 hours agopypi mlflow} Downloads
Splink ⭐ 663
Fast, accurate and scalable probabilistic data linkage using your choice of SQL backend
dependent packages 1total releases 72latest release June 28, 2022most recent commit 17 hours agopypi splink} Downloads
Lakefs ⭐ 3,434
lakeFS - Data version control for your data lake | Git for data
dependent packages 1total releases 62latest release June 15, 2022most recent commit 14 hours agopypi lakefs-client} Downloads
Spark ⭐ 1,878
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
dependent packages 16total releases 20latest release June 01, 2022most recent commit 4 months agonuget Microsoft.Spark} Downloads
Spark Rapids ⭐ 543
Spark RAPIDS plugin - accelerate Apache Spark with GPUs
total releases 10latest release April 14, 2022most recent commit 18 hours ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Classification
Data Science
Scraper
Transaction
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2023 Awesome Open Source.  All rights reserved.