Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Big Data Apache Spark Open Source Projects

Open source projects categorized as Big Data Apache Spark
Categories > Data Processing > Apache Spark
Categories > Data Processing > Big Data
Spark ⭐ 37,661
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 939total releases 46most recent commit 3 months ago
Data Engineer Handbook ⭐ 5,650
This is a repo with links to everything you'd ever want to learn about data engineering
most recent commit 3 months ago
Synapseml ⭐ 4,960
Simple and Distributed Machine Learning
dependent packages 6total releases 12most recent commit 7 days ago
Hudi ⭐ 4,901
Upserts, Deletes And Incremental Processing on Big Data.
dependent packages 58total releases 21most recent commit 3 months ago
Koalas ⭐ 3,291
Koalas: pandas API on Apache Spark
dependent packages 16total releases 47most recent commit 7 months agopypi koalas} Downloads
Spark ⭐ 1,963
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
dependent packages 18total releases 20most recent commit 5 months agonuget Microsoft.Spark} Downloads
Spark Doc Zh ⭐ 1,186
Apache Spark 官方文档中文版
dependent packages 1total releases 1most recent commit 9 months agonpm spark-doc-zh} Downloads
Mobius ⭐ 937
C# and F# language binding and extensions to Apache Spark
total releases 22most recent commit 3 months ago
Tispark ⭐ 872
TiSpark is built for running Apache Spark on top of TiDB/TiKV
dependent packages 10total releases 55most recent commit 3 months ago
Incubator Livy ⭐ 840
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
dependent packages 8total releases 6most recent commit 3 months ago
Spark Rapids ⭐ 619
Spark RAPIDS plugin - accelerate Apache Spark with GPUs
dependent packages 1total releases 19most recent commit 3 months ago
Spline ⭐ 553
Data Lineage Tracking And Visualization Solution
dependent packages 6total releases 14most recent commit 4 months ago
Morpheus ⭐ 329
Morpheus brings the leading graph query language, Cypher, onto the leading distributed processing platform, Spark.
dependent packages 7total releases 33most recent commit 4 years ago
Parquet Dotnet ⭐ 319
🏐 Apache Parquet for modern .NET
total releases 4most recent commit 2 years ago
Mist ⭐ 303
Serverless proxy for Spark cluster
dependent packages 4total releases 3most recent commit 3 years ago
Data Accelerator ⭐ 293
Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.
dependent packages 7total releases 26most recent commit 9 days agonpm datax-common} Downloads
Succinct ⭐ 239
Enabling queries on compressed data.
most recent commit 4 years ago
Azure Event Hubs Spark ⭐ 225
Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs
total releases 68most recent commit a year ago
Sparkrdma ⭐ 191
RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark
most recent commit 5 years ago
Spark.jl ⭐ 180
Julia binding for Apache Spark
most recent commit 2 years ago
Bigdata Playground ⭐ 154
A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
most recent commit 5 years ago
Spark On Lambda ⭐ 144
Apache Spark on AWS Lambda
most recent commit a year ago
Pyspark Cheatsheet ⭐ 140
PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster
most recent commit 2 years ago
The 10 Most Depended On Big Data Apache Spark Open Source Projects
Spark ⭐ 37,661
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 939total releases 46latest release May 09, 2021most recent commit 3 months ago
Hudi ⭐ 4,901
Upserts, Deletes And Incremental Processing on Big Data.
dependent packages 58total releases 21latest release November 11, 2023most recent commit 3 months ago
Spark ⭐ 1,963
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
dependent packages 18total releases 20latest release June 01, 2022most recent commit 5 months agonuget Microsoft.Spark} Downloads
Koalas ⭐ 3,291
Koalas: pandas API on Apache Spark
dependent packages 16total releases 47latest release October 19, 2021most recent commit 7 months agopypi koalas} Downloads
Tispark ⭐ 872
TiSpark is built for running Apache Spark on top of TiDB/TiKV
dependent packages 10total releases 55latest release August 08, 2023most recent commit 3 months ago
Incubator Livy ⭐ 840
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
dependent packages 8total releases 6latest release September 25, 2023most recent commit 3 months ago
Morpheus ⭐ 329
Morpheus brings the leading graph query language, Cypher, onto the leading distributed processing platform, Spark.
dependent packages 7total releases 33latest release June 03, 2019most recent commit 4 years ago
Spline ⭐ 553
Data Lineage Tracking And Visualization Solution
dependent packages 6total releases 14latest release July 25, 2019most recent commit 4 months ago
The 10 Latest Releases In Big Data Apache Spark Open Source Projects
Synapseml ⭐ 4,960
Simple and Distributed Machine Learning
dependent packages 6total releases 12latest release November 27, 2023most recent commit 7 days ago
Hudi ⭐ 4,901
Upserts, Deletes And Incremental Processing on Big Data.
dependent packages 58total releases 21latest release November 11, 2023most recent commit 3 months ago
Spark Rapids ⭐ 619
Spark RAPIDS plugin - accelerate Apache Spark with GPUs
dependent packages 1total releases 19latest release October 24, 2023most recent commit 3 months ago
Incubator Livy ⭐ 840
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
dependent packages 8total releases 6latest release September 25, 2023most recent commit 3 months ago
Tispark ⭐ 872
TiSpark is built for running Apache Spark on top of TiDB/TiKV
dependent packages 10total releases 55latest release August 08, 2023most recent commit 3 months ago
Aut ⭐ 128
The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
total releases 27latest release November 17, 2022most recent commit 9 months ago
Spark ⭐ 1,963
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
dependent packages 18total releases 20latest release June 01, 2022most recent commit 5 months agonuget Microsoft.Spark} Downloads
Koalas ⭐ 3,291
Koalas: pandas API on Apache Spark
dependent packages 16total releases 47latest release October 19, 2021most recent commit 7 months agopypi koalas} Downloads
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.