Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Scala Apache Spark Open Source Projects

Open source projects categorized as Scala Apache Spark
Spark ⭐ 37,661
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 939total releases 46most recent commit 3 months ago
Synapseml ⭐ 4,967
Simple and Distributed Machine Learning
dependent packages 6total releases 12most recent commit 2 days ago
Hudi ⭐ 4,901
Upserts, Deletes And Incremental Processing on Big Data.
dependent packages 58total releases 21most recent commit 3 months ago
Bigdl ⭐ 4,728
Accelerate LLM with low-bit (FP4 / INT4 / FP8 / INT8) optimizations using bigdl-llm
dependent packages 10total releases 16most recent commit 3 months ago
Spark Nlp ⭐ 3,578
State of the Art Natural Language Processing
dependent packages 30total releases 134most recent commit 3 months agopypi spark-nlp} Downloads
Spark Notebook ⭐ 3,147
Interactive and Reactive Data Science using Scala and Spark.
most recent commit a year ago
Deequ ⭐ 3,044
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
dependent packages 6total releases 37most recent commit 3 months ago
Analytics Zoo ⭐ 2,592
Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray
dependent packages 3total releases 1most recent commit 4 months agopypi bigdl-orca-spark321} Downloads
Ballista ⭐ 2,244
Distributed compute platform implemented in Rust, and powered by Apache Arrow.
dependent packages 13total releases 4most recent commit 3 years ago
Transmogrifai ⭐ 2,099
TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning
dependent packages 3total releases 9most recent commit 2 years ago
Feathr ⭐ 1,886
Feathr – A scalable, unified data and AI engineering platform for enterprise
dependent packages 4total releases 48most recent commit 3 months ago
Docker Spark ⭐ 1,783
Apache Spark docker image
most recent commit 2 years ago
Awesome Spark ⭐ 1,461
A curated list of awesome Apache Spark packages and resources.
most recent commit a year ago
Livy ⭐ 911
Livy is an open source REST interface for interacting with Apache Spark from anywhere
dependent packages 5total releases 4most recent commit 4 years ago
Tispark ⭐ 872
TiSpark is built for running Apache Spark on top of TiDB/TiKV
dependent packages 10total releases 55most recent commit 3 months ago
Incubator Livy ⭐ 840
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
dependent packages 8total releases 6most recent commit 3 months ago
Extraction Framework ⭐ 802
The software used to extract structured data from Wikipedia
dependent packages 3total releases 3most recent commit 7 months ago
Kafka Storm Starter ⭐ 729
[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.
most recent commit 2 years ago
Spark Rapids ⭐ 619
Spark RAPIDS plugin - accelerate Apache Spark with GPUs
dependent packages 1total releases 19most recent commit 3 months ago
Sparkmeasure ⭐ 603
This is the development repository for sparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task and stage metrics data.
dependent packages 2total releases 9most recent commit 5 months ago
Sparklearning ⭐ 573
Learning Apache spark,including code and data .Most part can run local.
most recent commit 2 years ago
Learningsparkv2 ⭐ 570
This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
most recent commit 2 years ago
Spline ⭐ 553
Data Lineage Tracking And Visualization Solution
dependent packages 6total releases 14most recent commit 5 months ago
The 10 Most Depended On Scala Apache Spark Open Source Projects
Spark ⭐ 37,661
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 939total releases 46latest release May 09, 2021most recent commit 3 months ago
Hudi ⭐ 4,901
Upserts, Deletes And Incremental Processing on Big Data.
dependent packages 58total releases 21latest release November 11, 2023most recent commit 3 months ago
Spark Nlp ⭐ 3,578
State of the Art Natural Language Processing
dependent packages 30total releases 134latest release December 08, 2023most recent commit 3 months agopypi spark-nlp} Downloads
Ballista ⭐ 2,244
Distributed compute platform implemented in Rust, and powered by Apache Arrow.
dependent packages 13total releases 4latest release May 10, 2020most recent commit 3 years ago
Tispark ⭐ 872
TiSpark is built for running Apache Spark on top of TiDB/TiKV
dependent packages 10total releases 55latest release August 08, 2023most recent commit 3 months ago
Bigdl ⭐ 4,728
Accelerate LLM with low-bit (FP4 / INT4 / FP8 / INT8) optimizations using bigdl-llm
dependent packages 10total releases 16latest release April 19, 2021most recent commit 3 months ago
Incubator Livy ⭐ 840
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
dependent packages 8total releases 6latest release September 25, 2023most recent commit 3 months ago
Morpheus ⭐ 329
Morpheus brings the leading graph query language, Cypher, onto the leading distributed processing platform, Spark.
dependent packages 7total releases 33latest release June 03, 2019most recent commit 4 years ago
Deequ ⭐ 3,044
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
dependent packages 6total releases 37latest release November 09, 2023most recent commit 3 months ago
The 10 Latest Releases In Scala Apache Spark Open Source Projects
Spark Nlp ⭐ 3,578
State of the Art Natural Language Processing
dependent packages 30total releases 134latest release December 08, 2023most recent commit 3 months agopypi spark-nlp} Downloads
Synapseml ⭐ 4,967
Simple and Distributed Machine Learning
dependent packages 6total releases 12latest release November 27, 2023most recent commit 2 days ago
Hudi ⭐ 4,901
Upserts, Deletes And Incremental Processing on Big Data.
dependent packages 58total releases 21latest release November 11, 2023most recent commit 3 months ago
Deequ ⭐ 3,044
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
dependent packages 6total releases 37latest release November 09, 2023most recent commit 3 months ago
Pulsar Spark ⭐ 103
Spark Connector to read and write with Pulsar
dependent packages 2total releases 10latest release November 06, 2023most recent commit 5 months ago
Spark Rapids ⭐ 619
Spark RAPIDS plugin - accelerate Apache Spark with GPUs
dependent packages 1total releases 19latest release October 24, 2023most recent commit 3 months ago
Cobrix ⭐ 131
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
dependent packages 2total releases 43latest release October 16, 2023most recent commit 3 months ago
Spark Snowflake ⭐ 196
Snowflake Data Source for Apache Spark.
dependent packages 3total releases 65latest release October 13, 2023most recent commit 3 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.