Awesome Open Source

Programming Languages

The Top 23 Spark Open Source Projects

Open source projects categorized as Spark

Categories > Data Processing > Spark

Edit Category

Aws Glue Samples ⭐ 1,334

AWS Glue code samples

most recent commit 6 months ago

Awesome Opensource Data Engineering ⭐ 1,331

An Awesome List of Open-Source Data Engineering Projects

most recent commit 4 months ago

Dr Elephant ⭐ 1,301

Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark

most recent commit a year ago

Ecosystem ⭐ 1,289

Integration of TensorFlow with other open-source frameworks

dependent packages 13total releases 13most recent commit 2 years ago

Caffeonspark ⭐ 1,272

Distributed deep learning on Hadoop and Spark clusters.

most recent commit 4 years ago

Sparkmagic ⭐ 1,272

Jupyter magics and kernels for working with remote Spark clusters

dependent packages 6total releases 54most recent commit 3 months ago

pypi autovizwidget} Downloads

Bigdata Growth ⭐ 1,256

大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。

most recent commit 7 days ago

Taier ⭐ 1,220

Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display

total releases 2most recent commit 3 months ago

Spark Doc Zh ⭐ 1,186

Apache Spark 官方文档中文版

dependent packages 1total releases 1most recent commit 9 months ago

npm spark-doc-zh} Downloads

Vue Trend ⭐ 1,178

🌈 Simple, elegant spark lines for Vue.js

dependent packages 11total releases 9most recent commit a year ago

npm vuetrend} Downloads

Killrweather ⭐ 1,174

KillrWeather is a reference application (work in progress) showing how to easily integrate streaming and batch data processing with Apache Spark Streaming, Apache Cassandra, Apache Kafka and Akka for fast, streaming computations on time series data in asynchronous event-driven environments.

most recent commit 7 years ago

Dockerfiles ⭐ 1,171

50+ DockerHub public images for Docker & Kubernetes - DevOps, CI/CD, GitHub Actions, CircleCI, Jenkins, TeamCity, Alpine, CentOS, Debian, Fedora, Ubuntu, Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak

most recent commit 5 months ago

Spark ⭐ 1,141

A simple Android sparkline chart view.

most recent commit 3 years ago

Bigflow ⭐ 1,122

Baidu Bigflow is an interface that allows for writing distributed computing programs and provides lots of simple, flexible, powerful APIs. Using Bigflow, you can easily handle data of any scale. Bigflow processes 4P+ data inside Baidu and runs about 10k jobs every day.

most recent commit a year ago

Machine Learning ⭐ 1,046

机器学习原理

most recent commit a year ago

Spark Sklearn ⭐ 1,039

(Deprecated) Scikit-learn integration package for Apache Spark

total releases 8most recent commit 4 years ago

Snappydata ⭐ 1,037

Project SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in one cluster

dependent packages 23total releases 17most recent commit a year ago

Pixiedust ⭐ 1,035

Python Helper library for Jupyter Notebooks

dependent packages 7total releases 92most recent commit 3 years ago

pypi pixiedust} Downloads

Kylo is a data lake management software platform and framework for enabling scalable enterprise-class data lakes on big data technologies such as Teradata, Apache Spark and/or Hadoop. Kylo is licensed under Apache 2.0. Contributed by Teradata Inc.

dependent packages 15total releases 10most recent commit a year ago

Pyspark Example Project ⭐ 1,034

Example project implementing best practices for PySpark ETL jobs and applications.

most recent commit a year ago

Utils4s ⭐ 1,033

scala、spark使用过程中，各种测试用例以及相关资料整理

most recent commit 5 years ago

Spark Csv ⭐ 1,009

CSV Data Source for Apache Spark 1.x

dependent packages 24total releases 13most recent commit 5 years ago

Spark Nlp Workshop ⭐ 977

Public runnable examples of using John Snow Labs' NLP for Apache Spark.

most recent commit 3 months ago

The 10 Most Depended On Spark Open Source Projects

Sparkling Water ⭐ 957

Sparkling Water provides H2O functionality inside Spark cluster

dependent packages 6total releases 195latest release October 26, 2023most recent commit 4 months ago

Deequ ⭐ 3,044

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

dependent packages 6total releases 37latest release November 09, 2023most recent commit 3 months ago

Magic Rdds ⭐ 20

Miscellaneous functionality for manipulating Apache Spark RDDs.

dependent packages 6total releases 28latest release December 29, 2018most recent commit 5 years ago

Codeigniter Installers ⭐ 28

Composer installers for CodeIgniter components, [sparks](http://getsparks.org/), and [modules](https://bitbucket.org/wiredesignz/codeig

dependent packages 6total releases 7latest release November 12, 2012most recent commit 11 years ago

packagist compwright/codeigniter-installers} Downloads

Ember Sparks ⭐ 19

✨ Ambitious UI components for your Ember app.

dependent packages 6total releases 3latest release January 26, 2017most recent commit 4 years ago

npm ember-sparks} Downloads

Python Spark ⭐ 44

An Earley-Algorithm Context-free grammar Parser Toolkit

dependent packages 6total releases 28latest release July 29, 2019most recent commit 3 months ago

pypi spark-parser} Downloads

Metrics Spark Reporter ⭐ 28

Dropwizard Metrics reporter for Apache Spark

dependent packages 6total releases 1latest release December 22, 2014most recent commit 9 years ago

Web Styleguide ⭐ 7

Coding style guide for how we develop for the web

dependent packages 6total releases 3latest release January 09, 2019most recent commit a year ago

npm @webex/eslint-config-base} Downloads

Stream computing platform for bigdata

dependent packages 6total releases 1latest release July 03, 2019most recent commit 2 years ago

Python PMML scoring library

dependent packages 6total releases 15latest release November 03, 2022most recent commit a year ago

pypi pypmml} Downloads

The 10 Latest Releases In Spark Open Source Projects

Azure Kusto Spark ⭐ 75

Apache Spark Connector for Azure Kusto

total releases 46latest release August 14, 2023most recent commit 3 months ago

Starters ⭐ 23

java python and punchlang samples to discover the punch platform FaaS

dependent packages 2total releases 6latest release August 09, 2023most recent commit 6 months ago

H2o 3 ⭐ 6,618

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

dependent packages 33total releases 49latest release August 09, 2023most recent commit 3 months ago

Tispark ⭐ 872

TiSpark is built for running Apache Spark on top of TiDB/TiKV

dependent packages 10total releases 55latest release August 08, 2023most recent commit 3 months ago

Dataspherestudio ⭐ 2,860

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

dependent packages 39total releases 7latest release August 07, 2023most recent commit 3 months ago

Spark Hats ⭐ 29

Nested array transformation helper extensions for Apache Spark

dependent packages 3total releases 5latest release August 03, 2023most recent commit 9 months ago

Qbeast Spark ⭐ 171

Qbeast-spark: DataSource enabling multi-dimensional indexing and efficient data sampling. Big Data, free from the unnecessary!

dependent packages 1total releases 9latest release August 03, 2023most recent commit 3 months ago

Linkis ⭐ 3,224

Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.

dependent packages 38total releases 3latest release July 29, 2023most recent commit 16 days ago

Cloudflow ⭐ 323

Cloudflow enables users to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes.

dependent packages 11total releases 609latest release July 27, 2023most recent commit 9 months ago

Kotlin Spark Api ⭐ 425

This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x

dependent packages 15total releases 6latest release July 25, 2023most recent commit 4 months ago

Popular Data Processing Categories

Jupyter Notebook

Categories

Application Programming Interfaces

Artificial Intelligence

Cloud Computing

Command Line Interface

Computer Science

Configuration Management

Content Management

Data Processing

Integrated Development Environments

Learning Resources

Lists Of Projects

Machine Learning

Operating Systems

Package Managers

Programming Languages

Runtime Environments

Software Architecture

Software Development

Software Performance

Software Quality

Text Processing

User Interface Components

Version Control

Web User Interface

Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source. All rights reserved.