Awesome Open Source

Programming Languages

The Top 23 Python Scala Spark Open Source Projects

Open source projects categorized as Python Scala Spark

Categories > Programming Languages > Python

Categories > Programming Languages > Scala

Categories > Data Processing > Spark

Spark ⭐ 37,661

Apache Spark - A unified analytics engine for large-scale data processing

dependent packages 939total releases 46most recent commit 3 months ago

Deeplearning4j ⭐ 13,397

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.

dependent packages 119total releases 54most recent commit a month ago

It_book ⭐ 8,543

本项目收藏这些年来看过或者听过的一些不错的常用的上千本书籍，没准你想找的书就在这里呢，包含了互联网行

most recent commit 3 years ago

Bigdl ⭐ 4,728

Accelerate LLM with low-bit (FP4 / INT4 / FP8 / INT8) optimizations using bigdl-llm

dependent packages 10total releases 16most recent commit 3 months ago

Tensorflowonspark ⭐ 3,851

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

total releases 32most recent commit 9 months ago

Analytics Zoo ⭐ 2,592

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray

dependent packages 3total releases 1most recent commit 4 months ago

pypi bigdl-orca-spark321} Downloads

Mleap ⭐ 1,479

MLeap: Deploy ML Pipelines to Production

dependent packages 12total releases 26most recent commit 5 months ago

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

dependent packages 17total releases 14most recent commit 3 months ago

Incubator Livy ⭐ 840

Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.

dependent packages 8total releases 6most recent commit 3 months ago

Pythondatascience Collections ⭐ 615

最全数据分析资料汇总（含python、爬虫、数据库、大数据、tableau、统计学等）

most recent commit 3 years ago

Sparkmeasure ⭐ 603

This is the development repository for sparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task and stage metrics data.

dependent packages 2total releases 9most recent commit 5 months ago

Eat_pyspark_in_10_days ⭐ 534

pyspark🍒🥭 is delicious，just eat it!😋😋

most recent commit 2 years ago

Spark Standalone Cluster On Docker ⭐ 311

Learn Apache Spark in Scala, Python (PySpark) and R (SparkR) by building your own cluster with a JupyterLab interface on Docker. ⚡

most recent commit a year ago

Sagemaker Spark ⭐ 285

A Spark library for Amazon SageMaker.

total releases 36most recent commit 7 months ago

Big Data Processing Framework - Unified Data API or SQL on Any Storage

most recent commit a year ago

Spark Extension ⭐ 152

A library that provides useful extensions to Apache Spark and PySpark.

total releases 65most recent commit 3 months ago

Sparknotebook ⭐ 142

An example of running Apache Spark using Scala in ipython notebook

most recent commit 9 years ago

A recommender system for discovering GitHub repos, built with Apache Spark

most recent commit 4 years ago

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

total releases 27most recent commit 10 months ago

Spark Website ⭐ 109

Apache Spark Website

most recent commit 3 months ago

Yelper_recommendation_system ⭐ 108

Yelper recommendation system

most recent commit 3 years ago

Spark_python_ml_examples ⭐ 81

Spark 2.0 Python Machine Learning examples

most recent commit 5 years ago

Jgit Spark Connector ⭐ 67

jgit-spark-connector is a library for running scalable data retrieval pipelines that process any number of Git repositories for source code analysis.

dependent packages 1total releases 40most recent commit 5 years ago

pypi sourced-engine} Downloads

The 10 Most Depended On Python Scala Spark Open Source Projects

Spark ⭐ 37,661

Apache Spark - A unified analytics engine for large-scale data processing

dependent packages 939total releases 46latest release May 09, 2021most recent commit 3 months ago

Deeplearning4j ⭐ 13,397

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.

dependent packages 119total releases 54latest release August 10, 2022most recent commit a month ago

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

dependent packages 17total releases 14latest release December 16, 2020most recent commit 3 months ago

Mleap ⭐ 1,479

MLeap: Deploy ML Pipelines to Production

dependent packages 12total releases 26latest release May 07, 2021most recent commit 5 months ago

Bigdl ⭐ 4,728

Accelerate LLM with low-bit (FP4 / INT4 / FP8 / INT8) optimizations using bigdl-llm

dependent packages 10total releases 16latest release April 19, 2021most recent commit 3 months ago

Incubator Livy ⭐ 840

Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.

dependent packages 8total releases 6latest release September 25, 2023most recent commit 3 months ago

Python PMML scoring library

dependent packages 6total releases 15latest release November 03, 2022most recent commit a year ago

pypi pypmml} Downloads

Analytics Zoo ⭐ 2,592

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray

dependent packages 3total releases 1latest release July 29, 2022most recent commit 4 months ago

pypi bigdl-orca-spark321} Downloads

Sparkmeasure ⭐ 603

This is the development repository for sparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task and stage metrics data.

dependent packages 2total releases 9latest release August 10, 2022most recent commit 5 months ago

Jgit Spark Connector ⭐ 67

jgit-spark-connector is a library for running scalable data retrieval pipelines that process any number of Git repositories for source code analysis.

dependent packages 1total releases 40latest release October 10, 2018most recent commit 5 years ago

pypi sourced-engine} Downloads

The 10 Latest Releases In Python Scala Spark Open Source Projects

Feature Store Api ⭐ 51

Python - Java/Scala API for the Hopsworks feature store

dependent packages 1total releases 136latest release November 28, 2023most recent commit 3 months ago

pypi hsfs} Downloads

Spark Extension ⭐ 152

A library that provides useful extensions to Apache Spark and PySpark.

total releases 65latest release September 28, 2023most recent commit 3 months ago

Incubator Livy ⭐ 840

Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.

dependent packages 8total releases 6latest release September 25, 2023most recent commit 3 months ago

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

total releases 27latest release November 17, 2022most recent commit 10 months ago

Python PMML scoring library

dependent packages 6total releases 15latest release November 03, 2022most recent commit a year ago

pypi pypmml} Downloads

Sagemaker Spark ⭐ 285

A Spark library for Amazon SageMaker.

total releases 36latest release August 26, 2022most recent commit 7 months ago

Deeplearning4j ⭐ 13,397

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.

dependent packages 119total releases 54latest release August 10, 2022most recent commit a month ago

Sparkmeasure ⭐ 603

This is the development repository for sparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task and stage metrics data.

dependent packages 2total releases 9latest release August 10, 2022most recent commit 5 months ago

Analytics Zoo ⭐ 2,592

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray

dependent packages 3total releases 1latest release July 29, 2022most recent commit 4 months ago

pypi bigdl-orca-spark321} Downloads

Dead Salmon Brain ⭐ 11

Apache Spark based framework for analysis A/B experiments

total releases 5latest release May 15, 2022most recent commit 2 years ago

Popular Programming Languages Categories

Categories

Application Programming Interfaces

Artificial Intelligence

Cloud Computing

Command Line Interface

Computer Science

Configuration Management

Content Management

Data Processing

Integrated Development Environments

Learning Resources

Lists Of Projects

Machine Learning

Operating Systems

Package Managers

Programming Languages

Runtime Environments

Software Architecture

Software Development

Software Performance

Software Quality

Text Processing

User Interface Components

Version Control

Web User Interface

Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source. All rights reserved.