Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Spark Open Source Projects

Open source projects categorized as Python Spark
Spark ⭐ 37,661
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 939total releases 46most recent commit 3 months ago
Data Science Ipython Notebooks ⭐ 25,668
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
most recent commit 7 months ago
Redash ⭐ 24,479
Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
dependent packages 3total releases 2most recent commit 3 months agonpm @redash/viz} Downloads
Deeplearning4j ⭐ 13,397
Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.
dependent packages 119total releases 54most recent commit a month ago
Ds Cheatsheets ⭐ 11,535
List of Data Science Cheatsheets to rule the world
most recent commit a year ago
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585most recent commit 3 months agopypi dagster} Downloads
It_book ⭐ 8,543
本项目收藏这些年来看过或者听过的一些不错的常用的上千本书籍，没准你想找的书就在这里呢，包含了互联网行
most recent commit 3 years ago
H2o 3 ⭐ 6,618
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
dependent packages 33total releases 49most recent commit 3 months ago
Mage Ai ⭐ 6,324
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
total releases 314most recent commit 3 months ago
Dev Setup ⭐ 5,802
macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
most recent commit 2 years ago
Technical Books ⭐ 5,519
😆 国内外互联网技术大牛们都写了哪些书籍：计算机基础、网络、前端、后端、数据库、架构、大数据、深度学习.
most recent commit 3 months ago
Bigdl ⭐ 4,728
Accelerate LLM with low-bit (FP4 / INT4 / FP8 / INT8) optimizations using bigdl-llm
dependent packages 10total releases 16most recent commit 3 months ago
Sqlglot ⭐ 4,652
Python SQL Parser and Transpiler
dependent packages 65total releases 461most recent commit 3 months agopypi sqlglot} Downloads
Tensorflowonspark ⭐ 3,851
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.
total releases 32most recent commit 10 months ago
Ibis ⭐ 3,404
The flexibility of Python with the scale and performance of modern SQL.
dependent packages 29total releases 68most recent commit 3 months agopypi ibis-framework} Downloads
Koalas ⭐ 3,291
Koalas: pandas API on Apache Spark
dependent packages 16total releases 47most recent commit 7 months agopypi koalas} Downloads
Dpark ⭐ 2,637
Python clone of Spark, a MapReduce alike framework in Python
total releases 19most recent commit 3 years ago
Analytics Zoo ⭐ 2,592
Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray
dependent packages 3total releases 1most recent commit 5 months agopypi bigdl-orca-spark321} Downloads
Lakesoul ⭐ 2,248
LakeSoul is an end-to-end, realtime and cloud native Lakehouse framework with fast data ingestion, concurrent update and incremental data analytics on cloud storages for both BI and AI applications.
dependent packages 1total releases 7most recent commit 3 months ago
Spark Deep Learning ⭐ 1,915
Deep Learning Pipelines for Apache Spark
most recent commit 2 years ago
Benchm Ml ⭐ 1,839
A minimal benchmark for scalability, speed and accuracy of commonly used open source implementations (R packages, Python scikit-learn, H2O, xgboost, Spark MLlib etc.) of the top machine learning algorithms for binary classification (random forests, gradient boosted trees, deep neural networks etc.).
most recent commit 2 years ago
Fugue ⭐ 1,821
A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.
dependent packages 23total releases 125most recent commit 3 months agopypi fugue} Downloads
.github ⭐ 1,722
ApacheCN 开源组织：公告、介绍、成员、活动、交流方式
most recent commit 4 months ago
The 10 Most Depended On Python Spark Open Source Projects
Spark ⭐ 37,661
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 939total releases 46latest release May 09, 2021most recent commit 3 months ago
Cloudpickle ⭐ 1,514
Extended pickling support for Python objects
dependent packages 897total releases 39latest release October 16, 2023most recent commit 4 months agopypi cloudpickle} Downloads
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Sqlglot ⭐ 4,652
Python SQL Parser and Transpiler
dependent packages 65total releases 461latest release December 07, 2023most recent commit 3 months agopypi sqlglot} Downloads
Findspark ⭐ 428
dependent packages 63total releases 14latest release February 11, 2022most recent commit 2 years agopypi findspark} Downloads
Cdap ⭐ 735
An open source framework for building data analytic applications.
dependent packages 56total releases 23latest release September 01, 2023most recent commit 3 months ago
Flytekit ⭐ 175
Extensible Python SDK for developing Flyte tasks and workflows. Simple to get started and learn and highly extensible.
dependent packages 55total releases 350latest release December 06, 2023most recent commit 3 months agopypi flytekit} Downloads
Ibis ⭐ 3,404
The flexibility of Python with the scale and performance of modern SQL.
dependent packages 29total releases 68latest release December 10, 2023most recent commit 3 months agopypi ibis-framework} Downloads
The 10 Latest Releases In Python Spark Open Source Projects
Ibis ⭐ 3,404
The flexibility of Python with the scale and performance of modern SQL.
dependent packages 29total releases 68latest release December 10, 2023most recent commit 3 months agopypi ibis-framework} Downloads
Sqlglot ⭐ 4,652
Python SQL Parser and Transpiler
dependent packages 65total releases 461latest release December 07, 2023most recent commit 3 months agopypi sqlglot} Downloads
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Flytekit ⭐ 175
Extensible Python SDK for developing Flyte tasks and workflows. Simple to get started and learn and highly extensible.
dependent packages 55total releases 350latest release December 06, 2023most recent commit 3 months agopypi flytekit} Downloads
Raydp ⭐ 265
RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.
dependent packages 1total releases 49latest release December 05, 2023most recent commit 4 months agopypi raydp} Downloads
E2eaiok ⭐ 30
Intel® End-to-End AI Optimization Kit
total releases 154latest release December 02, 2023most recent commit 3 months ago
Sageworks ⭐ 36
SageWorks: An easy to use Python API for creating and deploying SageMaker Models
total releases 60latest release November 29, 2023most recent commit 3 months ago
Popular Programming Languages Categories
Python
Javascript
Java
Typescript
C Plus Plus
Php
Golang
C
C Sharp
Shell
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.