Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Data Science Spark Open Source Projects

Open source projects categorized as Python Data Science Spark
Data Science Ipython Notebooks ⭐ 25,668
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
most recent commit 7 months ago
Ds Cheatsheets ⭐ 11,535
List of Data Science Cheatsheets to rule the world
most recent commit a year ago
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585most recent commit 3 months agopypi dagster} Downloads
H2o 3 ⭐ 6,618
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
dependent packages 33total releases 49most recent commit 3 months ago
Mage Ai ⭐ 6,324
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
total releases 314most recent commit 3 months ago
Koalas ⭐ 3,291
Koalas: pandas API on Apache Spark
dependent packages 16total releases 47most recent commit 7 months agopypi koalas} Downloads
Benchm Ml ⭐ 1,839
A minimal benchmark for scalability, speed and accuracy of commonly used open source implementations (R packages, Python scikit-learn, H2O, xgboost, Spark MLlib etc.) of the top machine learning algorithms for binary classification (random forests, gradient boosted trees, deep neural networks etc.).
most recent commit 2 years ago
Spark Py Notebooks ⭐ 1,515
Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
most recent commit a year ago
Pixiedust ⭐ 1,035
Python Helper library for Jupyter Notebooks
dependent packages 7total releases 92most recent commit 3 years agopypi pixiedust} Downloads
Pyspark Example Project ⭐ 1,034
Example project implementing best practices for PySpark ETL jobs and applications.
most recent commit a year ago
Splink ⭐ 939
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
dependent packages 2total releases 119most recent commit 3 months agopypi splink} Downloads
Around Dataengineering ⭐ 926
A Data Engineering & Machine Learning Knowledge Hub
most recent commit 2 years ago
Dist Keras ⭐ 611
Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.
total releases 2most recent commit 6 years ago
Data Science Learning Resources ⭐ 499
A collection of data science and machine learning resources that I've found helpful (I only post what I've read!)
most recent commit 10 months ago
Complete Life Cycle Of A Data Science Project ⭐ 499
Complete-Life-Cycle-of-a-Data-Science-Project
most recent commit 4 months ago
Popmon ⭐ 461
Monitor the stability of a Pandas or Spark dataframe ⚙︎
dependent packages 2total releases 36most recent commit 9 months agopypi popmon} Downloads
Agile_data_code_2 ⭐ 435
Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
most recent commit a year ago
Datacompy ⭐ 339
Pandas and Spark DataFrame comparison for humans and more!
dependent packages 10total releases 20most recent commit 3 months agopypi datacompy} Downloads
Sk Dist ⭐ 283
Distributed scikit-learn meta-estimators in PySpark
total releases 12most recent commit a year ago
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35most recent commit 5 months agopypi butterfree} Downloads
Openuba ⭐ 264
A robust, and flexible open source User & Entity Behavior Analytics (UEBA) framework used for Security Analytics. Developed with luv by Data Scientists & Security Analysts from the Cyber Security Industry. [PRE-ALPHA]
most recent commit a year ago
Pysparkling ⭐ 253
A pure Python implementation of Apache Spark's RDD and DStream interfaces.
dependent packages 1total releases 69most recent commit a year agopypi pysparkling} Downloads
Data_science_blogs ⭐ 232
A repository to keep track of all the code that I end up writing for my blog posts.
most recent commit a year ago
The 10 Most Depended On Python Data Science Spark Open Source Projects
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Flytekit ⭐ 175
Extensible Python SDK for developing Flyte tasks and workflows. Simple to get started and learn and highly extensible.
dependent packages 55total releases 350latest release December 06, 2023most recent commit 3 months agopypi flytekit} Downloads
Koalas ⭐ 3,291
Koalas: pandas API on Apache Spark
dependent packages 16total releases 47latest release October 19, 2021most recent commit 7 months agopypi koalas} Downloads
Datacompy ⭐ 339
Pandas and Spark DataFrame comparison for humans and more!
dependent packages 10total releases 20latest release November 15, 2023most recent commit 3 months agopypi datacompy} Downloads
Visions ⭐ 166
Type System for Data Analysis in Python
dependent packages 8total releases 27latest release December 05, 2021most recent commit a year agopypi visions} Downloads
Pixiedust ⭐ 1,035
Python Helper library for Jupyter Notebooks
dependent packages 7total releases 92latest release February 01, 2021most recent commit 3 years agopypi pixiedust} Downloads
Popmon ⭐ 461
Monitor the stability of a Pandas or Spark dataframe ⚙︎
dependent packages 2total releases 36latest release July 18, 2023most recent commit 9 months agopypi popmon} Downloads
Splink ⭐ 939
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
dependent packages 2total releases 119latest release November 14, 2023most recent commit 3 months agopypi splink} Downloads
Pysparkling ⭐ 253
A pure Python implementation of Apache Spark's RDD and DStream interfaces.
dependent packages 1total releases 69latest release November 13, 2022most recent commit a year agopypi pysparkling} Downloads
The 10 Latest Releases In Python Data Science Spark Open Source Projects
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Flytekit ⭐ 175
Extensible Python SDK for developing Flyte tasks and workflows. Simple to get started and learn and highly extensible.
dependent packages 55total releases 350latest release December 06, 2023most recent commit 3 months agopypi flytekit} Downloads
Datacompy ⭐ 339
Pandas and Spark DataFrame comparison for humans and more!
dependent packages 10total releases 20latest release November 15, 2023most recent commit 3 months agopypi datacompy} Downloads
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35latest release November 14, 2023most recent commit 5 months agopypi butterfree} Downloads
Splink ⭐ 939
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
dependent packages 2total releases 119latest release November 14, 2023most recent commit 3 months agopypi splink} Downloads
Popmon ⭐ 461
Monitor the stability of a Pandas or Spark dataframe ⚙︎
dependent packages 2total releases 36latest release July 18, 2023most recent commit 9 months agopypi popmon} Downloads
Pysparkling ⭐ 253
A pure Python implementation of Apache Spark's RDD and DStream interfaces.
dependent packages 1total releases 69latest release November 13, 2022most recent commit a year agopypi pysparkling} Downloads
Cowait ⭐ 54
Containerized distributed programming framework for Python
total releases 55latest release September 22, 2022most recent commit a year ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.