Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Spark Big Data Open Source Projects

Open source projects categorized as Python Spark Big Data
Spark ⭐ 37,661
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 939total releases 46most recent commit 3 months ago
Data Science Ipython Notebooks ⭐ 25,668
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
most recent commit 6 months ago
H2o 3 ⭐ 6,618
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
dependent packages 33total releases 49most recent commit 3 months ago
Koalas ⭐ 3,291
Koalas: pandas API on Apache Spark
dependent packages 16total releases 47most recent commit 7 months agopypi koalas} Downloads
Dpark ⭐ 2,637
Python clone of Spark, a MapReduce alike framework in Python
total releases 19most recent commit 3 years ago
Lakesoul ⭐ 2,248
LakeSoul is an end-to-end, realtime and cloud native Lakehouse framework with fast data ingestion, concurrent update and incremental data analytics on cloud storages for both BI and AI applications.
dependent packages 1total releases 7most recent commit 3 months ago
Spark Py Notebooks ⭐ 1,515
Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
most recent commit a year ago
Adam ⭐ 966
ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.
dependent packages 17total releases 14most recent commit 3 months ago
Coding Now ⭐ 925
学习记录的一些笔记，以及所看得一些电子书eBooks、视频资源和平常收纳的一些自己认为比较好的博客、
most recent commit 2 years ago
Incubator Livy ⭐ 840
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
dependent packages 8total releases 6most recent commit 3 months ago
Spark Movie Lens ⭐ 757
An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset
most recent commit 3 years ago
Listenbrainz Server ⭐ 613
Server for the ListenBrainz project, including the front-end (javascript/react) code that it serves and all of the data processing components that LB uses.
most recent commit 3 months ago
Gimel ⭐ 230
Big Data Processing Framework - Unified Data API or SQL on Any Storage
most recent commit a year ago
Juicy Bigdata ⭐ 162
🎉🎉🐳 Datawhale大数据处理导论教程 | 大数据技术方向的开篇课程🎉🎉
most recent commit a year ago
Lakehouse Engine ⭐ 154
The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.
most recent commit 3 months ago
Geopyspark ⭐ 151
GeoTrellis for PySpark
most recent commit 4 years ago
Data Algorithms With Spark ⭐ 151
O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian
most recent commit 10 months ago
Pyspark Cheatsheet ⭐ 140
PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster
most recent commit 2 years ago
Python Bigdata ⭐ 128
Data science and Big Data with Python
most recent commit 8 months ago
Aut ⭐ 128
The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
total releases 27most recent commit 10 months ago
Spark Website ⭐ 109
Apache Spark Website
most recent commit 3 months ago
Frank Kanes Taming Big Data With Apache Spark And Python ⭐ 106
Frank Kane's Taming Big Data with Apache Spark and Python, published by Packt
most recent commit a year ago
Spark With Python ⭐ 98
Fundamentals of Spark with Python (using PySpark), code examples
most recent commit 4 years ago
The 6 Most Depended On Python Spark Big Data Open Source Projects
Spark ⭐ 37,661
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 939total releases 46latest release May 09, 2021most recent commit 3 months ago
Koalas ⭐ 3,291
Koalas: pandas API on Apache Spark
dependent packages 16total releases 47latest release October 19, 2021most recent commit 7 months agopypi koalas} Downloads
Incubator Livy ⭐ 840
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
dependent packages 8total releases 6latest release September 25, 2023most recent commit 3 months ago
The 10 Latest Releases In Python Spark Big Data Open Source Projects
Sageworks ⭐ 36
SageWorks: An easy to use Python API for creating and deploying SageMaker Models
total releases 60latest release November 29, 2023most recent commit 3 months ago
Yaetos ⭐ 32
Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified
total releases 32latest release November 27, 2023most recent commit 3 months ago
Incubator Livy ⭐ 840
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
dependent packages 8total releases 6latest release September 25, 2023most recent commit 3 months ago
Aut ⭐ 128
The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
total releases 27latest release November 17, 2022most recent commit 10 months ago
Spooq ⭐ 8
total releases 10latest release August 16, 2022most recent commit a year ago
Koalas ⭐ 3,291
Koalas: pandas API on Apache Spark
dependent packages 16total releases 47latest release October 19, 2021most recent commit 7 months agopypi koalas} Downloads
Spark ⭐ 37,661
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 939total releases 46latest release May 09, 2021most recent commit 3 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.