Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Dataset Spark Open Source Projects

Open source projects categorized as Dataset Spark
Deequ ⭐ 3,044
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
dependent packages 6total releases 37most recent commit 3 months ago
Spark Cassandra Connector ⭐ 1,929
DataStax Connector for Apache Spark to Apache Cassandra
dependent packages 22total releases 81most recent commit 3 months ago
Petastorm ⭐ 1,693
Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
dependent packages 8total releases 86most recent commit 5 months agopypi petastorm} Downloads
Spark Py Notebooks ⭐ 1,515
Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
most recent commit a year ago
Mobius ⭐ 937
C# and F# language binding and extensions to Apache Spark
total releases 22most recent commit 3 months ago
Spark Movie Lens ⭐ 757
An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset
most recent commit 3 years ago
Cdap ⭐ 735
An open source framework for building data analytic applications.
dependent packages 56total releases 23most recent commit 3 months ago
Machinelearning ⭐ 684
Machine learning resources，including algorithm, paper, dataset, example and so on.
most recent commit 5 years ago
Complete Life Cycle Of A Data Science Project ⭐ 499
Complete-Life-Cycle-of-a-Data-Science-Project
most recent commit 4 months ago
Whylogs Java ⭐ 179
Profile and monitor your ML data pipeline end-to-end
dependent packages 2total releases 5most recent commit 3 years ago
Setl ⭐ 173
A simple Spark-powered ETL framework that just works 🍺
total releases 4most recent commit 5 months ago
Qb ⭐ 160
QANTA Quiz Bowl AI
most recent commit 2 years ago
Spark Iforest ⭐ 147
Isolation Forest on Spark
most recent commit 4 years ago
Spatialspark ⭐ 141
Big Spatial Data Processing using Spark
total releases 1most recent commit 7 years ago
Distributed Dataset ⭐ 107
A distributed data processing framework in Haskell.
most recent commit 4 years ago
Phrase At Scale ⭐ 84
Detect common phrases in large amounts of text using a data-driven approach. Size of discovered phrases can be arbitrary. Can be used in languages other than English
most recent commit 5 years ago
Tiledb Vcf ⭐ 79
Efficient variant-call data storage and retrieval library using the TileDB storage library.
most recent commit 3 months ago
Dataframecheatsheet ⭐ 74
Cheatsheet for Spark DataFrame
most recent commit 4 years ago
Mltoolkits ⭐ 65
learningOrchestra is a distributed Machine Learning integration tool that facilitates and streamlines iterative processes in a Data Science project.
most recent commit 2 years ago
Spark Tutorial ⭐ 55
This tutorial provides a quick introduction to using Spark
most recent commit 8 years ago
Spark Examples ⭐ 54
RAPIDS Spark examples
most recent commit 4 years ago
Examples ⭐ 47
These are some code examples
most recent commit 4 years ago
Spark Mail ⭐ 45
Tutorial on parsing Enron email to Avro and then explore the email set using Spark.
most recent commit 5 years ago
The 6 Most Depended On Dataset Spark Open Source Projects
Cdap ⭐ 735
An open source framework for building data analytic applications.
dependent packages 56total releases 23latest release September 01, 2023most recent commit 3 months ago
Spark Cassandra Connector ⭐ 1,929
DataStax Connector for Apache Spark to Apache Cassandra
dependent packages 22total releases 81latest release April 08, 2021most recent commit 3 months ago
Deequ ⭐ 3,044
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
dependent packages 6total releases 37latest release November 09, 2023most recent commit 3 months ago
Enceladus ⭐ 28
Dynamic Conformance Engine
dependent packages 4total releases 40latest release March 23, 2023most recent commit 4 months ago
Whylogs Java ⭐ 179
Profile and monitor your ML data pipeline end-to-end
dependent packages 2total releases 5latest release November 01, 2020most recent commit 3 years ago
The 10 Latest Releases In Dataset Spark Open Source Projects
Deequ ⭐ 3,044
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
dependent packages 6total releases 37latest release November 09, 2023most recent commit 3 months ago
Cdap ⭐ 735
An open source framework for building data analytic applications.
dependent packages 56total releases 23latest release September 01, 2023most recent commit 3 months ago
Enceladus ⭐ 28
Dynamic Conformance Engine
dependent packages 4total releases 40latest release March 23, 2023most recent commit 4 months ago
Sparkdataset ⭐ 28
Instant search for and access to many datasets in Pyspark.
total releases 3latest release November 01, 2021most recent commit 2 years ago
Spark Cassandra Connector ⭐ 1,929
DataStax Connector for Apache Spark to Apache Cassandra
dependent packages 22total releases 81latest release April 08, 2021most recent commit 3 months ago
Whylogs Java ⭐ 179
Profile and monitor your ML data pipeline end-to-end
dependent packages 2total releases 5latest release November 01, 2020most recent commit 3 years ago
Setl ⭐ 173
A simple Spark-powered ETL framework that just works 🍺
total releases 4latest release August 21, 2020most recent commit 5 months ago
Isarn Sketches Spark ⭐ 27
Routines and data structures for using isarn-sketches idiomatically in Apache Spark
total releases 19latest release June 20, 2020most recent commit 2 years ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.