Awesome Open Source
Awesome Open Source
Combined Topics
apache-spark x
machine-learning x

The Top 84 Machine Learning Apache Spark Open Source Projects

Categories > Data Processing > Apache Spark
Categories > Machine Learning > Machine Learning
Mlflow ⭐ 11,828
Open source platform for the machine learning lifecycle
dependent packages 235total releases 57most recent commit 3 hours agopypi mlflow} Downloads
Synapseml ⭐ 3,297
Simple and Distributed Machine Learning
dependent packages 1total releases 3most recent commit 5 days agopypi synapseml} Downloads
Spark Nlp ⭐ 2,727
State of the Art Natural Language Processing
dependent packages 2total releases 90most recent commit 12 hours ago
Transmogrifai ⭐ 2,099
TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning
dependent packages 3total releases 9most recent commit 3 months ago
Spark ⭐ 1,766
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
dependent packages 15total releases 19most recent commit 11 days agonuget Microsoft.Spark} Downloads
Oryx ⭐ 1,759
Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
dependent packages 2total releases 14most recent commit 10 months ago
Sparkit Learn ⭐ 1,054
PySpark + Scikit-learn = Sparkit-learn
total releases 13most recent commit a year ago
Spark Sklearn ⭐ 1,039
(Deprecated) Scikit-learn integration package for Apache Spark
total releases 8most recent commit 2 years ago
Sparklyr ⭐ 856
R interface for Apache Spark
dependent packages 26total releases 46most recent commit 21 hours ago
Dist Keras ⭐ 611
Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.
total releases 2most recent commit 4 years ago
Sparklearning ⭐ 573
Learning Apache spark,including code and data .Most part can run local.
most recent commit 6 months ago
Agile_data_code_2 ⭐ 423
Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
most recent commit 7 months ago
Machinelearning ⭐ 406
Machine Learning
most recent commit 5 years ago
Whylogs Java ⭐ 179
Profile and monitor your ML data pipeline end-to-end
dependent packages 2total releases 5most recent commit 8 months ago
Awesome Ai Infrastructures ⭐ 171
Infrastructures™ for Machine Learning Training/Inference in Production.
most recent commit 3 years ago
Rumble ⭐ 165
⛈️ RumbleDB 1.18.0 "Scarlet Ixora" 🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more
total releases 4most recent commit a month ago
Bigdata Playground ⭐ 154
A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
most recent commit 3 years ago
Spark Ext ⭐ 147
Spark Extension : ML transformers, SQL aggregations, etc that are missing in Apache Spark
most recent commit 6 years ago
Albedo ⭐ 142
A recommender system for discovering GitHub repos, built with Apache Spark
most recent commit 2 years ago
Spark With Python ⭐ 98
Fundamentals of Spark with Python (using PySpark), code examples
most recent commit 2 years ago
Pyspark Twitter Stream Mining ⭐ 63
Real-time Machine Learning with Apache Spark on Twitter Public Stream
most recent commit 5 years ago
Mmtf Pyspark ⭐ 60
Methods for the parallel and distributed analysis and mining of the Protein Data Bank using MMTF and Apache Spark.
most recent commit 2 days ago
Mmtf Workshop 2018 ⭐ 53
Structural Bioinformatics Training Workshop & Hackathon 2018
most recent commit a day ago
Aardpfark ⭐ 47
A library for exporting Spark ML models and pipelines to PFA
most recent commit 3 years ago
Spark Tda ⭐ 46
SparkTDA is a package for Apache Spark providing Topological Data Analysis Functionalities.
most recent commit 4 years ago
Liquidsvm ⭐ 45
Support vector machines (SVMs) and related kernel-based learning algorithms are a well-known class of machine learning algorithms, for non-parametric classification and regression. liquidSVM is an implementation of SVMs whose key features are: fully integrated hyper-parameter selection, extreme speed on both small and large data sets, full flexibility for experts, and inclusion of a variety of different learning scenarios: multi-class classification, ROC, and Neyman-Pearson learning, and least-s
most recent commit 2 years ago
Sparkxgb ⭐ 40
R interface for XGBoost on Spark
most recent commit 4 months ago
Jpmml Sparkml Lightgbm ⭐ 39
JPMML-SparkML plugin for converting LightGBM-Spark models to PMML
most recent commit 7 months ago
Spark Transformers ⭐ 37
Spark-Transformers: Library for exporting Apache Spark MLLIB models to use them in any Java application with no other dependencies.
most recent commit 4 years ago
Lstm Tensorspark ⭐ 35
Implementation of a LSTM with TensorFlow and distributed on Apache Spark
most recent commit 4 years ago
Bigclam Apachespark ⭐ 34
Overlapping community detection in Large-Scale Networks using BigCLAM model build on Apache Spark
most recent commit 4 years ago
Spark Notebook Ml Labs ⭐ 30
Data Science with Apache Spark and Spark Notebook
most recent commit 5 years ago
Baskerville ⭐ 27
Security Analytics Engine - Anomaly Detection in Web Traffic
most recent commit 4 months ago
Detecting Malicious Url Machine Learning ⭐ 23
most recent commit 4 years ago
Awesome Sparklyr ⭐ 22
An awesome sparklyr related package collection
most recent commit 2 years ago
Mmtf Spark ⭐ 19
Methods for the parallel and distributed analysis and mining of the Protein Data Bank using MMTF and Apache Spark.
most recent commit 3 years ago
Admml ⭐ 17
ADMM based Scalable Machine Learning on Spark
most recent commit 5 years ago
Explainer ⭐ 16
A machine learning model explainer that works on top of Apache Spark
most recent commit 5 years ago
Machine Learning ⭐ 14
Examples of all Machine Learning Algorithm in Apache Spark
most recent commit 5 years ago
Tedsds ⭐ 14
Apache Spark - Turbofan Engine Degradation Simulation Data Set example in Apache Spark
most recent commit 5 years ago
Spark Ensemble ⭐ 12
Ensemble Learning for Apache Spark 🌲
most recent commit a year ago
Pybda ⭐ 9
💻💻💻 A commandline tool for analysis of big biological data sets for distributed HPC clusters.
most recent commit 3 years ago
Spark Gp ⭐ 9
Gaussian Process Classification and Regression on Apache Spark
most recent commit a year ago
Sparkitecture ⭐ 9
A collection of “cookbook-style” scripts for simplifying data engineering and machine learning in Apache Spark.
most recent commit 7 months ago
Javarank ⭐ 8
Recommendation engine in Java. Based on an ALS algorithm (Apache Spark). Train a new model after N seconds.
most recent commit 3 years ago
Ma Inf 4223 Dbda Lab ⭐ 7
Repository for Lab “Distributed Big Data Analytics” (MA-INF 4223), University of Bonn
most recent commit 3 years ago
Railsewa Finalyearproject ⭐ 6
Automated Real-Time Indian Railway Twitter Complaint Administration System. It uses Apache Kafka, Spark, MySQL, PHP. The full project is deployed on AWS EC2 and RDS.
most recent commit 2 years ago
Databrickstraining ⭐ 6
Repository for Microsoft Databricks Training Events - Hosted by BlueGranite
most recent commit 3 years ago
Bigdata And Machine Learning ⭐ 6
Basics of Big Data and Machine Learning using Apache Spark and Scala
most recent commit 5 years ago
Cookie Datasets ⭐ 6
Read well-known ML datasets in Apache Spark
most recent commit 5 years ago
Sms Spam Filter Using Hortonworks ⭐ 5
Build Spam Filter Model on HDP using Watson Studio Local
most recent commit 3 years ago
Physonline ⭐ 5
PhysOnline: An Open Source Machine Learning Pipeline for Real-Time Analysis of Waveform Data
most recent commit 2 years ago
Docker Spark Anaconda ⭐ 5
Spark and Anaconda in Docker
most recent commit 5 years ago
Machine Learning Pipeline Lr Pyspark ⭐ 5
Power Plant ML Pipeline Application - Apache Spark
most recent commit 5 years ago
Kotlin Apache Spark Example ⭐ 5
kotlin-apache-spark-example
most recent commit 3 years ago
Irecognize ⭐ 5
iRecognize: Visual Intelligence Made Easy
most recent commit 4 months ago
Spark For Dummies ⭐ 5
Mastering Spark 2 from the very beginning
most recent commit 3 years ago
Spark Twitter Streaming ⭐ 4
Streaming Twitter Sentiment Analysis with Apache Spark
most recent commit 4 years ago
Fraud_transaction_detection_using_apache_spark_ml ⭐ 4
Analyzing data and provide insights on Financial Fraud Detection using Spark ML.
most recent commit 4 years ago
Spark Pipeline ⭐ 4
Machine learning pipeline for Apache Spark
most recent commit a month ago
Ibm Ai Engineering Professional Certificate ⭐ 4
IBM AI Engineering
most recent commit 2 years ago
Net.jgp.labs.sparkdq4ml ⭐ 4
Labs around Apache Spark fro DQ (Data Quality) and ML (Machine Learning)
most recent commit 2 years ago
Eeg_dataanalysispackage ⭐ 4
Application for analyzing EEG data stored on Hadoop using Apache Spark
most recent commit 5 years ago
Serene ⭐ 4
Serene Data Integration Platform
most recent commit 5 years ago
Rotation Forest Bd ⭐ 4
🔄🌳⚡ The Rotation Forest implementation for Big Data on Apache Spark
most recent commit a year ago
Professional Certificate Programs ⭐ 4
This repository contains all the projects and labs I worked on while pursuing professional certificate programs, specializations, and bootcamp. [Areas: Deep Learning, Machine Learning, Applied Data Science].
most recent commit 2 years ago
Spark Ml Examples ⭐ 3
most recent commit 5 years ago
Kagglecompetitions ⭐ 3
Repo for Kaggle Data Science Competitions
most recent commit 4 years ago
Multivac Kaggle Titanic ⭐ 3
Simple example of Titanic competition by Spark 2.2
most recent commit 3 years ago
Cdap Spark ⭐ 2
A wrapper for Apache Spark to make machine & deep learning available in Google CDAP data pipelines.
most recent commit 5 months ago
Cautious Potato ⭐ 2
Investigating the deequ
most recent commit 3 years ago
Multivac Fakenews ⭐ 2
Detecting users and communities which propagate fake news on Twitter by Apache Spark
most recent commit 3 years ago
Machine Learning In Spark With Pyspark ⭐ 2
[A repo to store some code and experiences about machine learning in spark with pyspark] Spark comes with a library containing common machine learning (ML) functionality, called MLlib. MLlib provides multiple types of machine learning algorithms, including classification, regression, clustering, and collaborative filtering, as well as supporting functionality such as model evaluation and data import. It also provides some lower-level ML primitives, including a generic gradient descent optimizati
most recent commit 3 years ago
Coursera Ibm Ai Engineering ⭐ 2
IBM - AI Engineering Professional Certificate
most recent commit 2 years ago
Machine Learning With Apache Spark 2.0 ⭐ 2
Source code repository for Machine Learning with Apache Spark 2.0
most recent commit 4 years ago
Spark Ifs ⭐ 2
Iterative filter-based feature selection on large datasets with Apache Spark
most recent commit 4 years ago
Detecting Hacking Activities With Streaming Data ⭐ 2
Integrating the machine learning models into the streaming platform to detect any real-time threats (Apache Kafka, Apache Spark Streaming)
most recent commit a year ago
Dataworkssummit2018_spark_ml ⭐ 2
hands on introduction to basic Machine Learning techniques with Apache Spark ML using the cloud.
most recent commit 4 years ago
Spark Scala ⭐ 2
Apache Spark with Scala for ML and Data Analytics
most recent commit 2 years ago
Spartann ⭐ 2
Hyper performant kNN using Annoy for Apache Spark.
most recent commit 6 months ago
Spark Api ⭐ 2
Apache Spark Recommendation/Machine Learning Api Service
most recent commit 5 years ago
Sparkmljavaexamples ⭐ 2
Apache Spark Machine Learning - Java Examples
most recent commit 6 years ago
Movierecommender ⭐ 2
Implementation of a basic movie recommendation system using Apache Spark and the MovieLens data set.
most recent commit 6 years ago
Chopin2 ⭐ 2
Domain-Agnostic Supervised Learning with Hyperdimensional Computing
most recent commit 8 days ago
1-84 of 84 projects
Related Awesome Lists
Python Machine Learning Projects (14,099)
Jupyter Notebook Machine Learning Projects (8,463)
Machine Learning Deep Learning Projects (6,034)
Machine Learning Data Science Projects (4,324)
Machine Learning Neural Network Projects (3,255)
Machine Learning Tensorflow Projects (2,638)
Machine Learning Artificial Intelligence Projects (2,027)
Machine Learning Natural Language Processing Projects (1,786)
Machine Learning Computer Vision Projects (1,763)
Machine Learning Pytorch Projects (1,688)
Machine Learning Scikit Learn Projects (1,381)
Machine Learning Algorithms Projects (1,372)
Machine Learning Neural Projects (1,332)
Javascript Machine Learning Projects (1,257)
Machine Learning Keras Projects (1,188)
Machine Learning Network Projects (1,148)
Html Machine Learning Projects (1,055)
R Machine Learning Projects (842)
Machine Learning Numpy Projects (803)
C Plus Plus Machine Learning Projects (758)
Machine Learning Data Analysis Projects (757)
Java Machine Learning Projects (692)
Machine Learning Sklearn Projects (679)
Machine Learning Paper Projects (629)
Machine Learning Data Visualization Projects (574)
Machine Learning Opencv Projects (549)
Scala Apache Spark Projects (495)
Machine Learning Svm Projects (493)
Machine Learning Reinforcement Projects (5)
Categories
Advertising 📦 8
All Projects
Application Programming Interfaces 📦 107
Applications 📦 174
Artificial Intelligence 📦 69
Blockchain 📦 66
Build Tools 📦 105
Cloud Computing 📦 68
Code Quality 📦 24
Collaboration 📦 27
Command Line Interface 📦 38
Community 📦 79
Companies 📦 60
Compilers 📦 59
Computer Science 📦 73
Configuration Management 📦 37
Content Management 📦 153
Control Flow 📦 187
Data Formats 📦 72
Data Processing 📦 249
Data Storage 📦 116
Economics 📦 56
Frameworks 📦 178
Games 📦 110
Graphics 📦 92
Hardware 📦 137
Integrated Development Environments 📦 43
Learning Resources 📦 139
Legal 📦 24
Libraries 📦 117
Lists Of Projects 📦 19
Machine Learning 📦 313
Mapping 📦 57
Marketing 📦 15
Mathematics 📦 54
Media 📦 214
Messaging 📦 96
Networking 📦 292
Operating Systems 📦 72
Operations 📦 114
Package Managers 📦 50
Programming Languages 📦 173
Runtime Environments 📦 90
Science 📦 42
Security 📦 342
Social Media 📦 23
Software Architecture 📦 65
Software Development 📦 60
Software Performance 📦 53
Software Quality 📦 116
Text Editors 📦 40
Text Processing 📦 118
User Interface 📦 291
User Interface Components 📦 440
Version Control 📦 26
Virtualization 📦 63
Web Browsers 📦 35
Web Servers 📦 23
Web User Interface 📦 167
Top Programming Languages
Javascript  (1,046,457)
Python  (794,068)
Java  (384,724)
Php  (284,330)
Typescript  (238,326)
C Plus Plus  (235,680)
Ruby  (221,645)
C  (180,013)
Shell  (169,262)
C Sharp  (161,654)
Golang  (155,676)
Swift  (64,583)
Objective C  (57,182)
R  (57,094)
Rust  (55,815)
Dart  (51,232)
Kotlin  (45,306)
Lua  (33,164)
Matlab  (30,828)
Perl  (30,482)
Language  (30,083)
Scala  (28,584)
Haskell  (20,224)
Clojure  (19,064)
Powershell  (18,767)
Bash  (18,581)
Coffeescript  (17,335)
Elixir  (16,552)
Assembly  (14,504)
Processing  (13,294)
Julia  (12,509)
Basic  (10,772)
Flow  (10,528)
Emacs Lisp  (10,358)
Erlang  (9,055)
Scheme  (8,442)
Groovy  (8,426)
Solidity  (7,546)
Ocaml  (6,657)
Pascal  (6,374)
Hack  (6,364)
Lisp  (6,346)
Fortran  (5,474)
Elm  (5,468)
Common Lisp  (5,287)
Dlang  (4,740)
Shell Script  (4,625)
Python Library  (4,446)
Crystal  (4,373)
Scripting  (4,199)
Top Projects
Freecodecamp ⭐ 346,041
996.icu ⭐ 262,179
Free Programming Books ⭐ 232,913
Coding Interview University ⭐ 219,669
Awesome ⭐ 201,499
Vue ⭐ 195,787
Developer Roadmap ⭐ 194,722
Public Apis ⭐ 191,889
React ⭐ 188,013
System Design Primer ⭐ 180,579
Tensorflow ⭐ 164,945
Bootstrap ⭐ 157,273
You Dont Know Js ⭐ 152,909
Cs Notes ⭐ 150,780
Ohmyzsh ⭐ 145,101
Build Your Own X ⭐ 141,391
Flutter ⭐ 140,622
Javascript Algorithms ⭐ 140,055
Python ⭐ 136,032
Gitignore ⭐ 133,164
Linux ⭐ 131,793
Vscode ⭐ 131,712
Awesome Python ⭐ 126,837
Javascript ⭐ 122,717
Javaguide ⭐ 121,455
Python 100 Days ⭐ 117,585
Computer Science ⭐ 112,893
Youtube Dl ⭐ 109,983
Fucking Algorithm ⭐ 106,048
The Art Of Command Line ⭐ 105,874
React Native ⭐ 102,729
Electron ⭐ 101,695
D3 ⭐ 100,901
Go ⭐ 99,196
30 Seconds Of Code ⭐ 95,528
Create React App ⭐ 95,021
Axios ⭐ 93,374
Awesome Selfhosted ⭐ 88,479
Kubernetes ⭐ 88,431
Free Programming Books Zh_cn ⭐ 88,386
Node ⭐ 87,633
Next.js ⭐ 86,760
Terminal ⭐ 83,115
Deno ⭐ 82,447
Three.js ⭐ 81,980
Angular ⭐ 81,364
Awesome Go ⭐ 80,677
Typescript ⭐ 80,654
Ant Design ⭐ 80,284
Material Ui ⭐ 78,434
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2022 Awesome Open Source.  All rights reserved.