Airflow
This set of code and instructions has the porpouse to instanciate a compiled environment with set of docker images like airflow webserver, airflow scheduler, postgresql, pyspark, Data Pipeline consuming data from weather api , processing with pyspark and storing in postgresql
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Mleap | 1,479 | 15 | 12 | 5 months ago | 26 | May 07, 2021 | 109 | apache-2.0 | Scala | |
| MLeap: Deploy ML Pipelines to Production | ||||||||||
| Butterfree | 269 |  | 1 | 5 months ago | 35 | November 14, 2023 | 6 | apache-2.0 | Python | |
| A tool for building feature stores. | ||||||||||
| Morphl Community Edition | 233 | 5 years ago | 7 | apache-2.0 | Python | |||||
| MorphL Community Edition uses big data and machine learning to predict user behaviors in digital products and services with the end goal of increasing KPIs (click-through rates, conversion rates, etc.) through personalization | ||||||||||
| Learn By Examples | 72 | 6 years ago | 2 | Scala | ||||||
| Real-world Spark pipelines examples | ||||||||||
| Jgit Spark Connector | 67 |  | 1 | 1 | 5 years ago | 40 | October 10, 2018 | 12 | apache-2.0 | Scala |
| jgit-spark-connector is a library for running scalable data retrieval pipelines that process any number of Git repositories for source code analysis. | ||||||||||
| Spark Nba Analytics | 41 | 7 years ago | mit | HTML | ||||||
| Analyzing NBA data using Spark 2.1 | ||||||||||
| Basin | 29 | a year ago | 42 | other | TypeScript | |||||
| Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser | ||||||||||
| Sparkdltrigger | 28 | a year ago | apache-2.0 | Jupyter Notebook | ||||||
| Repo for the article "Machine Learning Pipelines with Modern Big DataTools for High Energy Physics" | ||||||||||
| Odsc_india_2018 | 26 | 6 years ago | Jupyter Notebook | |||||||
| My presentation at ODSC India 2018 about Deep Learning with Apache Spark | ||||||||||
| Spark Movies Etl | 21 | 8 months ago | 2 | Python | ||||||
| Spark data pipeline that ingests and transforms movie ratings data. | ||||||||||
Alternatives To AirflowSelect To Compare
Alternative Project Comparisons
Popular Pipeline Projects
Popular Pyspark Projects
Popular Data Processing Categories
Related Searches
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Php
Postgresql
Pipeline
Spark
Data Engineering
Airflow
Pyspark