Aws Concurrent Data Orchestration Pipeline Emr Livy
This code demonstrates the architecture featured on the AWS Big Data blog (https://aws.amazon.com/blogs/big-data/ ) which creates a concurrent data pipeline by using Amazon EMR and Apache Livy. This pipeline is orchestrated by Apache Airflow.
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Airflow | 34,299 |  | 320 | 9 days ago | 169 | November 27, 2023 | 890 | apache-2.0 | Python | |
| Apache Airflow - A platform to programmatically author, schedule, and monitor workflows | ||||||||||
| Awesome Apache Airflow | 3,468 | 5 months ago | 2 | Shell | ||||||
| Curated list of resources about Apache Airflow | ||||||||||
| Dag Factory | 1,001 |  | 1 | 5 months ago | 36 | July 20, 2023 | 50 | mit | Python | |
| Dynamically generate Apache Airflow DAGs from YAML configuration files | ||||||||||
| Airflow Tutorial | 669 | 2 years ago | 18 | mit | Jupyter Notebook | |||||
| Apache Airflow tutorial | ||||||||||
| Data Pipelines With Apache Airflow | 587 | 3 months ago | 34 | other | Python | |||||
| Code for Data Pipelines with Apache Airflow | ||||||||||
| Airflow Code Editor | 347 | 3 months ago | 49 | October 25, 2023 | 6 | apache-2.0 | JavaScript | |||
| A plugin for Apache Airflow that allows you to edit DAGs in browser | ||||||||||
| Airflow Client Python | 289 |  | 3 | 3 months ago | 27 | November 14, 2023 | 5 | apache-2.0 | Python | |
| Apache Airflow - OpenApi Client for Python | ||||||||||
| Whirl | 183 | 3 months ago | 5 | apache-2.0 | Shell | |||||
| Fast iterative local development and testing of Apache Airflow workflows | ||||||||||
| Airflow_in_docker_compose | 182 | 5 months ago | 3 | mit | Python | |||||
| Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*) | ||||||||||
| Airflow_for_beginners | 166 | 2 years ago | 10 | mit | Python | |||||
Alternatives To Aws Concurrent Data Orchestration Pipeline Emr LivySelect To Compare
Alternative Project Comparisons
Popular Airflow Projects
Popular Apache Projects
Popular Control Flow Categories
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Amazon Web Services
Pipeline
Apache
Amazon
Concurrent
Dag
Airflow