Awesome Open Source

Programming Languages

Search results for airflow

512 search results found

Opencase ⭐ 26

Modular Open Source PC case designed for Mini-ITX motherboards.

Airflow Prometheus Exporter ⭐ 25

Export Airflow metrics (from mysql) in prometheus format

Airflow Jupyter Docker Compose ⭐ 24

Orchestration of data science and earth observation models in Apache Airflow, scale-up with Celery Executor, experiment with jupyter notebook using a docker containers composition

Microflow ⭐ 24

Microservice orchestration inspired by AWS Step functions and Apache Airflow

Spot Price Reporter ⭐ 24

Fetch and plot AWS spot pricing history

💨🥫 A Data Factory system for running data processing pipelines built on AirFlow and tailored to CKAN. Includes evolution of DataPusher and Xloader for loading data to DataStore.

Audiophile E2e Pipeline ⭐ 24

Pipeline that extracts data from Crinacle's Headphone and InEarMonitor databases and finalizes data for a Metabase Dashboard.

Terraform Aws Mwaa ⭐ 23

Terraform module for Amazon MWAA(Apache Airflow)

Airflow Breeze Gcp Extension ⭐ 23

Data Pipeline ⭐ 23

Tickit Data Lake Demo ⭐ 23

Resources for video demonstrations and blog posts related to DataOps on AWS

Wellcome tool to parse references scraped from policy documents using machine learning

Data Pipelining ⭐ 23

Simple Dag Editor ⭐ 23

Zero configuration Airflow plugin that let you manage your DAG files.

Telemetry-Aware Addon Recommender

Udacity Data Eng Proj2 ⭐ 23

A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract data from S3, apply a series of transformations and load into S3 and Redshift.

Awesome Kedro ⭐ 22

Clean Pods ⭐ 22

Kubernetes: Clean pods older than X days

Airflow On K8s Operator ⭐ 22

Airflow on Kubernetes Operator

Jobanalytics_and_search ⭐ 22

JobAnalytics system consumes data from multiple sources and provides valuable information to both job hunters and recruiters.

Airflow Snowflake ⭐ 22

Code to be contributed to the Apache Airflow (incubating) project for ETL workflow management for integrating with the Snowflake Data Warehouse.

Forklift ⭐ 22

🚚 ETL for Spark and Airflow

Typhoon Orchestrator ⭐ 22

Create elegant data pipelines and deploy to AWS Lambda or Airflow

Docker_datalake ⭐ 21

Dataops Platform Airflow Dbt ⭐ 21

Build DataOps platform with Apache Airflow and dbt on AWS

Heroku Airflow ⭐ 21

Demo of Author data workflows with Airflow on Heroku (not maintained)

Crypto_dash_app ⭐ 21

A web application built on Plotly Dash

Helpdesk ⭐ 20

Yet another helpdesk based on multiple providers

Ndexr Platform ⭐ 20

The NDEXR platform code

Terraform Aws Airflow ⭐ 20

Terraform module for a PostgreSQL-backed Apache Airflow instance

Terraform Infra ⭐ 20

Production Grade Terraform for Provisioning Infrastructure

Sf Evictiontracker ⭐ 20

Tracking and measuring neighborhood and district-level eviction rates in the city of San Francisco.

Propertyclassification ⭐ 20

Classifying the type of property given Real Estate, satellite and Street view Images

Airdatacomputer ⭐ 20

Air Data Computer

Airflow Gke ⭐ 20

Setting up Apache Airflow on GKE

Airflow Chart ⭐ 20

Helm chart for deploying Apache Airflow in kubernetes

Newrelic Airflow Plugin ⭐ 20

Send airflow metrics to New Relic!

Airflow Docker ⭐ 19

This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and workflows.

Tech_blog ⭐ 19

Follow the Lumiata Tech Blog on Medium!

Insight Gdelt Feed ⭐ 19

A way for home buyers to know about factors affecting a state

Airflow Averageface ⭐ 19

Creates a pipeline Airflow and Scrapy to output an average image composition of everyone's face in a given website

Airflow Tutorial ⭐ 19

Tutorial like code for how to deploy airflow using docker and how to use the DockerOperator.

Puppet Airflow ⭐ 19

Puppet module to provision Airbnb's Airflow

Superset Trading Dashboard ⭐ 19

Superset Trading Dashboard

Spotify Api ⭐ 19

Pipeline that extracts data from the Spotify API to build a more detailed version of Spotify Wrapped

Pipeline Editor ⭐ 19

Common pipeline-editor components used in different clients (e.g. Elyra application, Web browser extensions, etc)

Sentry_airflow ⭐ 19

Airflow integration with Sentry (https://sentry.io)

Airflow Elt Blueprint ⭐ 18

A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.

Final Project Level3 Recsys 06 ⭐ 18

🔥 당신을 추천하는 가장 좋은 방법 RecommendU @boostcampaitech4

Observatory Platform ⭐ 18

Observatory Platform Package

Airflow On Kubernetes ⭐ 18

Bare minimal Airflow on Kubernetes (Local, EKS, AKS)

Fab Oidc ⭐ 18

Flask-AppBuilder SecurityManager for OpenIDConnect

Airflow_demo ⭐ 18

Airflow script for incremental data import from Mysql to Hive using Sqoop.

Airflow Dbt Cloud ⭐ 18

dbt Cloud pipelines in airflow examples

Ansible Airflow ⭐ 18

Ansible role for the installation of Apache Airflow

Airflow Git Sync ⭐ 17

Sync DAG changes from Git to Airflow

Dbimport ⭐ 17

DBImport ingestion tool. Handle import, export and standard ETL flows in Hadoop/Hive

Airflow Dag Action ⭐ 17

GitHub Actions to Validate DAGs, Variables and Dependencies upon Pull Request

Airflow Provider Anomaly Detection ⭐ 17

An airflow provider for anomaly detection.

Dolthub Etl Jobs ⭐ 17

ETL jobs that DoltHub maintained that load public data into DoltHub.

Airflow Livy Operators ⭐ 17

Lets Airflow DAGs run Spark jobs via Livy: sessions and/or batches.

Serverless Datalake Example ⭐ 17

A serverless datalake project and framework based on AWS S3，Glue，Athena，MWAA and QuickSight. With a series of best practices, it guides you how to build a serverless datalake.

Phila Airflow ⭐ 16

Airflow Docker ⭐ 16

Apache Airflow Docker Image.

Airflowetl ⭐ 16

Blog post on ETL pipelines with Airflow

CoWorks is a unified compositional serverless microservices framework over AWS, Flask and Airflow technologies.

Package made to simplify Data Scientist work, on ML and production part.

Karpov_courses ⭐ 16

🐳 Проектная деятельность. Здесь хранятся лекции, практические задания и проекты с karpov_courses. Ссылка: https://karpov.courses/

Health Equity Tracker ⭐ 15

Health Equity Tracker is a free-to-use data visualization platform that is enabling new insights into the impact of COVID-19 and other social and political determinants of health on historically underrepresented groups in the United States.

Testbed of AI Systems Quality Management

Telemetry Streaming ⭐ 15

Spark Streaming ETL jobs for Mozilla Telemetry

Real Time Twitter Sentiment Analysis Product

Oci Airflow ⭐ 15

Apache Airflow on Oracle Cloud Infrastructure

Airflow Docker Compose Example ⭐ 15

Example implementation running Airflow as separate services with docker-compose.

Airflow Provider Hightouch ⭐ 15

Airflow operators, hooks, and sensors for interacting with the Hightouch API

Airflow Provider Fivetran Async ⭐ 15

A new Airflow Provider for Fivetran, maintained by Astronomer and Fivetran

A scalable, declarative, low-code framework for real-time and batch feature calculation/management (quant finance, anomaly/fraud detection, etc.), predictive ML training/inference and simulation. Built on top of Ray

Airflow Valohai Plugin ⭐ 15

🦈 Airflow plugin to scale machine learning tasks with Valohai and get automatic version control

Airflowdatapipeline ⭐ 15

Example of an ETL Pipeline using Airflow

Airflow Ci ⭐ 15

Apache Airflow CI pipeline

Opentrials Airflow ⭐ 15

Configuration and definitions of Airflow for OpenTrials

Db Airflow Dags ⭐ 15

Collection of DAGS for DCP database development and generation

Greenish ⭐ 15

Data monitoring tool, monitors the result, not the run

Etl Airflow S3 ⭐ 14

ETL of newspaper article keywords using Apache Airflow, Newspaper3k, Quilt T4 and AWS S3

Facebook_ads_plugin ⭐ 14

Airflow Plugins ⭐ 14

various airflow plugins

Airflow Helm ⭐ 14

A Helm Chart for Apache Airflow

Minikube for big data with Scala and Spark

Airflow Metrics ⭐ 14

Metrics for airflow

Airflow Imaging Plugins ⭐ 14

Set of plugins helping to work with imaging data in Airflow.

Apache Airflow Cloudera Parcel ⭐ 14

Parcel for Apache Airflow

Heroku_airflow ⭐ 14

Build for Apache Airflow running on Heroku

Airflow Docker ⭐ 14

An opinionated implementation of exclusively using airflow DockerOperators for all Operators

Dbt Airflow ⭐ 14

A Python package that creates fine-grained dbt tasks on Apache Airflow

Terraform Provider Astronomer ⭐ 14

Lifecycle management for Airflow clusters managed by Astronomer

Data Pipeline from the Global Historical Climatology Network DataSet

Airflow Ml Prediction ⭐ 13

Running ECS task for ML prediction orchestrated by Airflow

Ml Airflow ⭐ 13

Generalized project for running Airflow DAGs, with possibility of skipping tasks already done for some set of input parameters.

Thepipelinetool ⭐ 13

A pipeline orchestration tool

201-300 of 512 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.