Gcp Airflow Foundations

Opinionated framework based on Airflow 2.0 for building pipelines to ingest data into a BigQuery data warehouse
Alternatives To Gcp Airflow Foundations
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Optimus70738 months ago120July 25, 202353apache-2.0Go
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
Ethereum Etl Airflow378
2 months ago27mitPython
Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethereum-smart-contract-into-bigquery-in-8-mins-bab5db1fdeee
Astro Sdk30322 months ago49August 30, 2023153apache-2.0Python
Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
Starthinker156
8 months ago15April 07, 202218apache-2.0Python
Reference framework for building data workflows provided by Google. Accelerates authentication, logging, scheduling, and deployment of solutions using GCP. To borrow a tagline.. "The framework for professionals with deadlines."
Tensorflow Lifetime Value140
2 years ago9apache-2.0Jupyter Notebook
Predict customer lifetime value using AutoML Tables, or ML Engine with a TensorFlow neural network and the Lifetimes Python library.
Public Datasets Pipelines131
2 months ago56apache-2.0Python
Cloud-native, data onboarding architecture for Google Cloud Datasets
Cortex Data Foundation129
2 months ago3apache-2.0Python
Data Foundation - Google Cloud Cortex Framework
Tensorflow Recommendation Wals126
4 years ago8apache-2.0Python
An end-to-end solution for website article recommendations based on Google Analytics data. Uses WALS matrix-factorization in TensorFlow, trained on Cloud ML Engine. Recommendations served with App Engine Flex and Cloud Endpoints. Orchestration is performed using Airflow on Cloud Composer. See the solution tutorials at:
Polygon Etl93
4 months ago39November 21, 202310mitPython
ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub
Airflow Tutorial86
4 years ago1apache-2.0PLpgSQL
Use Airflow to move data from multiple MySQL databases to BigQuery
Alternatives To Gcp Airflow Foundations
Select To Compare


Alternative Project Comparisons
Popular Bigquery Projects
Popular Airflow Projects
Popular Data Processing Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Bigquery
Dag
Data Engineering
Airflow