Awesome Open Source

Programming Languages

Search results for bigquery data engineering

data-engineering x

24 search results found

Airbyte ⭐ 12,918

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

Cloudquery ⭐ 5,380

The open source high performance data integration platform built for developers.

Growthbook ⭐ 5,285

Open Source Feature Flagging and A/B Testing Platform

Dataform ⭐ 757

Dataform is a framework for managing SQL based data operations in BigQuery, Snowflake, and Redshift

Bigfunctions ⭐ 490

Supercharge BigQuery with BigFunctions

Mlcraft ⭐ 418

Synmetrix – open source semantic layer / Boost your LLM precision

Ethereum Etl Airflow ⭐ 378

Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethe

Awesome Bigquery Views ⭐ 322

Useful SQL queries for Blockchain ETL datasets in BigQuery.

Jupysql ⭐ 261

Better SQL in Jupyter. 📊

Public Datasets ⭐ 187

The list of public blockchain datasets in BigQuery

Public Datasets Pipelines ⭐ 131

Cloud-native, data onboarding architecture for Google Cloud Datasets

Polygon Etl ⭐ 93

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

Airbyte_serverless ⭐ 83

Airbyte made simple (no UI, no database, no cluster)

Prism is the easiest way to develop, orchestrate, and execute data pipelines in Python.

Amora Data Build Tool ⭐ 37

Amora Data Build Tool enables analysts and engineers to transform data on the data warehouse (BigQuery) by writing Amora Models that describe the data schema using Python's "PEP484 - Type Hints" and select statements with SQLAlchemy. Amora is able to transform Python code into SQL data transformation jobs that run inside the warehouse.

Odbc Scanner Duckdb Extension ⭐ 32

A DuckDB extension to read data directly from databases supporting the ODBC interface

Debussy_concert ⭐ 29

Debussy is an opinionated Data Architecture and Engineering framework, enabling data analysts and engineers to build better platforms and pipelines.

Stairlight ⭐ 25

A data lineage tool detects table dependencies from rendered SQL statements.

Data Engineering Mta Turnstile ⭐ 14

Data Engineering - Metropolitan Transportation Authority (MTA) Subway Data Analysis

Data Pipeline from the Global Historical Climatology Network DataSet

Social Media Analysis ⭐ 11

Social Media Analysis, scalable solution, flexible deployment that analyses social media contents

Data Pipeline With Dbt Using Airflow On Gcp ⭐ 10

This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such as Astro, DBT, GCP, Airflow, Metabase.

De Zoomcamp Project ⭐ 10

My personal project for data engineering zoomcamp

Scheduling Big Data Workloads and Data Pipelines in the Cloud with pyDag

Hedera Etl ⭐ 8

ETL scripts for Hedera Hashgraph

Reddit Data Engineering ⭐ 7

An end-to-end data engineering pipeline to create a dashboard for the latest content on the r/Stocks subreddit

Gcp Airflow Foundations ⭐ 5

Opinionated framework based on Airflow 2.0 for building pipelines to ingest data into a BigQuery data warehouse

1-24 of 24 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.