Awesome Open Source

Programming Languages

Search results for python bigquery

235 search results found

Redash ⭐ 24,479

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

Airbyte ⭐ 12,918

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

Sqlglot ⭐ 4,652

Python SQL Parser and Transpiler

Google Cloud Python ⭐ 4,557

Google Cloud Client Library for Python

The flexibility of Python with the scale and performance of modern SQL.

Ethereum Etl ⭐ 2,760

Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

Professional Services ⭐ 2,635

Common solutions and tools developed by Google Cloud's Professional Services team. This repository and its contents are not an officially supported Google product.

Swirl Search ⭐ 1,371

Swirl is open-source software that uses AI to simultaneously search multiple content and data sources, finds the best results using a reader LLM, then prompts Generative AI, enabling you to get answers from your own data.

pepy is a site to get statistics information about any Python package.

Python Bigquery ⭐ 690

Bigfunctions ⭐ 490

Supercharge BigQuery with BigFunctions

Dbt Checkpoint ⭐ 471

🎣 List of `pre-commit` hooks to ensure the quality of your `dbt` projects.

Bigquery Python ⭐ 414

Simple Python client for interacting with Google BigQuery.

Objectiv Analytics ⭐ 408

Powerful product analytics for data teams, with full control over data & models.

Python Bigquery Pandas ⭐ 406

Google BigQuery connector for pandas

Ethereum Etl Airflow ⭐ 378

Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethe

⚙️ Código de manutenção do datalake (metadados e pacotes de acesso) | 📖 Docs: https://basedosdados.github.io/mais/

Pypinfo ⭐ 373

Easily view PyPI download statistics via Google's BigQuery.

Astro Sdk ⭐ 303

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

Security Analytics ⭐ 268

Community Security Analytics provides a set of community-driven audit & threat queries for Google Cloud

Jupysql ⭐ 261

Better SQL in Jupyter. 📊

Bigquery Schema Generator ⭐ 228

Generates the BigQuery schema from newline-delimited JSON or CSV data records.

Bigquery Etl ⭐ 216

Geomancer ⭐ 194

Automated feature engineering for geospatial data

Dbt Coves ⭐ 193

CLI tool for dbt users to simplify creation of staging models (yml and sql) files

Dbt Ml Preprocessing ⭐ 162

A SQL port of python's scikit-learn preprocessing module, provided as cross-database dbt macros.

Starthinker ⭐ 156

Reference framework for building data workflows provided by Google. Accelerates authentication, logging, scheduling, and deployment of solutions using GCP. To borrow a tagline.. "The framework for professionals with deadlines."

Encrypted Bigquery Client ⭐ 156

An experimental version of the BQ client which adds client-side encryption

Keras Serving ⭐ 147

bring keras-models to production with tensorflow-serving and nodejs + docker 🍕

Simple Ddl Parser ⭐ 147

Simple DDL Parser to parse SQL (HQL, TSQL, AWS Redshift, BigQuery, Snowflake and other dialects) ddl files to json/python dict with full information about columns: types, defaults, primary keys, etc. & table properties, types, domains, etc.

Cueobserve ⭐ 144

Timeseries Anomaly detection and Root Cause Analysis on data in SQL data warehouses and databases

Public Datasets Pipelines ⭐ 131

Cloud-native, data onboarding architecture for Google Cloud Datasets

Twitter For Bigquery ⭐ 130

Simplest way to get Tweets into BigQuery. Uses Google Cloud & App Engine, as well as Python and D3.

Cortex Data Foundation ⭐ 129

Data Foundation - Google Cloud Cortex Framework

Tensorflow Recommendation Wals ⭐ 126

An end-to-end solution for website article recommendations based on Google Analytics data. Uses WALS matrix-factorization in TensorFlow, trained on Cloud ML Engine. Recommendations served with App Engine Flex and Cloud Endpoints. Orchestration is performed using Airflow on Cloud Composer. See the solution tutorials at:

Tinyquery ⭐ 119

A Python in-memory test stub for BigQuery

Bigquery Ml Templates ⭐ 116

BigQuery ML SQL templates for common marketing use cases

Gcp Variant Transforms ⭐ 114

GCP Variant Transforms

Python Bigquery Dataframes ⭐ 112

BigQuery DataFrames

Megalista ⭐ 109

First Party data integration solution built for marketing teams to enable audience and conversion onboarding into Google Marketing products (Google Ads, Campaign Manager, Google Analytics).

Bigflow ⭐ 108

A Python framework for data processing on GCP.

Kubernetes Bigquery Python ⭐ 106

Example Kubernetes app that shows how to build a 'pipeline' to stream data into BigQuery. Uses Redis or Google Cloud PubSub

Dataproc Templates ⭐ 103

Dataproc templates and pipelines for solving simple in-cloud data tasks

Get All Hacker News Submissions Comments ⭐ 99

Simple Python scripts to download all Hacker News submissions and comments and store them in a PostgreSQL database.

Python Bigquery Storage ⭐ 95

Polygon Etl ⭐ 93

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

Opensourcecontributors ⭐ 90

Find all contributions for a user through the GitHub Archive

Premier League ⭐ 88

A Data Engineering project. Repository for backend infrastructure and Streamlit app files for a Premier League Dashboard.

Bigquery Examples ⭐ 85

Advanced BigQuery examples on genomic data.

Airbyte_serverless ⭐ 83

Airbyte made simple (no UI, no database, no cluster)

Bigquery_fdw ⭐ 81

BigQuery Foreign Data Wrapper for PostgreSQL

Final Project Level3 Recsys 01 ⭐ 81

🍁 AI를 활용한 메이플스토리 코디 추천 서비스👚👕

Stream raw hit-level Google Analytics data into BigQuery

Data Pipeline ⭐ 79

Data pipeline is a tool to run Data loading pipelines. It is an open sourced app engine app that users can extend to suit their own needs. Out of the box it will load files from a source, transform them and then output them (output might be writing to a file or loading them into a data analysis tool). It is designed to be modular and support various sources, transformation technologies and output types. The transformations can be chained together to form complex pipelines.

Gitmostwanted.com ⭐ 76

Advanced explorer of github.com

Bigquery_helper ⭐ 76

Helper class to simplify common read-only BigQuery tasks.

Dataflow Prediction Example ⭐ 73

Dbt Dry Run ⭐ 72

Dry run capability for dbt projects using BigQuery

Prism is the easiest way to develop, orchestrate, and execute data pipelines in Python.

Python library for API access and data analysis in Product, BI, Revenue Operations (GAM, GA, Athena etc.)

Bigquery View Analyzer ⭐ 66

CLI tool for managing + visualising BigQuery authorised views

An upgraded and improved version of the Iris automatic GCP-labeling project

Python utilities for working with FHIR, including libraries to build simple, flat FHIR views in BigQuery.

Ddlparse ⭐ 58

DDL parase and Convert to BigQuery JSON schema and DDL statements

Tensorflow Kubernetes Art Classification ⭐ 50

Train a TensorFlow model on Kubernetes to recognize art culture based on the collection from the Metropolitan Museum of Art

Mozilla Pipeline Schemas ⭐ 47

Schemas for Mozilla's data ingestion pipeline and data lake outputs

Python Bigquery Test Kit ⭐ 46

BigQuery test kit is a framework written in python that allows you to be more confident in your SQL and check that they are ready to prod. Rendering SQL template is part of this framework as well if you rely, for instance, on Airflow to orchestrate your jobs and their macros.

Gcp Census ⭐ 45

[DEPRECATED] GAE python based app which regularly collects information about GCP resources and stores them in BigQuery

Bigquery Integrations ⭐ 45

Import files (data) from Intercom, FTP(S), SFTP, MySQL, etc. servers into BigQuery.

Bigquery Bokeh Dashboard ⭐ 43

Datacatalog Tag Engine ⭐ 43

Tag Engine lets you automate the process of creating and populating metadata tags with Google Cloud's Data Catalog. Tag Engine is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.

Cloud Opensource Python ⭐ 42

Dependency Management Toolkit for Google Cloud Python Projects

dump raw log data from google analytics

Create backups of BigQuery datasets/tables

Lightweight configuration and access to multiple databases in a single project

Amora Data Build Tool ⭐ 37

Amora Data Build Tool enables analysts and engineers to transform data on the data warehouse (BigQuery) by writing Amora Models that describe the data schema using Python's "PEP484 - Type Hints" and select statements with SQLAlchemy. Amora is able to transform Python code into SQL data transformation jobs that run inside the warehouse.

Make_github_great_again ⭐ 35

See if people on github are influenced by Trump

End-to-end DataOps platform deployed by Terraform.

Cloud Data Lake ⭐ 34

Data lake, data warehouse on GCP

Dataflow Contact Center Speech Analysis ⭐ 34

Speech Analysis Framework, a collection of components and code from Google Cloud that you can use to transcribe audio files to create analytics.

Automatebot ⭐ 33

This bot will automate your reporting task by querying the data to bigquery, doing visualization, and automatically send the image through Telegram Chat.

Stackdriver Metrics Export ⭐ 33

Alligator2 ⭐ 32

A sample integration between the Google My Business API and the Cloud Natural Language API

Managed_ml_systems_and_iot ⭐ 32

Managed Machine Learning Systems and Internet of Things Live Lesson

Gcs Inventory Loader ⭐ 31

List or load your GCS bucket inventory into BigQuery fast with this tool.

Blast is a data orchestration tool that can run SQL and Python against Google BigQuery and Snowflake. It supports templating with Jinja, data quality tests, query validation, environment management and more.

(pronounced BACK-wup) BigQuery backup scripts

End To End Machine Learning With Google Cloud ⭐ 29

End to End Machine Learning with Google Cloud Platform

Debussy_concert ⭐ 29

Debussy is an opinionated Data Architecture and Engineering framework, enabling data analysts and engineers to build better platforms and pipelines.

dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.

글또의 요정 지니 프로젝트

Big_data_demystified ⭐ 28

Big Data Demystified meetup and blog examples

Target Bigquery ⭐ 26

A Singer (https://singer.io) target that writes data to Google BigQuery.

Edx2bigquery ⭐ 26

Tool to convert & load data from edX platform into BigQuery

Magic The Gathering ⭐ 25

A complete pipeline to pull data from Scryfall's "Magic: The Gathering"-API, via Prefect orchestration and DBT transformation.

Pypi Download Stats ⭐ 25

ABANDONED - Calculate detailed download stats and generate HTML and badges for PyPI packages

Stairlight ⭐ 25

A data lineage tool detects table dependencies from rendered SQL statements.

Bazel Bench ⭐ 25

Gsc Logger ⭐ 24

Google Search Console Logger for Google App Engine

A lightweight tool to fetch tables from BigQuery as pandas DataFrame very fast using BigQuery Storage API combined with multiprocessing

Related Searches

Python Django (28,897)

Python Machine Learning (20,195)

Python Flask (17,643)

Python Script (17,004)

Python Dataset (14,792)

Python Docker (14,113)

Python Tensorflow (13,736)

Python Command Line (13,351)

Python Deep Learning (13,092)

Python Jupyter Notebook (12,976)

1-100 of 235 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.