Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for sql etl
etl
x
sql
x
137 search results found
Tidb
⭐
35,604
TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://tidbcloud.com/free-trial
Doris
⭐
11,243
Apache Doris is an easy-to-use, high performance and unified analytics database.
Mage Ai
⭐
6,324
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
Steampipe
⭐
6,061
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
Cloudquery
⭐
5,380
The open source high performance data integration platform built for developers.
Ethereum Etl
⭐
2,760
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ
Quadratic
⭐
2,485
Quadratic | Data Science Spreadsheet with Python & SQL
Awesome Business Intelligence
⭐
1,862
Actively curated list of awesome BI tools. PRs welcome!
Dozer
⭐
1,367
Dozer is a real-time data platform for building, deploying and maintaining data products.
Peerdb
⭐
1,315
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
Pgsync
⭐
1,003
Postgres to Elasticsearch/OpenSearch sync
Data Engineering Wiki
⭐
934
The best place to learn data engineering. Built and maintained by the data engineering community.
Sqlmesh
⭐
931
SQLMesh is a data transformation framework that brings the benefits of DevOps to data teams. It enables data scientists, analysts, and engineers to efficiently run and deploy data transformations written in SQL or Python.
Dataform
⭐
757
Dataform is a framework for managing SQL based data operations in BigQuery, Snowflake, and Redshift
Metorikku
⭐
536
A simplified, lightweight ETL Framework based on Apache Spark
Automate Dv
⭐
435
A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
Etlalchemy
⭐
414
Extract, Transform, Load: Any SQL Database in 4 lines of Code.
Versatile Data Kit
⭐
389
One framework to develop, deploy and operate data workflows with Python and SQL.
Etl
⭐
367
Extract, Transform, and Load data with Ruby
Big_data_architect_skills
⭐
353
一个大数据架构师应该掌握的技能
Webkettle
⭐
350
基于web版kettle开发的一套分布式综合调度,管理,ETL开发的用户专业版B/S架构工具
Replicadb
⭐
304
ReplicaDB is open source tool for database replication, designed for efficiently transferring bulk data between relational and non-relational databases
Astro Sdk
⭐
303
Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
Cuelake
⭐
266
Use SQL to build ELT pipelines on a data lakehouse.
Bulk Writer
⭐
218
Provides guidance for fast ETL jobs, an IDataReader implementation for SqlBulkCopy (or the MySql or Oracle equivalents) that wraps an IEnumerable, and libraries for mapping entites to table columns.
Feldera
⭐
199
Feldera Continuous Analytics Platform
Dbt Coves
⭐
193
CLI tool for dbt users to simplify creation of staging models (yml and sql) files
Mara Example Project 2
⭐
175
An example mini data warehouse for python project stats, template for new projects
Neo4j Etl
⭐
168
Data import from relational databases to Neo4j.
Dbt Databricks
⭐
165
A dbt adapter for Databricks.
Steampipe Plugin Aws
⭐
165
Use SQL to instantly query AWS resources across regions and accounts. Open source CLI. No DB required.
Analytics Readings
⭐
155
Readings for Analytics Engineers
Ethereum Etl Postgres
⭐
136
ETL for moving Ethereum data to PostgreSQL database
Easy_sql
⭐
126
A library developed to ease the data ETL development process.
Etl
⭐
124
R package to facilitate ETL operations
Datachecks
⭐
117
Open Source Data Quality Monitoring.
Sayn
⭐
117
Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).
Peaks Consolidation
⭐
102
The Peaks Consolidation is equipped with state-of-the-art algorithms and data structures that support high-performance databending exercises. It specializes in management accounting and consolidation, with some special topics in machine learning and bioinformatics.
Patterns Devkit
⭐
101
Data pipelines from re-usable components
Locopy
⭐
99
locopy: Loading/Unloading to Redshift and Snowflake using Python.
Flowman
⭐
85
Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.
Go Etl
⭐
83
go-etl是一个集数据源抽取,转化,加载的工具集,提供强大的离线数据同步能力。
Etlhelper
⭐
81
ETL Helper is a Python ETL library to simplify data transfer into and out of databases.
Dataengineeringpilipinas
⭐
80
Data Engineering Pilipinas is a community for data engineers, data analysts, data scientists, developers, AI / ML engineers, and users of closed and open source data tools and methods / techniques in the Philippines. Data Engineering Pilipinas is a PyData group.
Projects
⭐
76
Sample projects using Ploomber.
Sqlbucket
⭐
67
Lightweight library to write, orchestrate and test your SQL ETL. Writing ETL with data integrity in mind.
Beneath
⭐
64
Beneath is a serverless real-time data platform ⚡️
Apachespark
⭐
59
This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.
Steampipe Plugin Github
⭐
58
Use SQL to instantly query repositories, users, gists and more from GitHub. Open source CLI. No DB required.
Getl
⭐
51
A tool for developing and testing ETL and ELT processes for automating the capture, delivery and processing of information in data warehouses on the MicroFocus Vertica platform.
Steampipe Plugin Kubernetes
⭐
41
Use SQL to instantly query Kubernetes API resources. Open source CLI. No DB required.
Steampipe Sqlite
⭐
39
Steampipe SQLite is a zero-ETL engine for SQLite. Virtual tables translate queries into live API calls for cloud services and APIs. Hundreds of plugins with thousands of documented examples.
Csv Cruncher
⭐
38
Treats CSV and JSON files as SQL tables, and exports SQL SELECTs back to CSV or JSON.
Projeto_etl_rfb_ibge_anp
⭐
38
PYTHON E POSTGRESQL - EXTRACT TRANSFORM LOAD - ETL - DADOS PÚBLICOS DA RECEITA FEDERAL DO BRASIL - RFB, INSTITUTO BRASILEIRO DE GEOGRAFIA E ESTATÍSTICA - IBGE E AGÊNCIA NACIONAL DO PETRÓLEO, GÁS NATURAL E BIOCOMBUSTÍVEIS - ANP - PYTHON E POSTGRESQL
Sope
⭐
37
Apache Spark ETL Utilities
Sharpetl
⭐
36
Write ETL using your favorite SQL dialects
Steampipe Plugin Gcp
⭐
36
Use SQL to instantly query GCP resources across regions, projects and organizations. Open source CLI. No DB required.
Tablite
⭐
36
multiprocessing enabled out-of-memory data analysis library for tabular data.
Etl Cdmbuilder
⭐
35
ETL-CDMBuilder is a repo containing a .NET Core application to perform ETL to OMOP CDM for multiple databases
Ether_sql
⭐
35
A python library to push ethereum blockchain data into an sql database.
Mara Etl Tools
⭐
33
Utilities for creating ETL pipelines with mara
Ides
⭐
32
智能数据探索服务(Intelligent Data Exploration Service),一站式Data + AI数据解决方案!
Blast
⭐
31
Blast is a data orchestration tool that can run SQL and Python against Google BigQuery and Snowflake. It supports templating with Jinja, data quality tests, query validation, environment management and more.
Steampipe Plugin Azure
⭐
30
Use SQL to instantly query Azure resources across regions and subscriptions. Open source CLI. No DB required.
Dbd
⭐
29
dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.
Steampipe Plugin Terraform
⭐
27
Use SQL to instantly query resources, data sources and more from Terraform code. Open source CLI. No DB required.
Datayoga
⭐
27
streaming data pipeline platform
Avro
⭐
27
Apache AVRO for go
Awesome Vertica
⭐
26
A curated list of awesome Vertica libraries, tools and resources
Fhir Pipe
⭐
25
Populate FHIR-compliant objects using SQL databases and processing rules
Steampipe Plugin Sdk
⭐
25
Steampipe Plugin SDK is a simple abstraction layer to write a Steampipe plugin. Plugins automatically work across all engine types including the Steampipe CLI, Postgres FDW, SQLite extension and the export CLI.
Sql_to_ibis
⭐
25
A Python package that parses sql and converts it to ibis expressions
Steampipe Plugin Googlesheets
⭐
23
Use SQL to instantly query spreadsheets, sheets, and cell data from Google Sheets. Open source CLI. No DB required.
Mongodb Rdbms Sync
⭐
23
Bi-directional data sync between Relational Databases (Oracle, MySql) & MongoDB.
Steampipe Plugin Virustotal
⭐
22
Use SQL to instantly query file, domain, URL and IP scanning results from VirusTotal.
Id3c
⭐
22
Data logistics system enabling real-time pathogen surveillance. Built for the Seattle Flu Study.
Sqloogle
⭐
21
Crawl, Index, and Search Your SQL.
Steampipe Plugin Net
⭐
20
Use SQL to instantly query DNS records, certificates and other network information. Open source CLI. No DB required.
Steampipe Plugin Jira
⭐
19
Use SQL to instantly query Jira. Open source CLI. No DB required.
Steampipe Plugin Whois
⭐
18
Use SQL to instantly query WHOIS. Open source CLI. No DB required.
Steampipe Plugin Csv
⭐
18
Use SQL to instantly query data from CSV files. Open source CLI. No DB required.
Steampipe Plugin Oci
⭐
18
Use SQL to instantly query Oracle Cloud resources across regions and accounts. Open source CLI. No DB required.
Sql To Redis
⭐
17
Convert SQL tables to Redis as JSON
Sync Engine Example
⭐
17
Synchronization Algorithm Exploration: Techniques to synchronize a SQL database with external destinations.
Helium Etl Queries
⭐
17
A collection of SQL views used to enrich data produced by a Helium blockchain-etl
Airflowetl
⭐
16
Blog post on ETL pipelines with Airflow
Spark Etl
⭐
15
Set of ETL utils for Spark
Hadoop Data Ingestion Tool
⭐
15
OLAP and ETL of Big Data
Automated_etl_google_cloud Social_dashboard
⭐
15
A dashboard is worth a thousand words => https://datastudio.google.com/reporting/755f3183-d
Steampipe Plugin Hackernews
⭐
14
Use SQL to instantly query stories, users and other items from Hacker News. Open source CLI. No DB required.
Steampipe Plugin Code
⭐
14
Use SQL to instantly query secrets and more from source code. Open source CLI. No DB required.
Steampipe Plugin Reddit
⭐
14
Use SQL to instantly query Reddit posts, comments & more. Open source CLI. No DB required.
Cubetl
⭐
14
CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python
Postgresql To Mssql
⭐
14
Migrate postgresql data to sql server on the fly!
Sheetwork
⭐
14
A handy package to load Google Sheets to your database right from the CLI and with easy configuration via YAML files.
Bootcamp Igti Analista De Dados
⭐
13
Bootcamp online analista de dados disponibilizado pelo IGTI – Instituto de Gestão e Tecnologia da Informação
Steampipe Plugin Googleworkspace
⭐
13
Use SQL to instantly query calendar events, drive files, gmail messages, and more from Google Workspace. Open source CLI. No DB required.
Datalink
⭐
13
简单易用的ETL工具
Steampipe Plugin Twitter
⭐
13
Use SQL to instantly query tweets, users and followers from Twitter. Open source CLI. No DB required.
Hands On Data Warehousing With Azure Data Factory
⭐
12
Hands-On Data Warehousing with Azure Data Factory, published by Packt
Related Searches
Database Sql (5,501)
Python Sql (3,922)
Mysql Sql (2,867)
Java Sql (2,781)
Javascript Sql (2,662)
C Sharp Sql (2,429)
Postgresql Sql (2,411)
Php Sql (2,276)
Golang Sql (1,383)
Sql Table (1,358)
1-100 of 137 search results
Next >
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.