Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for elt
elt
x
66 search results found
Airflow
⭐
34,468
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Airbyte
⭐
12,918
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
Doris
⭐
11,243
Apache Doris is an easy-to-use, high performance and unified analytics database.
Dbt Core
⭐
8,985
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
Seatunnel
⭐
7,139
SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
Mage Ai
⭐
6,324
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
Cloudquery
⭐
5,380
The open source high performance data integration platform built for developers.
Kestra
⭐
5,257
Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
Meltano
⭐
1,460
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
Dlt
⭐
1,069
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
Sqlmesh
⭐
931
SQLMesh is a data transformation framework that brings the benefits of DevOps to data teams. It enables data scientists, analysts, and engineers to efficiently run and deploy data transformations written in SQL or Python.
Dataform
⭐
757
Dataform is a framework for managing SQL based data operations in BigQuery, Snowflake, and Redshift
Optimus
⭐
707
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
Kuwala
⭐
610
Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data science models and products with a focus on geospatial data. Currently, the following data connectors are available worldwide: a) High-resolution demograp
Transfer
⭐
495
Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.
Automate Dv
⭐
456
A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
Versatile Data Kit
⭐
389
One framework to develop, deploy and operate data workflows with Python and SQL.
Dbt Metabase
⭐
383
dbt + Metabase integration
Replicadb
⭐
304
ReplicaDB is open source tool for database replication, designed for efficiently transferring bulk data between relational and non-relational databases
Astro Sdk
⭐
303
Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
Nango Sync
⭐
294
Sync external APIs to your DB, fast.
Cuelake
⭐
266
Use SQL to build ELT pipelines on a data lakehouse.
Dbt Coves
⭐
193
CLI tool for dbt users to simplify creation of staging models (yml and sql) files
Aws Etl Orchestrator
⭐
185
A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.
Reddit Detective
⭐
160
Play detective on Reddit: Discover political disinformation campaigns, secret influencers and more
Sayn
⭐
117
Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).
Airbyte Connectors
⭐
90
Airbyte connectors (sources & destinations) + Airbyte CDK for JavaScript/TypeScript
Sling Cli
⭐
84
Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
Airbyte_serverless
⭐
83
Airbyte made simple (no UI, no database, no cluster)
Dbt Sqlite
⭐
59
A SQLite adapter plugin for dbt (data build tool)
Drivers
⭐
53
Low-code Python library enabling access to APIs, tools, data sources in seconds.
Getl
⭐
51
A tool for developing and testing ETL and ELT processes for automating the capture, delivery and processing of information in data warehouses on the MicroFocus Vertica platform.
Alphasql
⭐
39
AlphaSQL provides Integrated Type and Schema Check and Parallelization for SQL file set mainly for BigQuery
Datasphere Integration
⭐
38
an data-centric integration platform
Amora Data Build Tool
⭐
37
Amora Data Build Tool enables analysts and engineers to transform data on the data warehouse (BigQuery) by writing Amora Models that describe the data schema using Python's "PEP484 - Type Hints" and select statements with SQLAlchemy. Amora is able to transform Python code into SQL data transformation jobs that run inside the warehouse.
Wikirepo
⭐
36
Python based Wikidata framework for easy dataframe extraction
Dbd
⭐
29
dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.
Datayoga
⭐
27
streaming data pipeline platform
Dbt Firebolt
⭐
26
The dbt adapter for Firebolt
Arthur Redshift Etl
⭐
25
ELT Code for your Data Warehouse
Databricks Notebooks
⭐
22
Collection of Sample Databricks Spark Notebooks ( mostly for Azure Databricks )
Spark Movies Etl
⭐
21
Spark data pipeline that ingests and transforms movie ratings data.
Plugin Sdk
⭐
20
CloudQuery Go SDK for source and destination plugins
Cq Source Sharepoint
⭐
18
🔌 CloudQuery SharePoint Source Plugin
Rivery_cli
⭐
17
Rivery CLI
Dbt Teradata
⭐
16
dbt adapter for Teradata
Ghcn D
⭐
14
Data Pipeline from the Global Historical Climatology Network DataSet
Recce
⭐
12
PR review tool designed for DBT projects
Tap Dbt
⭐
12
Singer Tap for dbt API v2 built with the Meltano SDK
Data Brewery
⭐
12
Data Brewery is an ETL (Extract-Transform-Load) program that connect to many data sources (cloud services, databases, ...) and manage data warehouse workflow.
Singer Working Group
⭐
12
Working group for ongoing development and iteration of the Singer Spec, the de-facto protocol for open source data connectors. Please use "Issues" to create discussion items - or use "Discussions" for general questions.
Airflowjob
⭐
11
Airflow POC demo : 1) env set up 2) airflow DAG 3) Spark/ML pipeline | #DE
Greatex
⭐
10
A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in Airflow.
Nl_parser_using_spacy
⭐
9
NLP parser using NER and TDD
Elt Framework
⭐
8
Extract Load Transform (ELT) framework is a metadata based batch orchestration framework for modern data platforms. Implemented using Azure PaaS data services. Common ingestion and transformation patterns available out of box. Reusable code can be easily extended to cater to custom patterns.
Tap Dbt Artifacts
⭐
8
Singer Tap for dbt Artifacts built with the Meltano SDK
Meltano On Github Actions
⭐
6
Cookiecutter template for creating GitHub Actions orchestrated Meltano projects
Shift
⭐
5
Shift is a high performance better alternative to Airbyte, Singer, Meltano
Eruptr
⭐
5
Don't ETL or ELT. LET your data be free.
Analytics_data_where_house
⭐
5
An analytics engineering sandbox focusing on real estates prices in Cook County, IL
Trusted Data Pipeline
⭐
5
Building 3D Trusted Data Pipelines With Dagster, Dbt, and Duckdb
Yahoo Finance Warehouse
⭐
5
ELT for yahoo finance API
Bids2table
⭐
5
Efficiently index large-scale BIDS neuroimaging datasets and derivatives
Target Elasticsearch
⭐
5
A Meltano target for Elasticsearch
Meltano Dogfood
⭐
5
Personal dogfood Meltano project
Doris Sdk
⭐
5
SDK for Apache Doris
1-66 of 66 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.