Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for amazon web services etl
amazon-web-services
x
etl
x
3 search results found
Steampipe
⭐
6,061
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
Cloudquery
⭐
5,380
The open source high performance data integration platform built for developers.
Aws Sdk Pandas
⭐
3,813
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Ethereum Etl
⭐
2,760
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ
Aws Glue Samples
⭐
1,334
AWS Glue code samples
Aws Glue Libs
⭐
568
AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
Redun
⭐
464
Yet another redundant workflow engine
Aws Serverless Data Lake Framework
⭐
379
Enterprise-grade, production-hardened, serverless data lake on AWS
Data Engineering Projects
⭐
322
Personal Data Engineering Projects
Beginner_de_project
⭐
276
Beginner data engineering project - batch edition
Substation
⭐
242
Substation is a cloud-native, event-driven data pipeline toolkit built for security teams.
Aws Etl Orchestrator
⭐
185
A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.
Steampipe Plugin Aws
⭐
165
Use SQL to instantly query AWS resources across regions and accounts. Open source CLI. No DB required.
Aws Ecs Airflow
⭐
110
Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks
Locopy
⭐
99
locopy: Loading/Unloading to Redshift and Snowflake using Python.
Data Engineering Nanodegree
⭐
76
Projects done in the Data Engineering Nanodegree by Udacity.com
Luigi Warehouse
⭐
73
A luigi powered analytics / warehouse stack
Zeus
⭐
61
Zeus + SciFi = the power of the gods, meets the power of science fiction. Designing wisdom into intelligence, through intelligent design.
Rony
⭐
56
Data Engineering made simple - An opinionated Data Engineering framework
Etlflow
⭐
43
EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for running complex Auditable workflows which can interact with Google Cloud Platform, AWS, Kubernetes, Databases, SFTP servers, On-Prem Systems and more.
Udacity Data Engineering
⭐
42
Udacity Data Engineering Nano Degree (DEND)
Steampipe Sqlite
⭐
39
Steampipe SQLite is a zero-ETL engine for SQLite. Virtual tables translate queries into live API calls for cloud services and APIs. Hundreds of plugins with thousands of documented examples.
Yaetos
⭐
32
Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified
Aws Auto Terminate Idle Emr
⭐
26
AWS Auto Terminate Idle AWS EMR Clusters Framework is an AWS based solution using AWS CloudWatch and AWS Lambda using a Python script that is using Boto3 to terminate AWS EMR clusters that have been idle for a specified period of time.
Arthur Redshift Etl
⭐
25
ELT Code for your Data Warehouse
Hotsub
⭐
23
Command line tool to run batch jobs concurrently with ETL framework on AWS or other cloud computing resources
Redshift Ruby Tutorial
⭐
23
Using AWS Redshift and Ruby to setup your data warehouse
Nodestream
⭐
23
A Fast, Declarative, and Extensible ETL Framework for Graph Databases.
Aws Glue Docker
⭐
22
🐋 Docker image for AWS Glue Spark/Python
Irs990
⭐
21
ETL toolkit for 2.5 million electronic nonprofit tax returns released by the IRS.
Taskflow
⭐
20
An advanced yet simple system to run your background tasks and workflows
Aws_glue_etl_docker
⭐
20
Helper library to run AWS Glue ETL scripts docker container for local testing of development in a Jupyter notebook
Amazon S3 Step Functions Ingestion Orchestration
⭐
19
Design pattern for orchestrating an incremental data ingestion pipeline using AWS Step Functions from an on premise location into an Amazon S3 datalake bucket
Covalent Aws Plugins
⭐
16
Executor plugins interfacing Covalent with various AWS compute platforms
Dracula Covid19
⭐
16
An ETL tool for converting untyped CSV to parquet. Also triggers data lake updates.
Terraform Aws Glue
⭐
12
Terraform modules for provisioning and managing AWS Glue resources
Aws Glue Test Data Generator
⭐
12
AWS Glue Configurable Test Data Generator for S3 Data Lakes and DynamoDB
Serverless Python Workflow With Aws Lambda
⭐
11
A tutorial to setup and deploy a simple Serverless Python workflow with REST API endpoints in AWS Lambda.
Handoff
⭐
11
Single command serverless ETL orchestration.
Data Engineering Onboarding Starter
⭐
8
This repository contains a 10 step program to enter the world of Data Engineering
Aws Etl
⭐
7
This is an ETL application on AWS with general open sales and customer data that you can find here: https://github.com/camposvinicius/data/blob/main/A it's a zipped file with some .csvs inside that we will apply transformations.
Aws_glueetl_workshop
⭐
6
AWS_GlueETL_workshop
Umbrella
⭐
6
Minimal ETL (Python) framework that runs on AWS Lambda
Spark Databricks
⭐
6
🔥 Master Apache Spark & Databricks! Dive into a world of big data with exclusive insights from Udemy courses, personal notes, and practical guides. Whether you're starting out or scaling new heights in data engineering, this is your ultimate resource hub! 🌟🚀
Data Engineer Portfolio
⭐
6
This is a repository to demonstrate my details, skills, projects and to keep track of my progression in Data Analytics and Data Science topics.
Aws Redshift Matillion Workshop
⭐
5
Scripts, Instructions and Materials for AWS Redshift and Matillion ETL workshop
Udacity Data Engineering Nanodegree
⭐
5
This is a repository to hold the files and notebooks produced throughout my Udacity's Nanodegree Data Engineering program.
Stock Market Real Time Data Pipeline With Apache Kafka And Cassandra
⭐
5
A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apache Kafka and stored in a local Cassandra database.
Nutchpighive
⭐
5
crawl GooglePlay data with Nutch, ETL with Pig, analyze with Hive
Torianik Music Etl
⭐
5
ETL Pipeline for transforming 1 million Spotify playlists dataset into SQL database.
Related Searches
Python Amazon Web Services (7,973)
Amazon Web Services Lambda Functions (7,452)
Amazon Web Services Terraform (4,243)
Amazon Web Services Serverless (4,018)
Amazon Web Services Hcl (3,473)
Golang Amazon Web Services (2,930)
Docker Amazon Web Services (2,864)
Amazon Web Services Aws Lambda (2,628)
Amazon Web Services Cloudformation (2,431)
Typescript Amazon Web Services (2,319)
1-3 of 3 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.