Awesome Open Source

Programming Languages

Search results for amazon web services data engineering

amazon-web-services x

data-engineering x

0 search results found

Cloudquery ⭐ 5,380

The open source high performance data integration platform built for developers.

Aws Sdk Pandas ⭐ 3,779

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

Udacity Data Engineering Projects ⭐ 1,335

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Data Engineering Interview Questions ⭐ 554

More than 2000+ Data engineer interview questions.

Yet another redundant workflow engine

Learn Something Every Day ⭐ 409

📝 A compilation of everything that I learn; Computer Science, Software Development, Engineering, Math, and Coding in General. Read the rendered results here ->

Aws Serverless Data Lake Framework ⭐ 379

Enterprise-grade, production-hardened, serverless data lake on AWS

Everything Tech ⭐ 372

A collection of online resources to help you on your Tech journey.

Data Engineering Projects ⭐ 322

Personal Data Engineering Projects

Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!

Substation ⭐ 242

Substation is a cloud-native, event-driven data pipeline toolkit built for security teams.

Aws Ddk ⭐ 233

An open source development framework to help you build data workflows and modern data architecture on AWS.

Phidata ⭐ 220

Build AI Assistants using function calling

Aws Orbit Workbench ⭐ 127

A Data Platform built for AWS, powered by Kubernetes.

Dataflow Ops ⭐ 97

Project demonstrating how to automate Prefect 2.0 deployments to AWS ECS Fargate

Airflow Autoscaling Ecs ⭐ 87

Airflow Deployment on AWS ECS Fargate Using Cloudformation

Data Engineering Nanodegree ⭐ 76

Projects done in the Data Engineering Nanodegree by Udacity.com

Ansible Playbook ⭐ 59

Ansible playbook to deploy distributed technologies

Data Engineering made simple - An opinionated Data Engineering framework

Towardsdataengineering ⭐ 52

This repo contains commands that data engineers use in day to day work.

Prefect Deployment Patterns ⭐ 48

Code examples showing flow deployment to various types of infrastructure

Sageworks ⭐ 36

SageWorks: An easy to use Python API for creating and deploying SageMaker Models

Uber Expenses Tracking ⭐ 35

The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such as Apache Airflow, AWS Redshift and Power BI.

Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified

Arthur Redshift Etl ⭐ 25

ELT Code for your Data Warehouse

Audiophile E2e Pipeline ⭐ 24

Pipeline that extracts data from Crinacle's Headphone and InEarMonitor databases and finalizes data for a Metabase Dashboard.

Serverless Datahub ⭐ 24

🎯 Building Scalable Cloud-Native DataHub Serverless Application ⛅🚀

Nodestream ⭐ 23

A Fast, Declarative, and Extensible ETL Framework for Graph Databases.

Jobanalytics_and_search ⭐ 22

JobAnalytics system consumes data from multiple sources and provides valuable information to both job hunters and recruiters.

Aws Glue Docker ⭐ 22

🐋 Docker image for AWS Glue Spark/Python

Pyspark On Aws Emr ⭐ 13

The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on writing pyspark code.

Bootcamp_data Engineering ⭐ 13

Bootcamp to learn basics in Data Engineering

Data Engineering: Chapter 5 aws chapter for pragmatic ai. Creates an "real world" Data Engineering API using Flask,Click, Pandas and Swagger docs

Business_closures_de_pipeline ⭐ 10

Data Engineering pipeline hosted entirely in the AWS ecosystem utilizing DocumentDB as the database

Clusterless ⭐ 8

Clusterless is a tool for scheduling decentralized, scalable, and secure data pipelines for continuously arriving data, across clouds.

Data Ml Engineering Career Path ⭐ 8

Devops Mlops ⭐ 8

Tools for DevOps and MLOps. Materials and projects. New technologies and infrastructure review.

Data Engineering Onboarding Starter ⭐ 8

This repository contains a 10 step program to enter the world of Data Engineering

Kafka-like functionality in AWS Serverless Cloud

Data Engineering ⭐ 7

Code for my blogs on Data Engineering

Spark Databricks ⭐ 6

🔥 Master Apache Spark & Databricks! Dive into a world of big data with exclusive insights from Udemy courses, personal notes, and practical guides. Whether you're starting out or scaling new heights in data engineering, this is your ultimate resource hub! 🌟🚀

Data Engineer Portfolio ⭐ 6

This is a repository to demonstrate my details, skills, projects and to keep track of my progression in Data Analytics and Data Science topics.

Software Architect Mindmap ⭐ 6

🧠Mindmap of 🗺️Software Architecture, Software engineering: An Overview of Software Terminologies and Concepts.

Dataengineering Youtube Project ⭐ 6

Data Engineering Youtube Project

Udacity Data Engineering Nanodegree ⭐ 5

This is a repository to hold the files and notebooks produced throughout my Udacity's Nanodegree Data Engineering program.

Airflow Terraform ⭐ 5

Easily deploy airflow infrastructure on an AWS VPC using terraform.

Docker_spark_history_ui ⭐ 5

A dockerised version of the spark history server which enables us to access metrics in the spark ui from a log generated by AWS glue

Tessellate ⭐ 5

A data engineering cli for reading and writing data to/from multiple locations across multiple formats.

Stock Market Real Time Data Pipeline With Apache Kafka And Cassandra ⭐ 5

A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apache Kafka and stored in a local Cassandra database.

1-0 of 0 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.