Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 19 Data Engineering Pipeline Open Source Projects

Open source projects categorized as Data Engineering Pipeline
Categories > Data Engineering Pipeline
Udacity Data Engineering Projects ⭐ 1,335
Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
most recent commit 2 years ago
Goodreads_etl_pipeline ⭐ 593
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
most recent commit 4 years ago
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181most recent commit 3 months agopypi vdk-core} Downloads
Movalytics Data Warehouse ⭐ 114
Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow
most recent commit 4 years ago
Dataflow Ops ⭐ 97
Project demonstrating how to automate Prefect 2.0 deployments to AWS ECS Fargate
most recent commit 9 months ago
Udacity Data Engineer Nanodegree ⭐ 64
Classwork projects and home works done through Udacity data engineering nano degree
most recent commit 4 months ago
Prefect Deployment Patterns ⭐ 48
Code examples showing flow deployment to various types of infrastructure
most recent commit a year ago
Marshmallow Pyspark ⭐ 12
Marshmallow serializer integration with pyspark
total releases 5most recent commit 4 months ago
Social Media Analysis ⭐ 11
Social Media Analysis, scalable solution, flexible deployment that analyses social media contents
most recent commit a year ago
Business_closures_de_pipeline ⭐ 10
Data Engineering pipeline hosted entirely in the AWS ecosystem utilizing DocumentDB as the database
most recent commit 2 years ago
Predicting Retail Churn With Azure Ml Studio ⭐ 9
Challenge to job: Data Scientist
most recent commit 2 years ago
Disaster Response Pipeline ⭐ 7
ETL pipeline combined with supervised learning and grid search to classify text messages sent during a disaster event
most recent commit 5 years ago
F1 Data Pipeline ⭐ 7
F1 Data Pipeline
most recent commit 10 months ago
Av_ultimate_student_hunt ⭐ 7
Solution for the Ultimate Student Hunt Challenge (1st place).
most recent commit 2 years ago
Tweetanalyzer ⭐ 6
An environment for analyzing Twitter
most recent commit 2 years ago
Nyc Stats ⭐ 6
Analysis of 311 Service Requests for the City of NYC (from 2010 to 2023) Tech: Prefect cloud, dbt core, BigQuery, Compute Engine, CloudRun, Artifact Registry, Terraform, Docker
most recent commit a year ago
Stock Market Real Time Data Pipeline With Apache Kafka And Cassandra ⭐ 5
A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apache Kafka and stored in a local Cassandra database.
most recent commit 10 months ago
Data Engineering Project With Hdfs And Kafka ⭐ 5
Data Engineering Project with Hadoop HDFS and Kafka
most recent commit 5 months ago
Shift ⭐ 5
Shift is a high performance better alternative to Airbyte, Singer, Meltano
most recent commit 6 months ago
The 2 Latest Releases In Data Engineering Pipeline Open Source Projects
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181latest release November 28, 2023most recent commit 3 months agopypi vdk-core} Downloads
Marshmallow Pyspark ⭐ 12
Marshmallow serializer integration with pyspark
total releases 5latest release November 11, 2022most recent commit 4 months ago
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.