Awesome Open Source

Programming Languages

Search results for pipeline s3

20 search results found

Airbyte ⭐ 12,918

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

Goodreads_etl_pipeline ⭐ 593

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Datajoint Python ⭐ 158

Relational data pipelines for the science lab

Aws Pdf Textract Pipeline ⭐ 148

🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript

Rds Snapshot Export To S3 Pipeline ⭐ 102

RDS Snapshot Export to S3 Pipeline

Scrapy S3pipeline ⭐ 66

Scrapy pipeline to store chunked items into Amazon S3 or Google Cloud Storage bucket.

Automating Livestream Video Monitoring ⭐ 34

This repo presents a demo application for realtime livestream video quality monitoring using AWS serverless and AI/ML services.

Deploy a secured, clustered, auto-scaling NiFi service in AWS.

Spark Movies Etl ⭐ 21

Spark data pipeline that ingests and transforms movie ratings data.

Saisoku is a Python module that helps you build complex pipelines of batch file/directory transfer/sync jobs.

Nyc_taxi_pipeline ⭐ 12

Design/Implement stream/batch architecture on NYC taxi data | #DE

Jenkins Pipeline Cache Plugin ⭐ 11

A cloud native file cache for Jenkins build pipelines which uses an S3-Bucket as storage provider.

Cwlogs S3 ⭐ 11

Gem to download CloudWatch logs and upload to S3 for processing.

Terraform Static Site Pipeline ⭐ 10

Easy (🌴 🍺) and cheap (💸) way to deploy and scale frontend sites on AWS with Terraform 🌎

Memsql Loader ⭐ 9

Deprecated - Check out MemSQL Pipelines instead!

Go Triton ⭐ 9

Kinesis-backed Data Pipeline

Insight Zone Defense ⭐ 8

One-click automation of big data pipeline with monitoring

Aws S3 Orb ⭐ 7

Integrate Amazon AWS S3 with your CircleCI CI/CD pipeline easily with the aws-s3 orb.

Vela S3 Cache ⭐ 6

S3 Cache Plugin for Vela (Target's official Pipeline Automation Framework)

Datapipeline ⭐ 6

Faas Flow Minio Datastore ⭐ 6

faasflow DataStore implementation that uses either minio DB or amazon s3

Kubernetes Operator Roiergasias ⭐ 5

'Roiergasias' kubernetes operator is meant to address a fundamental requirement of any data science / machine learning project running their pipelines on Kubernetes - which is to quickly provision a declarative data pipeline (on demand) for their various project needs using simple kubectl commands. Basically, implementing the concept of No Ops. The fundamental principle is to utilise best of docker, kubernetes and programming language features to run a workflow with minimal workflow definition s

AWS Event Pipeline Service

Related Searches

Python Pipeline (4,199)

Amazon Web Services S3 (2,551)

Javascript Pipeline (1,369)

Javascript S3 (1,284)

Python S3 (1,171)

Pipeline Jenkins (1,150)

Shell Pipeline (1,143)

Amazon S3 (1,059)

Docker Pipeline (1,018)

Jupyter Notebook Pipeline (976)

1-20 of 20 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.