Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Pipeline Data Flow Open Source Projects

Open source projects categorized as Pipeline Data Flow
Categories > Control Flow > Data Flow
Categories > Data Processing > Pipeline
Beam ⭐ 7,355
Apache Beam is a unified programming model for Batch and Streaming data processing.
dependent packages 14total releases 568most recent commit 3 months ago
Professional Services ⭐ 2,635
Common solutions and tools developed by Google Cloud's Professional Services team. This repository and its contents are not an officially supported Google product.
most recent commit 3 months ago
Nextflow ⭐ 2,447
A DSL for data-driven computational pipelines
dependent packages 16total releases 250most recent commit 3 months ago
Datakit ⭐ 1,044
Connect processes into powerful data pipelines with a simple git-like filesystem interface
most recent commit 2 years ago
Scipipe ⭐ 1,010
Robust, flexible and resource-efficient pipelines using Go and the commandline
total releases 47most recent commit 9 months ago
Dataflowjavasdk ⭐ 853
Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines.
dependent packages 14total releases 38most recent commit 3 years ago
Purefunctionpipelinedataflow ⭐ 436
My Blog: The Math-based Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with principle-based Warehouse/Workshop Model
most recent commit 4 months ago
Blaze ⭐ 344
Blazing fast NIO microframework and Http Parser
dependent packages 19total releases 52most recent commit 5 months ago
Dffml ⭐ 235
The easiest way to use Machine Learning. Mix and match underlying ML libraries and data set sources. Generate new datasets or modify existing ones with ease.
dependent packages 22total releases 17most recent commit 3 months agopypi dffml} Downloads
Dspatch ⭐ 194
The Refreshingly Simple Cross-Platform C++ Dataflow / Patching / Pipelining / Graph Processing / Stream Processing / Reactive Programming Framework
most recent commit 3 months ago
Dataflow ⭐ 169
Efficient Data Loading Pipeline in Pure Python
most recent commit 4 years ago
Dataflowpythonsdk ⭐ 157
Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines.
most recent commit 7 years ago
Dataflowsdk Examples ⭐ 148
Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines. This repository hosts a few example pipelines to get you started with Dataflow.
most recent commit 6 years ago
Greenpipes ⭐ 143
An asynchronous pipe implementation for the Task Parallel Library
most recent commit a year ago
Awesome Beam ⭐ 130
A curated list of awesome resources for Apache Beam
most recent commit a year ago
Mlr3pipelines ⭐ 127
Dataflow Programming for Machine Learning in R
most recent commit 3 months ago
Eventkit ⭐ 119
Event-driven data pipelines
dependent packages 6total releases 12most recent commit 4 months agopypi eventkit} Downloads
Gcp Variant Transforms ⭐ 114
GCP Variant Transforms
most recent commit 2 years ago
Intelmq Manager ⭐ 101
IntelMQ Manager is a graphical interface to manage configurations for IntelMQ framework.
total releases 9most recent commit 9 months ago
Dataflow Ops ⭐ 97
Project demonstrating how to automate Prefect 2.0 deployments to AWS ECS Fargate
most recent commit 9 months ago
Thurber ⭐ 95
Clojure ++ Apache Beam ++ Google Cloud Dataflow
total releases 31most recent commit a year ago
Dlp Dataflow Deidentification ⭐ 80
Multi Cloud Data Tokenization Solution By Using Dataflow and Cloud DLP
most recent commit 4 months ago
Automaticanalysis ⭐ 70
Automatic Analysis (aa)
most recent commit 4 months ago
The 10 Most Depended On Pipeline Data Flow Open Source Projects
Blaze ⭐ 344
Blazing fast NIO microframework and Http Parser
dependent packages 19total releases 52latest release October 05, 2023most recent commit 5 months ago
Nextflow ⭐ 2,447
A DSL for data-driven computational pipelines
dependent packages 16total releases 250latest release November 24, 2023most recent commit 3 months ago
Beam ⭐ 7,355
Apache Beam is a unified programming model for Batch and Streaming data processing.
dependent packages 14total releases 568latest release November 13, 2023most recent commit 3 months ago
Oi ⭐ 12
Compile your pipeline once, run anywhere
dependent packages 9total releases 94latest release December 03, 2021most recent commit 2 years ago
Eventkit ⭐ 119
Event-driven data pipelines
dependent packages 6total releases 12latest release August 24, 2023most recent commit 4 months agopypi eventkit} Downloads
Swint Pipe ⭐ 22
Pipeline logic flow for Swint
dependent packages 2total releases 17latest release March 05, 2019most recent commit 5 years agonpm swint-pipe} Downloads
Nuts Flow ⭐ 17
A simple dataflow framework in Python
dependent packages 1total releases 44latest release January 06, 2021most recent commit 3 years agopypi nutsflow} Downloads
The 10 Latest Releases In Pipeline Data Flow Open Source Projects
Nextflow ⭐ 2,447
A DSL for data-driven computational pipelines
dependent packages 16total releases 250latest release November 24, 2023most recent commit 3 months ago
Beam ⭐ 7,355
Apache Beam is a unified programming model for Batch and Streaming data processing.
dependent packages 14total releases 568latest release November 13, 2023most recent commit 3 months ago
Blaze ⭐ 344
Blazing fast NIO microframework and Http Parser
dependent packages 19total releases 52latest release October 05, 2023most recent commit 5 months ago
Eventkit ⭐ 119
Event-driven data pipelines
dependent packages 6total releases 12latest release August 24, 2023most recent commit 4 months agopypi eventkit} Downloads
Intelmq Manager ⭐ 101
IntelMQ Manager is a graphical interface to manage configurations for IntelMQ framework.
total releases 9latest release July 19, 2023most recent commit 9 months ago
Oi ⭐ 12
Compile your pipeline once, run anywhere
dependent packages 9total releases 94latest release December 03, 2021most recent commit 2 years ago
Scipipe ⭐ 1,010
Robust, flexible and resource-efficient pipelines using Go and the commandline
total releases 47latest release October 14, 2021most recent commit 9 months ago
Nuts Flow ⭐ 17
A simple dataflow framework in Python
dependent packages 1total releases 44latest release January 06, 2021most recent commit 3 years agopypi nutsflow} Downloads
Thurber ⭐ 95
Clojure ++ Apache Beam ++ Google Cloud Dataflow
total releases 31latest release December 12, 2020most recent commit a year ago
Popular Control Flow Categories
Stream
Thread
Promise
Series
Webhook
Daemon
Parallel
Reactive
Rxjava
Cron
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.