Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Pipeline Apache Open Source Projects

Open source projects categorized as Pipeline Apache
Categories > Web Servers > Apache
Categories > Data Processing > Pipeline
Beam ⭐ 7,355
Apache Beam is a unified programming model for Batch and Streaming data processing.
dependent packages 14total releases 568most recent commit 3 months ago
Dataflowjavasdk ⭐ 853
Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines.
dependent packages 14total releases 38most recent commit 3 years ago
Hop ⭐ 802
Hop Orchestration Platform
most recent commit 3 months ago
Data Pipelines With Apache Airflow ⭐ 587
Code for Data Pipelines with Apache Airflow
most recent commit 4 months ago
Piper ⭐ 480
piper - a distributed workflow engine
most recent commit 8 months ago
Tez ⭐ 446
Apache Tez
most recent commit 3 months ago
Sparkflow ⭐ 301
Easy to use library to bring Tensorflow on Apache Spark
total releases 13most recent commit 6 months ago
Jpmml Sparkml ⭐ 265
Java library and command-line application for converting Apache Spark ML pipelines to PMML
total releases 127most recent commit 4 months ago
Whirl ⭐ 183
Fast iterative local development and testing of Apache Airflow workflows
most recent commit 3 months ago
Airflow_for_beginners ⭐ 166
most recent commit 2 years ago
Dataflowpythonsdk ⭐ 157
Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines.
most recent commit 7 years ago
Dataflowsdk Examples ⭐ 148
Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines. This repository hosts a few example pipelines to get you started with Dataflow.
most recent commit 6 years ago
Envelope ⭐ 133
Build configuration-driven ETL pipelines on Apache Spark
most recent commit 4 years ago
Incubator Liminal ⭐ 131
Apache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.
total releases 28most recent commit 8 months ago
Awesome Beam ⭐ 130
A curated list of awesome resources for Apache Beam
most recent commit a year ago
Crunch ⭐ 100
Mirror of Apache Crunch (Incubating)
most recent commit 3 years ago
Falcon ⭐ 95
Mirror of Apache Falcon
most recent commit 5 years ago
Flink Connectors ⭐ 93
Apache Flink connectors for Pravega.
total releases 9most recent commit 6 months ago
Pyspark2pmml ⭐ 93
Python library for converting Apache Spark ML pipelines to PMML
dependent packages 2total releases 2most recent commit 4 months agopypi pyspark2pmml} Downloads
Aws Concurrent Data Orchestration Pipeline Emr Livy ⭐ 66
This code demonstrates the architecture featured on the AWS Big Data blog (https://aws.amazon.com/blogs/big-data/ ) which creates a concurrent data pipeline by using Amazon EMR and Apache Livy. This pipeline is orchestrated by Apache Airflow.
most recent commit 5 years ago
Data Stream Development With Apache Spark Kafka And Spring Boot ⭐ 54
Data Stream Development with Apache Spark, Kafka and Spring Boot by Packt Publishing
most recent commit a year ago
Lighthouse ⭐ 54
Lighthouse is a library for data lakes built on top of Apache Spark. It provides high-level APIs in Scala to streamline data pipelines and apply best practices.
dependent packages 3total releases 10most recent commit 3 years ago
Deep Learning Pyspark ⭐ 40
Deep Learning with Apache Spark and Deep Cognition
most recent commit 6 years ago
The 4 Most Depended On Pipeline Apache Open Source Projects
Beam ⭐ 7,355
Apache Beam is a unified programming model for Batch and Streaming data processing.
dependent packages 14total releases 568latest release November 13, 2023most recent commit 3 months ago
Pyspark2pmml ⭐ 93
Python library for converting Apache Spark ML pipelines to PMML
dependent packages 2total releases 2latest release April 08, 2019most recent commit 4 months agopypi pyspark2pmml} Downloads
The 10 Latest Releases In Pipeline Apache Open Source Projects
Beam ⭐ 7,355
Apache Beam is a unified programming model for Batch and Streaming data processing.
dependent packages 14total releases 568latest release November 13, 2023most recent commit 3 months ago
Jpmml Sparkml ⭐ 265
Java library and command-line application for converting Apache Spark ML pipelines to PMML
total releases 127latest release April 27, 2023most recent commit 4 months ago
Flink Connectors ⭐ 93
Apache Flink connectors for Pravega.
total releases 9latest release July 01, 2021most recent commit 6 months ago
Sparkflow ⭐ 301
Easy to use library to bring Tensorflow on Apache Spark
total releases 13latest release May 18, 2019most recent commit 6 months ago
Pyspark2pmml ⭐ 93
Python library for converting Apache Spark ML pipelines to PMML
dependent packages 2total releases 2latest release April 08, 2019most recent commit 4 months agopypi pyspark2pmml} Downloads
Sparklanes ⭐ 16
A lightweight data processing framework for Apache Spark
total releases 5latest release January 31, 2019most recent commit 4 years ago
Morebeam ⭐ 8
Additional functions useful for building Apache Beam pipelines in Go.
total releases 1latest release December 16, 2018most recent commit 5 years ago
Popular Web Servers Categories
Nginx
Apache
Webserver
Http Server
Socket Io
Tomcat
Rack
Netty
Tornado
Jetty
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.