Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for pipeline dag
dag
x
pipeline
x
34 search results found
Argo Workflows
⭐
14,264
Workflow Engine for Kubernetes
Orchest
⭐
3,876
Build data pipelines, the easy way 🛠️
Polyaxon
⭐
3,438
MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle
Awesome Etl
⭐
3,041
A curated list of awesome ETL frameworks, libraries, and software.
Cyclone
⭐
1,042
Powerful workflow engine and end-to-end pipeline solutions implemented with native Kubernetes resources. https://cyclone.dev
Serving
⭐
834
A flexible, high-performance carrier for machine learning models(『飞桨』服务化部署框架)
Goodreads_etl_pipeline
⭐
593
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
Dagli
⭐
351
Framework for defining machine learning models, including feature generation and transformations, as directed acyclic graphs (DAGs).
Dffml
⭐
235
The easiest way to use Machine Learning. Mix and match underlying ML libraries and data set sources. Generate new datasets or modify existing ones with ease.
Data Testing With Airflow
⭐
190
Whirl
⭐
183
Fast iterative local development and testing of Apache Airflow workflows
Factotum
⭐
155
A system to programmatically run data pipelines
Kedro Airflow
⭐
120
Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.
Aws Concurrent Data Orchestration Pipeline Emr Livy
⭐
66
This code demonstrates the architecture featured on the AWS Big Data blog (https://aws.amazon.com/blogs/big-data/ ) which creates a concurrent data pipeline by using Amazon EMR and Apache Livy. This pipeline is orchestrated by Apache Airflow.
Dag
⭐
62
Simple DSL for executing functions in Go
Mlinspect
⭐
57
Inspect ML Pipelines in Python in the form of a DAG
Airflow Tutorial
⭐
51
🐍💨 Airflow tutorial for PyCon 2019
Hlatyping
⭐
34
Precision HLA typing from next-generation sequencing data
Hadron
⭐
25
The Kubernetes native and CloudEvents native distributed event streaming, event orchestration & messaging platform
Nomad Pipeline
⭐
24
Run pipeline-style workloads in Nomad
Glide
⭐
19
Easy ETL
Composable Logs
⭐
16
Python library to run ML/data pipelines on stateless compute infrastructure (that may be ephemeral or serverless). Please see the documentation site with more details and demo:
Phila Airflow
⭐
16
Airflowdatapipeline
⭐
15
Example of an ETL Pipeline using Airflow
Peapod
⭐
15
Dependency and data pipeline management framework for Spark and Scala
Airflow Imaging Plugins
⭐
14
Set of plugins helping to work with imaging data in Airflow.
Lineage
⭐
14
Generate beautiful documentation for your data pipelines in markdown format
Thepipelinetool
⭐
13
A pipeline orchestration tool
Graphtik
⭐
13
A Python-3.6+ lib for solving & executing graphs of functions on hierarchical data, with `pandas` in mind
Magnus Core
⭐
10
Core package of magnus
Dask Funk
⭐
9
Plumatic 'graph' in python using dask!
Python_packages_for_pipeline_workflow
⭐
9
Buildkite Graph
⭐
9
A graph-based generator for Buildkite pipelines
Accel Video Pipe
⭐
9
AVPipe :-)
Airflow_to_manage_talend_etl_jobs
⭐
8
Airflow, an open source platform, is used to orchestrate workflows as Directed Acyclic Graphs (DAGs) of tasks in a programmatic manner. An airflow scheduler is used to schedule workflows and data processing pipelines. Airflow user interface allows easy visualization of pipelines running in production environment, monitoring of the progress of the workflows, and troubleshooting issues when needed.
Seq2dagchainer
⭐
8
Seq2DagChainer - colinear chaining between a sequence and a DAG
Udacity Airflow
⭐
8
Udacity Data Pipeline Exercises
Pypedream
⭐
8
DAGPype looks like was abandoned. Move to github to see if can be revived.
Aerobio
⭐
7
Extensible full DAG streaming computation server with services and jobs for RNA-Seq, Tn-Seq, WG-Seq and Term-Seq.
Airflow Data Pipeline
⭐
7
Udacity Data Engineer Nanodegree - Airflow data pipeline
Airflow4ds
⭐
6
Using Apache Airflow to author, run and monitor complex data pipelines.
Sheenflow
⭐
6
A next-generation open source orchestration platform for the development, production, and observation of data assets.
Papercast
⭐
6
A Python pipeline tool and plugin ecosystem for processing technical documents. Process papers from arXiv, SemanticScholar, PDF, with GROBID, LangChain, listen as podcast. Customize your own pipelines.
Dagchainer
⭐
6
DAGChainer - colinear chaining between a sequence and a DAG
Mlinspect Cidr
⭐
5
Inspect ML Pipelines in Python in the form of a DAG (CIDR Submission version)
Pipeline Mvp
⭐
5
数据pipeline处理编排框架
Rgapeppipe
⭐
5
Reference Guided Assembly Pepperell Lab Pipeline
Airflow Ml Pipeline
⭐
5
The repository to learn Machine Learning with Airflow
Airflow Dags
⭐
5
Owl
⭐
5
OWL is a pipeline/workflow execution framework designed to be simple and lightweight. Its intended mode of operation is the repeated execution of fairly static workflows. This is the mode of operation of data centers and astronomical observatories, more than individual researchers.
Related Searches
Python Pipeline (4,255)
Javascript Pipeline (1,369)
Pipeline Jenkins (1,150)
Shell Pipeline (1,143)
Docker Pipeline (1,018)
Jupyter Notebook Pipeline (976)
Java Pipeline (868)
Pipeline Continuous Integration (808)
Golang Pipeline (682)
Machine Learning Pipeline (553)
1-34 of 34 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.