Luigi

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Alternatives To Luigi
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Luigi17,046338766 months ago80October 05, 2023124apache-2.0Python
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Digandburied645
8 years ago4GCC Machine Description
挖坑与填坑
Tez446
6 months ago67apache-2.0Java
Apache Tez
Shifu23512a year ago9April 03, 2019237apache-2.0Java
An end-to-end machine learning and data mining framework on Hadoop
Graphbuilder90
10 years ago1apache-2.0Java
The GraphBuilder library provides functions to construct large scale graphs. It is implemented on Apache Hadoop.
Smart Data Lake8786 months ago26October 25, 202364gpl-3.0Scala
Smart Automation Tool for building modern Data Lakes and Data Pipelines
Briefly85
6 years ago2apache-2.0Python
Briefly - A Python Meta-programming Library for Job Flow Control
Ni81
a year ago6mitPerl
Say "ni" to data of any size
Data Pipeline79
10 years ago2apache-2.0Python
Data pipeline is a tool to run Data loading pipelines. It is an open sourced app engine app that users can extend to suit their own needs. Out of the box it will load files from a source, transform them and then output them (output might be writing to a file or loading them into a data analysis tool). It is designed to be modular and support various sources, transformation technologies and output types. The transformations can be chained together to form complex pipelines.
Til51
2 years ago173gpl-3.0DIGITAL Command Language
Today I Learned
Alternatives To Luigi
Select To Compare


Alternative Project Comparisons
Popular Hadoop Projects
Popular Pipeline Projects
Popular Data Processing Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Pipeline
Hadoop