Data Pipeline

Data pipeline is a tool to run Data loading pipelines. It is an open sourced app engine app that users can extend to suit their own needs. Out of the box it will load files from a source, transform them and then output them (output might be writing to a file or loading them into a data analysis tool). It is designed to be modular and support various sources, transformation technologies and output types. The transformations can be chained together to form complex pipelines.
Alternatives To Data Pipeline
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Professional Services2,635
2 months ago41apache-2.0Python
Common solutions and tools developed by Google Cloud's Professional Services team. This repository and its contents are not an officially supported Google product.
Dataflowjavasdk853249143 years ago38June 26, 201854
Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines.
Googleml203
3 years ago11Python
Google机器学习教程笔记(基础版)
Dataflowpythonsdk157
7 years ago20
Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines.
Dataflowsdk Examples148
6 years ago5
Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines. This repository hosts a few example pipelines to get you started with Dataflow.
Hand_tracking113
4 years ago5apache-2.0Python
Minimal Python interface for Google's Mediapipe HandTracking pipeline
Kubernetes Bigquery Python106
3 years ago5apache-2.0Python
Example Kubernetes app that shows how to build a 'pipeline' to stream data into BigQuery. Uses Redis or Google Cloud PubSub
Data Pipeline79
10 years ago2apache-2.0Python
Data pipeline is a tool to run Data loading pipelines. It is an open sourced app engine app that users can extend to suit their own needs. Out of the box it will load files from a source, transform them and then output them (output might be writing to a file or loading them into a data analysis tool). It is designed to be modular and support various sources, transformation technologies and output types. The transformations can be chained together to form complex pipelines.
Pubsub To Bigquery64
6 years agoapache-2.0Java
A highly configurable Google Cloud Dataflow pipeline that writes data into Google Big Query table from Pub/Sub
Continuous Deployment Bitbucket60
4 years ago1apache-2.0Python
Alternatives To Data Pipeline
Select To Compare


Alternative Project Comparisons
Popular Pipeline Projects
Popular Google Projects
Popular Data Processing Categories

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Google
Cloud Computing
Pipeline
Hadoop
Bigquery