Awesome Open Source
Awesome Open Source
Combined Topics
data-transformation
x
Advertising
📦 9
All Projects
Application Programming Interfaces
📦 120
Applications
📦 181
Artificial Intelligence
📦 72
Blockchain
📦 70
Build Tools
📦 111
Cloud Computing
📦 79
Code Quality
📦 28
Collaboration
📦 30
Command Line Interface
📦 48
Community
📦 81
Companies
📦 60
Compilers
📦 60
Computer Science
📦 74
Configuration Management
📦 39
Content Management
📦 167
Control Flow
📦 197
Data Formats
📦 77
Data Processing
📦 266
Data Storage
📦 132
Economics
📦 60
Frameworks
📦 198
Games
📦 122
Graphics
📦 103
Hardware
📦 148
Integrated Development Environments
📦 47
Learning Resources
📦 147
Legal
📦 28
Libraries
📦 119
Lists Of Projects
📦 21
Machine Learning
📦 336
Mapping
📦 61
Marketing
📦 15
Mathematics
📦 55
Media
📦 228
Messaging
📦 97
Networking
📦 304
Operating Systems
📦 84
Operations
📦 120
Package Managers
📦 52
Programming Languages
📦 229
Runtime Environments
📦 96
Science
📦 42
Security
📦 375
Social Media
📦 26
Software Architecture
📦 70
Software Development
📦 68
Software Performance
📦 57
Software Quality
📦 127
Text Editors
📦 45
Text Processing
📦 131
User Interface
📦 310
User Interface Components
📦 465
Version Control
📦 29
Virtualization
📦 68
Web Browsers
📦 38
Web Servers
📦 25
Web User Interface
📦 194
The Top 91 Data Transformation Open Source Projects on Github
Categories
>
Data Processing
>
Data Transformation
Glom
⭐
1,354
☄️ Python's nested data operator (and CLI), for all your declarative restructuring needs. Got data? Glom it! ☄️
Optimus
⭐
1,108
🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
Transformerkit
⭐
831
A block-based API for NSValueTransformer, with a growing collection of useful examples.
Porter
⭐
551
💄 Scalable and durable all-purpose data import library for publishing APIs and SDKs.
Optimus
⭐
542
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
Pglogical
⭐
517
Logical Replication extension for PostgreSQL 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.
Prose
⭐
511
Microsoft Program Synthesis using Examples SDK is a framework of technologies for the automatic generation of programs from input-output examples. This repo includes samples and sample data for the Microsoft Program Synthesis using Example SDK.
Lightdash
⭐
386
An open source alternative to Looker built using dbt. Made for analysts ❤️
Sqawk
⭐
272
Like Awk but with SQL and table joins
Temme
⭐
258
📄 Concise selector to extract JSON from HTML.
Collapse
⭐
240
Advanced and Fast Data Transformation in R
Zingg
⭐
183
Scalable fuzzy matching for data mastering, deduplication and entity resolution.
Datagene
⭐
162
DataGene - Identify How Similar TS Datasets Are to One Another (by @firmai)
Sjmisc
⭐
143
Data transformation and utility functions for R
Big Data Mapreduce Course
⭐
107
Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University
Naas
⭐
105
⚙️ Schedule notebooks, run them like APIs, expose securely your assets: Jupyter as a viable ⚡️ Production environment
Setl
⭐
101
A simple Spark-powered ETL framework that just works 🍺
Blurr
⭐
95
Data transformations for the ML era
Clojure Dsl Resources
⭐
91
A curated list of Clojure resources for dealing with domain-specific languages.
Allie
⭐
85
🤖 A machine learning framework for audio, text, image, video, or .CSV files (50+ featurizers and 15+ model trainers).
Fastverse
⭐
59
An Extensible Suite of High-Performance and Low-Dependency Packages for Statistical Computing and Data Manipulation in R
Dry Transformer
⭐
52
Data transformation toolkit
Wrangler
⭐
49
Wrangler Transform: A DMD system for transforming Big Data
Aws Dbs Refarch Datalake
⭐
47
Reference Architectures for Datalakes on AWS
Pipe_envy
⭐
44
Elixir style pipe operator for Ruby
Php Serializer
⭐
44
Serialize PHP variables, including objects, in any format. Support to unserialize it too.
Daany
⭐
43
Daany - .NET DAta ANalYtics .NET 5 library with the implementation of DataFrame, Time series decompositions and Linear Algebra routines BLASS and LAPACK.
Semantic Bus
⭐
39
object flow treatment, data transformation
Datapackage M
⭐
24
Power Query M functions for working with Tabular Data Packages (Frictionless Data) in Power BI and Excel
Foofah
⭐
21
Foofah: programming-by-example data transformation program synthesizer
Serializer
⭐
21
A PHP serialization component focused on performance
Flowman
⭐
20
Flowman is a Spark based data build tool. By using high level flow specifications with YAML files, Flowman simplifies the development of data pipelines.
Pycsvw
⭐
20
A tool to read CSV files with CSVW metadata and transform them into other formats.
Bamboolib_binder_template
⭐
18
bamboolib - template for creating your own binder notebook
Field_mapper
⭐
17
Data mapping & transformation
Ldwizard
⭐
16
A generic framework for simplifying the creation of linked data.
Richflow
⭐
13
A Node.js and JavaScript synchronous data pipeline processing, data sharing and stream processing library. Actionable & Transformable Pipeline data processing.
Serializer Benchmark
⭐
13
A PHP benchmark application to compare PHP serializer libraries
Tutorials
⭐
13
Short programming tutorials pertaining to data analysis.
Object Mapper
⭐
13
Maps generically data from source to target object via extensible strategies and controls
Glide
⭐
12
Easy ETL
Dynamic.yaml
⭐
12
DEPRECATED: YAML-based data transformations
Vue Models
⭐
12
A plugin for handling models in Vue.js that includes schemas for data transformation
Hk Atm Locator
⭐
10
🏧 香港自動櫃員機定位器 🏧 Centralising Automated Teller Machine (ATM) Data in Hong Kong in a well-defined yet standardised format and display in a web portal for public use
Scrape
⭐
9
When you need those jobs hypersonic 🚀 scrape 🔪
Data Lens
⭐
8
Functional utilities for Common Lisp
I2o Transform
⭐
7
PCORnet Ontology to OMOP - beta!
Dixer Docs
⭐
6
Dixer documentation. A tool to make data transformation and pipelines with a lot of job types like dataflow send mail, http request, file and ftp operation tasks and more! With expression support. Developed in Golang!
Object Model Transform
⭐
6
Transforms objects and its object property into model.
Purescript Morello
⭐
6
Cherry-picking 🍒 for your data
Index Array By
⭐
5
A utility function to index arrays by any criteria
Datapreprocessingsystem
⭐
5
Preprocessing of data (e.g. filling missing values, normalization,etc.) in field of Data Mining (Knowledge Discovery).
Rds R
⭐
4
R package to simplify the usage of the RDS REST API and provide convenience in accessing data and metadata.
Datapackage Connector
⭐
4
Power BI Custom Connector for loading tables directly from Tabular Data Packages (Frictionless Data) into Power BI
Dtrans
⭐
4
Erlang data-transformation and validation micro library
Python Static Api Generator
⭐
4
Python static API generator
Gson Relationship
⭐
4
Transform json-data using relational concepts
Data Drift
⭐
3
Data streaming and scheduling library for Node.JS
Cuphic
⭐
3
Transform or scrape Hiccup with a declarative DSL.
Detl
⭐
3
Seamless data and meta-data tracking in Python
Calendar Shaper
⭐
3
🗓️ iCalendar proxy reshaping the data for your needs
Rds Js Examples
⭐
3
TypeScript/JavaScript example code using the RDS API
Catalystml
⭐
2
CatalystML is an open source specification for real-time feature processing, purpose built to transform data for machine learning models.
Rds Js
⭐
2
Simplify usage of the RDS API for TypeScript/JavaScript developers
Etl
⭐
2
Extract |> Transform |> Load
Odin3
⭐
2
Odin for Python 3. Ground up refresh built with Python 3 (>=3.5) in mind.
Supermaxi Receipt Analysis
⭐
2
A utility script to parse an ecuadorian supermarket receipt detail from the source XML
Mantau
⭐
2
🍞 mantau is a simple schema based golang data transformer for struct, map & slices or arrays.
Prolog Schemes
⭐
2
Optics and data transformation library
Appliedpredictivemodeling
⭐
1
Exercises From Book "Applied Predictive Modeling" by "Kuhn and Johnson (2013)"
Cds101 Course Materials
⭐
1
Course materials for CDS 101: Introduction to Computational and Data Sciences, offered at George Mason University
Data Interpreter
⭐
1
A library to transform / normalize / map a data source (JS Data Structure) using a template.
Linelist Aggregation Epimap
⭐
1
Data transformation and aggregation of Linelist for the Epimap
Df And Order
⭐
1
Using df-and-order your interactions with dataframes become very clean and predictable.
Inventory_prediction
⭐
1
The sale prediction models implement various machine learning tools to classify the WholeSale database and make predictions for future inventory. The goal is to focus on performing cluster analyst on an inventory of a wholesale database (data was just a small sample) that data scientists can draw hypothesis from there.
Data
⭐
1
Data extraction, transformation, processing and visualisation
Mcode
⭐
1
Functions to merge and recode across multiple variables
Data Transform
⭐
1
A collection of scripts to transform data into different formats and structures for use with the Tidal Migrations API.
Weather Visibility Prediction
⭐
1
This is a Project which uses live weather data using API, and predicts visibility in the weather.
Adaboost_classification_sklearn
⭐
1
AdaBoost classification using sklearn
Realtime_fraud_transaction_detection
⭐
1
The project focus on creating Fraud Detection Application to detect fraudulent credit card transactions. My application will be built from various data mining and model prediction methods. The combination of them will increase the change to prevent fraud detection. The disadvantage of this is the response time to end-users as many layers of detection will add more execution time to application O(log(n)).
Data Engineering Case
⭐
1
ETL Redshift-based workflow automated with AWS Step Funtions.
Convert Class Shape
⭐
1
Convert class shapes: polygons to rectangles, bitmaps to polygons and others transformations
Merge Classes
⭐
1
Class mapping to combine classes with the same shapes
Customer Segmentation Toolkit
⭐
1
Data transformations toolkit made from jupyter notebook: https://www.kaggle.com/fabiendaniel/customer-segmentation
Datasponge
⭐
1
Pluggable framework that can be used to spider websites and extract data.
Standoff2conll
⭐
0
An extended version of the conversion script from brat-flavored standoff to CoNLL format, with additional support for the D. Roth and W. Yih CoNLL 2004 Relation Extractor training format.
Reporttabledemo
⭐
0
Complex iOS Tableview Demo using a Transform, ViewModel and external Adapter using Enums
Reporttableadapterdemo
⭐
0
Complex iOS Tableview Demo with external Adapter
Dataframes
⭐
0
Explore all methods and feautures available for DataFrame based operation
Iot Central Compute
⭐
0
A simple way to do compute and data transformation on data sent to Azure IoT Central using Azure Functions and a slightly modified version of the Azure IoT Central Device Bridge.
1-91 of 91 projects
Advertising
📦 9
All Projects
Application Programming Interfaces
📦 120
Applications
📦 181
Artificial Intelligence
📦 72
Blockchain
📦 70
Build Tools
📦 111
Cloud Computing
📦 79
Code Quality
📦 28
Collaboration
📦 30
Command Line Interface
📦 48
Community
📦 81
Companies
📦 60
Compilers
📦 60
Computer Science
📦 74
Configuration Management
📦 39
Content Management
📦 167
Control Flow
📦 197
Data Formats
📦 77
Data Processing
📦 266
Data Storage
📦 132
Economics
📦 60
Frameworks
📦 198
Games
📦 122
Graphics
📦 103
Hardware
📦 148
Integrated Development Environments
📦 47
Learning Resources
📦 147
Legal
📦 28
Libraries
📦 119
Lists Of Projects
📦 21
Machine Learning
📦 336
Mapping
📦 61
Marketing
📦 15
Mathematics
📦 55
Media
📦 228
Messaging
📦 97
Networking
📦 304
Operating Systems
📦 84
Operations
📦 120
Package Managers
📦 52
Programming Languages
📦 229
Runtime Environments
📦 96
Science
📦 42
Security
📦 375
Social Media
📦 26
Software Architecture
📦 70
Software Development
📦 68
Software Performance
📦 57
Software Quality
📦 127
Text Editors
📦 45
Text Processing
📦 131
User Interface
📦 310
User Interface Components
📦 465
Version Control
📦 29
Virtualization
📦 68
Web Browsers
📦 38
Web Servers
📦 25
Web User Interface
📦 194
"GitHub" is a registered trademark of GitHub, Inc. Awesome Open Source is not affiliated with GitHub.
