Awesome Open Source
Awesome Open Source
Combined Topics
data-cleaning x
Advertising 📦 9
All Projects
Application Programming Interfaces 📦 120
Applications 📦 181
Artificial Intelligence 📦 72
Blockchain 📦 70
Build Tools 📦 111
Cloud Computing 📦 79
Code Quality 📦 28
Collaboration 📦 30
Command Line Interface 📦 48
Community 📦 81
Companies 📦 60
Compilers 📦 60
Computer Science 📦 74
Configuration Management 📦 39
Content Management 📦 167
Control Flow 📦 197
Data Formats 📦 77
Data Processing 📦 266
Data Storage 📦 132
Economics 📦 60
Frameworks 📦 198
Games 📦 122
Graphics 📦 103
Hardware 📦 148
Integrated Development Environments 📦 47
Learning Resources 📦 147
Legal 📦 28
Libraries 📦 119
Lists Of Projects 📦 21
Machine Learning 📦 336
Mapping 📦 61
Marketing 📦 15
Mathematics 📦 55
Media 📦 228
Messaging 📦 97
Networking 📦 304
Operating Systems 📦 84
Operations 📦 120
Package Managers 📦 52
Programming Languages 📦 229
Runtime Environments 📦 96
Science 📦 42
Security 📦 375
Social Media 📦 26
Software Architecture 📦 70
Software Development 📦 68
Software Performance 📦 57
Software Quality 📦 127
Text Editors 📦 45
Text Processing 📦 131
User Interface 📦 310
User Interface Components 📦 465
Version Control 📦 29
Virtualization 📦 68
Web Browsers 📦 38
Web Servers 📦 25
Web User Interface 📦 194

The Top 338 Data Cleaning Open Source Projects on Github

Categories > Data Processing > Data Cleaning
Openrefine ⭐ 8,581
OpenRefine is a free, open source power tool for working with messy data and improving it
Great_expectations ⭐ 5,968
Always know what to expect from your data.
Miller ⭐ 4,922
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
Cleanlab ⭐ 2,585
The standard package for machine learning with noisy labels, finding mislabeled data, and uncertainty quantification. Works with most datasets and models.
Dat8 ⭐ 1,439
General Assembly's 2015 Data Science course in Washington, DC
Optimus ⭐ 1,172
🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
Pandas Videos ⭐ 1,144
Jupyter notebook and datasets from the pandas Q&A video series
Dataprep ⭐ 1,137
DataPrep — The easiest way to prepare data in Python
Janitor ⭐ 1,083
simple tools for data cleaning in R
Data Forge Ts ⭐ 1,057
The JavaScript data transformation and analysis toolkit inspired by Pandas and LINQ.
Pandera ⭐ 938
A light-weight, flexible, and expressive data validation library for dataframes
Hypergbm ⭐ 634
A full pipeline AutoML tool for tabular data
Schema Inspector ⭐ 484
Schema-Inspector is a simple JavaScript object sanitization and validation module.
Dirty_cat ⭐ 367
Machine learning on dirty tabular data
Validate ⭐ 327
Professional data validation for the R environment
Nonechucks ⭐ 315
Deal with bad samples in your dataset dynamically, use Transforms as Filters, and more!
Klib ⭐ 277
Easy to use Python library of customized functions for cleaning and analyzing data.
Voicebook ⭐ 271
🗣️ A book and repo to get you started programming voice computing applications in Python (10 chapters and 200+ scripts).
Datamaid ⭐ 118
An R package for data screening
Feature Engineering Tutorials ⭐ 110
Data Science Feature Engineering and Selection Tutorials
Bumblebee ⭐ 108
🚕 A spreadsheet-like data preparation web app that works over Optimus (Pandas, Dask, cuDF, Dask-cuDF, Spark and Vaex)
Allie ⭐ 85
🤖 A machine learning framework for audio, text, image, video, or .CSV files (50+ featurizers and 15+ model trainers).
Refinr ⭐ 82
Cluster and merge similar char values: an R implementation of Open Refine clustering algorithms
Holoclean Legacy Deprecated ⭐ 74
A Machine Learning System for Data Enrichment.
Covid_19_jhu_data_web_scrap_and_cleaning ⭐ 61
This repository contains data and code used to get and clean data from https://github.com/CSSEGISandData/COVID-19 and https://www.worldometers.info/coronavirus/
Akvo Lumen ⭐ 60
Make sense of your data
Data Analysis Using Python ⭐ 58
Exploratory data analysis 📊using python 🐍of used car 🚘 database taken from ⓚ𝖆𝖌𝖌𝖑𝖊
Clean ⭐ 47
Fast and Easy Data Cleaning (in R)
Taxa ⭐ 41
taxonomic classes for R
Dtcleaner ⭐ 37
DTCleaner: data cleaning using multi-target decision trees.
Gratefuldata ⭐ 33
Grateful Data isn't programming code, but an online tutorial about data acquisition, cleaning and enriching, using publicly accessible data on the band the Grateful Dead as examples. Read the Wiki to find out how to use the sample data.
Lc Open Refine ⭐ 33
Library Carpentry: OpenRefine
Pgdedupe ⭐ 32
A simple command line interface to the datamade/dedupe library.
Redditcleaner ⭐ 31
Cleans Reddit Text Data 📜 :broom:
Scrubr ⭐ 30
Clean species occurrence records
Drugs Recommendation Using Reviews ⭐ 27
Analyzing the Drugs Descriptions, conditions, reviews and then recommending it using Deep Learning Models, for each Health Condition of a Patient.
Covid 19 Data Cleanup ⭐ 25
Scripts to cleanup data from https://github.com/CSSEGISandData/COVID-19
Covid 19 India Data ⭐ 25
data and code for scrapping and cleaning data on covid-19 in India from https://www.mohfw.gov.in/ and https://www.covid19india.org/
Nepali Translator ⭐ 23
Neural Machine Translation on the Nepali-English language pair
Fifa 2019 Analysis ⭐ 21
This is a project based on the FIFA World Cup 2019 and Analyzes the Performance and Efficiency of Teams, Players, Countries and other related things using Data Analysis and Data Visualizations
Foofah ⭐ 21
Foofah: programming-by-example data transformation program synthesizer
Boltzmannclean ⭐ 21
Fill missing values in Pandas DataFrames using Restricted Boltzmann Machines
Multimodal Sentiment Analysis ⭐ 20
Engaged in research to help improve to boost text sentiment analysis using facial features from video using machine learning.
Cleantext ⭐ 19
An open-source package for python to clean raw text data
Openrefine Ecology Lesson ⭐ 19
Data Cleaning with OpenRefine for Ecologists
Udacity Data Analyst Nanodegree ⭐ 19
Learn2clean ⭐ 18
Learn2Clean: Optimizing the Sequence of Tasks for Data Preparation and Cleaning
Machine Learning Projects 2 ⭐ 17
Errorlocate ⭐ 17
Find and replace erroneous fields in data using validation rules
Stata Economics ⭐ 16
Economics Lesson with Stata
Moodle Local_datacleaner ⭐ 16
Reduce, filter, and anonymize moodle data for non-prod environments
R Learning Journey ⭐ 16
Some of the projects i made when starting to learn R for Data Science at the university
Cleaner.jl ⭐ 16
A toolbox of simple solutions for common data cleaning problems.
Datacleanr ⭐ 15
Interactive and Reproducible Data Cleaning
Exemplary Ml Pipeline ⭐ 14
Exemplary, annotated machine learning pipeline for any tabular data problem.
Dockingml ⭐ 14
A package for MD, Docking and Machine learning drug discovery pipeline
Validatedb ⭐ 14
Validate on a table in a DB, using dbplyr
Ukrainian Stopwords ⭐ 13
the list of ~2000 ukrainian stopwords (with numbers)
Dedupe ⭐ 13
Java DSL for (online) deduplication
Titanic Survival In Depth Analysis ⭐ 12
Used Pandas , Matplotlib , Seaborn libraries to Analyze , Visualize and Explore the data of people travelling on Titanic, and Used Scikit-learn Modelling Algorithms to predict their probability of Survival.
Twitter Sentiment Analysis ⭐ 12
It is a Natural Language Processing Problem where Sentiment Analysis is done by Classifying the Positive tweets from negative tweets by machine learning models for classification, text mining, text analysis, data analysis and data visualization
World Food Production ⭐ 11
Comparing Top food and feed Producers around the globe and also seeking some interesting answers, solutions, patterns, hints and warnings through the power of Data Analysis and Data Visualization using Machine Learning.
Vulcan ⭐ 11
A high level deep learning framework for quickly prototyping networks with added tools in data visualisation, model interpretability and performance metrics
Animefaceranker ⭐ 11
Anime Face Quality Check
Openrefine Socialsci ⭐ 11
OpenRefine for Social Science Data
Students Performance Analytics ⭐ 11
Students Performance Evaluation using Feature Engineering, Feature Extraction, Manipulation of Data, Data Analysis, Data Visualization and at lat applying Classification Algorithms from Machine Learning to Separate Students with different grades
Tongdaxin Futures Data Clearing Database Operation ⭐ 10
对通达信数据进行去重和清洗处理，并将数据存入MongoDB，方便往后研究
Churn Modelling Dataset ⭐ 10
Predicting which set of the customers are gong to churn out from the organization by looking into some of the important attributes and applying Machine Learning and Deep Learning on it.
Xcel ⭐ 10
【项目已迁移到团队github】因此该 repository 只会同步最新的 README.md，若需要 watch、Star、Fork，则去团队的 github。谢谢。
Datawrangler ⭐ 10
Make quick and dirty data mining easier in Sublime Text
Ipydataclean ⭐ 10
Interactive cleaning for Pandas DataFrames
Data Forge Fs ⭐ 9
This library contains the file system extensions to Data-Forge that allow it to directly read and write CSV and JSON files in Node.js
Web Scraper Sentiment Analysis Tripadvisor ⭐ 9
Academic project for Advances in Data Science and Architecture course
Google Job Skills ⭐ 9
Having an Exploratory Analysis at what kind of Jobs and Job Locations are provided by Google and Youtube, also we look into some specific details which are important to get hired by youtube and google.
Black Friday Regression Analysis ⭐ 9
Predicting Prices for the products to be sold on Black Friday in US using Regression Analysis, Feature Engineering, Feature Selection, Feature Extraction and Data analysis - Data Visualizations.
Textcleaner ⭐ 9
text-data pre-processing utility
Udacity Bertelsmann Data Science Challenge Scholarship 2018 ⭐ 9
This is a repo for my Bertelsmann Data Science Scholarship Challenge: notes, exercises, quizzes.
Plane ⭐ 9
A text processing tool including tag(HTML, URL, Email) extraction and removing, punctuation normalization, simple segmentation, and so on.
Pywikimm ⭐ 9
Collects a multimodal dataset of Wikipedia articles and their images
Qualtricsdatacleaning ⭐ 9
R Tutorial: useful R codes for cleaning and filtering data from Qualtrics surveys, and for creating new variables in the dataframe. With step-by-step explanations.
Deductive ⭐ 9
Methods for deductive data correction and imputation
Graduate Admissions Analysis ⭐ 8
Analyzing the Factors on which Graduates get Admissions in Abroad and Visualizing some of the most intriguing and interesting patterns followed onto it using Data Analysis and Data Visualizations Using Machine Learning.
Python Agent ⭐ 8
数据结构化分析框架
Validatetools ⭐ 8
Datapipe ⭐ 7
dataPipe is a data processing and data analytics library for JavaScript. Inspired by LINQ (C#) and Pandas (Python)
Avito Demand Prediction Challenge ⭐ 7
It is a Competition for Regression Challenge held by Kaggle, It is based on a Avito Dataset whose size is 123GB which can be accessed from Kaggle, I have done Data Pre-processing, feature engineering, feature extraction, data visualization, machine learning, stacking and boosting
Allstate Claims Severity ⭐ 7
Udacity Machine Learning Engineer Nanodegree capstone proposal.
Analytics Vidhya Game Of Deep Learning Hackathon ⭐ 7
Contains my experiments for the Game of Deep Learning Hackathon conducted by Analytics Vidhya
Ntuoss Datascraping And Datacleaning Workshop ⭐ 7
This repository contains the reference scripts and the content presented in the NTU OSS Data scraping and Data cleaning workshop.
Data Cleaning ⭐ 7
Data cleaning tool.
James Bond Actors ⭐ 7
Script to grab Freebase data about James Bond actors and generate gexf data file.
Data_processing_and_feature_engineering_in_machine_learning ⭐ 7
This is an attempt to summarize feature engineering methods that I have learned over the course of my graduate school.
Bangalore House Prediction App ⭐ 7
Predicts home prices of Bangalore. Used Flutter, Flask and Jupyter Notebook.
Av_ultimate_student_hunt ⭐ 7
Solution for the Ultimate Student Hunt Challenge (1st place).
Big Mart Sales Prediction ⭐ 7
Using Machine Learning Algorithms for Regression Analysis to predict the sales pattern and Using Data Analysis and Data Visualizations to Support it.
Sars 2003 Outbreak Data Webscraping Code ⭐ 6
repository contains complete WHO data of 2003 outbreak with code used to web scrap, data mung and cleaning
Marshmallow Pyspark ⭐ 6
Marshmallow serializer integration with pyspark
Natural Language Processing ⭐ 6
text mining, regex, N-grams, fuzzy matching
Pakistan Suicide Bombing Dataset ⭐ 6
Analyzing the Suicide Bombing Patterns and seeking some of the most tangled questions with good visualizations with the help of Machine Learning and Data Science.
Sparklyclean ⭐ 6
Optimal distributed data deduplication and supervised learning pipeline using Apache Spark
1-100 of 338 projects

Related Projects

Jupyter Notebook Data Cleaning Projects (155)
Python Data Cleaning Projects (140)
Data Science Cleaning Projects (5)
Advertising 📦 9
All Projects
Application Programming Interfaces 📦 120
Applications 📦 181
Artificial Intelligence 📦 72
Blockchain 📦 70
Build Tools 📦 111
Cloud Computing 📦 79
Code Quality 📦 28
Collaboration 📦 30
Command Line Interface 📦 48
Community 📦 81
Companies 📦 60
Compilers 📦 60
Computer Science 📦 74
Configuration Management 📦 39
Content Management 📦 167
Control Flow 📦 197
Data Formats 📦 77
Data Processing 📦 266
Data Storage 📦 132
Economics 📦 60
Frameworks 📦 198
Games 📦 122
Graphics 📦 103
Hardware 📦 148
Integrated Development Environments 📦 47
Learning Resources 📦 147
Legal 📦 28
Libraries 📦 119
Lists Of Projects 📦 21
Machine Learning 📦 336
Mapping 📦 61
Marketing 📦 15
Mathematics 📦 55
Media 📦 228
Messaging 📦 97
Networking 📦 304
Operating Systems 📦 84
Operations 📦 120
Package Managers 📦 52
Programming Languages 📦 229
Runtime Environments 📦 96
Science 📦 42
Security 📦 375
Social Media 📦 26
Software Architecture 📦 70
Software Development 📦 68
Software Performance 📦 57
Software Quality 📦 127
Text Editors 📦 45
Text Processing 📦 131
User Interface 📦 310
User Interface Components 📦 465
Version Control 📦 29
Virtualization 📦 68
Web Browsers 📦 38
Web Servers 📦 25
Web User Interface 📦 194
Privacy policy
"GitHub" is a registered trademark of GitHub, Inc. Awesome Open Source is not affiliated with GitHub.

All non readme contents or Github based topics or project metadata copyright Awesome Open Source 2018-2022