Awesome Open Source
Awesome Open Source
Combined Topics
data-cleaning
x
Advertising
📦 9
All Projects
Application Programming Interfaces
📦 120
Applications
📦 181
Artificial Intelligence
📦 72
Blockchain
📦 70
Build Tools
📦 111
Cloud Computing
📦 79
Code Quality
📦 28
Collaboration
📦 30
Command Line Interface
📦 48
Community
📦 81
Companies
📦 60
Compilers
📦 60
Computer Science
📦 74
Configuration Management
📦 39
Content Management
📦 167
Control Flow
📦 197
Data Formats
📦 77
Data Processing
📦 266
Data Storage
📦 132
Economics
📦 60
Frameworks
📦 198
Games
📦 122
Graphics
📦 103
Hardware
📦 148
Integrated Development Environments
📦 47
Learning Resources
📦 147
Legal
📦 28
Libraries
📦 119
Lists Of Projects
📦 21
Machine Learning
📦 336
Mapping
📦 61
Marketing
📦 15
Mathematics
📦 55
Media
📦 228
Messaging
📦 97
Networking
📦 304
Operating Systems
📦 84
Operations
📦 120
Package Managers
📦 52
Programming Languages
📦 229
Runtime Environments
📦 96
Science
📦 42
Security
📦 375
Social Media
📦 26
Software Architecture
📦 70
Software Development
📦 68
Software Performance
📦 57
Software Quality
📦 127
Text Editors
📦 45
Text Processing
📦 131
User Interface
📦 310
User Interface Components
📦 465
Version Control
📦 29
Virtualization
📦 68
Web Browsers
📦 38
Web Servers
📦 25
Web User Interface
📦 194
The Top 312 Data Cleaning Open Source Projects on Github
Categories
>
Data Processing
>
Data Cleaning
Openrefine
⭐
8,355
OpenRefine is a free, open source power tool for working with messy data and improving it
Great_expectations
⭐
4,999
Always know what to expect from your data.
Miller
⭐
4,387
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
Cleanlab
⭐
2,217
The standard package for machine learning with noisy labels and finding mislabeled data. Works with most datasets and models.
Dat8
⭐
1,439
General Assembly's 2015 Data Science course in Washington, DC
My Journey In The Data Science World
⭐
1,175
📢 Ready to learn or review your knowledge!
Pandas Videos
⭐
1,144
Jupyter notebook and datasets from the pandas Q&A video series
Optimus
⭐
1,106
🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF and PySpark
Janitor
⭐
1,045
simple tools for data cleaning in R
Dataprep
⭐
962
DataPrep — The easiest way to prepare data in Python
Data Forge Ts
⭐
952
The JavaScript data transformation and analysis toolkit inspired by Pandas and LINQ.
Pandera
⭐
705
A light-weight, flexible, and expressive pandas data validation library
Hypergbm
⭐
577
A full pipeline AutoML tool for tabular data
Schema Inspector
⭐
484
Schema-Inspector is a simple JavaScript object sanitization and validation module.
Nonechucks
⭐
315
Deal with bad samples in your dataset dynamically, use Transforms as Filters, and more!
Validate
⭐
308
Professional data validation for the R environment
Dirty_cat
⭐
304
Encoding methods for dirty categorical variables
Voicebook
⭐
270
🗣️ A book and repo to get you started programming voice computing applications in Python (10 chapters and 200+ scripts).
Klib
⭐
254
Easy to use Python library of customized functions for cleaning and analyzing data.
Datamaid
⭐
118
An R package for data screening
Bumblebee
⭐
100
🚕 A spreadsheet-like data preparation web app that works over Optimus (Pandas, Dask, cuDF, Dask-cuDF and PySpark)
Allie
⭐
83
🤖 A machine learning framework for audio, text, image, video, or .CSV files (50+ featurizers and 15+ model trainers).
Refinr
⭐
82
Cluster and merge similar char values: an R implementation of Open Refine clustering algorithms
Holoclean Legacy Deprecated
⭐
74
A Machine Learning System for Data Enrichment.
Covid_19_jhu_data_web_scrap_and_cleaning
⭐
61
This repository contains data and code used to get and clean data from https://github.com/CSSEGISandData/COVID-19 and https://www.worldometers.info/coronavirus/
Akvo Lumen
⭐
59
Make sense of your data
Data Analysis Using Python
⭐
58
Exploratory data analysis 📊using python 🐍of used car 🚘 database taken from ⓚ𝖆𝖌𝖌𝖑𝖊
Clean
⭐
47
Fast and Easy Data Cleaning (in R)
Taxa
⭐
39
taxonomic classes for R
Dtcleaner
⭐
37
DTCleaner: data cleaning using multi-target decision trees.
Gratefuldata
⭐
33
Grateful Data isn't programming code, but an online tutorial about data acquisition, cleaning and enriching, using publicly accessible data on the band the Grateful Dead as examples. Read the Wiki to find out how to use the sample data.
Pgdedupe
⭐
32
A simple command line interface to the datamade/dedupe library.
Redditcleaner
⭐
31
Cleans Reddit Text Data 📜 :broom:
Scrubr
⭐
30
Clean species occurrence records
Lc Open Refine
⭐
28
Library Carpentry: OpenRefine
Drugs Recommendation Using Reviews
⭐
27
Analyzing the Drugs Descriptions, conditions, reviews and then recommending it using Deep Learning Models, for each Health Condition of a Patient.
Covid 19 Data Cleanup
⭐
25
Scripts to cleanup data from https://github.com/CSSEGISandData/COVID-19
Covid 19 India Data
⭐
25
data and code for scrapping and cleaning data on covid-19 in India from https://www.mohfw.gov.in/ and https://www.covid19india.org/
Nepali Translator
⭐
23
Neural Machine Translation on the Nepali-English language pair
Boltzmannclean
⭐
21
Fill missing values in Pandas DataFrames using Restricted Boltzmann Machines
Fifa 2019 Analysis
⭐
21
This is a project based on the FIFA World Cup 2019 and Analyzes the Performance and Efficiency of Teams, Players, Countries and other related things using Data Analysis and Data Visualizations
Foofah
⭐
21
Foofah: programming-by-example data transformation program synthesizer
Multimodal Sentiment Analysis
⭐
20
Engaged in research to help improve to boost text sentiment analysis using facial features from video using machine learning.
Udacity Data Analyst Nanodegree
⭐
19
Learn2clean
⭐
18
Learn2Clean: Optimizing the Sequence of Tasks for Data Preparation and Cleaning
Errorlocate
⭐
17
Find and replace erroneous fields in data using validation rules
Openrefine Ecology Lesson
⭐
17
Data Cleaning with OpenRefine for Ecologists
Machine Learning Projects 2
⭐
17
R Learning Journey
⭐
16
Some of the projects i made when starting to learn R for Data Science at the university
Moodle Local_datacleaner
⭐
16
Reduce, filter, and anonymize moodle data for non-prod environments
Dockingml
⭐
14
A package for MD, Docking and Machine learning drug discovery pipeline
Validatedb
⭐
14
Validate on a table in a DB, using dbplyr
Exemplary Ml Pipeline
⭐
14
Exemplary, annotated machine learning pipeline for any tabular data problem.
Ukrainian Stopwords
⭐
13
the list of ~2000 ukrainian stopwords (with numbers)
Twitter Sentiment Analysis
⭐
12
It is a Natural Language Processing Problem where Sentiment Analysis is done by Classifying the Positive tweets from negative tweets by machine learning models for classification, text mining, text analysis, data analysis and data visualization
Titanic Survival In Depth Analysis
⭐
12
Used Pandas , Matplotlib , Seaborn libraries to Analyze , Visualize and Explore the data of people travelling on Titanic, and Used Scikit-learn Modelling Algorithms to predict their probability of Survival.
Stata Economics
⭐
12
Economics Lesson with Stata
Datacleanr
⭐
12
Interactive and Reproducible Data Cleaning
World Food Production
⭐
11
Comparing Top food and feed Producers around the globe and also seeking some interesting answers, solutions, patterns, hints and warnings through the power of Data Analysis and Data Visualization using Machine Learning.
Vulcan
⭐
11
A high level deep learning framework for quickly prototyping networks with added tools in data visualisation, model interpretability and performance metrics
Students Performance Analytics
⭐
11
Students Performance Evaluation using Feature Engineering, Feature Extraction, Manipulation of Data, Data Analysis, Data Visualization and at lat applying Classification Algorithms from Machine Learning to Separate Students with different grades
Animefaceranker
⭐
11
Anime Face Quality Check
Xcel
⭐
10
【项目已迁移到团队github】因此该 repository 只会同步最新的 README.md，若需要 watch、Star、Fork，则去团队的 github。谢谢。
Ipydataclean
⭐
10
Interactive cleaning for Pandas DataFrames
Tongdaxin Futures Data Clearing Database Operation
⭐
10
对通达信数据进行去重和清洗处理，并将数据存入MongoDB，方便往后研究
Churn Modelling Dataset
⭐
10
Predicting which set of the customers are gong to churn out from the organization by looking into some of the important attributes and applying Machine Learning and Deep Learning on it.
Datawrangler
⭐
10
Make quick and dirty data mining easier in Sublime Text
Openrefine Socialsci
⭐
10
OpenRefine for Social Science Data
Google Job Skills
⭐
9
Having an Exploratory Analysis at what kind of Jobs and Job Locations are provided by Google and Youtube, also we look into some specific details which are important to get hired by youtube and google.
Black Friday Regression Analysis
⭐
9
Predicting Prices for the products to be sold on Black Friday in US using Regression Analysis, Feature Engineering, Feature Selection, Feature Extraction and Data analysis - Data Visualizations.
Textcleaner
⭐
9
text-data pre-processing utility
Udacity Bertelsmann Data Science Challenge Scholarship 2018
⭐
9
This is a repo for my Bertelsmann Data Science Scholarship Challenge: notes, exercises, quizzes.
Web Scraper Sentiment Analysis Tripadvisor
⭐
9
Academic project for Advances in Data Science and Architecture course
Deductive
⭐
9
Methods for deductive data correction and imputation
Plane
⭐
9
A text processing tool including tag(HTML, URL, Email) extraction and removing, punctuation normalization, simple segmentation, and so on.
Pywikimm
⭐
9
Collects a multimodal dataset of Wikipedia articles and their images
Graduate Admissions Analysis
⭐
8
Analyzing the Factors on which Graduates get Admissions in Abroad and Visualizing some of the most intriguing and interesting patterns followed onto it using Data Analysis and Data Visualizations Using Machine Learning.
Dedupe
⭐
8
Java DSL for (online) deduplication
Validatetools
⭐
8
Python Agent
⭐
8
数据结构化分析框架
Allstate Claims Severity
⭐
7
Udacity Machine Learning Engineer Nanodegree capstone proposal.
Ntuoss Datascraping And Datacleaning Workshop
⭐
7
This repository contains the reference scripts and the content presented in the NTU OSS Data scraping and Data cleaning workshop.
Analytics Vidhya Game Of Deep Learning Hackathon
⭐
7
Contains my experiments for the Game of Deep Learning Hackathon conducted by Analytics Vidhya
Data Cleaning
⭐
7
Data cleaning tool.
Avito Demand Prediction Challenge
⭐
7
It is a Competition for Regression Challenge held by Kaggle, It is based on a Avito Dataset whose size is 123GB which can be accessed from Kaggle, I have done Data Pre-processing, feature engineering, feature extraction, data visualization, machine learning, stacking and boosting
Big Mart Sales Prediction
⭐
7
Using Machine Learning Algorithms for Regression Analysis to predict the sales pattern and Using Data Analysis and Data Visualizations to Support it.
Bangalore House Prediction App
⭐
7
Predicts home prices of Bangalore. Used Flutter, Flask and Jupyter Notebook.
Data_processing_and_feature_engineering_in_machine_learning
⭐
7
This is an attempt to summarize feature engineering methods that I have learned over the course of my graduate school.
James Bond Actors
⭐
7
Script to grab Freebase data about James Bond actors and generate gexf data file.
Optimus Examples
⭐
6
Examples for Optimus a Data Cleansing Library for Big Data.
Sars 2003 Outbreak Data Webscraping Code
⭐
6
repository contains complete WHO data of 2003 outbreak with code used to web scrap, data mung and cleaning
Av_ultimate_student_hunt
⭐
6
Solution for the Ultimate Student Hunt Challenge (1st place).
Data Cleaning
⭐
6
Data Cleaning with Python
Natural Language Processing
⭐
6
text mining, regex, N-grams, fuzzy matching
Marshmallow Pyspark
⭐
6
Marshmallow serializer integration with pyspark
Pakistan Suicide Bombing Dataset
⭐
6
Analyzing the Suicide Bombing Patterns and seeking some of the most tangled questions with good visualizations with the help of Machine Learning and Data Science.
Pypandas
⭐
6
PyPandas, a data cleaning framework for Spark
Sparklyclean
⭐
6
Optimal distributed data deduplication and supervised learning pipeline using Apache Spark
Datasetops
⭐
5
Fluent dataset operations, compatible with your favorite libraries
Qualtricsdatacleaning
⭐
5
R Tutorial: useful R codes for cleaning and filtering data from Qualtrics surveys, and for creating new variables in the dataframe. With step-by-step explanations.
1-100 of 312 projects
Next >
Related Projects
Jupyter Notebook Data Cleaning Projects (137)
Python Data Cleaning Projects (124)
Advertising
📦 9
All Projects
Application Programming Interfaces
📦 120
Applications
📦 181
Artificial Intelligence
📦 72
Blockchain
📦 70
Build Tools
📦 111
Cloud Computing
📦 79
Code Quality
📦 28
Collaboration
📦 30
Command Line Interface
📦 48
Community
📦 81
Companies
📦 60
Compilers
📦 60
Computer Science
📦 74
Configuration Management
📦 39
Content Management
📦 167
Control Flow
📦 197
Data Formats
📦 77
Data Processing
📦 266
Data Storage
📦 132
Economics
📦 60
Frameworks
📦 198
Games
📦 122
Graphics
📦 103
Hardware
📦 148
Integrated Development Environments
📦 47
Learning Resources
📦 147
Legal
📦 28
Libraries
📦 119
Lists Of Projects
📦 21
Machine Learning
📦 336
Mapping
📦 61
Marketing
📦 15
Mathematics
📦 55
Media
📦 228
Messaging
📦 97
Networking
📦 304
Operating Systems
📦 84
Operations
📦 120
Package Managers
📦 52
Programming Languages
📦 229
Runtime Environments
📦 96
Science
📦 42
Security
📦 375
Social Media
📦 26
Software Architecture
📦 70
Software Development
📦 68
Software Performance
📦 57
Software Quality
📦 127
Text Editors
📦 45
Text Processing
📦 131
User Interface
📦 310
User Interface Components
📦 465
Version Control
📦 29
Virtualization
📦 68
Web Browsers
📦 38
Web Servers
📦 25
Web User Interface
📦 194
"GitHub" is a registered trademark of GitHub, Inc. Awesome Open Source is not affiliated with GitHub.
