Awesome Open Source
Awesome Open Source
Combined Topics
data-cleaning x

The Top 462 Data Cleaning Open Source Projects

Categories > Data Processing > Data Cleaning
Openrefine ⭐ 9,155
OpenRefine is a free, open source power tool for working with messy data and improving it
total releases 5most recent commit a day ago
Great_expectations ⭐ 7,714
Always know what to expect from your data.
dependent packages 33total releases 179most recent commit 19 hours agopypi great-expectations} Downloads
Miller ⭐ 5,880
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
total releases 64most recent commit 3 days ago
Cleanlab ⭐ 4,271
The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
dependent packages 3total releases 18most recent commit 2 days agopypi cleanlab} Downloads
Pandera ⭐ 1,817
A light-weight, flexible, and expressive statistical data testing library
most recent commit 8 days ago
Pandas Videos ⭐ 1,808
Jupyter notebook and datasets from the pandas Q&A video series
most recent commit 7 months ago
Dat8 ⭐ 1,549
General Assembly's 2015 Data Science course in Washington, DC
most recent commit 2 months ago
Dataprep ⭐ 1,436
Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
dependent packages 1total releases 34most recent commit 15 days agopypi dataprep} Downloads
Optimus ⭐ 1,313
🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
total releases 27most recent commit 21 days ago
Janitor ⭐ 1,185
simple tools for data cleaning in R
dependent packages 42total releases 12most recent commit 7 days ago
Data Forge Ts ⭐ 1,169
The JavaScript data transformation and analysis toolkit inspired by Pandas and LINQ.
dependent packages 20total releases 141most recent commit 5 days agonpm data-forge} Downloads
Dirty_cat ⭐ 634
Machine learning on dirty tabular data
total releases 2most recent commit 3 days ago
Schema Inspector ⭐ 494
Schema-Inspector is a simple JavaScript object sanitization and validation module.
dependent packages 67total releases 42most recent commit 17 days agonpm schema-inspector} Downloads
Objectiv Analytics ⭐ 407
Powerful product analytics for data teams, with full control over data & models.
dependent packages 17total releases 44most recent commit a day agonpm @objectiv/developer-tools} Downloads
Klib ⭐ 341
Easy to use Python library of customized functions for cleaning and analyzing data.
total releases 57most recent commit 10 days ago
Validate ⭐ 341
Professional data validation for the R environment
dependent packages 10total releases 17most recent commit 6 months ago
Voicebook ⭐ 325
🗣️ A book and repo to get you started programming voice computing applications in Python (10 chapters and 200+ scripts).
most recent commit 16 days ago
Nonechucks ⭐ 315
Deal with bad samples in your dataset dynamically, use Transforms as Filters, and more!
total releases 18most recent commit a year ago
Hypergbm ⭐ 232
A full pipeline AutoML tool for tabular data
total releases 6most recent commit 12 days ago
Feature Engineering Tutorials ⭐ 217
Data Science Feature Engineering and Selection Tutorials
most recent commit 14 days ago
Pclean ⭐ 167
A domain-specific probabilistic programming language for scalable Bayesian data cleaning
most recent commit 6 months ago
Datamaid ⭐ 128
An R package for data screening
total releases 10most recent commit 10 months ago
Bumblebee ⭐ 120
🚕 A spreadsheet-like data preparation web app that works over Optimus (Pandas, Dask, cuDF, Dask-cuDF, Spark and Vaex)
most recent commit 2 months ago
Mzutils ⭐ 111
total releases 150most recent commit 4 months ago
Allie ⭐ 101
🤖 A machine learning framework for audio, text, image, video, or .CSV files (50+ featurizers and 15+ model trainers).
most recent commit 4 months ago
Refinr ⭐ 96
Cluster and merge similar char values: an R implementation of Open Refine clustering algorithms
total releases 4most recent commit 7 months ago
Holoclean Legacy Deprecated ⭐ 74
A Machine Learning System for Data Enrichment.
most recent commit 4 years ago
Akvo Lumen ⭐ 61
Make sense of your data
most recent commit 3 months ago
Covid_19_jhu_data_web_scrap_and_cleaning ⭐ 61
This repository contains data and code used to get and clean data from https://github.com/CSSEGISandData/COVID-19 and https://www.worldometers.info/coronavirus/
most recent commit 2 years ago
Data Analysis Using Python ⭐ 58
Exploratory data analysis 📊using python 🐍of used car 🚘 database taken from ⓚ𝖆𝖌𝖌𝖑𝖊
most recent commit 4 years ago
Numer.ai ⭐ 49
most recent commit 6 years ago
Taxa ⭐ 44
taxonomic classes for R
dependent packages 2total releases 10most recent commit 8 months ago
Pythresh ⭐ 40
Outlier Detection Thresholding
most recent commit 14 days ago
Lc Open Refine ⭐ 39
Library Carpentry: OpenRefine
most recent commit 5 days ago
Dtcleaner ⭐ 37
DTCleaner: data cleaning using multi-target decision trees.
most recent commit 6 years ago
Nepali Translator ⭐ 37
Neural Machine Translation on the Nepali-English language pair
most recent commit 6 days ago
Desbordante ⭐ 34
Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
most recent commit 5 days ago
Gratefuldata ⭐ 33
Grateful Data isn't programming code, but an online tutorial about data acquisition, cleaning and enriching, using publicly accessible data on the band the Grateful Dead as examples. Read the Wiki to find out how to use the sample data.
most recent commit 3 years ago
Pgdedupe ⭐ 32
A simple command line interface to the datamade/dedupe library.
total releases 7most recent commit 3 years ago
Redditcleaner ⭐ 31
Cleans Reddit Text Data 📜 :broom:
dependent packages 1total releases 4most recent commit 3 years agopypi redditcleaner} Downloads
Triebeard ⭐ 29
Radix trees in Rcpp and R
dependent packages 2total releases 2most recent commit a year ago
Foil ⭐ 27
Utilities for data cleaning and ETL processing
dependent packages 1total releases 17most recent commit 5 years agopypi foil} Downloads
Drugs Recommendation Using Reviews ⭐ 27
Analyzing the Drugs Descriptions, conditions, reviews and then recommending it using Deep Learning Models, for each Health Condition of a Patient.
most recent commit 3 years ago
Data Purifier ⭐ 26
A Python library for Automated Exploratory Data Analysis, Automated Data Cleaning, and Automated Data Preprocessing For Machine Learning and Natural Language Processing Applications in Python.
most recent commit 7 months ago
Covid 19 India Data ⭐ 25
data and code for scrapping and cleaning data on covid-19 in India from https://www.mohfw.gov.in/ and https://www.covid19india.org/
most recent commit 2 years ago
Covid 19 Data Cleanup ⭐ 25
Scripts to cleanup data from https://github.com/CSSEGISandData/COVID-19
most recent commit 2 years ago
Boltzmannclean ⭐ 21
Fill missing values in Pandas DataFrames using Restricted Boltzmann Machines
most recent commit 3 years ago
Foofah ⭐ 21
Foofah: programming-by-example data transformation program synthesizer
most recent commit 5 years ago
Openrefine Ecology Lesson ⭐ 21
Data Cleaning with OpenRefine for Ecologists
most recent commit 24 days ago
Datareused ⭐ 21
Get Data Reused
most recent commit 6 years ago
Fifa 2019 Analysis ⭐ 21
This is a project based on the FIFA World Cup 2019 and Analyzes the Performance and Efficiency of Teams, Players, Countries and other related things using Data Analysis and Data Visualizations
most recent commit 4 years ago
Cleaner.jl ⭐ 20
A toolbox of simple solutions for common data cleaning problems.
most recent commit 10 months ago
Multimodal Sentiment Analysis ⭐ 20
Engaged in research to help improve to boost text sentiment analysis using facial features from video using machine learning.
most recent commit 5 years ago
Fungible ⭐ 19
A library for fast reflective updates to immutable data trees
dependent packages 1total releases 4most recent commit 5 years agonuget Fungible} Downloads
Udacity Data Analyst Nanodegree ⭐ 19
most recent commit 4 years ago
Cleantext ⭐ 19
An open-source package for python to clean raw text data
dependent packages 2total releases 9most recent commit a year agopypi cleantext} Downloads
Amora Data Build Tool ⭐ 18
Amora Data Build Tool enables analysts and engineers to transform data on the data warehouse (BigQuery) by writing Amora Models that describe the data schema using Python's "PEP484 - Type Hints" and select statements with SQLAlchemy. Amora is able to transform Python code into SQL data transformation jobs that run inside the warehouse.
most recent commit 2 days ago
Errorlocate ⭐ 18
Find and replace erroneous fields in data using validation rules
total releases 9most recent commit 5 months ago
Learn2clean ⭐ 18
Learn2Clean: Optimizing the Sequence of Tasks for Data Preparation and Cleaning
most recent commit 2 years ago
Datacleanr ⭐ 17
Interactive and Reproducible Data Cleaning
most recent commit 5 months ago
Stata Economics ⭐ 16
Economics Lesson with Stata
most recent commit a year ago
Moodle Local_datacleaner ⭐ 16
Reduce, filter, and anonymize moodle data for non-prod environments
most recent commit a year ago
R Learning Journey ⭐ 16
Some of the projects i made when starting to learn R for Data Science at the university
most recent commit 3 years ago
Validatedb ⭐ 16
Validate on a table in a DB, using dbplyr
most recent commit 6 months ago
Pytrack ⭐ 15
a Map-Matching-based Python Toolbox for Vehicle Trajectory Reconstruction
total releases 8most recent commit 12 days ago
Twitter_sentiment_analysis_part2 ⭐ 15
redefining data-cleaning, preparation for visualisation
most recent commit 5 years ago
Openrefine Socialsci ⭐ 14
OpenRefine for Social Science Data
most recent commit a month ago
Exemplary Ml Pipeline ⭐ 14
Exemplary, annotated machine learning pipeline for any tabular data problem.
most recent commit 3 years ago
Dockingml ⭐ 14
A package for MD, Docking and Machine learning drug discovery pipeline
most recent commit 2 years ago
Dedupe ⭐ 13
Java DSL for (online) deduplication
dependent packages 1total releases 7most recent commit a year ago
Textcleaner ⭐ 13
text-data pre-processing utility
most recent commit 5 months ago
Datapipe ⭐ 12
dataPipe is a data processing and data analytics library for JavaScript. Inspired by LINQ (C#) and Pandas (Python)
dependent packages 2total releases 59most recent commit a month agonpm datapipe-js} Downloads
Titanic Survival In Depth Analysis ⭐ 12
Used Pandas , Matplotlib , Seaborn libraries to Analyze , Visualize and Explore the data of people travelling on Titanic, and Used Scikit-learn Modelling Algorithms to predict their probability of Survival.
most recent commit 3 years ago
Twitter Sentiment Analysis ⭐ 12
It is a Natural Language Processing Problem where Sentiment Analysis is done by Classifying the Positive tweets from negative tweets by machine learning models for classification, text mining, text analysis, data analysis and data visualization
most recent commit 4 years ago
World Food Production ⭐ 11
Comparing Top food and feed Producers around the globe and also seeking some interesting answers, solutions, patterns, hints and warnings through the power of Data Analysis and Data Visualization using Machine Learning.
most recent commit 3 years ago
Students Performance Analytics ⭐ 11
Students Performance Evaluation using Feature Engineering, Feature Extraction, Manipulation of Data, Data Analysis, Data Visualization and at lat applying Classification Algorithms from Machine Learning to Separate Students with different grades
most recent commit 2 years ago
Animefaceranker ⭐ 11
Anime Face Quality Check
most recent commit 3 years ago
Datawrangler ⭐ 10
Make quick and dirty data mining easier in Sublime Text
most recent commit 2 years ago
Arkhe ⭐ 10
🏺 Tools for cleaning rectangular data
most recent commit 8 days ago
Datasetops ⭐ 10
Fluent dataset operations, compatible with your favorite libraries
total releases 4most recent commit 2 months ago
Plane ⭐ 10
A text processing tool including tag(HTML, URL, Email) extraction and removing, punctuation normalization, simple segmentation, and so on.
dependent packages 1total releases 20most recent commit 8 months agopypi plane} Downloads
Xcel ⭐ 10
【项目已迁移到团队github】因此该 repository 只会同步最新的 README.md，若需要 watch、Star、Fork，则去团队的 github。谢谢。
most recent commit 5 years ago
Churn Modelling Dataset ⭐ 10
Predicting which set of the customers are gong to churn out from the organization by looking into some of the important attributes and applying Machine Learning and Deep Learning on it.
most recent commit 4 years ago
Marshmallow Pyspark ⭐ 10
Marshmallow serializer integration with pyspark
most recent commit a month ago
Tongdaxin Futures Data Clearing Database Operation ⭐ 10
对通达信数据进行去重和清洗处理，并将数据存入MongoDB，方便往后研究
most recent commit 5 years ago
Ipydataclean ⭐ 10
Interactive cleaning for Pandas DataFrames
most recent commit 3 years ago
Bllflow ⭐ 10
Big Life Lab Flow (BLLFlow) - a workflow for open, reproducible research. Includes support for PMML, DDI.
most recent commit 3 months ago
Web Scraper Sentiment Analysis Tripadvisor ⭐ 9
Academic project for Advances in Data Science and Architecture course
most recent commit 5 years ago
Data Forge Fs ⭐ 9
This library contains the file system extensions to Data-Forge that allow it to directly read and write CSV and JSON files in Node.js
most recent commit a year ago
Pywikimm ⭐ 9
Collects a multimodal dataset of Wikipedia articles and their images
most recent commit 2 years ago
Qualtricsdatacleaning ⭐ 9
R Tutorial: useful R codes for cleaning and filtering data from Qualtrics surveys, and for creating new variables in the dataframe. With step-by-step explanations.
most recent commit a year ago
Udacity Bertelsmann Data Science Challenge Scholarship 2018 ⭐ 9
This is a repo for my Bertelsmann Data Science Scholarship Challenge: notes, exercises, quizzes.
most recent commit 4 years ago
Black Friday Regression Analysis ⭐ 9
Predicting Prices for the products to be sold on Black Friday in US using Regression Analysis, Feature Engineering, Feature Selection, Feature Extraction and Data analysis - Data Visualizations.
most recent commit 4 years ago
Data_processing_and_feature_engineering_in_machine_learning ⭐ 9
This is an attempt to summarize feature engineering methods that I have learned over the course of my graduate school.
most recent commit 9 months ago
Google Job Skills ⭐ 9
Having an Exploratory Analysis at what kind of Jobs and Job Locations are provided by Google and Youtube, also we look into some specific details which are important to get hired by youtube and google.
most recent commit 3 years ago
Nyc Summons Precinct Visualization ⭐ 9
most recent commit 6 years ago
Deductive ⭐ 9
Methods for deductive data correction and imputation
total releases 5most recent commit 2 years ago
Cleanml ⭐ 9
A Benchmark for Joint Data Cleaning and Machine Learning
most recent commit 2 years ago
Metabar ⭐ 8
metabaR is an R package to curate and visualise DNA metabarcoding data after basic bioinformatics analyses.
most recent commit 2 months ago
Data Science Interview Prep ⭐ 8
Interactive notebooks to prepare for data science interviews. These notebooks are free and available for anyone to use!
most recent commit 2 years ago
1-100 of 462 projects
Categories
Advertising 📦 8
All Projects
Application Programming Interfaces 📦 107
Applications 📦 174
Artificial Intelligence 📦 69
Blockchain 📦 66
Build Tools 📦 105
Cloud Computing 📦 68
Code Quality 📦 24
Collaboration 📦 27
Command Line Interface 📦 38
Community 📦 79
Companies 📦 60
Compilers 📦 59
Computer Science 📦 73
Configuration Management 📦 37
Content Management 📦 153
Control Flow 📦 187
Data Formats 📦 72
Data Processing 📦 249
Data Storage 📦 116
Economics 📦 56
Frameworks 📦 178
Games 📦 110
Graphics 📦 92
Hardware 📦 137
Integrated Development Environments 📦 43
Learning Resources 📦 139
Legal 📦 24
Libraries 📦 117
Lists Of Projects 📦 19
Machine Learning 📦 313
Mapping 📦 57
Marketing 📦 15
Mathematics 📦 54
Media 📦 214
Messaging 📦 96
Networking 📦 292
Operating Systems 📦 72
Operations 📦 114
Package Managers 📦 50
Programming Languages 📦 173
Runtime Environments 📦 90
Science 📦 42
Security 📦 342
Social Media 📦 23
Software Architecture 📦 65
Software Development 📦 60
Software Performance 📦 53
Software Quality 📦 116
Text Editors 📦 40
Text Processing 📦 118
User Interface 📦 291
User Interface Components 📦 440
Version Control 📦 26
Virtualization 📦 63
Web Browsers 📦 35
Web Servers 📦 23
Web User Interface 📦 167
Top Programming Languages
Javascript  (1,121,606)
Python  (865,695)
Java  (404,416)
Php  (293,699)
Typescript  (285,088)
C Plus Plus  (252,382)
Ruby  (223,378)
C  (190,138)
Shell  (179,132)
C Sharp  (174,070)
Golang  (168,135)
Swift  (67,905)
Rust  (65,448)
R  (61,063)
Dart  (59,151)
Objective C  (57,045)
Kotlin  (51,453)
Lua  (37,172)
Matlab  (32,393)
Perl  (30,738)
Language  (29,926)
Scala  (29,146)
Haskell  (20,812)
Powershell  (20,003)
Clojure  (19,428)
Bash  (19,043)
Elixir  (17,208)
Coffeescript  (17,173)
Assembly  (15,260)
Julia  (13,695)
Processing  (13,306)
Emacs Lisp  (10,699)
Basic  (10,677)
Flow  (10,516)
Solidity  (9,601)
Erlang  (9,104)
Groovy  (8,541)
Scheme  (8,428)
Ocaml  (6,885)
Pascal  (6,720)
Hack  (6,554)
Lisp  (6,351)
Fortran  (5,737)
Elm  (5,546)
Common Lisp  (5,431)
Dlang  (4,783)
Shell Script  (4,773)
Python Library  (4,499)
Crystal  (4,424)
Scripting  (4,233)
Privacy | About | Terms | Follow Us On Twitter | Top Projects

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2022 Awesome Open Source.  All rights reserved.