Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Deduplication Open Source Projects

Open source projects categorized as Deduplication
Categories > Data Processing > Deduplication
Restic ⭐ 21,272
Fast, secure, efficient backup program
dependent packages 2total releases 81most recent commit 2 days ago
Borg ⭐ 9,810
Deduplicating archiver with compression and authenticated encryption.
most recent commit 2 days ago
Alertmanager ⭐ 5,915
Prometheus Alertmanager
dependent packages 338total releases 153most recent commit a day ago
Duplicacy ⭐ 4,769
A new generation cloud backup tool
total releases 51most recent commit a day ago
Kopia ⭐ 4,690
Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
dependent packages 21total releases 230most recent commit 17 hours ago
Imagededup ⭐ 4,625
😎 Finding duplicate images made easy!
most recent commit 2 months ago
Dupeguru ⭐ 3,990
Find duplicate files
most recent commit 2 months ago
Libpostal ⭐ 3,776
A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.
most recent commit a month ago
Dedupe ⭐ 3,708
🆔 A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
dependent packages 10total releases 174most recent commit 5 months agopypi dedupe} Downloads
Node Dev ⭐ 2,234
Zero-conf Node.js reloading
dependent packages 379total releases 76most recent commit 25 days agonpm node-dev} Downloads
Jdupes ⭐ 1,668
A powerful duplicate file finder and an enhanced fork of 'fdupes'.
total releases 1most recent commit 10 days ago
Rmlint ⭐ 1,600
Extremely fast tool to remove duplicates and other lint from your filesystem
total releases 1most recent commit 2 months ago
Borgmatic ⭐ 1,486
Simple, configuration-driven backup software for servers and workstations
most recent commit a day ago
Yarn Deduplicate ⭐ 1,334
Deduplication tool for yarn.lock files
dependent packages 245total releases 20most recent commit a month agonpm yarn-deduplicate} Downloads
Zbox ⭐ 1,233
Zero-details, privacy-focused in-app file system.
dependent packages 3total releases 27most recent commit 2 years agocargo zbox} Downloads
Dwarfs ⭐ 1,158
A fast high compression read-only file system for Linux and Windows
most recent commit a day ago
Gifcap ⭐ 1,125
Capture your screen to a GIF in your browser
most recent commit a month ago
Attic ⭐ 1,083
Deduplicating backup program
most recent commit 5 years ago
Rustic ⭐ 1,078
rustic - fast, encrypted, and deduplicated backups powered by Rust
total releases 19most recent commit 6 days ago
Autorestic ⭐ 840
Config driven, easy backup cli for restic.
total releases 38most recent commit 2 months ago
Recordlinkage ⭐ 808
A powerful and modular toolkit for record linkage and duplicate detection in Python
dependent packages 1total releases 22most recent commit 2 months agopypi recordlinkage} Downloads
Splink ⭐ 788
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
dependent packages 1total releases 72most recent commit 2 days agopypi splink} Downloads
Zingg ⭐ 784
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
most recent commit 2 days ago
The 10 Most Depended On Deduplication Open Source Projects
Fast Sass Loader ⭐ 239
High performance sass loader for webpack
dependent packages 6,063total releases 27latest release November 18, 2021most recent commit 2 years agonpm fast-sass-loader} Downloads
Node Dev ⭐ 2,234
Zero-conf Node.js reloading
dependent packages 379total releases 76latest release December 30, 2022most recent commit 25 days agonpm node-dev} Downloads
Alertmanager ⭐ 5,915
Prometheus Alertmanager
dependent packages 338total releases 153latest release December 22, 2022most recent commit a day ago
Yarn Deduplicate ⭐ 1,334
Deduplication tool for yarn.lock files
dependent packages 245total releases 20latest release May 11, 2023most recent commit a month agonpm yarn-deduplicate} Downloads
Talisman ⭐ 666
Straightforward fuzzy matching, information retrieval and NLP building blocks for JavaScript.
dependent packages 43total releases 30latest release January 21, 2021most recent commit 6 months agonpm talisman} Downloads
Kuromojin ⭐ 69
Provide a high-level wrapper for kuromoji.js. Cache/Promise API
dependent packages 40total releases 16latest release April 24, 2021most recent commit 2 months agonpm kuromojin} Downloads
Free Style ⭐ 698
Make CSS easier and more maintainable by using JavaScript
dependent packages 24total releases 54latest release January 24, 2021most recent commit 2 years agonpm free-style} Downloads
Dedupe ⭐ 3,708
:id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
dependent packages 10total releases 174latest release February 17, 2023most recent commit 5 months agopypi dedupe} Downloads
Fastcdc Rs ⭐ 102
FastCDC implementation in Rust
dependent packages 5total releases 8latest release January 08, 2022most recent commit 14 days agocargo fastcdc} Downloads
The 10 Latest Releases In Deduplication Open Source Projects
Restic ⭐ 21,272
Fast, secure, efficient backup program
dependent packages 2total releases 81latest release July 31, 2023most recent commit 2 days ago
Acid Store ⭐ 87
A transactional and deduplicating virtual file system
dependent packages 5total releases 24latest release July 15, 2023most recent commit 3 months agocargo acid-store} Downloads
Rustic ⭐ 1,078
rustic - fast, encrypted, and deduplicated backups powered by Rust
total releases 19latest release June 07, 2023most recent commit 6 days ago
Yarn Deduplicate ⭐ 1,334
Deduplication tool for yarn.lock files
dependent packages 245total releases 20latest release May 11, 2023most recent commit a month agonpm yarn-deduplicate} Downloads
Dedupe ⭐ 3,708
:id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
dependent packages 10total releases 174latest release February 17, 2023most recent commit 5 months agopypi dedupe} Downloads
Node Dev ⭐ 2,234
Zero-conf Node.js reloading
dependent packages 379total releases 76latest release December 30, 2022most recent commit 25 days agonpm node-dev} Downloads
Alertmanager ⭐ 5,915
Prometheus Alertmanager
dependent packages 338total releases 153latest release December 22, 2022most recent commit a day ago
Nomenklatura ⭐ 165
Framework and command-line tools for integrating FollowTheMoney data streams from multiple sources
dependent packages 4total releases 67latest release December 19, 2022most recent commit 2 days agopypi nomenklatura} Downloads
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Classification
Data Science
Scraper
Transaction
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2023 Awesome Open Source.  All rights reserved.