Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Deduplication Open Source Projects

Open source projects categorized as Deduplication
Categories > Data Processing > Deduplication
Restic ⭐ 22,423
Fast, secure, efficient backup program
dependent packages 2total releases 83most recent commit 3 months ago
Borg ⭐ 10,158
Deduplicating archiver with compression and authenticated encryption.
most recent commit 3 months ago
Alertmanager ⭐ 6,107
Prometheus Alertmanager
dependent packages 361total releases 156most recent commit 3 months ago
Kopia ⭐ 5,678
Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
dependent packages 21total releases 233most recent commit 3 months ago
Duplicacy ⭐ 4,900
A new generation cloud backup tool
total releases 51most recent commit 3 months ago
Imagededup ⭐ 4,789
😎 Finding duplicate images made easy!
most recent commit 4 months ago
Dupeguru ⭐ 4,415
Find duplicate files
most recent commit 3 months ago
Libpostal ⭐ 3,897
A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.
most recent commit 3 months ago
Dedupe ⭐ 3,879
🆔 A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
dependent packages 10total releases 174most recent commit 4 months agopypi dedupe} Downloads
Node Dev ⭐ 2,234
Zero-conf Node.js reloading
dependent packages 389total releases 76most recent commit 7 months agonpm node-dev} Downloads
Rmlint ⭐ 1,692
Extremely fast tool to remove duplicates and other lint from your filesystem
total releases 1most recent commit 4 months ago
Jdupes ⭐ 1,679
A powerful duplicate file finder and an enhanced fork of 'fdupes'.
total releases 1most recent commit 6 months ago
Borgmatic ⭐ 1,637
Simple, configuration-driven backup software for servers and workstations
most recent commit 2 days ago
Yarn Deduplicate ⭐ 1,352
Deduplication tool for yarn.lock files
dependent packages 332total releases 20most recent commit 4 months agonpm yarn-deduplicate} Downloads
Rustic ⭐ 1,310
rustic - fast, encrypted, and deduplicated backups powered by Rust
total releases 21most recent commit 3 months ago
Zbox ⭐ 1,233
Zero-details, privacy-focused in-app file system.
dependent packages 3total releases 27most recent commit 3 years agocargo zbox} Downloads
Dwarfs ⭐ 1,207
A fast high compression read-only file system for Linux and Windows
most recent commit 3 months ago
Gifcap ⭐ 1,125
Capture your screen to a GIF in your browser
most recent commit 7 months ago
Attic ⭐ 1,083
Deduplicating backup program
most recent commit 5 years ago
Autorestic ⭐ 968
Config driven, easy backup cli for restic.
total releases 45most recent commit 3 months ago
Splink ⭐ 939
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
dependent packages 2total releases 119most recent commit 3 months agopypi splink} Downloads
Zingg ⭐ 828
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
total releases 1most recent commit 3 months ago
Recordlinkage ⭐ 808
A powerful and modular toolkit for record linkage and duplicate detection in Python
dependent packages 3total releases 23most recent commit 9 months agopypi recordlinkage} Downloads
The 10 Most Depended On Deduplication Open Source Projects
Fast Sass Loader ⭐ 239
High performance sass loader for webpack
dependent packages 8,103total releases 27latest release November 18, 2021most recent commit 2 years agonpm fast-sass-loader} Downloads
Node Dev ⭐ 2,234
Zero-conf Node.js reloading
dependent packages 389total releases 76latest release December 30, 2022most recent commit 7 months agonpm node-dev} Downloads
Alertmanager ⭐ 6,107
Prometheus Alertmanager
dependent packages 361total releases 156latest release August 24, 2023most recent commit 3 months ago
Yarn Deduplicate ⭐ 1,352
Deduplication tool for yarn.lock files
dependent packages 332total releases 20latest release May 11, 2023most recent commit 4 months agonpm yarn-deduplicate} Downloads
Unique Stream ⭐ 25
node.js through stream that emits a unique stream of objects based on criteria
dependent packages 77total releases 16latest release December 18, 2018most recent commit 3 years agonpm unique-stream} Downloads
Kuromojin ⭐ 78
Provide a high-level wrapper for kuromoji.js. Cache/Promise API
dependent packages 52total releases 16latest release April 24, 2021most recent commit 3 months agonpm kuromojin} Downloads
Gulp Dedupe ⭐ 6
Check for duplicates in the stream and filter them or throw an error
dependent packages 49total releases 2latest release October 03, 2014most recent commit 4 years agonpm gulp-dedupe} Downloads
Talisman ⭐ 666
Straightforward fuzzy matching, information retrieval and NLP building blocks for JavaScript.
dependent packages 48total releases 30latest release January 21, 2021most recent commit a year agonpm talisman} Downloads
Dedupe ⭐ 23
easy deduplication of array values
dependent packages 33total releases 24latest release December 23, 2022most recent commit a year agonpm dedupe} Downloads
Free Style ⭐ 697
Make CSS easier and more maintainable by using JavaScript
dependent packages 29total releases 54latest release January 24, 2021most recent commit 3 months agonpm free-style} Downloads
The 10 Latest Releases In Deduplication Open Source Projects
Nomenklatura ⭐ 171
Framework and command-line tools for integrating FollowTheMoney data streams from multiple sources
dependent packages 5total releases 122latest release December 01, 2023most recent commit 3 months agopypi nomenklatura} Downloads
Blobstash ⭐ 83
You personal database. Mirror of https://git.sr.ht/~tsileo/blobstash
latest release November 20, 2023most recent commit 4 years ago
Rustic ⭐ 1,310
rustic - fast, encrypted, and deduplicated backups powered by Rust
total releases 21latest release November 19, 2023most recent commit 3 months ago
Mail Deduplicate ⭐ 156
📧 CLI to deduplicate mails from mail boxes.
total releases 18latest release November 15, 2023most recent commit a month ago
Deduplicator ⭐ 225
Filter, Sort & Delete Duplicate Files Recursively
total releases 17latest release November 14, 2023most recent commit 5 months ago
Splink ⭐ 939
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
dependent packages 2total releases 119latest release November 14, 2023most recent commit 3 months agopypi splink} Downloads
Daxus ⭐ 87
Daxus is a server state management library for React that provides full control over data, leading to a better user experience.
dependent packages 1total releases 14latest release November 04, 2023most recent commit 3 months agonpm daxus} Downloads
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.