Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Deduplication Open Source Projects

Open source projects categorized as Deduplication
Categories > Data Processing > Deduplication
Blobfs ⭐ 11
New project: https://git.sr.ht/~tsileo/blobfs
total releases 1most recent commit 4 years ago
Rss Braider ⭐ 10
Braid/aggregate one or more RSS feeds (file or url) into a single feed (RSS or JSON output). Process resulting feed through specified plugins. Automatic deduplication.
dependent packages 3total releases 11most recent commit 6 years agonpm rss-braider} Downloads
Sdhash ⭐ 10
Python library for image hashing and deduplication
dependent packages 1total releases 4most recent commit 8 years agopypi sdhash} Downloads
Soph ⭐ 10
Efficiently import pictures while handling duplicates gracefully
most recent commit 2 years ago
Phrontend Webpack ⭐ 10
A webpack config maker for phrontend apps
dependent packages 3total releases 10most recent commit 8 years agonpm phrontend-webpack} Downloads
Unisim ⭐ 10
UniSim is a package for efficient similarity computation, fuzzy matching, and clustering of data.
most recent commit 5 months ago
Dedup ⭐ 10
Find duplicate text files.
most recent commit a year ago
Data Deduplication ⭐ 10
Multiple ways of chunking for data deduplication: Fixed size chunking, Content defined chunking, and File based chunking.
most recent commit 10 years ago
Patentsview Evaluation ⭐ 10
Evaluation and benchmarking of PatentsView disambiguation algorithms
most recent commit 4 months ago
Imgdup ⭐ 10
Visual similarity image finder and cleaner
total releases 6most recent commit 9 years ago
Dynamic Dedupe ⭐ 10
Dedupes node modules as they are being required which works even when dependencies are linked via ln -s or npm link.
dependent packages 14total releases 4most recent commit 7 years agonpm dynamic-dedupe} Downloads
Tweet Deduplication ⭐ 10
A stream of deduplicated tweets built using RxJava and Twitter4J
most recent commit 8 years ago
Asysd ⭐ 9
most recent commit 4 months ago
Oc_graphenricher ⭐ 9
A tool to enrich any OCDM compliant Knowledge Graph, finding new identifiers and deduplicating entities.
total releases 7most recent commit 7 months ago
Dedup Resolve Webpack Plugin ⭐ 9
Webpack plugin that resolves copies of the same module in different locations to a single path.
total releases 1most recent commit 7 years ago
Atomic Style ⭐ 9
Atomic class sets and dedupes
dependent packages 1total releases 2most recent commit 8 years agonpm atomic-style} Downloads
Dmdedup4.8 ⭐ 9
Device-mapper Deduplication Target for linux kernel 4.8
most recent commit 6 years ago
A Collection Of Oss Tools ⭐ 9
most recent commit 6 years ago
Lafs Backup Tool ⭐ 9
Tool to securely push incremental (think "rsync --link-dest") backups to tahoe-lafs
most recent commit 8 years ago
Archipelago ⭐ 9
A distributed storage layer that decouples Volume and File operations/logic from the actual underlying storage technology.
most recent commit 8 years ago
Spatiotemporalresampling ⭐ 9
Absolutely Eliminate Anime Stutters By SpatiotemporalResampling
most recent commit 6 months ago
Deduplication Detecting ⭐ 9
文档去重功能是为了解决搜索引擎的文档语义重复的问题，方法是多重哈希下的语义指纹算法。
most recent commit 11 years ago
Qpc ⭐ 8
A tool for discovery, inspection, collection/deduplication, and reporting on an IT environment
most recent commit 4 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.