deduplication
Entity resolution (also known as data matching, data linkage, record linkage, and many other terms) is the task of finding entities in a dataset that refer to the same entity across different data sources (e.g., data files, books, websites, and databases). Entity resolution is necessary when joining different data sets based on entities that may or may not share a common identifier (e.g., database key, URI, National identification number), which may be due to differences in record shape, storage location, or curator style or preference.
Here are 29 public repositories matching this topic...
A mirror of https://github.com/rustic-rs/rustic used to provide automatic docker images
-
Updated
Aug 10, 2024 - Rust
A pure rust library for parsing and generating zchunk file
-
Updated
Dec 28, 2023 - Rust
A file system that can be used to compare different deduplication algorithms.
-
Updated
Jun 5, 2024 - Rust
Remove local files that are duplicates of files in another path
-
Updated
Nov 24, 2020 - Rust
A tool to deduplicate backups. It builds a hash tree of all files and folders in the target directory. Optionally also traversing into archives like zip or tar files. The hash tree is then used to find duplicate files and folders.
-
Updated
Aug 12, 2024 - Rust
Deduplicate data by creating reflinks between identical files.
-
Updated
Apr 22, 2024 - Rust
Simple program that finds duplicate files, written in Rust.
-
Updated
Jun 4, 2017 - Rust
Yet Another Dupes Finder
-
Updated
Jul 31, 2024 - Rust
A command-line tool for deduplicating entries in a file or stream.
-
Updated
May 23, 2018 - Rust
A file deduplication app written in rust to be _blazingly fast_ with absolutely no unsafe code.
-
Updated
Jan 20, 2023 - Rust
Traveler: a file deduplication tool. 📂🔍🗂️
-
Updated
Mar 12, 2023 - Rust
🗃️🐊 fdupes-xenon is crposs-platform Rust version of fdupes utility with many cool features! (just for practice in Rust)
-
Updated
Sep 17, 2023 - Rust
Duplicate file/folder finder, can also scan in archives, HDD optimized
-
Updated
Jun 30, 2024 - Rust
A collection of algorithms to generate a signature/fingerprint/hash in order to be used for detecting duplicate/near duplicate documents.
-
Updated
Aug 9, 2017 - Rust
Created by Halbert L. Dunn
Released 1946
- Followers
- 38 followers
- Organization
- entity-resolution
- Wikipedia
- Wikipedia