Dedup - Program to clean up duplicate files

This program has born out of 2 goals :

• Clean up my media collection like family photos, etc
• Learn Rust programming in the process.

Usage

dedup 0.2.0
Sriharsha Mucheli <[email protected]>
Program to find duplicate files and take care of them

USAGE:
    rdedup [FLAGS] [OPTIONS] --remote-path <remote_path>

FLAGS:
    -c, --commit     Performs a dry run by default. Use this option to commit file deletions
    -d, --debug      Activate debug mode
    -h, --help       Prints help information
    -V, --version    Prints version information
    -v, --verbose    Verbose mode (-v, -vv, -vvv, etc.)

OPTIONS:
    -H, --hash-algo <hash_algo>        Type of Hashing algorigthm to use for checksumming. [default: Md5]
    -l, --local-path <local_path>      Local Path containing files that need to be checked for duplicates [default: .]
    -R, --remote-list <remote_list>    File containing list of remote files and hashes
    -r, --remote-path <remote_path>    Remote path to use as a reference to filter duplicates in local

Options

--remote-path and --remote-list are in conflict. Only one of them should be specified.

The hashes captured in the remote list file should match the hash algorithm. Otherwise, no duplicates would be found.

Commit changes (-c --commit)

By default, the program performs a dry run. Pass the flag -c or --commit to actually perform the deletions.

Hash Algorithm (-H --hash-algo)

Supports usage of MD5, SHA128, SHA256 and SHA512. Can be specified with the -H or --hash-algo option. MD5 is used by default if nothing is specified.

Remote File (-R --remote-list)

The file to be passed to --remote-list option expects content to be in a format that can be generated by a find-exec or find-xargs chain. The input can come from a file or from stdin to chain commands.

Example:

$ find Scout -type f -exec md5sum {} \;
ae58348db17d3bfcf2310b08b85f8109  Scout/IMG_20151209_125608.jpg
10985fdbba07e6cae7867b522edad028  Scout/IMG_20151209_122811.jpg
5a7de7fb546e92996dc373e396207685  Scout/IMG_20151209_122322.jpg
.
.
.

$ find Scout -type f | xargs -I {} md5sum {}
ae58348db17d3bfcf2310b08b85f8109  Scout/IMG_20151209_125608.jpg
10985fdbba07e6cae7867b522edad028  Scout/IMG_20151209_122811.jpg
5a7de7fb546e92996dc373e396207685  Scout/IMG_20151209_122322.jpg
.
.
.