Well, one problem with all my data migrations is how to know what’s really in a folder. I want a kind of graphical indication of what is different and it should be a low-level check first with date and time and size and finally with a CRC.

I’ve used GoodSync for this and you can also use rsync as well. There is a deep comparison possible with CRCcheckcopy as well and I tried hashdeep but it was just too slow.

Using GoodSync in Analysis only mode

The nice thing about GoodSync is that if you set it into Two Way mode you get rid of the GSync files and then just analyze the files, you can rapidly tell the differences. The current version is 12, but they have a trial mode which is nice for rapid comparisons. I used to use GoodSync extensively until Synology Replication and Drive Storage does that for me automatically.

The main issue I found is that it wants to create these _gsdata files so it can store things and it fails on network drives for some reason. I think because it doesn’t ask for a username and password, you have to go to Job > Job Options > Left Side > Folder Options > No _gsdata_ folder here and do the same for the Right Side. Alternatively, give GoodSync your credentials, its pretty hidden in the file browser, the is a small icon that is a person, put it there and the icon will change to your user name

The user interface is nice though compared with rsync and you get a nice display of the files that are not present. Note that this doesn’t do a CRC comparison so for things like video files and that.

Rhash more flexible and newer

Rhash is in Homebrew and do the same thing recursively go through a directory and generate a checksum which you can then verify

So with rhash, the incantations are where it is fun to pick the right hashing function and yes I’m not sure why the script below doesn’t use the default checksum CRC32 which is fast and not for security but. to look for changes, but the basic recommendation is crc32 is very fast 750ms in the benchmark vs Sha3-256 at 1450ms.

In my benchmarks over the network, the sha3-256 was running at 24MBps and crc32c at 23MBps because the network is the limitation, not the checksum, so the script below just uses SHA3-256 for grins as it doesn’t affect compute time for networked drives.

# create the checksum -r means recursive
# -P show percentages and --speed shows stats
# --sha3-256 is the hashing function if you want security
# --crc32c is fine for just hashing
# -u update an existing has file
cd /Volumes/Movies
rhash -r --sha3-256 -P --speed -u ./rhash.sha256.txt *
# then make a copy and check the files against the has
rsync -avnCP /Volumes/Movies /Volumes/Movies-backup
cd /Volumes/Movies-backup
# now check the hashes
rhash -r -c ./rhash.sha256.txt *

Hashdeep is older and not updated

The only thing about Hashdeep is that it is very popular but also very old, but the equivalent of creating a checksum is below and note you don’t get your choice of hashes:

# create the hash recursively
hashdeep -r /Volumes/Movies > /Volumes/hashdeep.txt
# copy the backup with a checksum
rsync -avncP /Volumes/Movies /Volumes/Movies-backup
# audit the backup with -a with the -k keyfile
hashdeep -a -k /Volumes/Movies/hashdeep.txt -r /Volumes/Movies-backup

Rsync for the command line version

The basic rsync incantation looks like this where -c means do the slow checksum. If you omit this then it will be very fast. Note that with Synology, the volumes are at the root and the shared folders are next, so the command below tells you which files are new in the Movies directory:

# ssh into a Synology server assuming you have ssh setup
ssh yourname@yourservername.local
# remember that Linux wildcards are great so to see similar directories
ls -ld /vol*/Movies*
# get ready for things to take a long time as it does a CRC
# with the -c option 
rsync -vnarcP /volume1/Movies/ /volume2/Movies-Backup

CRCCheckCopy is a third alternative

Like hashdeep, it constructs a file called CRCstamps.txt that has the CRC of every file you give it. And then you can make a copy and verify that the CRCs are identical. Note that unlike rsync you have to run on your Mac as it doesn’t support links. Also note that it is pretty clunky for a Mac person, but it seems to work.

# on your Mac, use finder to connect and then go to command line
cd /Volumes/Movies
# generates a CRCstamps in the current working directory
# assume you copied CRCCheckCopy into your local bin
~/bin/crcheckcopy /s .
# now do a copy and check CRC (this will be slow)
rsync -anvcP /Volumes/Movies /Volumes/Movies-Backup
# now compare the CRCs
~/bin/crccheckcopy /v /Volumes/Movies-Backup
# if you are a geek, stick CRCstamps.txt into the backup
cd /Volumes/Movies-backup
~/bin/crccheckcopy /s .

Synology DSM to remove the shared folder from the Control Panel

I always forget how to do this because you have to go to the Synology > Control Panel > Shared Folder and. look for the share and choose Delete