Advice on massive file copies...

Michael Torrie torriem at gmail.com
Tue Dec 31 09:25:51 MST 2019


On 12/31/19 8:22 AM, John Von Essen wrote:
> Any ideas what could be going on? Is this the best way to do this?
> Maybe just doing cp would be easier/cleaner, is there something
> better then rsync to use? I just dont want ot have to copy 800GB
> everytime I sync. Maybe I use rsync in combination with find to walk
> to file tree and rsync each file one by one?

Rsync over cifs could be quite slow, especially if it has to compute
hashes for all the files to see if they've changed. I think what might
be happening is cifs doesn't report file times in a resolution high
enough to satisfy rsync's algorithm.  Try the flags that are typically
recommended for use on Windows file systems, like

--modify-window 1

And maybe also

--size-only

But try to avoid that last one if you can.  You might look into
--ignore-errors also, but that really only applies to the --delete flags.

The best bet in my mind is to rsync over ssh.  That way you're running
on the real file system.  Pretty sure QNAP can do that out of the box.
https://wiki.qnap.com/wiki/Rsync


More information about the PLUG mailing list