How to copy large amounts of files in Windows - Chris Titus Tech
video description
Date: 2022-03-21
Related videos
Comments and reviews: 10
Steve
The GUI copy will not preserve security settings most of the time, usually just replacing the source ACL's with the target folders ACL instead. Very bad if you need those ACL's preserved intact.
I used robocopy extensively to migrate a file server with over 2 TB of roaming profiles years ago. Did one pass to get the bulk data mirrored to the new system (ran overnight, took hours), command saved for later use. Later we just ran it again and again to get the modified files (took minutes each time). The ability to save logs each run helped us sort out a number of access issues reliably in the end too (use /NDL and /NFL to only log errors and the summary).
You can also use it (in rare cases) to salvage data from unreliable systems or drives as it can resume or at least skip files already copied. Had to do this ONE TIME to get data off a system with failing raid drives (lesson here is to actually check your raid and hardware logs regularly and not wait to the alarms to sound, yellow and red drive lights don't always get noticed). We managed to save all but two files if I recall correctly but it was touch and go for sure.
reply
The GUI copy will not preserve security settings most of the time, usually just replacing the source ACL's with the target folders ACL instead. Very bad if you need those ACL's preserved intact.
I used robocopy extensively to migrate a file server with over 2 TB of roaming profiles years ago. Did one pass to get the bulk data mirrored to the new system (ran overnight, took hours), command saved for later use. Later we just ran it again and again to get the modified files (took minutes each time). The ability to save logs each run helped us sort out a number of access issues reliably in the end too (use /NDL and /NFL to only log errors and the summary).
You can also use it (in rare cases) to salvage data from unreliable systems or drives as it can resume or at least skip files already copied. Had to do this ONE TIME to get data off a system with failing raid drives (lesson here is to actually check your raid and hardware logs regularly and not wait to the alarms to sound, yellow and red drive lights don't always get noticed). We managed to save all but two files if I recall correctly but it was touch and go for sure.
reply
Lat
Excellent subject-- one week short lol. That explained all the duplicate files and why nothing could be found. Even when I located it that did not transfer to the very next file in same folder. Still going through all 7 ssd-s and finding whole directories 3 or 4x deep under user/name/etc. dupchecker helpful as long as you tell it where to look. I wish it would scan everything everywhere and show which are complete and working and which can be trashed. So far though I was able to get back 6Gb of space. All I wanted was keep some important program files and empty each drive start with reformatted and clean pc. That didn-t go as expected. Lol.
reply
Excellent subject-- one week short lol. That explained all the duplicate files and why nothing could be found. Even when I located it that did not transfer to the very next file in same folder. Still going through all 7 ssd-s and finding whole directories 3 or 4x deep under user/name/etc. dupchecker helpful as long as you tell it where to look. I wish it would scan everything everywhere and show which are complete and working and which can be trashed. So far though I was able to get back 6Gb of space. All I wanted was keep some important program files and empty each drive start with reformatted and clean pc. That didn-t go as expected. Lol.
reply
it_industry
Large numbers of files, maybe a large quantity of files, even a large volume of files, but not a large amount of files. Sorry - that triggered me. Amount is so often incorrectly used these days, and often in professional media, that the distinctions are often forgotten. (Simplistically - Number for countable objects, Amount for non-countable objects)
Useful video too. Personally I would prefer a GUI, but I completely get that a multi-threaded solution has benefits of the standard copy tool in Explorer
reply
Large numbers of files, maybe a large quantity of files, even a large volume of files, but not a large amount of files. Sorry - that triggered me. Amount is so often incorrectly used these days, and often in professional media, that the distinctions are often forgotten. (Simplistically - Number for countable objects, Amount for non-countable objects)
Useful video too. Personally I would prefer a GUI, but I completely get that a multi-threaded solution has benefits of the standard copy tool in Explorer
reply
Kevin
one thing i don't like is, the /MT flag is only for file copyings. when ever i do server migrations there are multi millions of files and lots and lots of folders. It would be really cool if the multithread could handle directory transversal as well. it never fails the initial population to the destination takes weeks then one needs to run a delta to catch new/modified files. Robocopy is a huge waste of resources when its only 1 thread to traverse the files/folders.
reply
one thing i don't like is, the /MT flag is only for file copyings. when ever i do server migrations there are multi millions of files and lots and lots of folders. It would be really cool if the multithread could handle directory transversal as well. it never fails the initial population to the destination takes weeks then one needs to run a delta to catch new/modified files. Robocopy is a huge waste of resources when its only 1 thread to traverse the files/folders.
reply
Chris
It should be noted and I missed it in the video. If you are copying TONs of very small 1-100kb files you can completely turn off logging with /NFL /NDL /NJH /NJS - This will 10-100x the speed, but at the cost of logging.
NFL = No File List
NDL = No Directory List
NJH = No Job Header
NJS = No Job Summary.
You can copy 100,000s to millions of small files extremely fast, but will have no summary or log.
reply
It should be noted and I missed it in the video. If you are copying TONs of very small 1-100kb files you can completely turn off logging with /NFL /NDL /NJH /NJS - This will 10-100x the speed, but at the cost of logging.
NFL = No File List
NDL = No Directory List
NJH = No Job Header
NJS = No Job Summary.
You can copy 100,000s to millions of small files extremely fast, but will have no summary or log.
reply
BagelBytes
Windows Explorer copy and paste on my PC seems to work faster than RoboCopy utility somehow? For instance, I am getting about 113MB/s with the standard explorer copy, and RoboCopy gets roughly the same, but sometimes lower. Another example, between my 2 NVME SSD's, Explorer copy gets 1.2GB/s, and RoboCopy gets -800MB/s so about 1/3 slower than just normal copy and paste. Any ideas why this might be?
reply
Windows Explorer copy and paste on my PC seems to work faster than RoboCopy utility somehow? For instance, I am getting about 113MB/s with the standard explorer copy, and RoboCopy gets roughly the same, but sometimes lower. Another example, between my 2 NVME SSD's, Explorer copy gets 1.2GB/s, and RoboCopy gets -800MB/s so about 1/3 slower than just normal copy and paste. Any ideas why this might be?
reply
Adam
Hi Chris, thanks for this video. I will be helping a relative transfer basically everything on her old laptop to a newer one. Is connecting them with a crossover Ethernet cable and then using robocopy the best way to approach this?
Her old laptop is win 7 and the new one is win 10.
reply
Hi Chris, thanks for this video. I will be helping a relative transfer basically everything on her old laptop to a newer one. Is connecting them with a crossover Ethernet cable and then using robocopy the best way to approach this?
Her old laptop is win 7 and the new one is win 10.
reply
CoolDudeClem
This looks more like a Linux way of doing things! It may be faster, but typing in all that stuff would actually take me a lot longer than just a couple of mouse clicks here and there, plus I'd mess up somewhere. That's why I can't use Linux. Give a GUI any day.
reply
This looks more like a Linux way of doing things! It may be faster, but typing in all that stuff would actually take me a lot longer than just a couple of mouse clicks here and there, plus I'd mess up somewhere. That's why I can't use Linux. Give a GUI any day.
reply
Simon
The video should be called -How to copy large numbers of files-. The title actually means that that you're going to show people how to partially copy their files. I would guess that's even more difficult to achieve but slightly less useful :-)
reply
The video should be called -How to copy large numbers of files-. The title actually means that that you're going to show people how to partially copy their files. I would guess that's even more difficult to achieve but slightly less useful :-)
reply
R-mulo
Not gonna lie, I tought that I would never use this when I saw your video, until I had to backup my sister's external hdd.-
This is a god sent, the only thing that I miss is a general progress bar to the whole process...
reply
Not gonna lie, I tought that I would never use this when I saw your video, until I had to backup my sister's external hdd.-
This is a god sent, the only thing that I miss is a general progress bar to the whole process...
reply
Add a review, comment
Other channel videos
