Rsync Backup on Linux - Chris Titus Tech
video description
Date: 2022-03-20
Related videos
Comments and reviews: 10
optionstraderdan
I have been using Linux Lite and Peppermint os Linux for about 4 years now and it's time to upgrade a few machines. I added a new 4TB WD Red Drive to my Master heaviest duty fastest PC that was running Linux Lite 14.04 LTS and added the new drive via fstab to mount to the /media/ folder at boot time. I then watched another one of your videos that showed how to set up a Samba Shared Folder and after a little tweaking, I was able to accomplish that. I then went over to my old Toshiba Laptop that has a 250GB HD in it and modified the fstab to auto -mount to the new shared folder on a 2TB Partition I made on the new 4TB drive on the Master PC. Working on the command line is a little tedious, but I managed to watch this video a couple of times and was able to read between the lines a little bit about how to get the rsync configured using my specific folder, share device and backup names. I'm still not totally convinced I did it correctly, but it seems to be attached and able to create, read and write files and folders to the Master PC Shared folder on the new HD.
I followed your command line almost exactly, but got a little confused about whether to use the actual Samba shared folder name or the actual names and tree structure to point to the folder. Anyway after a little testing and tweaking, it appears to point to the right folder seems to be working. I fired off the backup job and went to dinner, and when I came back everything was standing still, but there were no errors on the screen. Everything just seemed to stop for no apparent reason. There was plenty of space on the Master drive to store the files, and I really could not see any reason for the pause. Nothing I could do would move the job along. I went into task manager on the source PC and rsync appeared to be in use, but after waiting almost 30 min, no change in status of the backup job in terminal. I tried to kill the job with a Ctrl-Z in terminal, and that didn't yield any results. I finally closed the terminal and then restarted a new terminal and tried to run the rsync job again, but this time to a new directory. I got an error that I Googled, and it said that either the target drive was out of space (which it was not as it was a new 2TB Partition that I placed on the 4GB Drive), OR the connection simply timed out. I believe this was what happened. The author suggested adding --Timeout=120 to the rsync command to prevent timeouts from occuring for 2 minutes. But, I wanted to kill the existing job and start over, so I went into task manager and tried to kill the tasks from there, but no dice either. I don't believe I started task manager as a Root User. Unfortunately, after a few minutes the laptop locked up and mouse froze. The only way to shut it down was a hard power off and restart. It came up just fine, so I tried to run the job again, but this time to a new target directory and using the additional --timeout=120 command line variable, and it kicked off and has been running for over an hour just chugging along backing up this old laptop. It's using wired ethernet connection through my gigabit network, but this old Laptop has only a 100Mbit connection, so this will take a while as the drive was about 90% full. The first job backed up about 10% of the laptop's files when it stopped working, so I could tell by checking available disk space on the target drive on the Master PC that so far I have backed up much more data than the first time. My goal it to basically replace Peppermint OS 8 (Ubuntu 16.04) with the latest version of Peppermint OS which I believe is Peppermint OS 10 Respin. I was planning on re-partitioning the Laptop HD into multiple partitions as outlined in one of your other videos to make this job easier next time I update. I will have to do the upgrade on the Master PC as well because it is running ver 14.04 LTS of Linux Lite. It was my very first Linux installation back in the day, and I DID originally put multiple partitions on that one for the root, boot, swap and home directories. In fact the entire OS is on an SSD drive, and my data is on a 2nd traditional HD, so that one should be easier to update I hope.
Anyway, thanks a million for your hard work on these videos! It sure helps out people like me that enjoy a little challenge, but really love the stability of Linux compared to the old Windbloz Software we had to endure for many years in the corporate world. Thanks Again from Central Florida!!!
reply
I have been using Linux Lite and Peppermint os Linux for about 4 years now and it's time to upgrade a few machines. I added a new 4TB WD Red Drive to my Master heaviest duty fastest PC that was running Linux Lite 14.04 LTS and added the new drive via fstab to mount to the /media/ folder at boot time. I then watched another one of your videos that showed how to set up a Samba Shared Folder and after a little tweaking, I was able to accomplish that. I then went over to my old Toshiba Laptop that has a 250GB HD in it and modified the fstab to auto -mount to the new shared folder on a 2TB Partition I made on the new 4TB drive on the Master PC. Working on the command line is a little tedious, but I managed to watch this video a couple of times and was able to read between the lines a little bit about how to get the rsync configured using my specific folder, share device and backup names. I'm still not totally convinced I did it correctly, but it seems to be attached and able to create, read and write files and folders to the Master PC Shared folder on the new HD.
I followed your command line almost exactly, but got a little confused about whether to use the actual Samba shared folder name or the actual names and tree structure to point to the folder. Anyway after a little testing and tweaking, it appears to point to the right folder seems to be working. I fired off the backup job and went to dinner, and when I came back everything was standing still, but there were no errors on the screen. Everything just seemed to stop for no apparent reason. There was plenty of space on the Master drive to store the files, and I really could not see any reason for the pause. Nothing I could do would move the job along. I went into task manager on the source PC and rsync appeared to be in use, but after waiting almost 30 min, no change in status of the backup job in terminal. I tried to kill the job with a Ctrl-Z in terminal, and that didn't yield any results. I finally closed the terminal and then restarted a new terminal and tried to run the rsync job again, but this time to a new directory. I got an error that I Googled, and it said that either the target drive was out of space (which it was not as it was a new 2TB Partition that I placed on the 4GB Drive), OR the connection simply timed out. I believe this was what happened. The author suggested adding --Timeout=120 to the rsync command to prevent timeouts from occuring for 2 minutes. But, I wanted to kill the existing job and start over, so I went into task manager and tried to kill the tasks from there, but no dice either. I don't believe I started task manager as a Root User. Unfortunately, after a few minutes the laptop locked up and mouse froze. The only way to shut it down was a hard power off and restart. It came up just fine, so I tried to run the job again, but this time to a new target directory and using the additional --timeout=120 command line variable, and it kicked off and has been running for over an hour just chugging along backing up this old laptop. It's using wired ethernet connection through my gigabit network, but this old Laptop has only a 100Mbit connection, so this will take a while as the drive was about 90% full. The first job backed up about 10% of the laptop's files when it stopped working, so I could tell by checking available disk space on the target drive on the Master PC that so far I have backed up much more data than the first time. My goal it to basically replace Peppermint OS 8 (Ubuntu 16.04) with the latest version of Peppermint OS which I believe is Peppermint OS 10 Respin. I was planning on re-partitioning the Laptop HD into multiple partitions as outlined in one of your other videos to make this job easier next time I update. I will have to do the upgrade on the Master PC as well because it is running ver 14.04 LTS of Linux Lite. It was my very first Linux installation back in the day, and I DID originally put multiple partitions on that one for the root, boot, swap and home directories. In fact the entire OS is on an SSD drive, and my data is on a 2nd traditional HD, so that one should be easier to update I hope.
Anyway, thanks a million for your hard work on these videos! It sure helps out people like me that enjoy a little challenge, but really love the stability of Linux compared to the old Windbloz Software we had to endure for many years in the corporate world. Thanks Again from Central Florida!!!
reply
midplanewanderer
Question (relatively new user here, Linux Mint 19.1): my home folder on my desktop is less than 5 Gigs (some files -unreadable,- if that matters);
to save my personal data, firefox and GUI settings (in-case of a hard-drive melt-down) can I not just merely copy my home folder off the desktop, dump it on a thumb-drive, and call it -Done?- So that, in the event of a hard-drive failure, I can just dump all my settings and tweaks stored in the home folder on the thumb drive into another system running a fresh install of Mint 19.1, or whatever, and carry-on? Or is the Rsync route recommended for ease of re-installation?
My understanding is that TimeShift (which I use) is sorta hit-and-miss if you're using it to save to an external drive to save personal data and profiles to re-upload to a fresh install. Seems to be a lot of differing points-of-view here...
reply
Question (relatively new user here, Linux Mint 19.1): my home folder on my desktop is less than 5 Gigs (some files -unreadable,- if that matters);
to save my personal data, firefox and GUI settings (in-case of a hard-drive melt-down) can I not just merely copy my home folder off the desktop, dump it on a thumb-drive, and call it -Done?- So that, in the event of a hard-drive failure, I can just dump all my settings and tweaks stored in the home folder on the thumb drive into another system running a fresh install of Mint 19.1, or whatever, and carry-on? Or is the Rsync route recommended for ease of re-installation?
My understanding is that TimeShift (which I use) is sorta hit-and-miss if you're using it to save to an external drive to save personal data and profiles to re-upload to a fresh install. Seems to be a lot of differing points-of-view here...
reply
santiago
Thanks Chris, I was always a bit confused but -a so I took the time to crack it down:
-a, --archive archive mode; equals -rlptgoD
-r : recursive
-l: copy symlinks as symlinks
-p: preserve permissions
-t: preserve modification times
-g:preserve group
-o:preserve owner
-D: same as devices specials (don't really know what this is) x
So now I see what you were saying, -a is a sensible way to copy.
------
-AX i don't really understand what they are useful for, even though Chris explained it.
reply
Thanks Chris, I was always a bit confused but -a so I took the time to crack it down:
-a, --archive archive mode; equals -rlptgoD
-r : recursive
-l: copy symlinks as symlinks
-p: preserve permissions
-t: preserve modification times
-g:preserve group
-o:preserve owner
-D: same as devices specials (don't really know what this is) x
So now I see what you were saying, -a is a sensible way to copy.
------
-AX i don't really understand what they are useful for, even though Chris explained it.
reply
benriful
I might be wrong here, but I think you can do better with rsync. Instead of having it store onto a network mount, instead install the rsync server on that NAS of yours. Then point your rsync job to the server. This way when it tests for differences it need not download the entire file from the NAS - the NAS can calculate the hash locally and then the only network traffic become the difference of the file, not the entire thing back and forth.
reply
I might be wrong here, but I think you can do better with rsync. Instead of having it store onto a network mount, instead install the rsync server on that NAS of yours. Then point your rsync job to the server. This way when it tests for differences it need not download the entire file from the NAS - the NAS can calculate the hash locally and then the only network traffic become the difference of the file, not the entire thing back and forth.
reply
shoe68w
-homergfunk I know you try to keep your content general and helpful to new Linux users. Have you thought of doing a basic intro to scripting? For example: an auto-apt-update script that would run through update, upgrade, autoremove, full-upgrade/dist-upgrade for Debian based distros. The same can be done for pacman, yum, etc. The idea being to show people that scripting is really not that hard and maybe perk some interest.
reply
-homergfunk I know you try to keep your content general and helpful to new Linux users. Have you thought of doing a basic intro to scripting? For example: an auto-apt-update script that would run through update, upgrade, autoremove, full-upgrade/dist-upgrade for Debian based distros. The same can be done for pacman, yum, etc. The idea being to show people that scripting is really not that hard and maybe perk some interest.
reply
Pavel
Could you please explain the following case. I have virtual server Ubuntu on AWS, I need to backup data from it to my local FreeNAS which is behind NAT. So I need to pull data from remote rsync to local rsync. Is it possible to do it? Will it work behind the NAT? I need a bit more information about network interaction. Which part will be an initiator in this scheme?
reply
Could you please explain the following case. I have virtual server Ubuntu on AWS, I need to backup data from it to my local FreeNAS which is behind NAT. So I need to pull data from remote rsync to local rsync. Is it possible to do it? Will it work behind the NAT? I need a bit more information about network interaction. Which part will be an initiator in this scheme?
reply
Yelsin
Thank you for the video! Worked like a charm!
I am now trying to do the same with a network mounted driver ( from a Windows machine ). The files transferred successfully, but I got a few error before the transfer started:
'rsync: symlink -/mnt/SG2000/Backups/12-1-2021/bin- -> -usr/bin- failed: Operation not supported (95)'
What do ?
reply
Thank you for the video! Worked like a charm!
I am now trying to do the same with a network mounted driver ( from a Windows machine ). The files transferred successfully, but I got a few error before the transfer started:
'rsync: symlink -/mnt/SG2000/Backups/12-1-2021/bin- -> -usr/bin- failed: Operation not supported (95)'
What do ?
reply
Chris
I think you should probably have mentioned it was fast because you'd already run the command and copied all of the data over. Subsequent runs will only copy the deltas. Also, something to remember, if you use --delete and your source gets wiped, when you next run your backup job the destination will also get wiped.
reply
I think you should probably have mentioned it was fast because you'd already run the command and copied all of the data over. Subsequent runs will only copy the deltas. Also, something to remember, if you use --delete and your source gets wiped, when you next run your backup job the destination will also get wiped.
reply
Raymond
What you missed it doing it to a new work drive. Well I guess you just had one mounted but it you don't have to do it something like this commend:
rsync -avP /from/ root-server:/to
So the from and too and you could switch this if taking it the other way from a server some other place even in the cloud.
reply
What you missed it doing it to a new work drive. Well I guess you just had one mounted but it you don't have to do it something like this commend:
rsync -avP /from/ root-server:/to
So the from and too and you could switch this if taking it the other way from a server some other place even in the cloud.
reply
GZ
So this is for backing up personal data (you demonstrated backing up the home folder), but for a system backup, like before doing a system-wise arch upgrade, you still need timeshift, right? So if that upgrade fails you can restore from the timeshift backup, but the rsync backup won't do that?
reply
So this is for backing up personal data (you demonstrated backing up the home folder), but for a system backup, like before doing a system-wise arch upgrade, you still need timeshift, right? So if that upgrade fails you can restore from the timeshift backup, but the rsync backup won't do that?
reply
Add a review, comment
Other channel videos
