Of course I now have far too much data for that to be practical so had to find a process that could be automated.
rsync seemed a sensible option having installed a second drive (see other articles on using fstab to mount ntfs drives ;) ) for the purposes of backups. Well infact it was still in the machine from when I was still using Windows as main OS as it was the backup drive for Windows. So rather than move lots of data about, far easier to get Linux to read NTFS.
I found a program called grsync that provides a graphical inferface (GUI) for rsync, which was great but required either being at my machine or tunnelling X output over ssh to launch.
Find a few flags in grsync that caused it to output the actual rsync command it ran to perform the backup and make a note of that command.
rsync -r -t --progress --delete /home/dafoot/ /media/backup/home_dafoot/
This would run an incremental backup of my data. Lovely.
That's a pain to remember, so I copied that into a shell script:
#!/bin/bash
rsync -r -t --progress --delete /home/dafoot/ /media/backup/home_dafoot/
and created an alias in my .bashrc file so it was a bit easier to remember:
alias backup='/home/dafoot/scripts/backupHome.sh'
Now I can simply type backup from the command line to run an incremental backup of my home directory.
Within my home directory there are a lot of files I don't need to backup, things like .cache and other program specific stuff. How can I save the disk space that these files are using?
The answer comes in the form of a flag to the rsync command, namely --exclude-from=<filename>
This allows me to create a file that contains all the files/directories I want rsync to ignore, I named the file backupexcludes.
[dafoot@bigfoot scripts]$ cat backupexcludes
#directories
.cache/
.mozilla/
.xsession*
#files
.bash_history
[dafoot@bigfoot scripts]$
A tweak to the backupHome.sh script and it now reads:
[dafoot@bigfoot scripts]$ cat backupHome.sh
#!/bin/bash
rsync -r -t --progress --delete --exclude-from=/home/dafoot/scripts/backupexcludes /home/dafoot/ /media/backup/home_dafoot/
[dafoot@bigfoot scripts]$
So now I can run 'backup' from the command line and it will run an incremental backup of my home directory, ignoring files I don't need to backup. As I learn what various directories are for and wether I need to keep them I can extend the list of regexp in backupexcludes to reduce space used on the backup drive by files I don't need to keep.
Job's a good 'un. My old hardware will now last a little longer as it will be longer before my drives reach capacity (which was starting to become an issue!).