eRacks Systems Tech Blog

Useful commands for backing up filesystems

Have you ever needed to backup the contents of one or more filesystems to another machine, yet you only had a single hard drive in the machine being backed up and found that you lacked the temporary disk space necessary to create your backup before shuttling it across the network to its final destination?

As an example, when I backup my laptop, I have too many gigabytes of data to realistically store my data on DVD-R’s, and my only option is to create a tarball of the root filesystem and store it on another machine on my network. The problem is that if I try to create a backup of my laptop’s contents, I find that the resulting tarball backup is too large to fit on the hard drive along with all the data.

One solution that I’ve found to this problem is to avoid storing the backup on the source machine altogether. Through stdin and stdout, along with the magic of *NIX pipes, we can stream the data in realtime over to its destination, and only then write it to disk.

Before we begin, it is very important to note that in most situations, you’ll have to boot into another environment and manually mount your partition before proceeding, particularly when dealing with an operating system’s root filesystem. Otherwise, not only will tar choke on certain directories like /proc and /dev, the contents of the disk will also continue to change as the backup is being made, leading to inconsistencies between the data on your filesystem and the data in the backup.

With that in mind, assuming that you have ssh installed and configured correctly on both the source and destination computers, you can create a backup with the following commands (as root):

#cd /path/to/your/mounted/filesystem
#tar -jcvp | ssh username@destination “cat > /path/to/backup.tar.bz2”

If you prefer to use gzip as opposed to bzip2, replace the above tar command with the following:

#tar -zcvp | ssh username@destination “cat > /path/to/backup.tar.gz”

Now, let’s say that you’ve created a new partition and want to restore a previous backup. Again, assuming that ssh is configured properly on the source and the destination machines, and assuming that you’ve mounted your partition, you would recover your backup with the following commands (again, as root):

#cd /path/to/your/mounted/filesystem
#ssh username@destination “cat /path/to/backup.tar.bz2” | tar -jvxp

If the backup is a gzipped archive, then replace the above tar command with the following:

#ssh username@destination “cat /path/to/backup.tar.gz” | tar -zvxp

Note that the user specified by ‘username’ above should have read/write permissions on the directory where the backup is to be stored for this procedure to work.

The astute reader will probably notice the missing -f option, which one usually passes to tar. The reason for this is that it tells tar to write its data to, or read its data from, a file. However, by ommitting it, we tell tar to send its output to stdout, or to receive its data from stdin when reading from an archive, which allows us to make use of pipes. It’s situations like these where the power of *NIX really shines!

james May 28th, 2008

Posted In: Backups

Tags: backup, bzip, bzip2, filesystem, gzip, partition, ssh, zip