eRacks Systems Tech Blog

Open Source Experts Since 1999

Useful commands for backing up filesystems

Have you ever needed to backup the contents of one or more filesystems to another machine, yet you only had a single hard drive in the machine being backed up and found that you lacked the temporary disk space necessary to create your backup before shuttling it across the network to its final destination?

As an example, when I backup my laptop, I have too many gigabytes of data to realistically store my data on DVD-R’s, and my only option is to create a tarball of the root filesystem and store it on another machine on my network. The problem is that if I try to create a backup of my laptop’s contents, I find that the resulting tarball backup is too large to fit on the hard drive along with all the data.

One solution that I’ve found to this problem is to avoid storing the backup on the source machine altogether. Through stdin and stdout, along with the magic of *NIX pipes, we can stream the data in realtime over to its destination, and only then write it to disk.

Before we begin, it is very important to note that in most situations, you’ll have to boot into another environment and manually mount your partition before proceeding, particularly when dealing with an operating system’s root filesystem. Otherwise, not only will tar choke on certain directories like /proc and /dev, the contents of the disk will also continue to change as the backup is being made, leading to inconsistencies between the data on your filesystem and the data in the backup.

With that in mind, assuming that you have ssh installed and configured correctly on both the source and destination computers, you can create a backup with the following commands (as root):

#cd /path/to/your/mounted/filesystem
#tar -jcvp | ssh username@destination “cat > /path/to/backup.tar.bz2”

If you prefer to use gzip as opposed to bzip2, replace the above tar command with the following:

#tar -zcvp | ssh username@destination “cat > /path/to/backup.tar.gz”

Now, let’s say that you’ve created a new partition and want to restore a previous backup. Again, assuming that ssh is configured properly on the source and the destination machines, and assuming that you’ve mounted your partition, you would recover your backup with the following commands (again, as root):

#cd /path/to/your/mounted/filesystem
#ssh username@destination “cat /path/to/backup.tar.bz2” | tar -jvxp

If the backup is a gzipped archive, then replace the above tar command with the following:

#ssh username@destination “cat /path/to/backup.tar.gz” | tar -zvxp

Note that the user specified by ‘username’ above should have read/write permissions on the directory where the backup is to be stored for this procedure to work.

The astute reader will probably notice the missing -f option, which one usually passes to tar. The reason for this is that it tells tar to write its data to, or read its data from, a file. However, by ommitting it, we tell tar to send its output to stdout, or to receive its data from stdin when reading from an archive, which allows us to make use of pipes. It’s situations like these where the power of *NIX really shines!

May 28th, 2008

Posted In: Backups

Tags: , , , , , , ,

Leave a Comment

Asus eeePC: Hands-on from the Op Manager

Hello everyone out in the blogosphere (Look my vocabulary improved!) Allow me to introduce myself. I am Max, the Op Manager here at eRacks. Now that that’s out of the way, lets dig in!

I recently had the chance to lay my grubby mits on the latest ASUS eeePC, and am here to give my initial impressions. Now mind you, I am a very busy man (Darn starbucks being so far away!), so I only had a couple hours to play with this little PC, and I must say, I am impressed.

Now, Asus keeps a tight grip on the distribution of their eeePCs, and makes sure they get there asking price, so shopping around won’t net more than about a dollar in savings. I will chalk that up on the bad side of things. However, while a little on the high end of the price scale, for its functionality, let me tell you something that makes up for that 100x: it works flawlessly, it’s quick, and it gets a lot of looks (ladies, forget the new hairstyle. Pick up one of these bad boys and prepare for the geek onslaught!) The fact that I had no issues with it speaks volumes, because I always break something and have to have Tony, our Head Tech, come and save me.

It also comes with all the software you would need: open source applications, games, and media playing programs preinstalled and ready to go. You do have to sit through a quick registration screen at first to get to this, but hey, you have to do that with everything. When I started this mini-beast up, I was pleased to see that everything displayed quite nicely on the 7″ 800×480 res. screen. It even comes with a pretty nice Intel graphics chipset to boot. So, as far as visuals go, while you wont be seeing HD style graphics, you will get a clear, precise picture that makes working on it pretty easy. Not bad ASUS, not bad… But you could, ya know, boost the res up to maybe 1200? Maybe…please? C’mon…

Anyway, this is not by any means a replacement for a full fledged laptop, but it is a nice miniPC that will come in handy for a quick write up at a trade show, a place to store a few pictures, a checking of websites or emails from the airport or any number of road-warrior-like activities. The other thing it’s good for is KIDS! Kids love it; it comes in multiple colors, it plays games, it’s small, it’s neat, it makes noise on its 5.1 realtek HD sound card, it plays music AND it’s cool looking. The only problem I see with kids and this is that on the models we got, the keyboard is white (wash your hands, children, before touching it), so beware of dirty fingers! We actually had a customer call us and let us know that their children were hammering away on these things and that they stood the test of time (at that point, 1 week. But hey, it’s a miniPC and a child. Thats like platinum record status!) Another good feature is the card reader. This allows you to store plenty of files on the SD cards. Neat!

The few bad things I have to say are as follows: it only has 2 hours of battery life (I know, I know; laptops and such do not have amazing battery lives, but 2 hours?! I’ve had layovers longer then that on flights from OC to SF); it has no DVD or CD player, which is a bummer, even though I do understand that it’s a different category of PC — I still want to be able to throw a DVD in or listen to a CD I just bought (ok, that may be a lie; who really buys CD’s anymore, anyone? I admit it. I do. MP3’s be damned!); the graphics could be a bit better and the white keyboard is a parents nightmare, although at least the keys are stuck close enough together that food can’t hide in them. Overall, there weren’t enough bad things to warrant a bad review, or to take away from the coolness factor. pretty asus pc

In closing, I know this isn’t as in-depth or as technical as some people would like. But hey, I’m Max, and Max is allowed to write what he wants (you love the 3rd person, I know it!) Overall, I give this 4/5 stars for a mini pc on coolness factor, and 3.5/5 on tech factor. Take my opinion with a grain of salt though, for I am just an Op Manager doing my thing.

For the techies, here’s a rundown of the specs:

  • Processor: Intel Celeron M Processor 353 (800MHz, 512KB L2 Cache, FSB 400MHz)
  • Chipset: Mobile Intel 910 GML Exp ress
  • Memory: 512MB DDR2-400 Memory
  • Display: 7″ WVGA (800×4 80) TFT LCD
  • Graphics Module: Intel GMA 900 Graphics Controller
  • Hard Drive: 2GB Solid-State Disk
  • Optical Storage Device: None
  • Audio: Realtek ALC662 5.1-Channel High Definition Audio CODEC; Built-in Stereo Speak ers and Microphone
  • Connectivity: 10Base-T/100Base -TX Fast Ethernet; 802.11b/g Wireless LAN
  • Interface: 3x USB 2.0 Ports; 1x RJ45 LAN Port; 1x Headphone-out; 1x Microphone-in
  • Card Reader: SD Card Reader, Support SD/MMC
  • Camera: None
  • Battery Pack & Life: 4-cell 4400mAh Li-ion Battery, Upto 2.8 Hours Life Time
  • AC Adapter: Output – 9.5V, 22W; Input – 100~240V AC, 50/60Hz Universal
  • Dimensions (WxDxH): 8.86 x 6.46 x 0.846~1.4 inch / 22.5 x 16.4 x 2.15~3.5 cm
  • Color: Blush Pink, Blue, Green, White, Black
  • Weight: 2 lbs / 0.92 kg
  • Operating System: Linux

Visit www.eRacks.com for more info.

May 2nd, 2008

Posted In: New products

Tags: , , , ,

Leave a Comment

Using Open Source Tools to Encode Video for the Vidego28 MP4 Player

I recently purchased a Vidego28 handheld MP4 player with a 2.8” display – which is actually a rebranded ONDA VX858 – and discovered that it has very specific requirements for the videos it plays.

The manufacturer provides a Windows binary-only utility for converting your videos to something the Vidego28 can read, but because I don’t use Windows, and because I prefer using the command line for automating these sorts of tasks, I immediately set out to discover exactly what format my videos should adhere to.

Googling around, I found that the videos it plays should be 12bpp XviD encoded AVIs at or below 24 fps (frames per second) with a maximum bitrate (for the video) of 800kbit/s and a resolution of 320×240 (which is the size of the display, in pixels.) I also found that the audio portion of the AVI should be encoded in the MP2 format. Using mencoder (http://en.wikipedia.org/wiki/MEncoder), I cobbled together the following command to do the job, partly from experiment and partly from what I had found in someone else’s blog (all on one line):

mencoder -mc 0 -noodml inputfile.avi -of avi -o outputfile.avi -ofps 24 -vf-add scale=320:240 -vf-add expand=320:240:-1:-1:1 -srate 44100 -ovc xvid -xvidencopts bitrate=400:max_bframes=0:quant_type=h263 -oac twolame -twolameopts br=160

At this point, I had partial success. I was able to take my videos and convert them to a format the Vidego28 could read. The problem was that almost immediately during playback on either my computer or the MP4 player, the audio and video became so out of sync that it was impossible to make use of the files mencoder produced. I spent hours on Google looking for a solution and tweaking the above command, without success.

At some point during my experimenting, I discovered that ffmpeg (http://ffmpeg.mplayerhq.hu/) could produce AVI files as well, and decided to give it a shot. Unfortunately, the files it produced were not in a format the Vidego28 could read, but out of curiosity, I decided to see what would happen if I used mencoder to take the output of ffmpeg and re-encode it per the original command I had tried earlier, since the files ffmpeg produced were in sync when played back on my laptop. To my astonishment, for whatever reason, mencoder was able to output an AVI in which the audio and video remained at all times in sync when played on the Vidego28!

Finally, I decided to write a simple script so that I could automate the process of calling ffmpeg, followed by mencoder, the result of which is listed below:

vidego.bash:

#!/bin/bash
FPS=$1
INPUT=$2
OUTPUT=$3

# Make sure all the required arguments were passed
if [ -z $FPS ] || [ -z $INPUT ] || [ -z $OUTPUT ]; then
   echo
   echo "Usage: $0 [fps] [input file] [output file]"
   echo
   exit 1
fi

# Make sure $FPS is valid
if [ $FPS -lt 1 ] || [ $FPS -gt 24 ]; then
   echo
   echo "error: frames per second should be between 1 and 24"
   echo
   exit 1
fi

# Let's do our stuff!
ffmpeg -i $INPUT -r $FPS /tmp/$OUTPUT

mencoder -mc 0 -noodml /tmp/$OUTPUT -of avi -o $OUTPUT -ofps $FPS -vf-add \
scale=320:240 -vf-add expand=320:240:-1:-1:1 -srate 44100 -ovc xvid \
-xvidencopts bitrate=400:max_bframes=0:quant_type=h263 -oac twolame \
-twolameopts br=160

rm /tmp/$OUTPUT

The above script takes as parameters the desired FPS (which should be between 1 and 24), the source file (which can be any format that mencoder supports) and the destination (the file that will go on your MP4 device). A sample call would look like this:

vidego.bash 24 input.file output.avi

Just copy output.avi to your Vidego28 (or ONDA VX858), and you’ll be good to go!

April 28th, 2008

Posted In: multimedia

Tags: , , , , , , , ,

3 Comments

FreeBSD 6.3 and 7.0 Installation on the Compal IFL90 Laptop

NOTE: Do NOT install the ports tree during the installation. We’re going to install the latest ports tree (complete with security updates) later on.

*** Step 1: Install FBSD 6.3 or 7.0

1. Insert the CD
2. Choose the standard install
3. When the installation is finished, you’re done!

*** Step 2: Installing the Nvidia Drivers

NOTE: The open source ‘nv’ driver doesn’t work on the GeForce 8 series of cards, so don’t even bother trying.

1. Type the command ‘sysconfig’ and install the ‘compat5x’ package via FTP. This is a pre-requisite for the nvidia driver package.

2. Download the FreeBSD x86 drivers from www.nvidia.com for the GeForce 8 series.

3. Untar, cd to the resulting directory and type the command ‘make install’

4. Type the command ‘nvidia-xconfig.’ You’ll get some errors about Gnome libraries not being installed, but it still manages to create a good xorg.conf anyway.

5. Open /etc/X11/xorg.conf and change the following:

In Section “Input Device:”
Change Option “Device” “/dev/sysmouse” to “/dev/psm0”
Change Option “Emulate3Buttons” “no” to “yes”

6. Test the xorg server by typing ‘startx’ and verify that you get a working display

7. Close xorg and pat yourself on the back for a job well done 😉

*** Step 3: Install the Ports Tree

1. Go to ftp.freebsd.org and download the latest 6.3 or 7.0 ports.tgz file (under /pub/releases/6.3 or /pub/releases/7.0 I believe, but I can’t remember exactly.)

2. Cd to /usr and extract ports.tgz

*** Step 4: Configuring the Soundcard

1. Edit the file /boot/loader.conf and add the line “snd_hda_load=”YES”

2. Reboot and grep dmesg for Realtek. If it shows up, try to play an MP3 to make sure sound is really working.

*** Step 5: Wireless Configuration

We no longer need to do anything. As of FreeBSD 7.0, the wpi driver works by default! Wireless will not work,
however, under 6.3

March 31st, 2008

Posted In: Laptop cookbooks

One Comment

Ubuntu Install for Compal IFL90 laptop 

Ubuntu 7.10 Installation on the Micro Express IFL90 Laptop

1. Go into the BIOS by hitting the F2 key during post and make sure it can boot from CD

2. Insert the Ubuntu CD and reboot the system (make sure you use the amd64 version, since Intel's Core 2 Duo is x86_64.)

3. When the install menu for Ubuntu comes up, choose "Safe graphics mode install."  DO NOT try the normal install, or the system will hang.

4. Once the LiveCD is fully booted, you will see a desktop with an icon labeled "Installer."  Double click on it and wait for the installation application to open.

5. You will be presented with a series of questions related to the configuration of the system.  Making sure you setup the proper timezone and the sysadmin username, leave everything else at their default values.  When it tells you it's ready to install, click "ok" and wait for it to finish.

6. When the installation is complete, click on the "reboot" button.  The disk will be ejected, and the system will boot from the CD.

7. When you have booted successfully from the hard drive, and are presented with the graphical login prompt, hit ALT-F1 to drop down to virtual terminal and login as sysadmin.

8. As the sysadmin user, type the command "sudo passwd root."  Enter sysadmin's password when prompted to do so, then enter the root password for the system (eracks.)

9. Logout of the virtual terminal and hit F7 to go back to the graphical console and login as sysadmin.

10. You will be notified that there are updates available.  Go ahead and install them.

11. You will be notified that there are "restricted drivers" available.  Click on the little icon on the top right that looks like a little circuit board and click on the "Restricted Drivers" tab.

12. Make sure the intel wireless adapter is enabled, and also enable the Nvidia accelerated graphics driver.

13. On the system menu at the top, click on "System -> Administration -> Synaptic Package Manager"

14. Install the package "linux-backport-modules."  This is required to make the sound card work.

15. Edit the file /etc/modprobe.d/alsa-base, and add the following line at the bottom: "options snd-hda-intel model=toshiba"

16. Reboot the computer, and make sure to test the wireless adapter and sound card (by playing a sound) 

You're done!

March 31st, 2008

Posted In: Laptop cookbooks

Tags: , , ,

Leave a Comment

« Previous Page — Next Page »