We've recently got a small FitPC for an IRC server and firewall on our LAN. They're very small being only the size of a stack of five CD cases, but powerful enough to run a normal Windows or Linux operating system. They're very silent and only consume 4-6W.
Hardy upgrade problems
The unit came installed with Ubuntu 7.10 but it seemed to have broken APT sources, so rather than mess around I thought I'd instead do a dist-upgrade to 8.04. When it rebooted after the upgrade it would fail to boot and end up at a busybox prompt saying it couldn't find the device to boot off.
This can be fixed by hitting ESC when the grub boot-loader starts and editing the first item (by pressing "e") then changing the "root" entry from the long UUID value to "/dev/sda1" (then hit enter and "b" to boot the new setting).
It still fails to boot into X, but thats a problem for another day because we're only using it headless as a firewall and IRC server. When it reports problems booting X, select the option to drop into a root prompt and login.
The first thing to do is make the /dev/sda1 boot device permenant by editing the /boot/grub/menu.lst file, then install openssh-server and configure it so you can log in remotely from here on.
USB memory stick installation
I haven't been able to get it to boot from a memory stick that most normal PC's can boot without problem. There are various notes around the place suggesting that this is very difficult to get working. The proper procedure is supposed to be to set the C drive to "USB Hard Drive" in the BIOS and then set the boot order to "C drive" first.
Interestingly if you remove the power, the FitPC's main board (but not the hard drive) can keep running if there are powered devices like a hub in one of its USB ports.