Herman Bos open source guerilla

Playing around with Xen Express

Since the Xen PV drivers for Windows only work on the commercial/closed source Xen versions I ended up doing a test run of Xen Express. Its a bit nasty as you can expect but its much better then I expected.

The installation

You boot up the Xen Express install disk, pass a simplified text based installer (redhat style) and reboot. After this you can run the Windows or Linux client to connect to your fresh Xen express server and get to work.

The shipped GUI for Linux works fine (not bad for java). If you use Ubuntu you can easily use `alien` to turn the rpms in debs. If you install the packages everything ends up in /opt.

Looking at Xen Express without a GUI

Fortunately its still possible to login on the system on the console or through ssh. A customized redhat system will present itself to you. There is a small root partition and the rest of the data is in LVM. For every disk there will be an LVM block device (which will presented as disks to the started xen domains). If you login you can also place installation iso's on the filesystem, which would make you able to select the iso's in the GUI as cdrom volume and use them as installation source (else you have to insert a cd in the server).

The crap part, which was to be expected: Your limited if you want to work from a shell. The logical volumes in LVM are named with a long UUIDs. I can imagine why thats likeable the way they build it, but it makes a quick snap shot of a volume difficult (especially when you have a lot of volumes). Together with DomU config files that are screwed up (every thing on one line and UUID filenames) this kills off the fun stuff. The one thing that still works is the "xm" tool, didn't test it extensive but at least `xm list` still works. So you can probably still do the basic operations (start, restart, shutdown, destroy, etc.).

Xen Express GUI, Windows PV drivers that work and the rest of it

When I started this post I was still mildly positive, but unfortunately it has been in my drafts for a week now. Now finishing it up a week later.

The windows PV drivers basicly give you native performance. Thats great. The GUI is not _that_ bad either. Annoying at some points but you can live with it. A few screenshots: (uh, got 'm at the office, wait a day).
Now that rest of it: Things that suck:

• Dom0 kernel is mutilated
  Many many drivers are missing. It doesn't even have the iptables physdev module to filter on a bridge. On the support pages I also found a lot of complaints about missing RAID drivers, etc.
• Software RAID big PITA
  The installer doesn't support making any software raid devices. Its not impossible to do so afterward (except for the root filesystem I guess) but its certainly a PITA. I tried to move the LVM partition to an md device, succeeded but when I removed the non raid LVM pv I got errors after reboot. Too bad. However I'm sure its possible. Another way I didn't try since I didn't want to waste more time is to make a new "SR" or something like that. Its to be able to swap disks to one machine to another. Quite sure you could fix up a raid volume and make it used by Xen. This way you would be able to have software raid for your VM's at least.
• Only HVM for Windows. Linux is still para-virtualized
  Maybe nice for performance but I want the real virtualization so I can do anything I want in the guest domain. You can get it to install but its more like tricking the GUI. Which also means you can forget about paravirtualized drivers (crap performance).
• Can't define vif-interface names
  Every DomU gets a virtual network interface (vif), unfortunately in Xen express you can't define it. So you can never make any firewall rules (if the DomU gets restarted the vifname changes accordingly with the dom-id. Please fix that you can set dom-id or vifname.
• The OS templates are really messed up
  If you install debian you'll be surprised its so quick to install. Nice? No. The installation is so screwed over, it sucks balls. Besides that you get gdm and vnc for free. I didn't login with VNC it might hold more surprises. The crap thing of these templates is that the ones shipped are useless. In unofficial docs you can find how to make these templates but I didn't bother since we already found the whole Xen Express system too limited for our situation.
• Firewalling your VM's is a PITA
  Mostly because of the implications mentioned above its a pain to firewall your Windows machine in Linux. From Dom0 it would be impossible, so we got this fix: Create a second bridge. Create a VM from the debian template system with two nics. Put the first interface in a bridge with the peth0, put the second one in a bridge with your Windows VM. Do the filtering in the debian instance. Most transparant way is to create a bridge device (put eth0 and eth1 in it) and filter the traffic with the physdev iptables module. If you want to reach this machine from the outside you can give an IP to the bridge device. I hope it helps.

I recommend to try Xen Express out if you like a lot of pain and like to see if it really stinks for yourself or if your a sore Windows user which doesn't realise the limitations and is surprisingly happy when buttons do work.

If you were doing advanced stuff with the open source Xen version and like your tools and freedom, stay away from this. If you have Linux experience and want to run Linux VM's: use the open source version. IMO the only reason you can consider using Xen Express is that the Windows PV drivers work. Even with Windows for real production work, dunno, you should weigh the options vs. a dedicated server.