Virtualization: Xen and KVM
We use virtualization *a lot* in our infrastructure. The advantages are numerous but for sure it saves costs, makes management easier and improves security by isolating services on their own virtual machines (and you can make a very specific configuration for the service).
Putting that aside we were moderately happy about Xen. We have been using Xen since 3.0 and lets say the battle for the optimal configuration was not an easy one. Xen has regularly given us bad magic in the past. In the end it the magic mostly disappeared but it still does have a nasty side (i hate the xen network scripts, even though we disable them nowadays) and many bugs. The latest version 3.2 is definitely the best till now and Xen is stable nowadays, at least in our "optimal configuration".
The most unpractical things with Xen after you get rid of the bad magic effect are the limited available choice of kernels (a pain with new hardware support and security). We sticked to paravirtualization till now because of its performance advantages. Which brings another pain with Xen: upgrading to new versions or migrating guests between Xen servers running different version (you can copy new kernel modules in the guest everytime).
A curse here and there is definitely on its place.
Then meet KVM. If your used to Xen (open source version) then it seems like virtualization like god intented. We tested KVM for a while now and everything is just like you expect and no nasty scripts everywhere (at least I didn't see them yet).
Virsch is quite handy too (although you also seem to able to use it with Xen, we never did). Although I must curse shortly about virt-install for its bugs and ubuntu-vm-builder for the lack of support for installing debian (which are both not part of kvm, but can be used to "easy install" guests).
In any case we are giving kvm a proper testing now (for at least a month) and it looks something that might replace Xen entirely in our infrastructure on the long term. It is just sweet.
One reason to blog about this is the following:
- What are you experiences with running kvm in production?
I am also very interested in experiences with running a windows *server* guest with continuous reasonable load. Notes about any quirks or limitations encountered by experience are very welcome as well.
Weak SSL key vulnerabilities not so funny
Yesterday evening I had the pleasure to pick up the following three security notices:
- [USN-612-1] OpenSSL vulnerability
- [USN-612-2] OpenSSH vulnerability
- [USN-612-3] OpenVPN vulnerability
I can tell you these are really not funny. They really generate a lot of work indirectly. Annoying but doable are things like regeneration SSH keys. The PITA situation is with OpenVPN, looks like we have the push new keys out to all the clients in a few setups. That really deserves a curse or two.
As a special bonus:
Once the update is applied, weak shared encryption keys and
SSL/TLS certificates will be rejected where possible (though
they cannot be detected in all cases). If you are using such
keys or certificates, OpenVPN will not start and the keys or
certificates will need to be regenerated.
Which means I am happy to read security notices. Just updating might result in a broken setup.
If anybody could give extra information on how weak those keys actually are and how easy they are to crack, I would be delighted. In the meanwhile, looking at the amount of servers, I guess I can schedule a teambuilding event where we can mass regenerate keys.
ps. I can tell you I'm quite happy that our backup machines which use SSH+RSYNC for automatic incremental backups are not vulnerable.
Comments broke after upgrading to wordpress 2.5
I borked something when I upgraded to wordpress 2.5. Ofcourse things break down together once again:
1) The cute little "challenge" plugin stopped working in the sense it blocked all comments even when the "challenge" got properly solved (not just spam and stupid people).
2) After disabling the plugin I found that the admin interface couldn't view comments anymore.
Thankfully the following error showed up in the apache2 logs:
WordPress database error Key 'comment_date_gmt' doesn't exist in table 'wp_comments' for query SELECT SQL_CALC_FOUND_ROWS * FROM wp_comments USE INDEX (comment_date_gmt) WHERE comment_approved != 'spam' ORDER BY comment_date_gmt DESC LIMIT 0, 25 made by _wp_get_comment_list, referer: http://dev.osso.nl/herman/blog/wp-admin/edit-comments.php?comment_status=approved
Apparently Wordpress 2.5 expects an INDEX on comment_date_gmt, which was not there.
ALTER TABLE `wp_comments` ADD INDEX ( `comment_date_gmt` ) ;
Fixed the problem. Maybe a side effect of of continuosly upgrading wordpress (I would guess adding the index was missing in an upgrade script along the way). Else lets blame it without any proof on MySQL for losing data randomly.
So feel free to leave comments once again!
ps. If someone still has tips to show off virtualisation in Ubuntu on the desktop I would love to hear it. I was kind of hoping on some responses on my last post.
dhcpd 3.0.2 horribly broken
This week we upgraded one of our routers from hoary (Ubuntu 5.04) to breezy (Ubuntu 5.10). However dhcpd didn't work properly anymore. Thinclients got several leases: one IP for PXE dhcp request and another ip for the normal startup. Not really lease friendly. I'm not sure this has anything to do with the "no free leases" problem on the edubuntu list but I would not be surprised.
With other options, the dhcpd server just looped (filling up the leases file) which made it stop functioning completely. Ask Peter for details. Anyway dhcp3-server in breezy is totally borked.
We solved the problem by backporting the dapper version which works flawlessly with the config with we had in hoary.
We have a pretty complex dhcpd.conf file, not something you get installed by default. However if you run into wierd problems on breezy. Don't waste your time and backport dhcp3-server (dhcpd 3.0.3) from dapper.
Firefox: Lets hog more memory!
Firefox apparantly has a new feature which caches pages in memory. YEAH LETS HOG EVEN MORE MEMORY to make the end user experience faster.
I sure hope its possible to disable this feature by default. Firefox seems to calculate the amount of crap it will cache based on available RAM.
Maybe OTHER processes might like a little memory as well? Nooo, ofcourse not, Firefox is all you need!
I also wonder if they ever heard of a thing called a TERMINAL SERVER. Can you imagine this little memory eating scheme with a Terminal server with 4GB of ram. With lets say 20 users. That must be some memory massacer.
This is undoubtly the most NOT WELL THOUGHT ABOUT feature of Firefox I know off. I certainly hope Ubuntu/Edubuntu will disable it by default in their packages. If not I guess its bye-bye Firefox.
Read details on: http://weblogs.mozillazine.org/ben/archives/009749.html
Update: Besides the ranting I did something useful, I filed a bug. 
Twitter: hpbos
- Gisteren the men who stare at goats gezien. :-) jeff bridges++
- @Ronald_R: Anders heb je nog Zombie Strippers! als optie.
- @Ronald_R Lesbian Vampire Killers... http://www.pajiba.com/trailers/new-lesbian-vampire-killers-trailer.php
- @joostdrijver stink voetbal, mijn reistijd naar kantoor verdubbeld in de laatste 500m.
- wie wil er vanavond "the men who stare at goats" met mij kijken? :P
- Rabobank internetbankieren is buggy as hell.
- Net film the fountain gezien. Wel mooi gemaakt maar niet mijn ding.
- Groningen binnen. Niet bijzonder. Nu coma tijd.
- Wat een kak verhaal van rouvoet. Ook met 4 procent geteld, lame.
Categories
- Advocacy
- Development
- Entertainment
- Evil
- Humor
- In the news
- Orbit Ubuntu
- Personal life
- Security
- Tech
- Travel
- Work
