Wordpress got a bit slow
Today I noticed my wordpress installation got a bit slow. Apparently MySQL was very busy... Why?
...
SELECT * FROM wp_comments ORDER BY comment_date_gmt DESC LIMIT 32500, 50
SELECT * FROM wp_comments ORDER BY comment_date_gmt DESC LIMIT 32650, 50
SELECT * FROM wp_comments ORDER BY comment_date_gmt DESC LIMIT 32800, 50
... etc.
The need to run very many queries like that at the moment you open your wordpress dashboard page is not obvious to me.
Anyway "33062 comments deleted" helped a lot.
Weak SSL key vulnerabilities not so funny
Yesterday evening I had the pleasure to pick up the following three security notices:
- [USN-612-1] OpenSSL vulnerability
- [USN-612-2] OpenSSH vulnerability
- [USN-612-3] OpenVPN vulnerability
I can tell you these are really not funny. They really generate a lot of work indirectly. Annoying but doable are things like regeneration SSH keys. The PITA situation is with OpenVPN, looks like we have the push new keys out to all the clients in a few setups. That really deserves a curse or two.
As a special bonus:
Once the update is applied, weak shared encryption keys and
SSL/TLS certificates will be rejected where possible (though
they cannot be detected in all cases). If you are using such
keys or certificates, OpenVPN will not start and the keys or
certificates will need to be regenerated.
Which means I am happy to read security notices. Just updating might result in a broken setup.
If anybody could give extra information on how weak those keys actually are and how easy they are to crack, I would be delighted. In the meanwhile, looking at the amount of servers, I guess I can schedule a teambuilding event where we can mass regenerate keys.
ps. I can tell you I'm quite happy that our backup machines which use SSH+RSYNC for automatic incremental backups are not vulnerable.
Wordpress is a pain
I'm happy I hardened my webserver setup a bit and all our blogs are running on a seperate virtual machine but damn, wordpress is a pain to maintain. I updated it less then a year ago and I'm already hopelessly out of date!
Also this time there are a whole lot of security updates of course. Enough fun for the weekend. I would probably have updated more often for security updates but everytime i try to get subscribed to the wordpress mailinglist i see no results (maybe it looks too much like spam;)).
We develop all new projects with Python and Django at the moment and it feels really attractive to just kick out Wordpress and swap it in with a simple django app. Ofcourse that would require some time I don't want to put in at the moment, but its definitely in my wishlist.
Going from wordpress 2.1.3 to 2.3.3 this weekend! yay!
ps. I really don't feel like to prepare dinner today! (painfully obvious when I even start blogging about wordpress)
Vista vies bah
Last week I got my first look on Windows Vista. I guess they outdid themselves on making something horrible. The interface is freaking slow, lots of popups too. It looks much more like the Mac now btw, just in a more failed way. I sure hope Ubuntu doesn't pursue something like Vista else you'll find me still using dapper in six years.
Funny you can see that most of the Windows users will start using Vista because it will be shipped on their new PC. Besides that its a big opportunity for the commercial Linux players: Vista sucks so bad it should even get interesting for Windows fanboys to switch to Linux. In the business world its most interesting ofcourse, since they tend to switch a bit less reckless even to the new Windows version which makes Linux a very good alternative.
Edit: My negative rant is based on the look & feel of vista. Just my first impression, I didn't start using it. I don't judge its capabilitities. I don't care, I wouldn't be able to get any work (at least not mine) done on Windows. Its made for mom, pop and the guy who wanks on his smooth animated menus.
Playing around with Xen Express
Since the Xen PV drivers for Windows only work on the commercial/closed source Xen versions I ended up doing a test run of Xen Express. Its a bit nasty as you can expect but its much better then I expected.
The installation
You boot up the Xen Express install disk, pass a simplified text based installer (redhat style) and reboot. After this you can run the Windows or Linux client to connect to your fresh Xen express server and get to work.
The shipped GUI for Linux works fine (not bad for java). If you use Ubuntu you can easily use `alien` to turn the rpms in debs. If you install the packages everything ends up in /opt.
Looking at Xen Express without a GUI
Fortunately its still possible to login on the system on the console or through ssh. A customized redhat system will present itself to you. There is a small root partition and the rest of the data is in LVM. For every disk there will be an LVM block device (which will presented as disks to the started xen domains). If you login you can also place installation iso's on the filesystem, which would make you able to select the iso's in the GUI as cdrom volume and use them as installation source (else you have to insert a cd in the server).
The crap part, which was to be expected: Your limited if you want to work from a shell. The logical volumes in LVM are named with a long UUIDs. I can imagine why thats likeable the way they build it, but it makes a quick snap shot of a volume difficult (especially when you have a lot of volumes). Together with DomU config files that are screwed up (every thing on one line and UUID filenames) this kills off the fun stuff. The one thing that still works is the "xm" tool, didn't test it extensive but at least `xm list` still works. So you can probably still do the basic operations (start, restart, shutdown, destroy, etc.).
Xen Express GUI, Windows PV drivers that work and the rest of it
When I started this post I was still mildly positive, but unfortunately it has been in my drafts for a week now. Now finishing it up a week later.
The windows PV drivers basicly give you native performance. Thats great. The GUI is not _that_ bad either. Annoying at some points but you can live with it. A few screenshots: (uh, got 'm at the office, wait a day).
Now that rest of it: Things that suck:
- Dom0 kernel is mutilated
Many many drivers are missing. It doesn't even have the iptables physdev module to filter on a bridge. On the support pages I also found a lot of complaints about missing RAID drivers, etc. - Software RAID big PITA
The installer doesn't support making any software raid devices. Its not impossible to do so afterward (except for the root filesystem I guess) but its certainly a PITA. I tried to move the LVM partition to an md device, succeeded but when I removed the non raid LVM pv I got errors after reboot. Too bad. However I'm sure its possible. Another way I didn't try since I didn't want to waste more time is to make a new "SR" or something like that. Its to be able to swap disks to one machine to another. Quite sure you could fix up a raid volume and make it used by Xen. This way you would be able to have software raid for your VM's at least. - Only HVM for Windows. Linux is still para-virtualized
Maybe nice for performance but I want the real virtualization so I can do anything I want in the guest domain. You can get it to install but its more like tricking the GUI. Which also means you can forget about paravirtualized drivers (crap performance). - Can't define vif-interface names
Every DomU gets a virtual network interface (vif), unfortunately in Xen express you can't define it. So you can never make any firewall rules (if the DomU gets restarted the vifname changes accordingly with the dom-id. Please fix that you can set dom-id or vifname. - The OS templates are really messed up
If you install debian you'll be surprised its so quick to install. Nice? No. The installation is so screwed over, it sucks balls. Besides that you get gdm and vnc for free. I didn't login with VNC it might hold more surprises. The crap thing of these templates is that the ones shipped are useless. In unofficial docs you can find how to make these templates but I didn't bother since we already found the whole Xen Express system too limited for our situation. - Firewalling your VM's is a PITA
Mostly because of the implications mentioned above its a pain to firewall your Windows machine in Linux. From Dom0 it would be impossible, so we got this fix: Create a second bridge. Create a VM from the debian template system with two nics. Put the first interface in a bridge with the peth0, put the second one in a bridge with your Windows VM. Do the filtering in the debian instance. Most transparant way is to create a bridge device (put eth0 and eth1 in it) and filter the traffic with the physdev iptables module. If you want to reach this machine from the outside you can give an IP to the bridge device. I hope it helps.
I recommend to try Xen Express out if you like a lot of pain and like to see if it really stinks for yourself or if your a sore Windows user which doesn't realise the limitations and is surprisingly happy when buttons do work.
If you were doing advanced stuff with the open source Xen version and like your tools and freedom, stay away from this. If you have Linux experience and want to run Linux VM's: use the open source version. IMO the only reason you can consider using Xen Express is that the Windows PV drivers work. Even with Windows for real production work, dunno, you should weigh the options vs. a dedicated server.
Nice start – Thank you KLM
Nice start of my travel to canada. I wanted to check in online, but somehow the webshite couldn't find my flight/booking. Gives you the "oh oh" feeling.
Calling KLM.... it took the klm lady awhile but she found out that schiphol decided to cancel the flight and they booked my flight to one two hours later and its not a direct flight anymore but i have to transfer in montreal. Great, and I can only get a new ticket tomorrow at the airport. Nice comment of her was that it said "please inform customers", good thing I was calling. Apparently they like to do such things one day before flight. Go KLM.
So now instead of a flight which leaves at 13:25 and arrive 8 hours later (15:30 localtime or something) I have to leave at 15:25 fly to montreal, arrive there at 17:00 (8 hours flight) and then leave for toronto at 18:00 where I will arrive at 19:20.
Little update (31/12): So indeed yesterday I arrived in Toronto. Quite a bit later however. The KLM flight to Montreal arrived on schedule (even a bit earlier). Really quick through customs, yay! perfect, to the transfer desk. Where is your luggage? Uh, on schiphol they said it would be transferred directly to Toronto... No! thats not true you have pick it up anyway. I had to bring it through customs myself. So a problem, since I already passed customs. Had to find some baggage phones, where you can call your airline and ask them to bring your luggage out at an alternative custom entrance/exit. Too bad noone popped up. Then I asked a someone from some nearby luggage department, he went looking... i waited... now it was 30 minutes till departure time. Some guy from air canada and he went looking for me as well. Luggage guy came back with nothing. 20 minutes till departure time. Few minutes later the air canada guy popped up with my luggage.. yay! After explaining everything to the customs lady I finally got my luggage but now they told me I missed my flight at the checkin. Fortunately there was more then just one check-in lady. My flight appeared delayed! yay! So I made it, I actually had to wait for another hour. In the end I arrived at Toronto airport at 10:00 with 3 hours delay. The only good thing about this last part is that KLM didn't put me in economy class for the Air Canada flight, so I had many buttons on my chair.
A definite don’t buy if you like hardware support
We had a night of computer terror:
- A terrible badly supported intel mainbord (dg965ot). Basicly its chips are not supported in Linux and it just won't boot. http://dev.osso.nl/peter/blog/2006/12/28/why-intel-dg965ot-sucks-balls/, nice specs, but just buy another board (at least for now).
- Besides that fedora core 6 x86_64 doesn't seem to work well with software raid1, it sucks balls: http://dev.osso.nl/peter/blog/2006/12/28/why-fedora-core-6-64-bit-x86-raid-1-suck-balls/
- Too finish off the night peters laptop died as well. hehe.
Sometimes I hate computers so much.
Challenge
I gave up on akismet, it just doesn't work properly in my setup. Fortunately spam is nagging so many people there is plenty of choice for plugins. Although I always found it pretty annoying to do something extra if I wanted to leave a comment: read some malformed letters in a image, guess what they suppose to be and type it in for verification. I picked the plugin "Challenge". The primary reason for my choice:
This plugin can also be used to ensure the commenter's basic intelligence before he/she/it leaves a comment.
The only disadvantage is that if I don't get any comments I don't know if its because people find my blog uninteresting or they are too retarded or lazy to do a simple calculation.
Update:
Thanks for the feedback regarding the new anti-spam comment plugin. Its exactly what you expect from the average PHP code you pluck of the web. Yes, its very NASTY.
The only thing I can say for it is that: no spam came through. Now I'm being lazy and really happy that I don't get over 1000 comments in my moderation queue every day. Sorry it sucks, it will probably change in the future when some spammer easily circumvents this crappy plugin or I just get sick of it.
Wordpress Akismet plugin broken with mod_chroot enabled
Since it was overly simple for Kim Chee to get his akismet thing to work (it kept telling me invalid key) it was probably a difference between webhost. I am a lazy bastard but this had to be simple, after 10 seconds my suspicions were clear. mod_chroot! I absolutely love mod_chroot, I wouldn't dare to host any php site on my servers without it. But it breaks the most nasty php apps.
And yes after disabling mod_chroot my akismet api key was found valid. Nice to know. Ofcourse I re-enabled mod_chroot by now, I am running php afterall. I just so hope it keeps working. My popularity achieved new heights today: 1098 comments. I don't like to be popular.
Important update: I already didn't like the way Akismet works, but it seemed a lazy way out of the problem, think again. When I recheck my moderation queue (even with mod_chroot disabled) it just happily tells me I'm so lucky I don't have spam. Akismet: Die.
2 days
662 comments marked as spam
Update: after another two days with the double amount of spam I decided to disable comments. Another update: Wordpress sucks, how can disable comments globally? I can only make it disabled by default for new articles in the option - discussion menu. That akismet sucks as well, in wordpress it says my key is invalid. Spam makes blogging annoying.
