Lighttpd performance
We use Lighttpd a lot these days to offload our Apache2 servers. More specific we use Lighttpd to serve all the static content (images, stylesheets, javascript files and such).
Since one of our customers wanted a clear view I did some performance testing. I used siege with 500 concurrent connections with a list of urls with all the available static files (little over 3000 files).
The results:
Lifting the server siege... done.
Transactions: 279582 hits
Availability: 100.00 %
Elapsed time: 300.51 secs
Data transferred: 2401.35 MB
Response time: 0.03 secs
Transaction rate: 930.36 trans/sec
Throughput: 7.99 MB/sec
Concurrency: 27.50
Successful transactions: 279499
Failed transactions: 0
Longest transaction: 3.20
Shortest transaction: 0.00
While sending out with a rate around 80Mbit and receiving around 7Mbit, the old Xeon server didn't get over a load of 0.20.
I knew lighttpd was good at its job, but this still surprised me positively. I didn't try to push it any futher but I would expect the server could have saturated a 1Gbit/s link if you look at these results. I bet Apache2 would not be able to come even close.
Commodity hardware++: Storage server
Great what you can do with commodity hardware these days:
storage-mc:/data# dd if=/dev/zero of=test.img bs=1M count=102400
102400+0 records in
102400+0 records out
107374182400 bytes (107 GB) copied, 698.145 seconds, 154 MB/s
4 x WD 500GB RE2 disks in RAID1+0. With a 4-port Areca 1210 hardware raid card. Sure beats Linux software RAID1 with performance.
Nothing bad about Linux software RAID1, we use it in many places. Just when you need that bit more I/O and SAS/SCSI is not really justified hardware SATA RAID can really make the difference. The Western Digital RE2 (Raid Edition 2) disks are also pretty good. These disks have 92MB/s buffer to disk and NCQ (Native Commandline Queuing). No complains so far!
Ubuntu software raid performance problems
We recently experienced some performance problems with an Ubuntu machine with software raid1. To localize it a bit further I ran some tests on different setups. For comparison I also included a setup with hardware raid and a setup with software raid on CentOS, I will also post the specs of the machines in question but I don't believe the specs have a noteable influence.
I tested with the transfer speeds with the following commands:
Write: dd if=/dev/zero of=test.img count=300 bs=10M
Read: dd if=test.img of=/dev/null bs=10M
Setup 1:
HP DL145 (Dual-Core AMD Opteron(tm) Processor 2210 HE, 1GB ECC reg)
RAID: 3ware Escalade 8006-2LP, PCI-X in RAID1
Disks: 2x Western Digital 80GB, 7200rpm, SATAII, 8MB cache
Distribution: Ubuntu 6.06 LTS, kernel 2.6.15-28-amd64-server
Write Results: 3145728000 bytes (3.1 GB) copied, 113.433 seconds, 27.7 MB/s
Read Results: 3145728000 bytes (3.1 GB) copied, 63.1678 seconds, 49.8 MB/s
Setup 2:
HP DL145 (Dual-Core AMD Opteron(tm) Processor 2210 HE, 1GB ECC reg)
RAID: md software raid1
Disks: 2x HP certified 80GB Seagate disks, 7200rpm, SATAII, 8MB cache
Distribution: Ubuntu 6.06 LTS, kernel 2.6.15-28-amd64-server
Write results: 3145728000 bytes (3.1 GB) copied, 753.233 seconds, 4.2 MB/s
Read results: 3145728000 bytes (3.1 GB) copied, 85.2566 seconds, 36.9 MB/s
Setup 3:
HP DL145 (Dual-Core AMD Opteron(tm) Processor 2210 HE, 1GB ECC reg)
RAID: md software raid1
Disks: 2x HP certified 80GB Seagate disks, 7200rpm, SATAII, 8MB cache
Distribution: Ubuntu 6.10, kernel 2.6.17-11-server (x86_64)
Write results: 3145728000 bytes (3.1 GB) copied, 729.397 seconds, 4.3 MB/s
Read results: 3145728000 bytes (3.1 GB) copied, 49.0884 seconds, 64.1 MB/s
Setup 4:
HP DL145 (Dual-Core AMD Opteron(tm) Processor 2210 HE, 1GB ECC reg)
RAID: md software raid1
Disks: 2x HP certified 80GB Seagate disks, 7200rpm, SATAII, 8MB cache
Distribution: Ubuntu 7.04, kernel 2.6.20-15-server (x86_64)
Write results: 3145728000 bytes (3.1 GB) copied, 528.52 seconds, 6.0 MB/s
Read results: 3145728000 bytes (3.1 GB) copied, 58.7009 seconds, 53.6 MB/s
Setup 5:
Intel(R) Celeron(R) CPU 2.66GHz, Intel 3000 serverboard, 1GB DDR2 RAM
RAID: md software raid1
Disks: 2x Westerm Digital 500GB disks, 7200rpm, SATAII, 16MB cache
Distribution: CentOS 4, kernel 2.6.9-42.0.10.EL (x86_64)
Write results: 71 seconds should be ~44MB/s
Read results: 47 seconds should be ~67MB/s
Notes: The CentOS4 time is measured with `time` since dd version in CentOS doesn't give stats itself. It might differ from the way dd measures the time. The hardware raid setup has western digital disks vs. seagate disks in the Ubuntu software raid setups. Personally we have the feeling the 80GB seagate disks are a bit faster then then 80GB WD disks but that might be an illusion. The Ubuntu setups all use JFS as filesystem and the CentOS ext3.
Conclusion: We can see that the Ubuntu software raid at least has a write performance issue. The software raid performance in CentOS seems to be much more decent. We would conclude that its something in the Ubuntu kernel area which makes the software raid suck balls and not software raid perse. The only other factor which might make a difference is the filesystem, however we never had JFS performance problems and filesystems are a different layer in the kernel AFAIK.
Suspicions: We have the feeling that if you have concurrent write actions, the performance will go down much more then the crappy 4-6MB/s. We also noticed that the load goes skyhigh if you do disk stuff with software raid on Ubuntu (breaks 15.00 easily with a 3GB mysql import with binlog on, leaving most things unresponsive). The load on CentOS doesn't seem to go up too much. IIRC we got the same crap performance in Debian which we gave a quick test (etch).
Twitter: hpbos
- Gisteren the men who stare at goats gezien. :-) jeff bridges++
- @Ronald_R: Anders heb je nog Zombie Strippers! als optie.
- @Ronald_R Lesbian Vampire Killers... http://www.pajiba.com/trailers/new-lesbian-vampire-killers-trailer.php
- @joostdrijver stink voetbal, mijn reistijd naar kantoor verdubbeld in de laatste 500m.
- wie wil er vanavond "the men who stare at goats" met mij kijken? :P
- Rabobank internetbankieren is buggy as hell.
- Net film the fountain gezien. Wel mooi gemaakt maar niet mijn ding.
- Groningen binnen. Niet bijzonder. Nu coma tijd.
- Wat een kak verhaal van rouvoet. Ook met 4 procent geteld, lame.
Categories
- Advocacy
- Development
- Entertainment
- Evil
- Humor
- In the news
- Orbit Ubuntu
- Personal life
- Security
- Tech
- Travel
- Work
