Linux Performance Benchmark - System

I decided to start by enduring a simple and generalist benchmark to all distributions, using hdparm, gzip, gunzip, lame, oggenc and heavier one where I mixed all I mentioned before plus a standard PHP compilation.

There were many surprises and a few details I couldn’t really understand but we’ll get to that.

Configuration

As you know, I’ve tested Ubuntu Server, Debian, CentOS and Gentoo. I’ve used the default configuration for all of them, except for Gentoo where I’ve changed the default kernel configuration (since it’s Desktop-oriented) tweaking a few options to be similar to those found on Ubuntu Server’s kernel configuration.

I also didn’t enforce the usage of similar package versions: My goal was to benchmark all distributions in a given time (which were updated at the same time I submitted my previous post). So, let’s see what we have in each of them:

Benchmark

I’ve used a small script for testing purposes, which runs 3 rounds of each of these tests:

The multiple runs had the goal of canceling circumstantial issues, except for the last one which is pretty long and does that by itself. To make all measurements (except for the hdparm test), I used GNU time.

Results

Results amazed me a bit. First of all, the default installation of CentOS must have some kind of problem on my system - despite it winning one of the tests, it lost by a huge margin on the ones involving gzip (including the heaviest one). A second consideration point regards Gentoo: It needs to be properly configured and I barely touched it. Gentoo’s power rests in it’s configurability because without proper configuration, it’s somewhat a standard Linux installation. Since my purpose was not to benchmark a fully configured environment, it was OK not to have Gentoo at full power - actually, none of the distros were. Bottom line: no tweakings were made, just pure performance out of each distro’s standard installation.

If you check my benchmarking script you can see that it creates a result file for each one of the tested distributions - Ubuntu, Debian, CentOS and Gentoo.

A summary of all results and relevant output will follow:

Benchmark Results

All results represent averages of all 3 runs (except for the last one which was only ran once, as I mentioned). Let’s go one by one and possibly do some result analysis.

hdparm

The performance achieved by all 4 was quite similar. Anyway, Gentoo wins this round.

This test measured the speed of cached reads so the higher the better.

hdparm benchmark

gzip

Gentoo wins this round as well, followed closely by Debian. It’s the first benchmark where the CentOS problems I mentioned start being serious. CPU is stressed and there are many IO operations (keep in mind that the file is huge).

This test measures the time needed to compress a 2.5GB file so the lower the better.

gzip benchmark

gunzip

Gentoo wins again and this time it is by a fair margin. CentOS’s problems keep getting noticed. Similarly to the gzip benchmark, the CPU is also stressed but in a smaller scale while the IO operations increase since we’re extracting a fairly big archive which will result in a larger file.

This test measures the time needed to extract the previously compressed file so the lower the better.

gunzip benchmark

lame

This one was an absolute surprise. After seeing CentOS falling to it’s knees on previous tests it actually won this round by a great margin - more than 9% comparing to the second best, which was Debian. Noticeably, the most up-to-date distributions - Ubuntu and Gentoo - were the ones to perform worse. Could there be a slight performance degradation from lame 3.96/3.97 to 3.98? This benchmark uses a lot of CPU and that was the main target.

This test measures the time needed to convert a 214MB file so the lower the better.

lame benchmark

oggenc

Ubuntu won this one, closely followed by Debian and then Gentoo. CentOS stayed behind but this is probably related to the outdated version of vorbis-tools which were used. Similarly to the lame benchmark, this one also uses a lot of CPU and that was also the main target.

This test also measures the time needed to convert a 214MB file so the lower the better.

oggenc benchmark

workload

Many strange things happened. First of all, CentOS showed a huge performance hit, probably related with the possible misconfiguration I mentioned previously. Another thing I’ve noticed is that sys+user time is higher than real time, which I find extremely awkward. Besides, Gentoo did a lot of voluntary context switching (>100000), 17 times more than Debian and around 6 times more than Ubuntu/CentOS. I can’t understand why this happened - hasn’t the source been correctly patched by Gentoo’s developers? Is the kernel badly configured? Is gcc behaving incorrectly? I couldn’t figure this out, even though the impact wasn’t really deep. If you have any ideas regarding this issues, please leave a comment.

On the same subject, I really think something went wrong with this benchmark - time measurements didn’t seem accurate. The only conclusion I can make out of this is the excellent Debian performance when comparing it to Ubuntu and Gentoo. Huge win!

This test measures the time needed to parallely run all tests mentioned before plus PHP 5.2.12 compilation so the lower the better.

workload benchmark

Conclusions

First of all, let’s give some medals:

I can safely assume that Gentoo wins when it comes to CPU usage + IO operations on a single instance. After all, it won the hdparm, gzip and gunzip benchmarks. When it comes to almost pure CPU processing power CentOS and Ubuntu get the glory since they won the track conversion benchmarks. At last but not least, Debian did an excellent job in the workload benchmark, leaving it’s opponents behind by a safe margin.

Now to some considerations:

What now?

CentOS has been ruled out of future benchmarking. It came with an obvious configuration defect and I don’t want to spend precious benchmarking time in hunting it down.

Now that we’re down to 3 distribution: Ubuntu, Debian and Gentoo, I’ll make a few rounds of new web-oriented benchmarks since that’s my main purpose - I’ll start with apache/nginx and then move forward to many webservers such as Passenger and Thin.

If you have any questions left about the procedure I’ve used or you think you have an answer to any of the problems/doubts I’ve encountered, feel free to comment bellow.

Posted 2 years ago • Comments •
blog comments powered by Disqus
Page 1 of 1