benchmarks; Xen vs OpenVZ 23 March 2007 @ 08:02 pm
Kir Kolyshkin
k001
[openvz]

There is a somewhat interesting article at the Infoworld blog, discussing the VMware vs. Xen and Xen vs. VMware benchmarks. It appears VMware did a not-quite-good job comparing their ESX to Xen, so Xen came back and presented another comparison, where it is either on par or a bit better than ESX.

From my experience working as Virtuozzo QA team leader (a few years ago) doing all sorts of performance and stress tests for Virtuozzo kernel, I know that there are very many factors influencing the results. Consider this: if you happen to run your test while cron is running daily jobs like slocate's update, log rotation routines etc., your performance could be 10 to 50 per cent slower. This was a very simple and obvious example -- just disable cron daemon before you do your testing. A trickier example is when networking performance increase by 10 to 15% if you bound a NIC interrupt to a single CPU on an two or four-way SMP box.

So, my suggestion is to take those benchmarks and comparisons with a grain of salt. Better yet, do your own comparison using your hardware and your workloads -- and make sure you understand all the results. So if something is slow -- find out why. If something is faster than it should be -- find out why, find out what you did wrong. Perhaps this part -- results analysis -- is the most complex part in the performance testing field.

Having said that, I'd like to point out a Xen vs. OpenVZ comparison, done by a German student Björn Gross-Hohnacker who I met at last year's LinuxWorld Cologne. Björn graciously allowed us to publish his results, so we have translated part of it into English.

Here is the bottom line summary: IPC and disk I/O performance is better (or much better) for OpenVZ than Xen, CPU-intensive tasks are about the same for both, networking is a bit better in OpenVZ. Conclusion: for homogeneous (i. e. Linux-only) environments, OpenVZ is way better -- as it was designed to be.

You are taking this with a grain of salt, aren't you? ;)

Tags: , , , , ,
Previous | Mem | Next
Permanent Link6 comments | Leave a comment
Post a new comment
[info]anyher on March 29th, 2007 - 10:07 am
Yep, i knowing about cron(well, in my case windows scheduler) tricks first-hand.
About results of OpenVz vs Xen - i think it's something that is looks realistic to me. I can't smell something completely rotten in that benchmarks.
(Anonymous) on May 14th, 2007 - 05:30 pm
IPC, disk IO.. it is not very required info.. I think, that best way - real benchmarks with mysql/apache with different configs (python,php,perl)/mongrel/nginx and other software =(
(Anonymous) on August 30th, 2007 - 04:17 am
Xen vs OpenVZ
HP has a "real world" app test here: http://www.hpl.hp.com/techreports/2007/HPL-2007-59.html
with similar conclusions...
(Anonymous) on November 15th, 2007 - 08:12 pm
Thanks
I was about to move my openVZ account to xen until I say your article.
Now I am having second thought after reading your article.

I read somewhere that java server performs better on xen (in terms of memory requirement). What is opinion on this?

Thanks,

surajz (www.himalayansouvenirs.com)
(Anonymous) on January 5th, 2008 - 12:07 pm
how about memory usage?
i just read here:
http://blog.craz8.com/articles/2006/12/25/openvz-redefines-ram/

how you need twice ram more than xen for webhosting vps application.
is that true?

tia.
Kir Kolyshkin[info]k001 on January 5th, 2008 - 02:03 pm
Re: how about memory usage?
how you need twice ram more than xen for webhosting vps application.
is that true?


No, it is not.

In fact, OpenVZ requires less memory than Xen, which is shown, for example, in HP labs report comparing OpenVZ and Xen, see my blog posting about it here, and please read the full report itself.

The main influence of RAM usage is density, i.e. the more RAM you use for an application, the less applications you can run on a server. HP labs tell in their report Xen was able to run 4 instances of a particular application they tested, while OpenVZ was fine with at least 6 instances of that app (they haven't tried running more). This means that OpenVZ density is at least x1.5 times higher.

Speaking of the blog post that you refer to -- yes, it is a problem, but rather than OpenVZ problem, this is a problem of hosting service providers which either oversell (i.e. putting way too much VPSs on a single system) or do not understand various limits and guarantees provided by OpenVZ's user beancounters subsystem. To mitigate that, we wrote a lot of detailed documentation regarding OpenVZ UBC, see http://wiki.openvz.org/UBC. Note that we never state that privvmpages is RAM -- rather than that we explain each UBC parameter in great details.

So, yes, maybe OpenVZ UBC is a bit complex, and it certainly reqiuires a bit of expertise to operate. You see, it's like a car with a manual gearbox -- you have to learn when and how to change gears in order to drive efficiently.