Free Hypervisors Comparison

I often read debates and opinions on the internet concerning the pros and cons of the free versions of the three most widely-spread hypervisors.

Here's my view on them:


ESXi:


Pros:

• It's the most popular bare-metal hypervisor, so whatever problem you encounter you can just google for the answer.  
• Supports a huge number of Guest OSes 

Cons:

• It's not free; this is not even a clever marketing technique. It's a bait and switch, plain and simple. Let me elaborate here. ESXi is free, which is the hypervisor part. Great. But its management tools aren't. So you can't do absolutely anything with it. If you want to actually install a guest OS, you'll need the vSphere client which costs thousands of dollars, or VMware workstation that you will need to use to connect to the ESXi remotely (a solution which will set you back a few hundred dollars instead). 

• I have found its command line to be obtuse. I know, eye of the beholder and all, but it would be much easier to learn if it had completion for its commands as well as the command parameters, just like XenServer.

Hyper-V:


Pros:

• If you're planning to install a Windows OS, look no further. Microsoft has done a great job with the optimization and performance of Windows-based VMs. 

 
Cons:

• Even though it is dead simple to actually install the hypervisor you can't do anything unless you have set up and configured users, domains, firewall rules, etc. from the command line. If you're not already familiar with PowerShell, this can be a daunting and time consuming task (actually, it is a time-consuming task even if you are familiar with the PowerShell). Luckily, there is a script out there that takes care of most of these tasks, reducing the time taken to actually set up everything. After that, you'll need to install the RSAT tools and Hyper-V tools on your PC. Finally, if you're running windows 8, for some strange reason, Microsoft has also decided that you will need the Pro Edition to manage your Hyper-V server.
• It does not provide zero-downtime live VM migration  
• Dynamic Memory is only supported with VMs running Windows Vista SP1 and above, or Windows Server 2008 SP2/2008R2 SP1, 2003 R2 SP2 and 2003 SP2.
• If you're planning to use Linux, you'll need to install the legacy network adapters, and after you're done with the installation of the Linux Integration Services, you'll need to substitute the network adapters for better ones; somewhat minor irritation, I know. 

XenServer:


Pros:

• It's a fully-fledged Operating System based on Red-Hat 5, which means you can tweak it and do whatever you want. For instance, you will need to have specialized software to take backups of your VMs if you're running Hyper-V or ESXi. You can if you want with XenServer, but personally I've just written a perl script. And it works great. And guess what, too: my backups are written to a GlusterFS volume. If I had ESXi, I would have to use the insecure and deprecated NFS protocol for that. On XenServer, I just had to install GlusterFS, like any other Linux node.

• Its command line is unbelievably good. And extremely easy to learn. You can do everything with it.

• The free version is not a watered-down version of the paid one. Almost anything you can do with the XenServer paid version you can with the XenServer free version, except features like GPU virtualization, in-memory read caching, support and patching (you'll have to patch using the command line if you're using the free version).

Cons:

• XenServer used to have a great networking management interface called the Distributed Virtual Switch Controller. Unfortunately, this tool became deprecated as soon as the product was decided to become open-source. Of course, since XenServer is a fully-fledged Linux distro, this means that it is trivial to actually solve any network-related issue by switching our networking stack to Linux bridge, and taking it from there. 

Zen Load Balancer Performance Tests Part 2

All right. Time to upgrade the entire thing as described in the "Zen Load Balancer 3.0.3 Perfomance and Security Customization"series.

In summary:

Zen Load Balancer 3.0.3 Perfomance and Security Customization Part 5 was skipped, no NIC bonding was necessary;
Zen Load Balancer 3.0.3 Perfomance and Security Customization Part 4 was implemented, with the 1GB+ recommended settings and no high-memory usage settings;
Zen Load Balancer 3.0.3 Perfomance and Security Customization Part 3 was skipped, no point of checking for invalid packets or rate limit. This is just a test server;
Zen Load Balancer 3.0.3 Perfomance and Security Customization Part 2 was implemented;
Zen Load Balancer 3.0.3 Perfomance and Security Customization Part 1 was implemented (no filesystem tuning though), although the system's memory is 2GB, which means the only benefit here is the NX protection which won't be necessary since this is a test server (really, this should actually produce some overhead).

Let's go over my Zen Load Balancer's stats again:

Memory: 2048MB
CPU #1: 3.0GHz

I decided to use G-WAN as the backend, mainly because:

a) I hadn't used it before and it was a good opportunity to do so;
b) They brag a ridiculously high performance throughput -you know you have to at least check if it's partly true, and if it is, all the better as I'll stress Zen even more;
c) They have an enormous amount of haters; from other webserver fanbois who refuse to accept that a different webserver than the one they use might be better, to company plants that ticket non-existant vulnerabilities to try to prove that G-WAN isn't as good as its publishers claim. This alone can make me install software just out of spite.

So G-WAN it is, then!

The webpage that I will serve will be generated by this little C++ servlet here:

root@gwan:/opt/gwan_linux64-bit/0.0.0.0_8080/#0.0.0.0/csp# cat bench.cpp
// ============================================================================
// C++ servlet example to benchmark Zen and G-WAN Web Application Server   (http://gwan.ch/)
// ----------------------------------------------------------------------------
// bench.cpp: Concatenate a bunch of "Hello world!" until it gets to 400KB (encodings included).
//
// This code is based on the hello.cpp example by Thomas Meitz (gwan at jproxx dot com)
// ============================================================================
// imported functions:
//   get_reply(): get a pointer on the 'reply' dynamic buffer from the server
//   xbuf_cat(): like strcat(), but it works in the specified dynamic buffer
// ----------------------------------------------------------------------------
#include "gwan.h" // G-WAN definitions
#include <string>

using namespace std;

class Hello
{
public:
   void whatsup(xbuf_t* reply, string& str)
   {
      xbuf_cat(reply, (char*)str.c_str());
   }
};
// ----------------------------------------------------------------------------
int main(int argc, char *argv[])
{
   string h("Hello World! ");
   do {
   h+=h;
   } while (h.length() < 240000);
   Hello hello;
   hello.whatsup(get_reply(argv), h);

   return 200; // return an HTTP code (200:'OK')
}
// ============================================================================
// End of Source Code
// ============================================================================

These were the farm settings:

Tests with Zen Load Balancer, customized as per the recommendations in my blog

The test will be the same as before; we'll send 10, 50 and 100 requests concurrently and 10000 in total, all trying to access the output of our script.

The tests will be run 5 times each. After that, we'll find the median value according to that point in time so as to eliminate any statistical aberrations.

D:\apache\Apache24\bin>abs -c10 -n10000 -f TLS1 https://192.168.0.99:443/?bench.cpp
This is ApacheBench, Version 2.3 <$Revision: 1554214 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.0.99 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        G-WAN
Server Hostname:        192.168.0.99
Server Port:            443
SSL/TLS Protocol:       TLSv1,RC4-SHA,1024,128

Document Path:          /?bench.cpp
Document Length:        425984 bytes

Concurrency Level:      10
Time taken for tests:   110.865 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      4262600020 bytes
HTML transferred:       4259840000 bytes
Requests per second:    90.20 [#/sec] (mean)
Time per request:       110.865 [ms] (mean)
Time per request:       11.086 [ms] (mean, across all concurrent requests)
Transfer rate:          37547.54 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        4   13   6.9     11     106
Processing:    45   98  24.3     97     756
Waiting:        1   11   8.0     10     103
Total:         57  111  25.0    109     775

Percentage of the requests served within a certain time (ms)
  50%    109
  66%    114
  75%    117
  80%    120
  90%    127
  95%    134
  98%    146
  99%    161
 100%    775 (longest request)

Whoa, got to say, this looks extremely promising. Let's do this four more times, find the median values and create a chart out of it.

This is our Zen's CPU usage for 10 concurrent connections:

This is not bad at all. No excessive CPU usage, and good performance as well for an HTTPS farm; apachebench reported around 90 requests per second.

For 50 concurrent connections:

Insane. Once again, no excessive CPU usage, and good performance for an HTTPS farm; apachebench reported almost 89 requests per second!

For 100 concurrent connections:

And this seals it. With our recommended performance upgrades, Zen is both faster and uses a lot less CPU. Just compare this with our previous results! For the record, apachebench reported around 80 requests per second.