Category Archives: performance

Quand le marketing dépense des sous qui ne vont pas à l’innovation

Pas facile le load-balancing !

J’ai d’ordinaire un très grand respect pour nos concurrents, que je qualifie plutôt de confrères et qu’il m’est même déjà arrivé d’aider en privé pour au moins deux d’entre eux. Etant à l’origine du répartiteur de charge libre le plus répandu qui donna naissance à l’Aloha, je suis très bien placé pour savoir à quel point cette tâche est difficile. Aussi j’éprouve une certaine admiration pour quiconque s’engage avec succès dans cette voie et plus particulièrement pour ceux qui parviennent à enrichir leurs produits sur le long terme, chose encore plus difficile que d’en recréer un de zéro.

C’est ainsi qu’il m’arrive de rappeller à mes collègues qu’on ne doit jamais se moquer de nos confrères quand il leur arrive des expériences très désagréables comme par exemple le fait de laisser traîner une clé SSH root sur des appliances, parce que ça arrive même aux plus grands, que nous ne sommes pas à l’abri d’une boulette similaire malgré tout le soin apporté à chaque nouvelle version, et que face à une telle situation nous serions également très embarrassés.

Toutefois il en est un qui ne semble pas connaître ces règles de bonne conduite, probablement parce qu’il ne connait pas la valeur du travail de recherche et développement apporté aux produits de ses concurrents. Je ne le nommerai pas, ce serait lui faire l’économie d’une publicité alors que c’est sa seule spécialité.

En effet, depuis que ce concurrent a touché $16M d’investissements, il n’a de cesse de débiner nos produits ainsi que ceux de quelques autres de nos confrères auprès de nos partenaires, et ce de manière complètement gratuite et sans aucun fondement (ce que les anglo-saxons appellent le “FUD”), juste pour essayer de se placer.

Je pense que leur première motivation vient sans doute de leur amertume d’avoir vu leur produit systématiquement éliminé par nos clients lors des tests comparatifs sur les critères de facilité d’intégration, de performance et de stabilité. C’est d’ailleurs la cause la plus probable vu que ce concurrent s’en prend à plusieurs éditeurs à la fois et que nous rencontrons nous-mêmes sur le terrain des confrères offrant des produits de qualité tels que Cisco, et F5 pour les produits matériels, ou load-balancer.org pour le logiciel. Bon et bien sûr il y a aussi ce concurrent agressif que je ne nomme pas.

Il est vrai que cela ne doit pas être plaisant pour lui de perdre les tests à chaque fois, mais lorsque nous perdons un test face à un confrère, ce qui nous arrive comme à tous, nous nous efforçons d’améliorer notre produit sur le critère qui nous a fait défaut, dans l’objectif de gagner la fois suivante, au lieu d’investir lourdement dans des campagnes de désinformation sur le vainqueur.

A mon avis, ce concurrent ne sait pas par où commencer pour améliorer sa solution, ce qui explique qu’il s’attaque à plusieurs concurrents en même temps. Je pense que ça relèverait un peu le débat de lui donner quelques bases pour améliorer ses solutions et se donner un peu plus de chances de se placer.

Déjà, il gagnerait du temps en commençant par regarder un peu comment fonctionne notre produit et à s’en inspirer. Je n’ai vraiment pas pour habitude de copier la concurrence et je préfère très largement l’innovation. Mais pour eux ça ne devrait pas être un problème vu que déjà ils ont choisi les mêmes matériels que notre milieu de gamme, à part qu’ils en ont changé la couleur, préférant celle des cocus. Les malheureux ne savaient pas qu’un matériel de bonne qualité ne fait pas tout et que le logiciel ainsi que la qualité de l’intégration comptent énormément (sinon les VM seraient toutes les mêmes). C’est comme cela qu’ils se sont retrouvés avec un décalage dans la gamme par rapport à nous : ils ont systématiquement besoin du boîtier plus gros pour atteindre un niveau de performances comparable (je parle de performances réelles mesurées sur le terrain avec les applications des clients et les méthodologies de test des clients, pas celles de la fiche produit qu’on trouve sur leur site et qui n’intéressent pas les clients).

Et oui messieurs, il faudrait aussi se pencher un peu sur la partie logicielle, le véritable savoir faire d’un éditeur. Déjà, utiliser une distribution Linux générique de type poste de travail pour faire une appliance optimisée réseau, ce n’était pas très fin, mais s’affranchir du tuning système, ça relève de la paresse. Au lieu de perdre votre temps à chercher des manuels d’optimisation sur Internet, prenez donc les paramètres directement dans notre Aloha, vous savez qu’ils sont bons et vous gagnerez du temps. Sans doute que certains paramétrages n’existeront pas chez vous vu que nous ajoutons constamment des fonctionnalités pour mieux répondre aux besoins, mais ce sera déjà un bon début. Ne comptez quand même pas sur nous pour vous attendre, pendant que vous nous copiez, nous innovons et conserverons toujours cette longueur d’avance :-). Mais au moins vous aurez l’air moins ridicules en avant-vente et éviterez de mettre vos partenaires dans l’embarras chez le client avec un produit qui ne fonctionne toujours pas au bout de 6 heures passées sur un test simple.

Volontairement je vais publier cet article en français. Cela leur fera un petit exercice de traduction qui leur sera bénéfique pour s’implanter sur le territoire français où les clients sont très exigeants sur l’usage de la langue française que leur support ne pratique toujours pas d’ailleurs.

Ah dernier point, j’invite tous les lecteurs de cet article à chercher “Exceliance” sur Google, par exemple en cliquant sur ce lien : http://www.google.com/search?q=exceliance

Vous noterez que notre concurrent préféré a même été jusqu’à payer des Google Adwords pour faire apparaître ses publicités lorsqu’on cherche notre nom, il faut croire qu’il nous en veut vraiment! C’est le seul à déployer autant d’efforts pour essayer de nous faire de l’ombre, comme si c’était absolument stratégique pour lui. Vous ne verrez pas cela de la part d’A10, Brocade, F5 ou Cisco (ni même Exceliance bien sûr) : ces produits ont chacun des atouts sur le terrain et n’ont pas besoin de recourir à des méthodes pareilles pour exister. Pensez à cliquer sur leur lien de pub, ça leur fera plaisir, ça leur coûte un petit peu à chaque clic, et puis ça vous donnera l’occasion d’admirer leurs beaux produits :-).

Links

HAProxy, Varnish and the single hostname website

As explained in a previous article, HAProxy and Varnish are two great OpenSource software which aim to improve performance, resilience and scalability of web applications.
We saw also that these two softwares are not competitors. Instead of that, they can work properly together, each one bringing the other one its features, making any web infrastructure more agile and robust at the same time.

In the current article, I’m going to explain how to use both of them on a web application hosted on a single domain name.

Main advantages of each soft


As a reminder, here are the main features each product owns.

HAProxy


HAProxy‘s main features:

  • Real load-balancer with smart persistence
  • Request queueing
  • Transparent proxy

Varnish


Varnish‘s main features:

  • Cache server with stale content delivery
  • Content compression
  • Edge Side Includes

Common features


HAProxy and Varnish both have the features below:

  • Content switching
  • URL rewritting
  • DDOS protection

So if we need any of them, we could use either HAProxy or Varnish.

Why a single domain

In web application, there are two types of content: static and dynamic.

By dynamic, I mean content which is generated on the fly and which is dedicated to a single user based on its current browsing on the application. Anything which is not in this category, can be considered as static. Even a page which is generated by PHP and whose content does change every minutes or few seconds (like the CMS WordPress or drupal). I call these pages “pseudo-static

The biggest strength of Varnish is that it can cache static objects, delivering them on behalf of the server, offloading most of the traffic from the server.



An object is identified by a Host header and its URL. When you have a single domain name, you have a single Host header for all your requests: static, pseudo static or dynamic.

You can’t split your traffic: everything requests must arrive on a single type of device: the LB, the cache, etc…

A good practise to split dynamic and static content is to use one domain name per type of object: www.domain.tld for dynamic and static.domain.tld for static content. Doing that you could forward dynamic traffic to the LB and static traffic to the caches directly.



Now, I guess you understand that the web application host naming can have an impact on the platform you’re going to build.

In the current article, I’ll only focus on applications using a single domain name. We’ll see how we can route traffic to the right product despite the limitation of the single domain name.



Don’t worry, I’ll write an other article later about the fun we could have when building a platform for an application hosted on multiple domain names.

Available architectures

Considering I summarize the “web application” as a single brick called “APPSERVER“, we have 2 main architectures available:

  1. CLIENT ==> HAPROXY ==> VARNISH ==> APPSERVER
  2. CLIENT ==> VARNISH ==> HAPROXY ==> APPSERVER

Pro and cons of HAProxy in front of Varnish


Pros:

  • Use HAProxy‘s smart load-balancing algorithm such as uri, url_param to make varnish caching more efficient and improve the hit rate
  • Make the Varnish layer scalable, since load-balanced
  • Protect Varnish ramp up when starting up (related to thread pool creation)
  • HAProxy can protect against DDOS and slowloris
  • Varnish can be used as a WAF

Cons:

  • no easy way do do application layer persistence
  • HAProxy queueing system can hardly protect the application hidden by Varnish
  • The client IP will be mandatory forwwarded on the X-Forwarded-For header (or any header you want)

Pro and cons of Varnish in front of HAProxy


Pros:

  • Smart layer 7 persistence with HAProxy
  • HAProxy layer scalable (with persistence preserved) since load-balanced by Varnish
  • APPSERVER protection through HAProxy request queueing
  • Varnish can be used as a WAF
  • HAProxy can use the client IP address (provided by Varnish in a HTTP header) to do Transparent proying (getting connected on APPSERVER with the client ip)

Cons:

  • HAProxy can’t protect against DDOS, Varnish will do
  • Cache size must be big enough to store all objects
  • Varnish layer not scalable

Finally, which is the best architecture??


No need to choose between both architecture above which one is the less worst for you.

It would be better to build a platform where there are no negative points.

The Architecture


The diagram below shows the architecture we’re going to work on.
haproxy_varnish
Legend:

  • H: HAProxy Load-Balancers (could be ALOHA Load-Balancer or any home made)
  • V: Varnish servers
  • S: Web application servers, whatever the product used here (tomcat, jboss, etc…)…
  • C: Client or end user

Main roles of each layers:

  • HAProxy: Layer 7 traffic routing, first row of protection against DDOS (syn flood, slowloris, etc…), application request flow optimiation
  • Varnish: Caching, compression. Could be used later as a WAF to protect the application
  • Server: hosts the application and the static content
  • Client: browse and use the web application

traffic flow


Basically, the client will send all the requests to HAProxy, then HAProxy, based on URL or file extension will take a routing decision:

  • If the request looks to be for a (pseudo) static object, then forward it to Varnish
    If Varnish misses the object, it will use HAProxy to get the content from the server.
  • Send all the other requests to the appserver. If we’ve done our job properly, there should be only dynamic traffic here.

I don’t want to use Varnish as the default option in the flow, cause a dynamic content could be cached, which could lead to somebody’s personal information sent to everybody

Furthermore, in case of massive misses or purposely built request to bypass the caches, I don’t the servers to be hammered by Varnish, so HAProxy protects them with a tight traffic regulation between Varnish and appservers..

Dynamic traffic flow


The diagram below shows how the request requiring dynamic content should be ideally routed through the platform:
haproxy_varnish_dynamic_flow
Legend:

  1. The client sends its request to HAProxy
  2. HAProxy chooses a server based on cookie persistence or Load-Balancing Algorithm if there is no cookie.
    The server processes the request and send the response back to HAPRoxy which forwards it to the client

Static traffic flow


The diagram below shows how the request requiring static content should be ideally routed through the platform:
haproxy_varnish_static_flow

  1. The client sends its request to HAProxy which sees it asks for a static content
  2. HAProxy forward the request to Varnish. If Varnish has the object in Cache (a HIT), it forwards it directly to HAProxy.
  3. If Varnish doesn’t have the object in cache or if the cache has expired, then Varnish forwards the request to HAProxy
  4. HAProxy randomly chooses a server. The response goes back to the client through Varnish.

In case of a MISS, the flow looks heavy 🙂 I want to do it that way to use the HAProxy traffic regulation features to prevent Varnish to flood the servers. Furthermore, since Varnish sees only static content, its HIT rate is over 98%… So the overhead is very low and the protection is improved.

Pros of such architecture

  • Use smart load-balancing algorithm such as uri, url_param to make varnish caching more efficient and improve the hit rate
  • Make the Varnish layer scalable, since load-balanced
  • Startup protection for Varnish and APPSERVER, allowing server reboot or farm expansion even under heavy load
  • HAProxy can protect against DDOS and slowloris
  • Smart layer 7 persistence with HAProxy
  • APPSERVER protection through HAProxy request queueing
  • HAProxy can use the client IP address to do Transparent proxying (getting connected on APPSERVER with the client ip)
  • Cache farm failure detection and routing to application servers (worst case management)
  • Can load-balance any type of TCP based protocol hosted on APPSERVER

Cons of such architecture


To be totally fair, there are a few “non-blocking” issues:

  • HAProxy layer is hardly scalable (must use 2 crossed Virtual IPs declared in the DNS)
  • Varnish can’t be used as a WAF since it will see only static traffic passing through. This can be updated very easily

Configuration

HAProxy Configuration

# On Aloha, the global section is already setup for you
# and the haproxy stats socket is available at /var/run/haproxy.stats
global
  stats socket ./haproxy.stats level admin
  log 10.0.1.10 local3

# default options
defaults
  option http-server-close
  mode http
  log global
  option httplog
  timeout connect 5s
  timeout client 20s
  timeout server 15s
  timeout check 1s
  timeout http-keep-alive 1s
  timeout http-request 10s  # slowloris protection
  default-server inter 3s fall 2 rise 2 slowstart 60s

# HAProxy's stats
listen stats
  bind 10.0.1.3:8880
  stats enable
  stats hide-version
  stats uri     /
  stats realm   HAProxy Statistics
  stats auth    admin:admin

# main frontend dedicated to end users
frontend ft_web
  bind 10.0.0.3:80
  acl static_content path_end .jpg .gif .png .css .js .htm .html
  acl pseudo_static path_end .php ! path_beg /dynamic/
  acl image_php path_beg /images.php
  acl varnish_available nbsrv(bk_varnish_uri) ge 1
  # Caches health detection + routing decision
  use_backend bk_varnish_uri if varnish_available static_content
  use_backend bk_varnish_uri if varnish_available pseudo_static
  use_backend bk_varnish_url_param if varnish_available image_php
  # dynamic content or all caches are unavailable
  default_backend bk_appsrv

# appsrv backend for dynamic content
backend bk_appsrv
  balance roundrobin
  # app servers must say if everything is fine on their side
  # and they can process requests
  option httpchk
  option httpchk GET /appcheck
  http-check expect rstring [oO][kK]
  cookie SERVERID insert indirect nocache
  # Transparent proxying using the client IP from the TCP connection
  source 10.0.1.1 usesrc clientip
  server s1 10.0.1.101:80 cookie s1 check maxconn 250
  server s2 10.0.1.102:80 cookie s2 check maxconn 250

# static backend with balance based on the uri, including the query string
# to avoid caching an object on several caches
backend bk_varnish_uri
  balance uri # in latest HAProxy version, one can add 'whole' keyword
  # Varnish must tell it's ready to accept traffic
  option httpchk HEAD /varnishcheck
  http-check expect status 200
  # client IP information
  option forwardfor
  # avoid request redistribution when the number of caches changes (crash or start up)
  hash-type consistent
  server varnish1 10.0.1.201:80 check maxconn 1000
  server varnish2 10.0.1.202:80 check maxconn 1000

# cache backend with balance based on the value of the URL parameter called "id"
# to avoid caching an object on several caches
backend bk_varnish_url_param
  balance url_param id
  # client IP information
  option forwardfor
  # avoid request redistribution when the number of caches changes (crash or start up)
  hash-type consistent
  server varnish1 10.0.1.201:80 maxconn 1000 track bk_varnish_uri/varnish1
  server varnish2 10.0.1.202:80 maxconn 1000 track bk_varnish_uri/varnish2

# frontend used by Varnish servers when updating their cache
frontend ft_web_static
  bind 10.0.1.3:80
  monitor-uri /haproxycheck
  # Tells Varnish to stop asking for static content when servers are dead
  # Varnish would deliver staled content
  monitor fail if nbsrv(bk_appsrv_static) eq 0
  default_backend bk_appsrv_static

# appsrv backend used by Varnish to update their cache
backend bk_appsrv_static
  balance roundrobin
  # anything different than a status code 200 on the URL /staticcheck.txt
  # must be considered as an error
  option httpchk
  option httpchk HEAD /staticcheck.txt
  http-check expect status 200
  # Transparent proxying using the client IP provided by X-Forwarded-For header
  source 10.0.1.1 usesrc hdr_ip(X-Forwarded-For)
  server s1 10.0.1.101:80 check maxconn 50 slowstart 10s
  server s2 10.0.1.102:80 check maxconn 50 slowstart 10s

Varnish Configuration

backend bk_appsrv_static {
        .host = "10.0.1.3";
        .port = "80";
        .connect_timeout = 3s;
        .first_byte_timeout = 10s;
        .between_bytes_timeout = 5s;
        .probe = {
                .url = "/haproxycheck";
                .expected_response = 200;
                .timeout = 1s;
                .interval = 3s;
                .window = 2;
                .threshold = 2;
                .initial = 2;
        }
}

acl purge {
        "localhost";
}

sub vcl_recv {
### Default options

        # Health Checking
        if (req.url == /varnishcheck) {
                error 751 "health check OK!";
        }

        # Set default backend
        set req.backend = bk_appsrv_static;

        # grace period (stale content delivery while revalidating)
        set req.grace = 30s;

        # Purge request
        if (req.request == "PURGE") {
                if (!client.ip ~ purge) {
                        error 405 "Not allowed.";
                }
                return (lookup);
        }

        # Accept-Encoding header clean-up
        if (req.http.Accept-Encoding) {
                # use gzip when possible, otherwise use deflate
                if (req.http.Accept-Encoding ~ "gzip") {
                        set req.http.Accept-Encoding = "gzip";
                } elsif (req.http.Accept-Encoding ~ "deflate") {
                        set req.http.Accept-Encoding = "deflate";
                } else {
                        # unknown algorithm, remove accept-encoding header
                        unset req.http.Accept-Encoding;
                }

                # Microsoft Internet Explorer 6 is well know to be buggy with compression and css / js
                if (req.url ~ ".(css|js)" && req.http.User-Agent ~ "MSIE 6") {
                        remove req.http.Accept-Encoding;
                }
        }

### Per host/application configuration
        # bk_appsrv_static
        # Stale content delivery
        if (req.backend.healthy) {
                set req.grace = 30s;
        } else {
                set req.grace = 1d;
        }

        # Cookie ignored in these static pages
        unset req.http.cookie;

### Common options
         # Static objects are first looked up in the cache
        if (req.url ~ ".(png|gif|jpg|swf|css|js)(?.*|)$") {
                return (lookup);
        }

        # if we arrive here, we look for the object in the cache
        return (lookup);
}

sub vcl_hash {
        hash_data(req.url);
        if (req.http.host) {
                hash_data(req.http.host);
        } else {
                hash_data(server.ip);
        }
        return (hash);
}

sub vcl_hit {
        # Purge
        if (req.request == "PURGE") {
                set obj.ttl = 0s;
                error 200 "Purged.";
        }

        return (deliver);
}

sub vcl_miss {
        # Purge
        if (req.request == "PURGE") {
                error 404 "Not in cache.";
        }

        return (fetch);
}

sub vcl_fetch {
        # Stale content delivery
        set beresp.grace = 1d;

        # Hide Server information
        unset beresp.http.Server;

        # Store compressed objects in memory
        # They would be uncompressed on the fly by Varnish if the client doesn't support compression
        if (beresp.http.content-type ~ "(text|application)") {
                set beresp.do_gzip = true;
        }

        # remove any cookie on static or pseudo-static objects
        unset beresp.http.set-cookie;

        return (deliver);
}

sub vcl_deliver {
        unset resp.http.via;
        unset resp.http.x-varnish;

        # could be useful to know if the object was in cache or not
        if (obj.hits > 0) {
                set resp.http.X-Cache = "HIT";
        } else {
                set resp.http.X-Cache = "MISS";
        }

        return (deliver);
}

sub vcl_error {
        # Health check
        if (obj.status == 751) {
                set obj.status = 200;
                return (deliver);
        }
}
  

Related links

Links

HAProxy and Varnish comparison

In opensource world, there are some very smart products which are very often used to build a high performance, reliable and scalable architecture.
HAProxy and Varnish are both in this category.

Since we can’t really compare a reverse-proxy cache and a reverse-proxy load-balancer, I’m just going to focus in common for both software as well as the advantage of each of them.
The list is not exhaustive, but must only focus on most used / interesting features. So feel free to add a comment if you want me to complete the list.

Common points between HAProxy and Varnish


Before comparing the differences, we can summarize the points in common:

  • reverse-proxy mode
  • advanced HTTP features
  • no SSL offloading
  • client-side HTTP 1.1 with keepalive
  • tunnel mode available
  • high performance
  • basic load-balancing
  • server health checking
  • IPv6 ready
  • Management socket (CLI)
  • Professional services and training available

Features available in HAProxy and not in Varnish


The features below are available in HAProxy, but aren’t in Varnish:

  • advanced load-balancer
  • multiple persistence methods
  • DOS and DDOS mitigation
  • Advanced and custom logging
  • Web interface
  • Server / application protection through queue management, slow start, etc…
  • SNI content switching
  • Named ACLs
  • Full HTTP 1.1 support on server side, but keep-alive
  • Can work at TCP level with any L7 protocol
  • Proxy protocol for both client and server
  • powerful log analyzer tool (halog)
  • <private joke> 2002 website design </private joke>

Features available in Varnish and not in HAProxy


The features below are available in Varnish, but aren’t in HAProxy:

  • caching
  • grace mode (stale content delivery)
  • saint mode (manages origin server errors)
  • modular software (with a lot of modules available)
  • intuitive VCL configuration language
  • HTTP 1.1 on server side
  • TCP connection re-use
  • Edge side includes (ESI)
  • a few command line tools for stats (varnishstat, varnishhist, etc…)
  • powerful live traffic analyzer (varnishlog)
  • <private joke> 2012 website design </private joke>

Conclusion


Even if HAProxy can do TCP proxying, it is often used in front of web application, exactly where we find Varnish :).
They complete very well together: Varnish will make the website faster by offloading static object delivery to itself, while HAProxy can ensure a smooth load-balancing with smart persistence and DDOS mitigation.

Basically, HAProxy and Varnish completes very well, despite being “competitors” on a few features, each on them has its own domain of expertise where it performs very well: HAProxy is a reverse-proxy Load-Balancer and Varnish is a Reverse-proxy cache.

To be honest, when, at HAProxy Technologies, we work on infrastructures where Aloha Load balancer or HAProxy is deployed, we often see Varnish deployed. And if it is not the case, we often recommend the customer to deploy one if we feel it would improve its website performance.
Recently, I had a discussion with Ruben and Kristian when they came to Paris and they told me that they also often see an HAProxy when they work on infrastructure where Varnish is deployed.

So the real question is: Since Varnish and HAProxy are a bit similar but complete so well, how can we use them together???
The response could be very long, so stay tuned, I’ll try to answer this question in an article coming soon.

Related Links

Links

Hypervisors virtual network performance comparison from a Virtualized load-balancer point of view

Introduction

At HAProxy Technologies, we edit and sell a Load-Balancer appliance called ALOHA (stands for Application Layer Optimisation and High-Availability).
A few month ago, we managed to make it run on the most common hypervisors available:

  • VMWare (ESX, vsphere)
  • Citrix XenServer
  • HyperV
  • Xen OpenSource
  • KVM

< ADVERTISEMENT>So whatever your hypervisor is, you can runan Aloha on top of it 🙂 </ADVERTISEMENT>

Since a Load-Balancer appliance is Network IO intensive, we thought it was a good opportunity to bench each Hypervisor from a virtual network performance point of view.

Well, more and more companies use Virtualization in their infrastructures, so we guessed that a lot of people would be interested by the results of this bench, that’s why we decided to publish them on our blog.

Things to bear in mind about virtualization

One of the interesting feature of Virtualization is to be able to consolidate several servers onto a single hardware.
As a consequence, the resources (CPU, memory, disk and network IOs) are shared between several virtual machines.
An other issue to take into account is that the Hypervisor is like a new “layer” between the hardware and the OS inside the VM, which means that it may have an impact on the performance.

Purpose of benchmarking Hypervisors

First of all: WE ARE TOTALLY NEUTRAL AND HAVE NO INTEREST SAYING GOOD OR BAD THINGS ABOUT ANY HYPERVISORS.

Our main goal here is to check if each Hypervisor performs well enough to allow us to sell our Virtual Appliance on top of it.
From the tests we’ll run, we want to be able to measure the impact of the Hypervisor on the Virtual Machine performance

Benchmark platform and procedure

To run these tests, we use the same server for all Hypervisors, just swapping the hard-drive, to run each hypervisor independently.

Hypervisor Hardware summarized below:

  • CPU quad core i7 @3.4GHz
  • 16G of memory
  • Network card 1G copper e1000e

NOTE: we benched some other network cards and we got UGLY results. (Cf conclusions)
NOTE: there is a single VM running on the hypervisor: The Aloha.

The Aloha Virtual Appliance used is the Aloha VA 4.2.5 with 1G of memory and 2 vCPUs.
The client and WWW servers are physical machines plugged on the same LAN than the Hypervisor.
The client tool is inject and the web server behind the Aloha VA is httpterm.
So basically, the only thing that will change during these tests is the Hypervisor.

The Aloha is configured in reverse-proxy mode (using HAProxy) between the client and the server, load-balancing and analyzing HTTP requests.
We focused mainly on virtual networking performance: number of HTTP connections per seconds and associated bandwidth.
We ran the benchmark with different object size: 0, 1K, 2K, 4K, 8K, 16K, 32K, 48K, 64K.
NOTE: by “HTTP connection”, we mean a single HTTP request with its response over a single TCP connection, like in HTTP/1.0.

Basically, the 0K object test is used to get the number of connections per second the VA can do and the 64K object is used to measure the maximum bandwidth.

NOTE: the maximum bandwith will be 1G anyway, since we’re limitated by the physical NIC.

We are going to bench Network IO only, since this is the intensive usage a load-balancer does.
We won’t bench disks IOs…

Tested Hypervisors


We benched a native Aloha against Aloha VA embedded in the Hypervisors listed below:

  • HyperV
  • RHEV (KVM based)
  • vshpere 5.0
  • Xen 4.1 on Ubuntu 11.10
  • XenServer 6.0

Benchmark results


Raw server performance (native tests, without any hypervisor)

For the first test, we run the Aloha on the server itself without any Hypervisor.
That way, we’ll have some figures on the capacity of the server itself. We’ll use those numbers later in the article to compare the impact of each Hypervisor on performance.

native_performance

Microsoft HyperV


We tested HyperV on a Windows 2008 r2 server.
For this hypervisor 2 network cards are available:

  1. Legacy network adapteur: which emulates the network layer through the tulip driver.
    ==> With this driver, we got around 1.5K requests per seconds, which is really poor…
  2. Network adapteur: requires the hv_netvsc driver supplied by Microsoft in open source since Linux Kernel 2.6.32.
    ==> this is the driver we used for the tests

hyperv_performance

RHEV 3.0 Beta (KVM based)

RHEV is Red Hat Hypervisor, based on KVM.
For the Virtualization of the Network Layer, RHEV uses the virtio drivers.
Note that RHEV was still in Beta version when running this test.

VMWare Vsphere 5

There are 3 types of network cards available for Vsphere 5.0
1. Intel e1000: e1000 driver, emulates network layer into the VM.
2. VMxNET 2: allows network layer virtualization
3. VMxNET 3: allows network layer virtualization
The best results were obtained with the vmxnet2 driver.

Note: We have not tested Vsphere 4 neither ESX 3.5.

vsphere_performance

Xen OpenSource 4.1 on Ubuntu 11.10

Since CentOS 6.0 does not provide Xen OpenSource in its official repositories, we decided to use the latest (Oneiric Ocelot) Ubuntu server distribution, with Xen 4.1 on top of it.
Xen provides two network interfaces:

  1. emulated one, based on 8139too driver
  2. virtualized network layer, xen-vnif

Of course, the results are much better with xen-vnif, so we’re going to use this driver for the test.

xen41_performance

Citrix Xenserver 6.0


The network driver used for XenServer is the same one than the Xen OpenSource: xen-vnif.

xenserver60_performance

Hypervisors comparison


HTTP connections per second


The graph below summarizes the http connections per second capacity for each Hypervisor.
It shows us the Hypervisor overhead by comparing the light blue line which represents the server capacity without any Hypervisor to each hypervisor line..

http_connections_comparison

Bandwith usage


The graph below summarizes the http connections per second capacity for each Hypervisor.
It shows us the Hypervisor overhead by comparing the light blue line which represents the server capacity without any Hypervisor to each hypervisor line..

bandwith_comparison

Performance loss


Well, comparing Hypervisors to each others is nice, but remember, we wanted to know how much performance was lost in the hypervisor layer.
The graph below shows, in percentage, the loss generated by each hypervisor when compared to the native Aloha.
The highest the percentage, the worste for the hypervisor…

performance_loss_comparison

Conclusion

  • the Hypervisor layer has a non-negligible impact on networking performance on a Virtualized Load-Balancer running in reverse-proxy mode.
    But I guess it would be the same for any VM which is Networking IO intensive
  • The shortest the connections, the biggest the impact is.
    For very long connection like TSE, IMAP, etc… virtualization might make sense
  • Vsphere seems in advanced compared to its competitors on a performance point of view.
  • HyperV and Citrix XenServer have interesting performance.
  • RHEV (KVM) and Xen open source can still improve performance.
    Unless this is related to our procedure.
  • Even if the hardware layer is not accessed by the VM anymore, it still has a huge impact on performance.
    IE, on vsphere, we could not go higher than 20K connections per second with a Realtek NIC in the server…
    With Intel e1000e driver, we got up to 55K connections per second….
    So, even when you use virtualization, hardware counts!

Links

How to play with maxconn to avoid server slowness or crash

Using Aloha load balancer and HAProxy, it is easy to protect any application or web server against unexpected high load.

Introduction

The response time of web servers is directly related to the number of requests they have to manage at the same time. And the response time is not linearly linked to the number of requests, it looks like exponential.
The graph below shows a server response time compared to the number of simultaneous users browsing the website:

Simultaneous connections limiting

Simultaneous connections limiting is basically a number (aka the limit) a load balancer will consider as the maximum number of requests to send to a backend server at the same time.
Of course, since HAProxy has such a function, Aloha load-balancer does.

Smart handling of requests peak with HAProxy

The meaning is too prevent too many requests to be forwarded to an application server, by adding a limit for simultaneous requests for each server of the backend.

Fortunately, HAProxy would not reject any request over the limit, unlike some other load balancer does.

HAProxy use a queueing system and will wait for the backend server to be able to answer. This mechanism will add slow delays to request in the queue, but it has a few advantages :

  • no client request are rejected
  • every request can be faster served than with an overloaded backend server
  • the delay is still acceptable (a few ms in queue)
  • your server won’t crash because of the spike

simultaneous requests limiting occurs on the server side: HAProxy will limit the number of concurrent request to the server despite what happens on the client side.
HAProxy will never refuse any client connection until the underlying server runs out of capacity.

Concrete numbers

If you read carefully the graph above, you can easily see that the more your server has to process requests at the same time, the longer each request will take to process.
The table below summarize the time spent by our example server to process 250 requests with different simultaneous requests limiting value:

Number of requests Simultaneous requests limit Average time per request Longuest response time in ms
250 10 9 225
250 20 9 112
250 30 9 75
250 50 25 125
250 100 100 250
250 150 225 305
250 250 625 625

It’s up to the website owner to know what will be the best limit to setup on HAProxy.
You can approximate it by using HTTP benchmark tools and by comparing average response time to constant number of request you send to your backend server.

From the example above, we can see we would get the best of this backend server by setting up the limit to 30.
Setting up a limit too low would implies queueing request for a longer time and setting it too high would be counter-productive by slowing down each request because of server capacity.

HAProxy simultaneous requests limiting configuration

The simultaneous requests limiting configuration is made with the maxconn keyword on the server line definition.
Example:

frontend APPLI1
	bind :80
	mode http
	option http-server-close
	default_backend APPLI1

backend APPLI1
	balance roundrobin
	mode http
	server server1 srv1:80 maxconn 30
	server server2 srv2:80 maxconn 30

Links

Smart content switching for news website

Purpose

Build a scalable architecture for news website using below components:

  • load balancer with content switching capability
  • cache server
  • application server

Definition

  • Content switching: the ability to route traffic based on the content of the HTTP request: URI, parameters, headers, etc…
    HAproxy is a good example of OpenSource reverse proxy load-balancer with content switching capability.
  • cache server: a server able to quickly deliver static content.
    Squid, Varnish and Apache Traffic Server are OpenSource cache reverse proxy.
  • application server: the server which build the pages for your news website.
    This can be either Apache+PHP, Tomcat+Java, IIS+asp .net, etc…

Target Network Diagram

Architecture

Context

All the traffic pass through the Aloha load-balancer.
HAproxy, the layer 7 load-balancer included in Aloha, will do the content switching to route request either to cache servers or to application servers.
If cache server misses an object, it will get it from the application servers.

Haproxy configuration

Service configuration:

frontend public
	bind :80
	acl DYN path_beg /user
	acl DYN path_beg /profile
	acl DYN method POST
	use_backend APPLICATION if DYN
	default_backend CACHE

The content switching is achieved by the few lines beginning with the keyword acl.
If a URI starts with /user or /profile or if the method is a POSTthen, the traffic will be redirected to the APPLICATION server pool, otherwise the CACHE pool will be used.

Application pool configuration:

backend APPLICATION
	balance roundrobin
	cookie PHPSESSID prefix
	option httpchk /health
	http-check expect string GOOD
	server APP1 1.1.1.1:80 cookie app1 check
	server APP2 1.1.1.2:80 cookie app2 check

We maintain backend server persistence using the cookie sent by the application server, named PHPSESSID in this example. You can change this cookie name to the cookie provided by your application, like JSESSIONIDASP.NET_SessionId or anything else.
Note the health check URL: /health. The script executed on the backend will check server health (database availability, CPU usage, memory usage, etc…) and will return a GOOD if everything looks fine and a WRONG if not. With this, HAproxy will consider the server as ready only if the server returns a GOOD.

Cache pool configuration

backend CACHE
	balance url
	hash-type consistent
	option httpchk /
	http-check expect KEYWORD
	reqidel ^Accept-Encoding unless { hdr_sub(Accept-Encoding) gzip }
	reqirep ^Accept-Encoding: .*gzip.* Accept-Encoding: gzip
	server CACHE1 1.1.1.3:80 check
	server CACHE2 1.1.1.4:80 check

Here, we balance requests according to the URL. The purpose of this metric is to “force” a single URL to always be retrieved from the same server.
Main benefits are :

  1. less objects in caches memory
  2. less requests to the application server for static and pseudo-static content

In order to lower the impact of the Vary: header on Content Encoding, we added the two lines reqidel / reqirep to normalize a bit the Accept-Encoding header.

Links