Category Archives: HAProxy

HAProxy’s load-balancing algorithm for static content delivery with Varnish

HAProxy’s load-balancing algorithms

HAProxy supports many load-balancing algorithms which may be used in many different type of cases.
That said, cache servers, which deliver most of the time the static content from your web applications, may require some specific load-balancing algorithms.

HAProxy stands in front of your cache server for some good reasons:

  • SSL offloading (read PHK’s feeling about SSL, Varnish and HAProxy)
  • HTTP content switching capabilities
  • advanced load-balancing algorithms

The main purpose of this article is to show how HAProxy can be used to aggregate Varnish servers memory storage in some kind of “JBOD” mode (like the “Just a Bunch Of Disks“).
Main purpose of the examples delivered here are to optimize the resources on the cache, mainly its memory, in order to improve the HIT rate. This will also improve your application response time and make your site top ranked on google 🙂

Content Switching in HAProxy

This has been covered many times on this blog.
As a quick introduction for readers who are not familiar with HAProxy, let’s explain how it works.

Clients will get connected to HAProxy through a Frontend. Then HAProxy routes traffic to a backend (server farm) where the load-balancing algorithm is used to choose a server.
A frontend can points to multiple backends and the choice of a backend is made through acls and use_backend rules..
Acls can be formed using fetches. A fetch is a directive which instructs HAProxy where to get content from.

Enough theory, let’s make a practical example: splitting static and dynamic traffic using the following rules:

  • Static content is hosted on domain names starting by ‘static.’ and ‘images.’
  • Static content files extensions are ‘.jpg’ ‘.png’ ‘.gif’ ‘.css’ ‘.js’
  • Static content can match any of the rule above
  • anything which is not static is considered as dynamic

The configuration sniplet below should be integrated into the HAProxy frontend. It matches the rules above to do traffic splitting. The varnish servers will stands in the bk_static farm.

frontend ft_public
 <frontend settings>
 acl static_domain  req.hdr_beg(Host) -i static. images.
 acl static_content path_end          -i .jpg .png .gif .css .js
 use_backend bk_static if static_domain or static_content
 default_backend bk_dynamic
backend bk_static
 <parameters related to static content delivery>

The configuration above creates 2 named acls ‘static_domain‘ and ‘static_content‘ which are used by the used_backend rule to route the traffic to varnish servers.

HAProxy and hash based load-balancing algotithm

Later in this article, we’ll heavily used the hash based load-balancing algorithms from HAProxy.
So a few information here (non exhaustive, it would deserve a long blog article) which will be useful for people wanting to understand what happens deep inside HAProxy.

The following parameters are taken into account when computing a hash algorithm:

  • number of servers in the farm
  • weight of each server in the farm
  • status of the servers (UP or DOWN)

If any of the parameter above changes, the whole hash computation also changes, hence request may hit an other server. This may lead to a negative impact on the response time of the application (during a short period of time).
Fortunately, HAProxy allows ‘consistent’ hashing, which means that only the traffic related to the change will be impacted.
That’s why you’ll see a lot of hash-type consistent directives in the configuration samples below.

Load-Balancing varnish cache server

Now, let’s focus on the magic we can add in the bk_static server farm.

Hashing the URL

HAProxy can hash the URL to pick up a server. With this load-balancing algorithm, we guarantee that a single URL will always hit the same Varnish server.

hashing the URL path only

In the example below, HAProxy hashes the URL path, which is from the first slash ‘/’ character up to the question mark ‘?’:

backend bk_static
  balance uri
  hash-type consistent

hashing the whole url, including the query string

In some cases, the query string may contain some variables in the query string, which means we must include the query string in the hash:

backend bk_static
  balance uri whole
  hash-type consistent

Query string parameter hash

That said, in some cases (API, etc…), hashing the whole URL is not enough. We may want to hash only on a particular query string parameter.
This applies well in cases where the client can forge itself the URL and all the parameters may be randomly ordered.
The configuration below tells HAProxy to apply the hash to the query string parameter named ‘id’ (IE: /image.php?width=512&id=12&height=256)

backend bk_static
  balance url_param id
  hash-type consistent

hash on a HTTP header

HAProxy can apply the hash to a specific HTTP header field.
The example below applies it on the Host header. This can be used for people hosting many domain names with a few pages, like users dedicated pages.

backend bk_static
  balance hdr(Host)
  hash-type consistent

Compose your own hash: concatenation of Host header and URL

Nowadays, HAProxy becomes more and more flexible and we can use this flexibility in its configuration.
Imagine, in your varnish configuration, you have a storage hash key based on the concatenation of the host header and the URI, then you may want to apply the same load-balancing algorithm into HAProxy, to optimize your caches.

The configuration below creates a new HTTP header field named X-LB which contains the host header (converted to lowercase) concatenated to the request uri (converted in lowercase too).

backend bk_static
  http-request set-header X-LB %[req.hdr(Host),lower]%[req.uri,lower]
  balance hdr(X-LB)
  hash-type consistent


HAProxy and Varnish works very well together. Each soft can benefit from performance and flexibility of the other one.


Microsoft Remote Desktop Services (RDS) Load-Balancing

Microsoft Remote Desktop services (RDS)

Remote Desktop Services, formerly Terminal Services, is a technology from Microsoft that allows users to access remotely to a session-based desktop, virtual machine-based desktop or applications hosted in a datacenter from their corporate network or from the internet.

Multiple RDS servers can be used in a farm. Hence we need to balance the load against them.
To achieve this purpose, we have different ways:
* using a connection broker
* using a load-balancer with the connection broker
* using a load-balancer without the connection broker

Of course, our load-balancer of choice is HAProxy!
In this blog article, we’re going to focus only on the case where a load-balancer is used.

The main issue when load-balancing multiple Remote Desktop Services servers is to ensure a user the continuity of his session in case of a network outage.

Current article will focus on session high availability for an optimal end user experience.

HAProxy with a connection broker

The connection broker, formerly Session broker, main purpose is to reconnect a user to his existing session. Since Windows 2008, the connection broker also have some load-balancing mechanism.

So, why using a load-balancer if the connection broker can do load-balance?

Answer is simple: security. Since HAProxy is a Reverse-Proxy, it breaks the TCP connection between the client and the server. HAProxy can be deployed in DMZ to give access to users coming from internet to a RDS farm deployed in the VLAN dedicated to servers.

HAProxy configuration

Note: this configuration works for the ALOHA 6.0 and above and HAPEE (HAProxy Enterprise Edition) 1.5 and above.

frontend ft_rdp
  mode tcp
  bind name rdp
  timeout client 1h
  log global
  option tcplog
  tcp-request inspect-delay 2s
  tcp-request content accept if RDP_COOKIE
  default_backend bk_rdp

backend bk_rdp
  mode tcp
  balance leastconn
  persist rdp-cookie
  timeout server 1h
  timeout connect 4s
  log global
  option tcplog
  option tcp-check
  tcp-check connect port 3389 ssl
  default-server inter 3s rise 2 fall 3
  server srv01 weight 10 check
  server srv02 weight 10 check

HAProxy without a connection broker

HAProxy can be used on its own to perform session load-balancing and resumption. For this purpose, it needs a stick-table where the user-server association is stored.
A peers section is added to the configuration. So we can share session persistence information between a cluster of ALOHAs or HAPEE servers.

peers aloha
 peer aloha1
 peer aloha2

frontend ft_rdp
  mode tcp
  bind name rdp
  timeout client 1h
  log global
  option tcplog
  tcp-request inspect-delay 2s
  tcp-request content accept if RDP_COOKIE
  default_backend bk_rdp

backend bk_rdp
  mode tcp
  balance leastconn
  timeout server 1h
  timeout connect 4s
  log global
  option tcplog
  stick-table type string len 32 size 10k expire 8h peers aloha
  stick on rdp_cookie(mstshash)
  option tcp-check
  tcp-check connect port 3389 ssl
  default-server inter 3s rise 2 fall 3
  server srv01 weight 10 check
  server srv02 weight 10 check

To know the user-server association, we can simply read the content of the stick-table:

echo show table bk_rdp | socat /var/run/haproxy.stat -
# table: bk_rdp, type: string, size:10240, used:5
0x21c7eac: key=Administrator use=0 exp=83332288 server_id=1
0x21c7eac: key=test-001 use=0 exp=83332288 server_id=2

We can easily read the login used by the user, the expiation date (in milliseconds) and the server ID used for the session.


HAProxy 1.6-dev1 and LUA

HAProxy 1.6-dev1

Yesterday, Willy has released HAProxy 1.6-dev1: ANNOUNCE HAProxy 1.6-dev1.
This version contains many new features and core improvements. Amongst the new features, one is LUA, contributed by Thierry (HAProxy Technologies developer).

NOTE: We invite everyone to download and test HAProxy 1.6-dev1 and to report bugs to the mailing list.

Please note that there may be a code freeze of HAProxy 1.6 in May. Purpose is to release a stable version in September.


LUA is a powerful, fast, lightweight, embeddable scripting language.

More about lua

The main advantage of LUA for HAProxy is to give the ability to anyone to write their own features without having to know writing C code and patching HAProxy…

About this article

Please note that LUA integration in HAProxy is brand new. So the information provided here are true today but may be obsolete later, because of internal changes and updates.

Compiling HAProxy and LUA

Installaing LUA 5.3

For now, HAProxy requires LUA 5.3 to work, here is the procedure to install it:

cd /usr/src
curl -R -O
tar zxf lua-5.3.0.tar.gz
cd lua-5.3.0
make linux
sudo make INSTALL_TOP=/opt/lua53 install

LUA 5.3 library and include files are now installed in /opt/lua53.

Compiling HAProxy for lua

Since HAProxy is in a dev state, we may encounter some bugs. Let’s enable some debugging features:

make DEBUG=-ggdb CFLAGS=-O0 TARGET=linux2628 
USE_LUA=yes LUA_LIB=/opt/lua53/lib/ 
LUA_INC=/opt/lua53/include/ LDFLAGS=-ldl

Using LUA in HAProxy

turning HAProxy as a mirror web server

By mirror web server, I mean a web server which returns the request in the body of the response, without any modification.
It may be useful to use in HAProxy when you want to setup some HTTP header manipulation in HAProxy and you want to see the result. (It is more convenient than tcpdump).

* First, the HAProxy configuration:

 lua-load ./mylua.lua

 mode http
 timeout connect 1s
 timeout client 1s
 timeout server 1s

frontend fe
 bind name fe
 acl debugme req.hdr_cnt(X-debug-me) ge 1
 # add your http manipulation rules here
 http-request lua mirror if debugme
 default_backend be

backend be
 server s

* And the LUA which does the mirroring of the request:

-- a simple mirror web server
-- it generates a response whose body contains the requests headers
function mirror(txn)
local buffer = ""
local response = ""
local mydate =

buffer = buffer .. "You sent the following headersrn"
buffer = buffer .. "===============================================rn"
buffer = buffer .. txn.req:dup()
buffer = buffer .. "===============================================rn"

response = response .. "HTTP/1.0 200 OKrn"
response = response .. "Server: haproxy-lua/mirrorrn"
response = response .. "Content-Type: text/htmlrn"
response = response .. "Date: " .. mydate .. "rn"
response = response .. "Content-Length: " .. buffer:len() .. "rn"
response = response .. "Connection: closern"
response = response .. "rn"
response = response .. buffer


Some explanations:
* mydate is the result of an HAProxy converter applied to an HAProxy fetch txn.f:date()
* The call to txn.req:dup returns the content of HAProxy’s buffer request.

Testing: simply run a curl against HAProxy, don’t forget to send a X-debug-me HTTP header with the parameter -H "X-debug-me: yes". HAProxy should return your request in the response body:

HTTP/1.0 200 OK
Server: haproxy-lua/mirror
Content-Type: text/html
Date: Fri, 12 Mar 2015 13:06:44 GMT
Content-Length: 208
Connection: keep-alive

You sent the following headers
GET / HTTP/1.1
User-Agent: curl/7.41.0
Accept: */*



A HTTP monitor which matches multiple conditions in HAProxy

Load-Balancing and health checking

Health checking is the method to check a service availability on a server.
It is one of the most important feature of a load-balancer. How could we balance traffic amongst servers if we don’t know if the service is alive???

HAProxy and HTTP check

HAProxy can probe HTTP applications using httpchk option.
This option can be customized using the http-check expect directive to match different status codes or content.
That said, a single http-check expect rule can match.
So we can’t match a status code and the presence of a string in the page, for example.

Make HAProxy match multiple conditions for HTTP health checking

The solution is to use to the raw tcp-check and write a health check script sequence which match all the conditions.

For example, you want to ensure the server’s response has:

  • HTTP status code is 200
  • absence of keyword Error
backend myapp
 option tcp-check
 tcp-check send GET /my/check/url HTTP/1.1rn
 tcp-check send Host: myhostrn
 tcp-check send Connection: closern
 tcp-check send rn
 tcp-check expect string HTTP/1.1 200 OK
 tcp-check expect ! string Error


Web application name to backend mapping in HAProxy


Let’s take a web application platform where many HTTP Host header points to.
Of course, this platform hosts many backends and HAProxy is used to perform content switching based on the Host header to route HTTP traffic to each backend.

HAProxy map

HAProxy 1.5 introduced a cool feature: converters. One converter type is map.
Long story made short: a map allows to map a data in input to an other one on output.

A map is stored in a flat file which is loaded by HAProxy on startup. It is composed by 2 columns, on the left the input string, on the right the output one:

in out

Basically, if you call the map above and give it the in strings, it will return out.


Now, the interesting part of the article 🙂

As stated in introduction, we want to map hundreds of Host headers to tens of backends.

The old way of mapping: acl and use_backend rules

Before the map, we had to use acls and use_backend rules.

like below:

frontend ft_allapps
 use_backend bk_app1 if { hdr(Host) -i }
 use_backend bk_app2 if { hdr(Host) -i }
 default_backend bk_default

Add one statement per use_backend rule.

This works nicely for a few backends and a few domain names. But this type of configuration is hardly scallable…

The new way of mapping: one map and one use_backend rule

Now we can use map to achieve the same purpose.

First, let’s create a map file called, with the following content: on the left, the domain name, on the right, the backend name:

#domainname  backendname bk_app1 bk_app1 bk_app2 bk_app2

And now, HAProxy configuration:

frontend ft_allapps
 use_backend %[req.hdr(host),lower,map_dom(/etc/hapee-1.5/,bk_default)]

Here is what HAProxy will do:

  1. req.hdr(host) ==> fetch the Host header from the HTTP request
  2. lower ==> convert the string into lowercase
  3. map_dom(/etc/hapee-1.5/ ==> look for the lowercase Host header in the map and return the backend name if found. If not found, the name of a default backend is returned
  4. route traffic to the backend name returned by the map

Now, adding a new content switching rule means just add one new line in the map content (and reload HAProxy). No regexes, map data is stored in a tree, so processing time is very low compared to matching many string in many ACLs for many use_backend rules.

simple is beautiful!!!

HAProxy map content auto update

If you are an HAPEE user (and soon available for the ALOHA), you can use the lb-update content to download the content of the map automatically.
Add the following statement in your configuration:

 update id url delay 60s timeout 5s retries 3 map


HAProxy and sslv3 poodle vulnerability

SSLv3 poodle vulnerability

Yesterday, Google security researchers have disclosed a new vulnerability on SSL protocol.
Fortunately, this vulnerability is only on an old version of the SSL protocol: SSLv3 (15 years old protocol).
An attacker can force a browser to downgrade the protocol version used to cipher traffic to SSLv3 in order to exploit the POODLE vulnerability and access to data in clear.

Some reading about SSLv3 Poodle vulnerability:

Today’s article is going to explain how to use HAProxy to simply prevent using SSLv3 or to prevent those users to reach your applications and print them a message.

Disable SSLv3 in HAProxy

In SSL offloading mode

In this mode, HAProxy is the SSL endpoint of the connection.
It’s a simple keyword on the frontend bind directive:

  bind ssl crt /pat/to/cert.pem no-sslv3

In SSL forward mode

In this mode, HAProxy forwards the SSL traffic to the server without deciphering it.
We must setup an ACL to match the SSL protocol version, then we can refuse the connection. This must be added in a **frontend** section:

  tcp-request inspect-delay 2s
  acl sslv3 req.ssl_ver 3
  tcp-request content reject if sslv3

Communicate a message to users

Denying sslv3 is a good way, but a better one would to educate as well users who are using this protocol.
The configuration below shows how to redirect a user to a specific page when they want to use your application over an SSLv3 connection. Of course, HAProxy must allow itself SSLv3:

frontend ft_www
  bind ssl crt /pat/to/cert.pem
  acl sslv3 ssl_fc_protocol SSLv3
# first rule after all your 'http-request deny' and
# before all the redirect, rewrite, etc....
  http-request allow if sslv3
# first content switching rule
  use_backend bk_sslv3 if sslv3

backend bk_sslv3
  mode http
  errorfile 503 /etc/haproxy/pages/sslv3.http

And the content of the file /etc/haproxy/pages/sslv3.http:

HTTP/1.0 200 OK
Cache-Control: no-cache
Connection: close
Content-Type: text/html

<title>SSLv3 spotted</title>
<body><h1>SSLv3 spotted</h1></body>
SSLv3 forbidden for your safety:<BR><BR>
If you want to browse this website, you should upgrade your browser.


Mitigating the shellshock vulnerability with HAProxy

Bash Shellshock vulnerability (CVE-2014-6271 and CVE-2014-7169)

Last week, a vulnerability in bash has been discovered. It is possible, under some circumstances, to inject code into a bash shell script.
It could be very dangerous if bash is used to process request sent remotely.
For now, you’re safe if no bash scripts are called by services with remote accesses.

Some reading about bash shellshock vulnerability:

Today’s article is going to explain how to use HAProxy to protect your application from bash shellshock vulnerability if you’re in the case where you have to be protected.


The diagram is pretty simple. Our purpose will to detect any purposely built requests and to prevent them to reach the server:

|                                                 |
| +----------+     +---------+     +------------+ |
| |          |     |         |     |            | |
| | Attacker | +-> | HAProxy | +-> | Vulnerable | |
| |          |     |         |     | server     | |
| +----------+     +---------+     |            | |
|                                  +------------+ |
|                                                 |


Place the configuration sniplet into your HAProxy frontend configuration:

  reqdeny  ^[^:]+:s*(s*)s+{
  reqdeny  ^[^:]+:s+.*?(<<[^<;]+){5,}

Of course, your frontend must be in http mode and HAProxy must have been compiled with USE_PCRE option.

HAProxy will return a 403 if a request matches the shellshock attack.

Note: greeting to Thomas for providing the tip on HAProxy’s mailing list