Tag Archives: reverse-proxy

HAProxy and container IP changes in Docker

HAProxy and Docker containers

Docker is a nice tool to handle containers: it allows building and running your apps in a simple and efficient way.

When used in production together with HAProxy, devops teams face a big challenge: how to followup a container IP change when restarting a container?

This blog article aims at giving a first answer to this question.

The version of docker used for this article is 1.8.1 (very important, since docker’s default behavior has changed in 1.9.0…)

HAProxy, webapp and docker diagram

The diagram below shows how docker runs on my laptop:

  • A docker0 network interface, with IP
  • all containers run in the subnet
|                                                                               |
|  +-----------------------docker-----------------------+         |
|  |                                                    | docker0:   |
|  |  +---------+  +---------+       +----------+       |                       |
|  |  | HAProxy |  | appsrv1 |       | rsyslogd |       |                       |
|  |  +---------+  +---------+       +----------+       |                       |
|  |                                                    |                       |
|  +----------------------------------------------------+                       |
|                                                                               |

The IP address associated to docker0 will be used to export some services.

In this article, we’ll start up 3 containers:

  1. rsyslogd: where HAProxy will send all its logs
  2. appsrv1: our application server, which may be restarted at any time
  3. haproxy: our load-balancer, which must follow-up appsrv1‘s IP

Building and running the lab


First, we need rsyslogd and haproxy containers. They can be build from the following Dockerfiles:

Then run:

docker build -t blog:haproxy_dns ~/tmp/haproxy/blog/haproxy_docker_dns_link/blog_haproxy_dns/
docker build -t blog:rsyslogd ~/tmp/haproxy/blog/haproxy_docker_dns_link/blog_rsyslogd/

I consider appsrv container as yours: it’s your application.

Starting up our lab

Docker assign IPs to containers in the order they are started up, incrementing last byte for each new container.

To make it simpler, let’s restart docker first, so our container IPs are predictible:

sudo /etc/rc.d/docker restart

Then let’s start up our rsyslogd container:

docker run --detach --name rsyslogd --hostname=rsyslogd \
	--publish= \

And let’s attach a terminal to it:

docker attach rsyslogd

Now, run appsrv container as appsrv1:

docker run --detach --name appsrv1 --hostname=appsrv1 demo:appsrv

And finally, let’s start HAProxy, with a docker link to appsrv1:

docker run --detach --name haproxy --hostname=haproxy \
	--link appsrv1:appsrv1 \

Docker links, /etc/hosts file updated and DNS

When using the ”–link” option, docker creates a new entry in the containers /etc/hosts file with the IP address and name provided by the ”link” directive.
Docker will also update this file when the remote container (here appsrv1) IP address is changed (IE when restarting the container).

If you’re familiar with HAProxy, you already know it doesn’t do file system IOs at run time. Furthermore, HAProxy doesn’t use /etc/hosts file directly. The glibc might use it when HAProxy asks for DNS resolution when parsing the configuration file. (read below for DNS resolution at runtime)

That said, if appsrv1 IP get changed, then /etc/hosts file is updated accordingly, then HAProxy is not aware of the change and the application may fail.
A quick solution would be to reload HAProxy process in its container, to force it taking into account the new IP.

A more reliable solution, is to use HAProxy 1.6 DNS resolution capability to follow-up the IP change. With this purpose in mind, we added 2 tools into our HAProxy container:

  1. dnsmasq: tiny software which can act as a DNS server which takes /etc/hosts file as its database
  2. inotifytools: watch changes on /etc/hosts file and force dnsmasq to reload it when necessary

I guess now you got it:

  • when appsrv1 is restarted, then docker gives it a new IP
  • Docker populates then this IP address into all /etc/hosts file required (those using ”link” directives)
  • Once populated, inotify tool detect the file change and triggers a dnsmasq reload
  • HAProxy periodically (can be configured) probes DNS and will get the new IP address quickly from dnsmasq

Docker container restart and HAProxy followup in action

At this stage, we should have a container attached to rsyslogd and we should be able to see HAProxy logging. Let’s give it a try:


Nov 17 09:29:09 haproxy[10]: [17/Nov/2015:09:29:09.729] f_myapp b_myapp/appsrv1 0/0/0/1/1 200 858 - - ---- 1/1/0/1/0 0/0 "GET / HTTP/1.1"

Now, let’s consider your dev team delivered a new version of your application, so you build it and need to restart its running container:

docker restart appsrv1

and voilĂ :

==> /var/log/haproxy/events <==
Nov 17 09:29:29 haproxy[10]: b_myapp/appsrv1 changed its IP from to by docker/dnsmasq.
Nov 17 09:29:29 haproxy[10]: b_myapp/appsrv1 changed its IP from to by docker/dnsmasq.

Let’s test the application again:


Nov 17 09:29:31 haproxy[10]: [17/Nov/2015:09:29:31.013] f_myapp b_myapp/appsrv1 0/0/0/0/0 200 858 - - ---- 1/1/0/1/0 0/0 "GET / HTTP/1.1"


There are a few limitations in this mechanism:

  • it’s painful to maintaint ”link” directive when you have a 10s or 100s or more of containers….
  • the host computer, and computers in the host network can’t easily access our containers, because we don’t know their IPs and their hostnames are resolved in HAProxy container only
  • if we want to add more appserver in HAProxy‘s farm we still need to restart HAProxy‘s container (and update configuration accordingly)

To fix some of the issues above, we can dedicate a container to perform DNS resolution within our docker world and deliver responses to any running containers or hosts in the network. We’ll see that in a next blog article


What’s new in HAProxy 1.6

[ANNOUNCE] HAProxy 1.6.0 released

Yesterday, 13th of October, Willy has announced the release of HAProxy 1.6.0, after 16 months of development!
First good news is that release cycle goes a bit faster and we aim to carry on making it as fast.
A total of 1156 commits from 59 people were committed since the release of 1.5.0, 16 months ago.

Please find here the official announce: [ANNOUNCE] haproxy-1.6.0 now released!.

In his mail, Willy detailed all the features that have been added to this release. The purpose of this blog article is to highlight a few of them, providing the benefits and some configuration examples.

NOTE: Most of the features below were already backported and integrated into our HAProxy Enterprise and ALOHA products.

HAProxy Enterprise is our Open Source version of HAProxy based on HAProxy community stable branch where we backport many features from dev branch and we package it to make the most stable, reliable, advanced and secured version of HAProxy. It also comes with third party software to fill the gap between a simple HAProxy process and a load-balancer (VRRP, syslog, SNMP, Route Health Injection, etc…). Cherry on the cake, we provide “enterprise” support on top of it.

NOTE 2: the list of new features introduced here is not exhaustive. Example are proposed in a quick and dirty way to teach you how to start with the feature. Don’t run those examples in production 🙂

It’s 2015, let’s use QUOTE in configuration file

Those who uses HAProxy for a long time will be happy to know that ‘\ ‘ (backslash-space) sequence is an old painful souvenir with 1.6 🙂
We can now write:

reqirep "^Host: www.(.*)" "Host: foobar\1"


option httpchk GET / "HTTP/1.1\r\nHost: www.domain.com\r\nConnection: close"

Lua Scripting

Maybe the biggest change that occurred is the integration of Lua.
Quote from Lua’s website: “Lua is a powerful, fast, lightweight, embeddable scripting language.“.

Basically, everyone has now the ability to extend HAProxy by writing and running their own Lua scripts. No need to write C code, maintain patches, etc…
If some Lua snipplets are very popular, we may write the equivalent feature in C and make it available in HAProxy mainline.

One of the biggest challenge Thierry faced when integrating Lua is to give it the ability to propose non-blocking processing of Lua code and non-blocking network socket management.

HAProxy requires Lua 5.3 or above.

With Lua, we can add new functions to the following HAProxy elements:

  • Service
  • Action
  • Sample-fetch
  • Converter

Compiling HAProxy and Lua

Installing LUA 5.3

cd /usr/src
curl -R -O http://www.lua.org/ftp/lua-5.3.0.tar.gz
tar zxf lua-5.3.0.tar.gz
cd lua-5.3.0
make linux
sudo make INSTALL_TOP=/opt/lua53 install

LUA 5.3 library and include files are now installed in /opt/lua53.

Compiling HAProxy with Lua support

make TARGET=linux2628 USE_OPENSSL=1 USE_PCRE=1 USE_LUA=1 LUA_LIB=/opt/lua53/lib/ LUA_INC=/opt/lua53/include/

HAProxy/Lua simple Hello world example

A simple Hello world! in Lua could be written like this:

  • The lua code in a file called hello_world.lua:
    core.register_service("hello_world", "tcp", function(applet)
       applet:send("hello world\n")
  • The haproxy configuration
       lua-load hello_world.lua
    listen proxy
       tcp-request content use-service lua.hello_world

More with Lua

Please read the doc, ask your questions on HAProxy‘s ML (no registration needed): haproxy@formilux.org.
Of course, we’ll write more articles later on this blog about Lua integration.


HAProxy‘s running context is very important when writing configuration. In HAProxy, each context is isolated. IE: you can’t use a request header when processing the response.
With HAProxy 1.6, this is now possible: you can declare capture slots, store data in it and use it at any time during a session.

 mode http

frontend f_myapp
 bind :9001
 declare capture request len 32 # id=0 to store Host header
 declare capture request len 64 # id=1 to store User-Agent header
 http-request capture req.hdr(Host) id 0
 http-request capture req.hdr(User-Agent) id 1
 default_backend b_myapp

backend b_myapp
 http-response set-header Your-Host %[capture.req.hdr(0)]
 http-response set-header Your-User-Agent %[capture.req.hdr(1)]
 server s1 check

Two new headers are inserted in the response:
Your-User-Agent: curl/7.44.0

Multiprocess, peers and stick-tables

In 1.5, we introduced “peers” to synchronize stick-table content between HAProxy servers. This feature was not compatible with multi-process mode.
We can now, in 1.6, synchronize content of the table it is stick to one process. It allows creating configurations for massive SSL processing pointing to a single backend sticked on a single process where we can use stick tables and synchronize its content.

peers article
 peer itchy

 pidfile /tmp/haproxy.pid
 nbproc 3

 mode http

frontend f_scalessl
 bind-process 1,2
 bind :9001 ssl crt /home/bassmann/haproxy/ssl/server.pem
 default_backend bk_lo

backend bk_lo
 bind-process 1,2
 server f_myapp unix@/tmp/f_myapp send-proxy-v2

frontend f_myapp
 bind-process 3
 bind unix@/tmp/f_myapp accept-proxy
 default_backend b_myapp

backend b_myapp
 bind-process 3
 stick-table type ip size 10k peers article
 stick on src
 server s1 check



It is now possible to position a syslog tag per process, frontend or backend. Purpose is to ease the job of syslog servers when classifying logs.
If no log-tag is provided, the default value is the program name.

Example applied to the configuration snipplet right above:

frontend f_scalessl
 log-tag SSL

frontend f_myapp
 log-tag CLEAR

New log format variables

New log format variables have appeared:

  • %HM: HTTP method (ex: POST)
  • %HP: HTTP request URI without query string (path)
  • %HQ: HTTP request URI query string (ex: ?bar=baz)
  • %HU: HTTP request URI (ex: /foo?bar=baz)
  • %HV: HTTP version (ex: HTTP/1.0)

Server IP resolution using DNS at runtime

In 1.5 and before, HAProxy performed DNS resolution when parsing configuration, in a synchronous mode and using the glibc (hence /etc/resolv.conf file).
Now, HAProxy can perform DNS resolution at runtime, in an asynchronous way and update server IP on the fly. This is very convenient in environment like Docker or Amazon Web Service where server IPs can be changed at any time.
Configuration example applied to docker. A dnsmasq is used as an interface between /etc/hosts file (where docker stores server IPs) and HAProxy:

resolvers docker
 nameserver dnsmasq

 mode http
 log global
 option httplog

frontend f_myapp
 bind :80
 default_backend b_myapp

backend b_myapp
 server s1 nginx1:80 check resolvers docker resolve-prefer ipv4

Then, let’s restart s1 with the command “docker restart nginx1” and let’s have a look at the magic in the logs:
(...) haproxy[15]: b_myapp/nginx1 changed its IP from to by docker/dnsmasq.

HTTP rules

New HTTP rules have appeared:

  • http-request: capture, set-method, set-uri, set-map, set-var, track-scX, sc-in-gpc0, sc-inc-gpt0, silent-drop
  • http-response: capture, set-map, set-var, sc-inc-gpc0, sc-set-gpt0, silent-drop, redirect


We often used HTTP header fields to store temporary data in HAProxy. With 1.6, we can now define variables.
A variable is available for a scope: session, transaction (request or response), request, response.
A variable name is prefixed by its scope (sess, txn, req, res), a dot ‘.’ and a tag composed by ‘a-z’, ‘A-Z’, ‘0-9’ and ‘_’.

Let’s rewrite the capture example using variables

 # variables memory consumption, in bytes
 tune.vars.global-max-size 1048576
 tune.vars.reqres-max-size     512
 tune.vars.sess-max-size      2048
 tune.vars.txn-max-size        256

 mode http

frontend f_myapp
 bind :9001
 http-request set-var(txn.host) req.hdr(Host)
 http-request set-var(txn.ua) req.hdr(User-Agent)
 default_backend b_myapp

backend b_myapp
 http-response set-header Your-Host %[var(txn.host)]
 http-response set-header Your-User-Agent %[var(txn.ua)]
 server s1 check


Now, HAProxy can send emails when server states change (mainly goes DOWN), so your sysadmins/devos won’t sleep anymore :). It used to be able to log only before.

mailers mymailers
 mailer smtp1 
 mailer smtp2 
backend mybackend 
 mode tcp 
 balance roundrobin
 email-alert mailers mymailers
 email-alert from test1@horms.org
 email-alert to test2@horms.org
 server srv1
 server srv2

Processing of HTTP request body

Until 1.5 included, HAProxy could process only HTTP request headers. It can now access request body.
Simply enable the statement below in your frontend or backend to give HAProxy this ability:

option http-buffer-request

Protection against slow-POST attacks

slow-POST attacks are like the slowlorys one, except that HTTP header are sent quicly, but request body are sent very slowly.
Once enabled, the timeout http-request parameters also apply to the POSTED data.

Fetch methods

A few new fetch methods now exists to play with the body: req.body, req.body_param, req.body_len, req.body_size, etc…

Short example to detect “SELECT *” string in a request POST body, and of course how to deny it:

 mode http

frontend f_mywaf
 bind :9001
 option http-buffer-request
 http-request deny if { req.body -m reg "SELECT \*" }
 default_backend b_myapp

backend b_myapp
 server s1 check

New converters

1.5 introduced the converters but only a very few of them were available.
1.6 adds many of them. The list is too long, but let’s give the most important ones: json, in_table, field, reg_sub, table_* (to access counters from stick-tables), etc…

Device Identification

Through our company, we have some customer who want us to integrate into HAProxy the ability to detect device type and characteristics and report it to the backend server.
We got a couple of contributions from 2 companies experts in this domain: 51 degrees and deviceatlas.
You can now load those libraries in HAProxy in order to fully qualify a client capabilities and set up some headers your application server can rely on to adapt content delivered to the client or let the varnish cache server use it to cache multiple flavor of the same object based on client capabilities.

More on this blog later on how to integrate each product.

Seamless server states

Prior 1.6, when being reloaded, HAProxy considers all the servers are UP until the first check is performed.
Since 1.6, we can dump server states into a flat file right before performing the reload and let the new process know where the states are stored. That way, the old and new processes owns exactly the same server states (hence seamless).
The following information are reported:

  • server IP address when resolved by DNS
  • operational state (UP/DOWN/…)
  • administrative state (MAINT/DRAIN/…)
  • Weight (including slowstart relative weight)
  • health check status
  • rise / fall current counter
  • check state (ENABLED/PAUSED/…)
  • agent check state (ENABLED/PAUSED/…)

The state could be applied globally (all server found) or per backend.
Simple HAProxy configuration:

 stats socket /tmp/socket
 server-state-file /tmp/server_state

backend bk
 load-server-state-from-file global
 server s1 check weight 11
 server s2 check weight 12

Before reloading HAProxy, we save the server states using the following command:

socat /tmp/socket - <<< "show servers state" > /tmp/server_state

Here is the content of /tmp/server_state file:

# <field names skipped for the blog article>
1 bk 1 s1 2 0 11 11 4 6 3 4 6 0 0
1 bk 2 s2 2 0 12 12 4 6 3 4 6 0 0

Now, let’s proceed with reload as usual.
Of course, the best option is to export the server states using the init script.

External check

HAProxy can run a script to perform complicated health checks.
Just be aware about the security concerns when enabling this feature!!



backend b_myapp
 external-check path "/usr/bin:/bin"
 external-check command /bin/true
 server s1 check


NOTE: some of the features introduced here may need a recent openssl library.

Detection of ECDSA-able clients

This has already been documented by Nenad on this blog: Serving ECC and RSA certificates on same IP with HAProxy

SSL certificate forgery on the fly

Since 1.6, HAProxy can forge SSL certificate on the fly!
Yes, you can use HAProxy with your company’s CA to inspect content.

Support of Certificate Transparency (RFC6962) TLS extension

When loading PEM files, HAProxy also checks for the presence of file at the same path suffixed by “.sctl”. If such file is found, support for Certificate Transparency (RFC6962) TLS extension is enabled.
The file must contain a valid Signed Certificate Timestamp List, as described in RFC. File is parsed to check basic syntax, but no signatures are verified.

TLS Tickets key load through stats socket

A new stats socket command is available to update TLS ticket keys at runtime. The new key is used for encryption/decryption while the old ones are used for decryption only.

Server side SNI

Many application servers now takes benefit from Server Name Indication TLS extension.

backend b_myapp_ssl
 mode http
 server s1 check ssl sni req.hdr(Host)

Peers v2

The peers protocol, used to synchronized data from stick-tables between two HAProxy servers can now synchronise more than just the sample and the server-id.
It can synchronize all the tracked counters. Note that each node push its local counter to a peer. So this must be used for safe reload and server failover only.
Don’t expect to see 10 HAProxy servers to sync and aggregate counters in real time.

That said, this protocol has been extended to support different data type, so we may see more features soon relying on it 😉

HTTP connection sharing

On the road to HTTP/2, HAProxy must be able to support connection pooling.
And on the road to the connection pools, we have the ability to share a server side connection between multiple clients.
The server side connection may be used by multiple clients, until the owner (the client side connection) of this session dies. Then new connections may be established.

The new keyword is http-reuse and have different level of sharing connections:

  • never: no connections are shared
  • safe: first request of each client is sent over its own connection, subsequent request may used an other connection. It works like a regular HTTP keepalive
  • aggressive: send request to connections that has proved reliably support connection reuse (no quick connection close after a response has been sent)
  • always: send request to established connections, whatever happens. If the server was closing the connection in the mean time, the request is lost and the client must resend it.

Get rid of 408 in logs

Simply use the new option
option http-ignore-probes


Packetshield: quand votre load-balancer vous protĂšge contre les DDOS!

Les attaques par DDOS

Il y a quelques temps, nous avions publié sur ce blog, un article expliquant comment utiliser un load-balancer pour se protéger contre les attaques de type DDOS applicatif: http://blog.haproxy.com/2012/02/27/use-a-load-balancer-as-a-first-row-of-defense-against-ddos/.

Malheureusement, les attaques sont aujourd’hui sur plusieurs vecteurs, notamment rĂ©seau et applicatif.
Leur but est toujours le mĂȘme: trouver un point faible dans l’architecture et le saturer:

  1. un firewall, nombre de sessions ou paquets par secondes
  2. un load-balancer, nombre sessions Ă©tablies
  3. un serveur applicatif, nombre de requĂȘtes par secondes
  4. etc…

Dans l’article prĂ©cĂ©dent, nous avions vu comment protĂ©ger les serveurs d’applications en utilisant le load-balancer, ce qui correspond Ă  la partie applicative de l’architecture.

Pour protĂ©ger la tĂȘte de l’architecture, le firewall, il faut en gĂ©nĂ©ral investir beaucoup d’argent dans des Ă©quipements coĂ»teux.
Par ailleurs, dans certains cas, lorsque l’ALOHA Ă©tait connectĂ© en direct sur Internet, il ne savait se protĂ©ger que modĂ©rĂ©ment contre les attaques “rĂ©seau”.

C’est pourquoi, HAProxy Technologies a dĂ©veloppĂ© sa propre solution de protection contre les DDOS de niveau rĂ©seau. Cet outil s’appelle Packetshield.

Les attaques de niveau rĂ©seau sont assez simples Ă  mettre en oeuvre pour l’attaquant, mais peuvent ĂȘtre dĂ©vastatrices pour la cible: lorsqu’un Ă©quipement est saturĂ©, il bloque tout le trafic.
Un équipement peut se saturer de différentes maniÚres:

  • un trop grand nombre de paquets par secondes
  • un trop grand nombre se sessions ouvertes (TCP et/ou UDP)
  • un trop gros volume de trafic entrant

(liste non exhaustive)

NOTE: pour ce dernier cas, il faut passer par un service externe de “nettoyage” du flux, qui renverra ensuite un trafic lĂ©gitime vers votre architecture.

Packetshield: Protection contre les DDOS réseau

L’ALOHA, Ă©quipĂ© de packetshield, se place en point d’entrĂ©e de l’infrastructure.
Packetshield va agir comme un Firewall stateful en analysant les flux réseaux le traversant. Seuls le trafic considéré comme légitime sera autorisé à passer.
Différents mécanismes rentrent en jeu lors de la protection:

  • analyse des paquets: les paquets non valides sont automatiquement bloquĂ©s
  • filtrage de flux: l’administrateur peut dĂ©finir ce qui est autorisĂ© ou pas
  • analyse des flux autorisĂ©s: Packetshield fait le trie pour savoir ce qui est lĂ©gitime ou pas

Contrairement aux solutions existantes qui fonctionnent dans un mode stateless et qui gĂ©nĂšrent beaucoup de faux positifs, Packetshield garantie qu’aucune session valide ne sera bloquĂ©e.

Disponibilité de Packetshield

Packetshield est disponible dans l’ALOHA en version appliance physique depuis le firmware 7.0.
La protection de l’interface gigabit (Max 1 million de paquets par secondes) est inclu gratuitement dans l’ALOHA.
Nos clients existants n’ont qu’Ă  mettre Ă  jour le firmware de leur appliance pour en bĂ©nĂ©ficier. Tout nouveau client achetant l’une de nos appliances bĂ©nĂ©ficiera lui aussi de cette protection, sans surcoĂ»t.

Packetshield est aussi disponible en version 10G (14 millions de paquets traités par secondes) sur nos toutes appliances dernier modÚle ALB5100, peut importe la licence choisie, du 8K au 64K.

Mode de déploiement

De part son fonctionnement stateful, Packetshield est utilisé de maniÚre optimal dans les déploiement suivant:

Packetshield peut fonctionner en mode stateless, sans ses fonctionnalités avancé. Dans ce mode, Packetshield est compatible avec les type de déploiement suivants:

Pour plus d’information

La documentation sur la configuration de Packetshield est disponible Ă  cette adresse: http://haproxy.com/doc/aloha/7.0/packet_flood_protection/.

Pour plus d’information, un petit mail Ă  contact (at) haproxy (point) com.


HAProxy’s load-balancing algorithm for static content delivery with Varnish

HAProxy’s load-balancing algorithms

HAProxy supports many load-balancing algorithms which may be used in many different type of cases.
That said, cache servers, which deliver most of the time the static content from your web applications, may require some specific load-balancing algorithms.

HAProxy stands in front of your cache server for some good reasons:

  • SSL offloading (read PHK’s feeling about SSL, Varnish and HAProxy)
  • HTTP content switching capabilities
  • advanced load-balancing algorithms

The main purpose of this article is to show how HAProxy can be used to aggregate Varnish servers memory storage in some kind of “JBOD” mode (like the “Just a Bunch Of Disks“).
Main purpose of the examples delivered here are to optimize the resources on the cache, mainly its memory, in order to improve the HIT rate. This will also improve your application response time and make your site top ranked on google 🙂

Content Switching in HAProxy

This has been covered many times on this blog.
As a quick introduction for readers who are not familiar with HAProxy, let’s explain how it works.

Clients will get connected to HAProxy through a Frontend. Then HAProxy routes traffic to a backend (server farm) where the load-balancing algorithm is used to choose a server.
A frontend can points to multiple backends and the choice of a backend is made through acls and use_backend rules..
Acls can be formed using fetches. A fetch is a directive which instructs HAProxy where to get content from.

Enough theory, let’s make a practical example: splitting static and dynamic traffic using the following rules:

  • Static content is hosted on domain names starting by ‘static.’ and ‘images.’
  • Static content files extensions are ‘.jpg’ ‘.png’ ‘.gif’ ‘.css’ ‘.js’
  • Static content can match any of the rule above
  • anything which is not static is considered as dynamic

The configuration sniplet below should be integrated into the HAProxy frontend. It matches the rules above to do traffic splitting. The varnish servers will stands in the bk_static farm.

frontend ft_public
 <frontend settings>
 acl static_domain  req.hdr_beg(Host) -i static. images.
 acl static_content path_end          -i .jpg .png .gif .css .js
 use_backend bk_static if static_domain or static_content
 default_backend bk_dynamic
backend bk_static
 <parameters related to static content delivery>

The configuration above creates 2 named acls ‘static_domain‘ and ‘static_content‘ which are used by the used_backend rule to route the traffic to varnish servers.

HAProxy and hash based load-balancing algotithm

Later in this article, we’ll heavily used the hash based load-balancing algorithms from HAProxy.
So a few information here (non exhaustive, it would deserve a long blog article) which will be useful for people wanting to understand what happens deep inside HAProxy.

The following parameters are taken into account when computing a hash algorithm:

  • number of servers in the farm
  • weight of each server in the farm
  • status of the servers (UP or DOWN)

If any of the parameter above changes, the whole hash computation also changes, hence request may hit an other server. This may lead to a negative impact on the response time of the application (during a short period of time).
Fortunately, HAProxy allows ‘consistent’ hashing, which means that only the traffic related to the change will be impacted.
That’s why you’ll see a lot of hash-type consistent directives in the configuration samples below.

Load-Balancing varnish cache server

Now, let’s focus on the magic we can add in the bk_static server farm.

Hashing the URL

HAProxy can hash the URL to pick up a server. With this load-balancing algorithm, we guarantee that a single URL will always hit the same Varnish server.

hashing the URL path only

In the example below, HAProxy hashes the URL path, which is from the first slash ‘/’ character up to the question mark ‘?’:

backend bk_static
  balance uri
  hash-type consistent

hashing the whole url, including the query string

In some cases, the query string may contain some variables in the query string, which means we must include the query string in the hash:

backend bk_static
  balance uri whole
  hash-type consistent

Query string parameter hash

That said, in some cases (API, etc…), hashing the whole URL is not enough. We may want to hash only on a particular query string parameter.
This applies well in cases where the client can forge itself the URL and all the parameters may be randomly ordered.
The configuration below tells HAProxy to apply the hash to the query string parameter named ‘id’ (IE: /image.php?width=512&id=12&height=256)

backend bk_static
  balance url_param id
  hash-type consistent

hash on a HTTP header

HAProxy can apply the hash to a specific HTTP header field.
The example below applies it on the Host header. This can be used for people hosting many domain names with a few pages, like users dedicated pages.

backend bk_static
  balance hdr(Host)
  hash-type consistent

Compose your own hash: concatenation of Host header and URL

Nowadays, HAProxy becomes more and more flexible and we can use this flexibility in its configuration.
Imagine, in your varnish configuration, you have a storage hash key based on the concatenation of the host header and the URI, then you may want to apply the same load-balancing algorithm into HAProxy, to optimize your caches.

The configuration below creates a new HTTP header field named X-LB which contains the host header (converted to lowercase) concatenated to the request uri (converted in lowercase too).

backend bk_static
  http-request set-header X-LB %[req.hdr(Host),lower]%[req.uri,lower]
  balance hdr(X-LB)
  hash-type consistent


HAProxy and Varnish works very well together. Each soft can benefit from performance and flexibility of the other one.


HAProxy 1.6-dev1 and LUA

HAProxy 1.6-dev1

Yesterday, Willy has released HAProxy 1.6-dev1: ANNOUNCE HAProxy 1.6-dev1.
This version contains many new features and core improvements. Amongst the new features, one is LUA, contributed by Thierry (HAProxy Technologies developer).

NOTE: We invite everyone to download and test HAProxy 1.6-dev1 and to report bugs to the mailing list.

Please note that there may be a code freeze of HAProxy 1.6 in May. Purpose is to release a stable version in September.


LUA is a powerful, fast, lightweight, embeddable scripting language.

More about lua

The main advantage of LUA for HAProxy is to give the ability to anyone to write their own features without having to know writing C code and patching HAProxy…

About this article

Please note that LUA integration in HAProxy is brand new. So the information provided here are true today but may be obsolete later, because of internal changes and updates.

Compiling HAProxy and LUA

Installaing LUA 5.3

For now, HAProxy requires LUA 5.3 to work, here is the procedure to install it:

cd /usr/src
curl -R -O http://www.lua.org/ftp/lua-5.3.0.tar.gz
tar zxf lua-5.3.0.tar.gz
cd lua-5.3.0
make linux
sudo make INSTALL_TOP=/opt/lua53 install

LUA 5.3 library and include files are now installed in /opt/lua53.

Compiling HAProxy for lua

Since HAProxy is in a dev state, we may encounter some bugs. Let’s enable some debugging features:

make DEBUG=-ggdb CFLAGS=-O0 TARGET=linux2628 
USE_LUA=yes LUA_LIB=/opt/lua53/lib/ 
LUA_INC=/opt/lua53/include/ LDFLAGS=-ldl

Using LUA in HAProxy

turning HAProxy as a mirror web server

By mirror web server, I mean a web server which returns the request in the body of the response, without any modification.
It may be useful to use in HAProxy when you want to setup some HTTP header manipulation in HAProxy and you want to see the result. (It is more convenient than tcpdump).

* First, the HAProxy configuration:

 lua-load ./mylua.lua

 mode http
 timeout connect 1s
 timeout client 1s
 timeout server 1s

frontend fe
 bind name fe
 acl debugme req.hdr_cnt(X-debug-me) ge 1
 # add your http manipulation rules here
 http-request lua mirror if debugme
 default_backend be

backend be
 server s

* And the LUA which does the mirroring of the request:

-- a simple mirror web server
-- it generates a response whose body contains the requests headers
function mirror(txn)
local buffer = ""
local response = ""
local mydate = txn.sc:http_date(txn.f:date())

buffer = buffer .. "You sent the following headersrn"
buffer = buffer .. "===============================================rn"
buffer = buffer .. txn.req:dup()
buffer = buffer .. "===============================================rn"

response = response .. "HTTP/1.0 200 OKrn"
response = response .. "Server: haproxy-lua/mirrorrn"
response = response .. "Content-Type: text/htmlrn"
response = response .. "Date: " .. mydate .. "rn"
response = response .. "Content-Length: " .. buffer:len() .. "rn"
response = response .. "Connection: closern"
response = response .. "rn"
response = response .. buffer


Some explanations:
* mydate is the result of an HAProxy converter txn.sc:http_date applied to an HAProxy fetch txn.f:date()
* The call to txn.req:dup returns the content of HAProxy’s buffer request.

Testing: simply run a curl against HAProxy, don’t forget to send a X-debug-me HTTP header with the parameter -H "X-debug-me: yes". HAProxy should return your request in the response body:

HTTP/1.0 200 OK
Server: haproxy-lua/mirror
Content-Type: text/html
Date: Fri, 12 Mar 2015 13:06:44 GMT
Content-Length: 208
Connection: keep-alive

You sent the following headers
GET / HTTP/1.1
User-Agent: curl/7.41.0
Accept: */*



Web application name to backend mapping in HAProxy


Let’s take a web application platform where many HTTP Host header points to.
Of course, this platform hosts many backends and HAProxy is used to perform content switching based on the Host header to route HTTP traffic to each backend.

HAProxy map

HAProxy 1.5 introduced a cool feature: converters. One converter type is map.
Long story made short: a map allows to map a data in input to an other one on output.

A map is stored in a flat file which is loaded by HAProxy on startup. It is composed by 2 columns, on the left the input string, on the right the output one:

in out

Basically, if you call the map above and give it the in strings, it will return out.


Now, the interesting part of the article 🙂

As stated in introduction, we want to map hundreds of Host headers to tens of backends.

The old way of mapping: acl and use_backend rules

Before the map, we had to use acls and use_backend rules.

like below:

frontend ft_allapps
 use_backend bk_app1 if { hdr(Host) -i app1.domain1.com app1.domain2.com }
 use_backend bk_app2 if { hdr(Host) -i app2.domain1.com app2.domain2.com }
 default_backend bk_default

Add one statement per use_backend rule.

This works nicely for a few backends and a few domain names. But this type of configuration is hardly scallable…

The new way of mapping: one map and one use_backend rule

Now we can use map to achieve the same purpose.

First, let’s create a map file called domain2backend.map, with the following content: on the left, the domain name, on the right, the backend name:

#domainname  backendname
app1.domain1.com bk_app1
app1.domain2.com bk_app1
app2.domain1.com bk_app2
app2.domain2.com bk_app2

And now, HAProxy configuration:

frontend ft_allapps
 use_backend %[req.hdr(host),lower,map_dom(/etc/hapee-1.5/domain2backend.map,bk_default)]

Here is what HAProxy will do:

  1. req.hdr(host) ==> fetch the Host header from the HTTP request
  2. lower ==> convert the string into lowercase
  3. map_dom(/etc/hapee-1.5/domain2backend.map) ==> look for the lowercase Host header in the map and return the backend name if found. If not found, the name of a default backend is returned
  4. route traffic to the backend name returned by the map

Now, adding a new content switching rule means just add one new line in the map content (and reload HAProxy). No regexes, map data is stored in a tree, so processing time is very low compared to matching many string in many ACLs for many use_backend rules.

simple is beautiful!!!

HAProxy map content auto update

If you are an HAPEE user (and soon available for the ALOHA), you can use the lb-update content to download the content of the map automatically.
Add the following statement in your configuration:

 update id domain2backend.map url delay 60s timeout 5s retries 3 map


HAProxy and sslv3 poodle vulnerability

SSLv3 poodle vulnerability

Yesterday, Google security researchers have disclosed a new vulnerability on SSL protocol.
Fortunately, this vulnerability is only on an old version of the SSL protocol: SSLv3 (15 years old protocol).
An attacker can force a browser to downgrade the protocol version used to cipher traffic to SSLv3 in order to exploit the POODLE vulnerability and access to data in clear.

Some reading about SSLv3 Poodle vulnerability:
* http://googleonlinesecurity.blogspot.fr/2014/10/this-poodle-bites-exploiting-ssl-30.html
* https://www.imperialviolet.org/2014/10/14/poodle.html
* https://www.poodletest.com/

Today’s article is going to explain how to use HAProxy to simply prevent using SSLv3 or to prevent those users to reach your applications and print them a message.

Disable SSLv3 in HAProxy

In SSL offloading mode

In this mode, HAProxy is the SSL endpoint of the connection.
It’s a simple keyword on the frontend bind directive:

  bind ssl crt /pat/to/cert.pem no-sslv3

In SSL forward mode

In this mode, HAProxy forwards the SSL traffic to the server without deciphering it.
We must setup an ACL to match the SSL protocol version, then we can refuse the connection. This must be added in a **frontend** section:

  tcp-request inspect-delay 2s
  acl sslv3 req.ssl_ver 3
  tcp-request content reject if sslv3

Communicate a message to users

Denying sslv3 is a good way, but a better one would to educate as well users who are using this protocol.
The configuration below shows how to redirect a user to a specific page when they want to use your application over an SSLv3 connection. Of course, HAProxy must allow itself SSLv3:

frontend ft_www
  bind ssl crt /pat/to/cert.pem
  acl sslv3 ssl_fc_protocol SSLv3
# first rule after all your 'http-request deny' and
# before all the redirect, rewrite, etc....
  http-request allow if sslv3
# first content switching rule
  use_backend bk_sslv3 if sslv3

backend bk_sslv3
  mode http
  errorfile 503 /etc/haproxy/pages/sslv3.http

And the content of the file /etc/haproxy/pages/sslv3.http:

HTTP/1.0 200 OK
Cache-Control: no-cache
Connection: close
Content-Type: text/html

<title>SSLv3 spotted</title>
<body><h1>SSLv3 spotted</h1></body>
SSLv3 forbidden for your safety:<BR>
If you want to browse this website, you should upgrade your browser.