Tag Archives: webperf

HAProxy and gzip compression

Synopsis

Compression is a Technic to reduce object size to reduce delivery delay for objects over HTTP protocol.
Until now, HAProxy did not include such feature. But the guys at HAProxy Technologies worked hard on it (mainly David Du Colombier and @wlallemand).
HAProxy can now be considered an new option to compress HTTP streams, as well as nginx, apache or IIS which already does it.

Note that this is in early beta, so use it with care.

Compilation


Get the latest HAProxy git version, by running a “git pull” in your HAProxy git directory.
If you don’t already have such directory, then run the a:

git clone http://git.1wt.eu/git/haproxy.git

Once your HAProxy sources are updated, then you can compile HAProxy:

make TARGET=linux26 USE_ZLIB=yes

Configuration

this is a very simple configuration test:

listen ft_web
 option http-server-close
 mode http
 bind 127.0.0.1:8090 name http
 default_backend bk_web

backend bk_web
 option http-server-close
 mode http
 compression algo gzip
 compression type text/html text/plain text/css
 server localhost 127.0.0.1:80

Compression test

On my localost, I have an apache with compression disabled and a style.css object whose size is 16302 bytes.

Download without compression requested

curl -o/dev/null -D - "http://127.0.0.1:8090/style.css" 
HTTP/1.1 200 OK
Date: Fri, 26 Oct 2012 08:55:42 GMT
Server: Apache/2.2.16 (Debian)
Last-Modified: Sun, 11 Mar 2012 17:01:39 GMT
ETag: "a35d6-3fae-4bafa944542c0"
Accept-Ranges: bytes
Content-Length: 16302
Content-Type: text/css

100 16302  100 16302    0     0  5722k      0 --:--:-- --:--:-- --:--:-- 7959k

Download with compression requested

 curl -o/dev/null -D - "http://127.0.0.1:8090/style.css" -H "Accept-Encoding: gzip"
HTTP/1.1 200 OK
Date: Fri, 26 Oct 2012 08:56:28 GMT
Server: Apache/2.2.16 (Debian)
Last-Modified: Sun, 11 Mar 2012 17:01:39 GMT
ETag: "a35d6-3fae-4bafa944542c0"
Accept-Ranges: bytes
Content-Type: text/css
Transfer-Encoding: chunked
Content-Encoding: gzip

100  4036    0  4036    0     0  1169k      0 --:--:-- --:--:-- --:--:-- 1970k

In this example, object size passed from 16302 bytes to 4036 bytes.

Have fun !!!!

Links

Application Delivery Controller and ecommerce websites

Synopsis

Today, almost any ecommerce website uses a load-balancer or an application delivery controller in front of it, in order to improve its availability and reliability.
In today’s article, I’ll explain how we can take advantage of ADCs’ layer 7 features to improve an ecommerce website performance and give the best experience to end-user in order to increase the revenue.
The points on which we can work are:

  • Network optimization
  • Traffic regulation
  • Overusage protection
  • User “tagging” based on cart content
  • User “tagging” based purchase phase
  • Blackout prevention
  • SEO optimization
  • Partner slowness protection

Note: the list is not exhaustive and the given example will be very simple. My purpose is not to create a very complicated configuration but give the reader clues on how he can take advantage of our product.


Note2: I won’t discuss about static content, there is already one article with a lot of details about it on this blog.


As Usual, the configuration example below applies on our ALOHA ADC appliance, but should work as well on HAProxy 1.5.

Network optimization

Client-side network latency have a negative impact on websites: the slowest the user connectivity is, the longest the connection will remain opened on the web server, the time for the client to download the object. This could last much longer if the client and server uses HTTP Keepalives.
Basically, this is what happens with basic layer 4 load-balancers like LVS or some other appliance vendors, when the TCP connection is established between the client and the server directly.
Since HAProxy works as a HTTP reverse-proxy, it breaks the TCP connection and enables TCP buffering between both connections. It means HAProxy reads the response at the speed of the server and delivers it at the speed of the client.
Slow clients with high latency will have no impact anymore on application servers because HAProxy “hides” it by its own latency to the server.
An other good point is that you can enable HTTP Keepalives on the client side and disable it on the server side: it allows a client to re-use a connection to download several objects, with no impact on server resources.
TCP buffering does not require any configuration, while enabling client side HTTP keep-alive is achieved by the line option http-server-close.
And The configuration is pretty simple:

# default options
defaults
  option http-server-close
  mode http
  log 10.0.0.1 local2
  option httplog
  timeout connect 5s
  timeout client 20s
  timeout server 15s
  timeout check 1s
  timeout http-keep-alive 1s
  timeout http-request 10s  # slowloris protection
  default-server inter 3s fall 2 rise 2 slowstart 60s

# main frontend
frontend ft_web
  bind 10.0.0.3:80
  default_backend bk_appsrv

# application server farm
backend bk_appsrv
  balance roundrobin
  # app servers must say if everything is fine on their side and 
  # they are ready to process traffic
  option httpchk GET /appcheck
  http-check expect rstring [oO][kK]
  cookie SERVERID insert indirect nocache
  server s1 10.0.1.101:80 cookie s1 check
  server s2 10.0.1.102:80 cookie s2 check

Traffic Regulation


Any server has a maximum capacity. The more it handles requests, the slower it will be to process each request. And if it has too many requests to process, it can even crash and won’t obviously be able to answer to anybody!
HAProxy can regulate request streams to servers in order to prevent them from crashing or even slowing down. Note that when well set up, it can allow you to use your server at their maximum capacity without never being in trouble.
Basically, HAProxy is able to manage request queues.
You can configure traffic regulation with fullconn and maxconn parameters in the backend and with minconn and maxconn parameters on the server line description.
Let’s update our server line description above with a simple maxconn parameter:

  server s1 10.0.1.101:80 cookie s1 check maxconn 250
  server s2 10.0.1.102:80 cookie s2 check maxconn 250

Note: there would be many many things to say about queueing and the HAProxy parameter cited above, but this is not the purpose of the current article.

Over usage protection

By over usage, I mean that you want to be able to handle an unexpected flow of users and be able to classify users in 2 categories:

  1. Those who have already been identified by the website and are using it
  2. Those who have just arrived and wants to use it

The difference between both type of users can be done through the ecommerce CMS cookie: identified users owns a Cookie while brand new users doesn’t.
If you know your server farm has the capacity to manage 10000 users, then you don’t want to allow more than this number until you expand the farm.
Here is the configuration to protect against over-usage (The application Cookie is “MYCOOKIE”.):

# default options
defaults
  option http-server-close
  mode http
  log 10.0.0.2 local2
  option httplog
  timeout connect 5s
  timeout client 20s
  timeout server 15s
  timeout check 1s
  timeout http-keep-alive 1s
  timeout http-request 10s  # slowloris protection
  default-server inter 3s fall 2 rise 2 slowstart 60s

# main frontend
frontend ft_web
  bind 10.0.0.3:80
  # update the number below to the number of people you want to allow
  acl maxcapacity table_cnt(bk_appsrv) ge 10000
  acl knownuser hdr_sub(Cookie) MYCOOK
  # route any unknown user to the sorry page if we reached the maximum number
  # of allowed users and the request does not have a cookie
  use_backend bk_sorrypage if maxcapacity !knownuser
  default_backend bk_appsrv

# appsrv backend for dynamic content
backend bk_appsrv
  balance roundrobin
  # define a stick-table with at most 10K entries
  # cookie value would be cleared from the table if not used for 10 mn
  stick-table type string len 32 size 10K expire 10m nopurge
  stick store-response set-cookie(MYCOOK)
  # Reset the TTL in the stick table each time a request comes in
  stick store-request cookie(MYCOOK)
  # app servers must say if everything is fine on their side and 
  # they are ready to process traffic
  option httpchk GET /appcheck
  http-check expect rstring [oO][kK]
  cookie SERVERID insert indirect nocache
  server s1 10.0.1.101:80 cookie s1 check maxconn 250
  server s2 10.0.1.102:80 cookie s2 check maxconn 250

# sorry page management
backend bk_sorrypage
  balance roundrobin
  server s1 10.0.1.103:80 check maxconn 1000
  server s2 10.0.1.104:80 check maxconn 1000

User tagging based on cart content

When your architecture has enough capacity, you don’t need to classify users. But imagine if your platform runs out of capacity, you want to be able to reserve resources for users who have no article in the cart, that way the website looks very fast, hopefully these users will buy some articles.
Just configure your ecommerce application to setup a cookie with some information about the cart: either the number of article, the whole value, etc…
In the example below, we’ll consider the application creates a cookie named CART and the number of articles as a value.
Based on the information provided by this cookie, we’ll take routing decision and choose different farms with different capacity.

# default options
defaults
  option http-server-close
  mode http
  log 10.0.0.2 local2
  option httplog
  timeout connect 5s
  timeout client 20s
  timeout server 15s
  timeout check 1s
  timeout http-keep-alive 1s
  timeout http-request 10s  # slowloris protection
  default-server inter 3s fall 2 rise 2 slowstart 60s

# main frontend
frontend ft_web
  bind 10.0.0.3:80
  # update the number below to the number of people you want to allow
  acl maxcapacity table_cnt(bk_appsrv) ge 10000
  acl knownuser hdr_sub(Cookie) MYCOOK
  acl empty_cart hdr_sub(Cookie) CART=0
  # route any unknown user to the sorry page if we reached the maximum number
  # of allowed users and the request does not own a cookie
  use_backend bk_sorrypage if maxcapacity !knownuser
  # Once the user have something in the cart, move it to a farm with less resources
  # only when there are too many users on the website
  use_backend bk_appsrv if maxcapacity !empty_cart 
  default_backend bk_appsrv_empty_cart

# Default farm when everything goes well
backend bk_appsrv_empty_cart
  balance roundrobin
  # create the entry in the table when the server generates the cookie
  stick store-response set-cookie(MYCOOK) table bk_appsrv
  # Reset the TTL in the stick table each time a request comes in
  stick store-request cookie(MYCOOK) table bk_appsrv
  # app servers must say if everything is fine on their side
  # and they can process requests
  option httpchk GET /appcheck
  http-check expect rstring [oO][kK]
  cookie SERVERID insert indirect nocache
  server s1 10.0.1.101:80 cookie s1 check maxconn 200
  server s2 10.0.1.102:80 cookie s2 check maxconn 200

# Reserve resources for the few users which have something in their cart
backend bk_appsrv
  balance roundrobin
  # define a stick-table with at most 10K entries
  # cookie would be cleared from the table if not used for 10  mn
  stick-table type string len 32 size 10K expire 10m nopurge
  # create the entry in the table when the server generates the cookie
  stick store-response set-cookie(MYCOOK)
  # Reset the TTL in the stick table each time a request comes in
  stick store-request cookie(MYCOOK)
  cookie SERVERID insert indirect nocache
  server s1 10.0.1.101:80 cookie s1 track bk_appsrv_empty_cart/s1 maxconn 50
  server s2 10.0.1.102:80 cookie s2 track bk_appsrv_empty_cart/s2 maxconn 50

backend bk_sorrypage
  balance roundrobin
  server s1 10.0.1.103:80 check maxconn 1000
  server s2 10.0.1.104:80 check maxconn 1000

User tagging based on purchase phase

The synopsis of this chapter is the same as the precedent chapter: behing able to classify users and ability to reserve resources.
But this time, we’ll identify users based on the phase they are. Basically, we’ll consider two phases:

  1. browsing phase, when people add articles in the cart
  2. purchasing phase, when people have finished filling up the cart and start providing billing, delivery and payment information

In order to classify users, we’ll use the URL path. It starts by /purchase/ when the user is in the purchasing phase. Any other URLs are considered as browsing.
Based on the information provided by requested URL, we’ll take routing decision and choose different farms with different capacity.

# defaults options
defaults
  option http-server-close
  mode http
  log 10.0.0.2 local2
  option httplog
  timeout connect 5s
  timeout client 20s
  timeout server 15s
  timeout check 1s
  timeout http-keep-alive 1s
  timeout http-request 10s  # slowloris protection
  default-server inter 3s fall 2 rise 2 slowstart 60s

# main frontend
frontend ft_web
  bind 10.0.0.3:80
  # update the number below to the number of people you want to allow
  acl maxcapacity table_cnt(bk_appsrv) ge 10000
  acl knownuser hdr_sub(Cookie) MYCOOK
  acl purchase_phase path_beg /purchase/
  # route any unknown user to the sorry page if we reached the maximum number
  # of allowed users and the request does not own a cookie
  use_backend bk_sorrypage if maxcapacity !knownuser
  # Once the user is in the purchase phase, move it to a farm with less resources
  # only when there are too many users on the website
  use_backend bk_appsrv if maxcapacity purchase_phase 
  default_backend bk_appsrv_browse

# Default farm when everything goes well
backend bk_appsrv_browse
  balance roundrobin
  # create the entry in the table when the server generates the cookie
  stick store-response set-cookie(MYCOOK) table bk_appsrv
  # Reset the TTL in the stick table each time a request comes in
  stick store-request cookie(MYCOOK) table bk_appsrv
  # app servers must say if everything is fine on their side
  # and they can process requests
  option httpchk GET /appcheck
  http-check expect rstring [oO][kK]
  cookie SERVERID insert indirect nocache
  server s1 10.0.1.101:80 cookie s1 check maxconn 200
  server s2 10.0.1.102:80 cookie s2 check maxconn 200

# Reserve resources for the few users in the purchase phase
backend bk_appsrv
  balance roundrobin
  # define a stick-table with at most 10K entries
  # cookie would be cleared from the table if not used for 10  mn
  stick-table type string len 32 size 10K expire 10m nopurge
  # create the entry in the table when the server generates the cookie
  stick store-response set-cookie(MYCOOK)
  # Reset the TTL in the stick table each time a request comes in
  stick store-request cookie(MYCOOK)
  cookie SERVERID insert indirect nocache
  server s1 10.0.1.101:80 cookie s1 track bk_appsrv_browse/s1 maxconn 50
  server s2 10.0.1.102:80 cookie s2 track bk_appsrv_browse/s2 maxconn 50

backend bk_sorrypage
  balance roundrobin
  server s1 10.0.1.103:80 check maxconn 1000
  server s2 10.0.1.104:80 check maxconn 1000

Blackout prevention

A website blackout is the worst thing that could happen: something has crashed and the application does not work anymore, or none of the servers are reachable.
When such thing occurs, it is common to get 503 errors or a blank page after 30 seconds.
In both cases, end users have a negative feeling about the website. At least an excuse page with an estimated recovery date would be appreciated. HAProxy allows to communicate to end user even if none of the servers are available.
The configuration below shows how to do it:

# defaults options
defaults
  option http-server-close
  mode http
  log 10.0.0.2 local2
  option httplog
  timeout connect 5s
  timeout client 20s
  timeout server 15s
  timeout check 1s
  timeout http-keep-alive 1s
  timeout http-request 10s  # slowloris protection
  default-server inter 3s fall 2 rise 2 slowstart 60s

# main frontend
frontend ft_web
  bind 10.0.0.3:80
  # update the number below to the number of people you want to allow
  acl maxcapacity table_cnt(bk_appsrv) ge 10000
  acl knownuser hdr_sub(Cookie) MYCOOK
  acl purchase_phase path_beg /purchase/
  acl no_appsrv nbsrv(bk_appsrv_browse) eq 0
  acl no_sorrysrv nbsrv(bk_sorrypage) eq 0
  # worst case management
  use_backend bk_worst_case_management if no_appsrv no_sorrysrv
  # use sorry servers if available
  use_backend bk_sorrypage if no_appsrv !no_sorrysrv
  # route any unknown user to the sorry page if we reached the maximum number
  # of allowed users and the request does not own a cookie
  use_backend bk_sorrypage if maxcapacity !knownuser
  # Once the user is in the purchase phase, move it to a farm with less resources
  # only when there are too many users on the website
  use_backend bk_appsrv if maxcapacity purchase_phase 
  default_backend bk_appsrv_browse

# Default farm when everything goes well
backend bk_appsrv_browse
  balance roundrobin
  # create the entry in the table when the server generates the cookie
  stick store-response set-cookie(MYCOOK) table bk_appsrv
  # Reset the TTL in the stick table each time a request comes in
  stick store-request cookie(MYCOOK) table bk_appsrv
  # app servers must say if everything is fine on their side
  # and they can process requests
  option httpchk GET /appcheck
  http-check expect rstring [oO][kK]
  cookie SERVERID insert indirect nocache
  server s1 10.0.1.101:80 cookie s1 check maxconn 200
  server s2 10.0.1.102:80 cookie s2 check maxconn 200

# Reserve resources for the few users in the purchase phase
backend bk_appsrv
  balance roundrobin
  # define a stick-table with at most 10K entries
  # cookie would be cleared from the table if not used for 10  mn
  stick-table type string len 32 size 10K expire 10m nopurge
  # create the entry in the table when the server generates the cookie
  stick store-response set-cookie(MYCOOK)
  # Reset the TTL in the stick table each time a request comes in
  stick store-request cookie(MYCOOK)
  cookie SERVERID insert indirect nocache
  server s1 10.0.1.101:80 cookie s1 track bk_appsrv_browse/s1 maxconn 50
  server s2 10.0.1.102:80 cookie s2 track bk_appsrv_browse/s2 maxconn 50

backend bk_sorrypage
  balance roundrobin
  server s1 10.0.1.103:80 check maxconn 1000
  server s2 10.0.1.104:80 check maxconn 1000

backend bk_worst_case_management
  errorfile 503 /etc/haproxy/errors/503.txt

And the content of the file /etc/haproxy/errors/503.txt could look like:

HTTP/1.0 200 OK
Cache-Control: no-cache
Connection: close
Content-Type: text/html
Content-Length: 246

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Maintenance</title>
</head>
<body>
<h1>Maintenance</h1>
We're sorry, ecommerce.com is currently under maintenance and will come back soon.
</body>
</html>

SEO optimization

Most search engines takes now into account pages response time.
The configuration below redirects search engine bots to a dedicated server and if it’s not available, then it is forwarded to the default farm. The bot is identified by its User-Agent header.

# defaults options
defaults
  option http-server-close
  mode http
  log 10.0.0.2 local2
  option httplog
  timeout connect 5s
  timeout client 20s
  timeout server 15s
  timeout check 1s
  timeout http-keep-alive 1s
  timeout http-request 10s  # slowloris protection
  default-server inter 3s fall 2 rise 2 slowstart 60s

# main frontend
frontend ft_web
  bind 10.0.0.3:80
  # update the number below to the number of people you want to allow
  acl maxcapacity table_cnt(bk_appsrv) ge 10000
  acl knownuser hdr_sub(Cookie) MYCOOK
  acl purchase_phase path_beg /purchase/
  acl bot hdr_sub(User-Agent) -i googlebot bingbot slurp
  acl no_appsrv nbsrv(bk_appsrv_browse) eq 0
  acl no_sorrysrv nbsrv(bk_sorrypage) eq 0
  acl no_seosrv nbsrv(bk_seo) eq 0
  # worst caperformancese management
  use_backend bk_worst_case_management if no_appsrv no_sorrysrv
  # use sorry servers if available
  use_backend bk_sorrypage if no_appsrv !no_sorrysrv
  # redirect bots
  use_backend bk_seo if bot !no_seosrv
  use_backend bk_appsrv if bot no_seosrv
  # route any unknown user to the sorry page if we reached the maximum number
  # of allowed users and the request does not own a cookie
  use_backend bk_sorrypage if maxcapacity !knownuser
  # Once the user is in the purchase phase, move it to a farm with less resources
  # only when there are too many users on the website
  use_backend bk_appsrv if maxcapacity purchase_phase 
  default_backend bk_appsrv_browse

# Default farm when everything goes well
backend bk_appsrv_browse
  balance roundrobin
  # create the entry in the table when the server generates the cookie
  stick store-response set-cookie(MYCOOK) table bk_appsrv
  # Reset the TTL in the stick table each time a request comes in
  stick store-request cookie(MYCOOK) table bk_appsrv
  # app servers must say if everything is fine on their side
  # and they can process requests
  option httpchk GET /appcheck
  http-check expect rstring [oO][kK]
  cookie SERVERID insert indirect nocache
  server s1 10.0.1.101:80 cookie s1 check maxconn 200
  server s2 10.0.1.102:80 cookie s2 check maxconn 200

# Reserve resources for the few users in the purchase phase
backend bk_appsrv
  balance roundrobin
  # define a stick-table with at most 10K entries
  # cookie would be cleared from the table if not used for 10  mn
  stick-table type string len 32 size 10K expire 10m nopurge
  # create the entry in the table when the server generates the cookie
  stick store-response set-cookie(MYCOOK)
  # Reset the TTL in the stick table each time a request comes in
  stick store-request cookie(MYCOOK)
  cookie SERVERID insert indirect nocache
  server s1 10.0.1.101:80 cookie s1 track bk_appsrv_browse/s1 maxconn 50
  server s2 10.0.1.102:80 cookie s2 track bk_appsrv_browse/s2 maxconn 50

# Reserve resources search engines bot
backend bk_seo
  option httpchk GET /appcheck
  http-check expect rstring [oO][kK]
  server s3 10.0.1.103:80 check

backend bk_sorrypage
  balance roundrobin
  server s1 10.0.1.103:80 check maxconn 1000
  server s2 10.0.1.104:80 check maxconn 1000

backend bk_worst_case_management
  errorfile 503 /etc/haproxy/errors/503.txt

Partner slowness protection

Some ecommerce website relies on partners for some product or services. Unfortunately, if the partner’s webservice application slows down, then our own application will slow down. Even worst, we may see sessions pilling up and server crashes due to lack of resources…
In order to prevent this, just configure your appserver to pass through HAProxy to reach your partners’ webservices. HAProxy can shut a session if a partner is too slow to answer. If the partner complain you don’t send them enough deals, just tell him to improve his platform, maybe using a ADC like HAProxy / ALOHA Load-Balancer ūüėČ

frontend ft_partner1
  bind 10.0.0.3:8001
  use_backend bk_partner1

backend bk_partner1
  # the partner has 2 seconds to answer each requests
  timeout server 2s
  # you can add a maxconn here if you're not supposed to open 
  # too many connections on the partner application
  server partner1 1.2.3.4:80 check

Related links

Links

HAProxy and Varnish comparison

In opensource world, there are some very smart products which are very often used to build a high performance, reliable and scalable architecture.
HAProxy and Varnish are both in this category.

Since we can’t really compare a reverse-proxy cache and a reverse-proxy load-balancer, I’m just going to focus in common for both software as well as the advantage of each of them.
The list is not exhaustive, but must only focus on most used / interesting features. So feel free to add a comment if you want me to complete the list.

Common points between HAProxy and Varnish


Before comparing the differences, we can summarize the points in common:

  • reverse-proxy mode
  • advanced HTTP features
  • no SSL offloading
  • client-side HTTP 1.1 with keepalive
  • tunnel mode available
  • high performance
  • basic load-balancing
  • server health checking
  • IPv6 ready
  • Management socket (CLI)
  • Professional services and training available

Features available in HAProxy and not in Varnish


The features below are available in HAProxy, but aren’t in Varnish:

  • advanced load-balancer
  • multiple persistence methods
  • DOS and DDOS mitigation
  • Advanced and custom logging
  • Web interface
  • Server / application protection through queue management, slow start, etc…
  • SNI content switching
  • Named ACLs
  • Full HTTP 1.1 support on server side, but keep-alive
  • Can work at TCP level with any L7 protocol
  • Proxy protocol for both client and server
  • powerful log analyzer tool (halog)
  • <private joke> 2002 website design </private joke>

Features available in Varnish and not in HAProxy


The features below are available in Varnish, but aren’t in HAProxy:

  • caching
  • grace mode (stale content delivery)
  • saint mode (manages origin server errors)
  • modular software (with a lot of modules available)
  • intuitive VCL configuration language
  • HTTP 1.1 on server side
  • TCP connection re-use
  • Edge side includes (ESI)
  • a few command line tools for stats (varnishstat, varnishhist, etc…)
  • powerful live traffic analyzer (varnishlog)
  • <private joke> 2012 website design </private joke>

Conclusion


Even if HAProxy can do TCP proxying, it is often used in front of web application, exactly where we find Varnish :).
They complete very well together: Varnish will make the website faster by offloading static object delivery to itself, while HAProxy can ensure a smooth load-balancing with smart persistence and DDOS mitigation.

Basically, HAProxy and Varnish completes very well, despite being “competitors” on a few features, each on them has its own domain of expertise where it performs very well: HAProxy is a reverse-proxy Load-Balancer and Varnish is a Reverse-proxy cache.

To be honest, when, at HAProxy Technologies, we work on infrastructures where Aloha Load balancer or HAProxy is deployed, we often see Varnish deployed. And if it is not the case, we often recommend the customer to deploy one if we feel it would improve its website performance.
Recently, I had a discussion with Ruben and Kristian when they came to Paris and they told me that they also often see an HAProxy when they work on infrastructure where Varnish is deployed.

So the real question is: Since Varnish and HAProxy are a bit similar but complete so well, how can we use them together???
The response could be very long, so stay tuned, I’ll try to answer this question in an article coming soon.

Related Links

Links

Layer 4 load balancing tunnel mode

The tunnel mode looks like the Direct Server Return mode, except that traffic between the load-balancer and the server can be routed.

The load-balancer encapsulates the request in an IP tunnel to the server.
The server recover the client request from the loadbalancer, process it and forward the response directly to the client.

TCP connection overview

layer4_tunnel_tcp_connection
The loadbalancer takes client requests then encapsulate them into an IP tunnel to forward them to the server.

Data flow

layer4_tunnel_data_flow
The client traffic between the server and the load-balancer is tunneled and can be routed between both of them.
The server will answer directly to the client.

Pros and cons

Pros

  • backends from multiple datacenters can be used
  • load-balancer network bandwith is not a bottleneck anymore
  • total output bandwith is the sum of each backend bandwith

Cons

  • requires patched backend to be able to tunnel IP traffic
  • no layer 7 advanced features are available

When use this architecture?

  • when the only way to reach backends is routing.
  • where no intelligence is required
  • when output capacity of the load-balancer could be the bottleneck

Links

layer 4 load balancing Direct Server Return mode

Direct server return is usually shortened to DSR.

In DSR mode, the load-balancer routes packets to the backends without changing anything in it but the destination MAC address.
The backends process the requests and answer directly to the clients, without passing through the load-balancer.

The backends must have the service IP configured on a loopback to be able to accept the requests.

TCP connection overview

layer4_dsr_tcp_connection
As usual when performing layer 4 load-balancing, the TCP connection is established directly between the client and the backend.
Note that the the requests pass through the load-balancer while the responses not.

Data flow

layer4_dsr_data_flow
As explained above, the load-balancer sees only the requests and just change the destination MAC address of the packets.
The backends answers directly to the client using the service IP configured on a loopback interface.

Pros and cons

Pros

  • very fast load-balancing mode
  • load-balancer network bandwith is not a bottleneck anymore
  • total output bandwith is the sum of each backend bandwith
  • less intrusive than the layer 4 load-balancing NAT mode

Cons

  • the service VIP must be configured on a loopback interface on each backend and must not answer to ARP requests
  • no layer 7 advanced features are available

When use this architecture?

  • where response time matters
  • where no intelligence is required
  • when output capacity of the load-balancer could be the bottleneck

Links

Layer 4 load balancing NAT mode

NAT stands for Network Address Translation.

In the NAT mode, the load-balancer will route traffic between user and server by changing destination IP address of the packets.

TCP connection overview

TCP connection is established between the client and the server.
The loadbalancer just ensures a client is always forwarded to the same server.
layer4_nat_tcp_connection

Data flow

As shown below, the clients get connected to the service VIP.
The load balancer chooses a server in the pool then forwards packets to it by changing destination IP address.
layer4_nat_data_flow

Pros and cons

Pros

  • fast load balancing
  • easy to deploy

Cons

  • infrastructure intrusive: need to change the default gateway of the servers
  • The server default gateway must use the load balancer, in order to do reverse NAT operation.
  • output bandwith is limitated by loadbalancer output capacity

When use this architecture?

  • where response time matters
  • where no intelligence is required
  • when output capacity of the load-balancer won’t be a bottleneck in a near future
  • when nothing but the default gateway of the servers can be changed

Links

How to play with maxconn to avoid server slowness or crash

Using Aloha load balancer and HAProxy, it is easy to protect any application or web server against unexpected high load.

Introduction

The response time of web servers is directly related to the number of requests they have to manage at the same time. And the response time is not linearly linked to the number of requests, it looks like exponential.
The graph below shows a server response time compared to the number of simultaneous users browsing the website:

Simultaneous connections limiting

Simultaneous connections limiting is basically a number (aka the limit) a load balancer will consider as the maximum number of requests to send to a backend server at the same time.
Of course, since HAProxy has such a function, Aloha load-balancer does.

Smart handling of requests peak with HAProxy

The meaning is too prevent too many requests to be forwarded to an application server, by adding a limit for simultaneous requests for each server of the backend.

Fortunately, HAProxy would not reject any request over the limit, unlike some other load balancer does.

HAProxy use a queueing system and will wait for the backend server to be able to answer. This mechanism will add slow delays to request in the queue, but it has a few advantages :

  • no client request are rejected
  • every request can be faster¬†served than with an overloaded backend server
  • the delay is still acceptable (a few ms in queue)
  • your server won’t crash because of the spike

simultaneous requests limiting occurs on the server side: HAProxy will limit the number of concurrent request to the server despite what happens on the client side.
HAProxy will never refuse any client connection until the underlying server runs out of capacity.

Concrete numbers

If you read carefully the graph above, you can easily see that the more your server has to process requests at the same time, the longer each request will take to process.
The table below summarize the time spent by our example server to process 250 requests with different simultaneous requests limiting value:

Number of requests Simultaneous requests limit Average time per request Longuest response time in ms
250 10 9 225
250 20 9 112
250 30 9 75
250 50 25 125
250 100 100 250
250 150 225 305
250 250 625 625

It’s up to the website owner to know what will be the best limit to setup on HAProxy.
You can approximate it by using HTTP benchmark tools and by comparing average response time to constant number of request you send to your backend server.

From the example above, we can see we would get the best of this backend server by setting up the limit to 30.
Setting up a limit too low would implies queueing request for a longer time and setting it too high would be counter-productive by slowing down each request because of server capacity.

HAProxy simultaneous requests limiting configuration

The simultaneous requests limiting configuration is made with the maxconn keyword on the server line definition.
Example:

frontend APPLI1
	bind :80
	mode http
	option http-server-close
	default_backend APPLI1

backend APPLI1
	balance roundrobin
	mode http
	server server1 srv1:80 maxconn 30
	server server2 srv2:80 maxconn 30

Links