Open in app

Sign in

Write

Sign in

Mastodon

DevOps: “Configuration of HAProxy​ Reverse Proxy Server for Load Balancing running on EC2 Instance on the top of AWS using Ansible”

Shobhit Sharma
8 min readDec 4, 2020

--

HAProxy is free, open-source software that provides a high availability load balancer and proxy server for TCP and HTTP-based applications that spread requests across multiple servers. It is written in C and has a reputation for being fast and efficient. Wikipedia

HAProxy, a Reverse Proxy Server is one of the powerful load balancers which can migrate the traffic using IP forwarding to the different and available backend servers.

Ansible, A powerful IT automation tool as well as the revolutionary invention of the Industry. It can solve almost every task and can reduce time complexity by running a single task on thousands of machines simultaneously. According to Wikipedia “Ansible is an open-source software provisioning, configuration management, and application-deployment tool enabling infrastructure as code. It runs on many Unix-like systems and can configure both Unix-like systems as well as Microsoft Windows.

The challenge is to configure HAProxy for load balancing with multiple backend servers using Ansible. Using the following scenario, we can achieve the goal or can solve this challenge very easily.

Setting Up Nodes

First of all, we need to set up the control node and managed nodes, the following requirement we need to get for this operation

EC2 Instances

Well, I am performing this task on AWS, the following Instances I am gonna use with their respective Public IP Addresses as

Control Node

Control Node : 3.137.168.187

Managed Nodes

Reverse Proxy Server: 18.191.192.66

Backend Server A: 3.139.69.93

Backend Server B: 3.21.159.243

The above requirement can be seen in the AWS EC2 Dashboard as

Configuration of Control Node

In this section, we need to follow the main steps for this operation as follows

Creating Host Groups in Inventory

In this section, we need to create a couple of groups for the load balancer and for backend servers. This will customize the operations according to managed nodes.

To create host groups, we need to add the following syntax according to our requirement in the inventory file as follows

[load_balancer]
18.191.192.66 ansible_user=ec2-user ansible_ssh_private_key_file=/root/SSHKeys/shobhitKeyNov2020.pem ansible_connection=ssh[http_server]
3.139.69.93 ansible_user=ec2-user ansible_ssh_private_key_file=/root/SSHKeys/shobhitKeyNov2020.pem ansible_connection=ssh
3.21.159.243 ansible_user=ec2-user ansible_ssh_private_key_file=/root/SSHKeys/shobhitKeyNov2020.pem ansible_connection=ssh

& in this file, I’ve created two groups — load_balancer & http_server

Writing Playbook to solve this challenge

Now, this is the main part of this entire operation, writing a playbook to solve this challenge. The following content should be written in the playbook for solving this challenge as follows

- hosts: http_server
  tasks:
  - name: Getting Public IP
    ipify_facts:
    register: public_ip
    
  - name: Installing HTTPD
    package:
      name: "httpd"
      state: present
    become: true  - name: Installing PHP
    package:
      name: "php"
      state: present
    become: true  - name: Copying Webpage
    copy:
      content: <title>0812 Server </title> <h1>Reverse Proxy - Load Balanced Site</h1> <br> <h1>0812 Server</h1> <h2 style="color:blue"><pre> <?php print `/usr/sbin/ifconfig`; ?></pre> </h2>
      dest: "/var/www/html/index.php"
    become: true  - name: Starting HTTPD
    service:
      name: "httpd"
      state: restarted
    become: true- hosts: load_balancer
  tasks:
  - name: Installing haproxy
    package:
      name: "haproxy"
      state: present
    become: true  - name: Changing server
    template:
      src: "/root/haproxy.cfg"
      dest: "/etc/haproxy/haproxy.cfg"
    become: true  - name: starting service
    service:
      name: haproxy
      state: restarted
    become: true

In this module, I’ve used an extra module named ipify_facts, this will tell the Public IP Address of the managed node. (This is just for information or optional thing).

In this playbook, there is a file that is being used in the template module with the address “/root/” called “haproxy.cfg”. This particular file is working as a template that is used to configure the haproxy dynamically according to the requirement. This file contains the following content

#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   https://www.haproxy.org/download/1.8/doc/configuration.txt
#
#---------------------------------------------------------------------#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats    # utilize system-wide crypto-policies
    ssl-default-bind-ciphers PROFILE=SYSTEM
    ssl-default-server-ciphers PROFILE=SYSTEM#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend main
    bind *:5000
    acl url_static       path_beg       -i /static /images /javascript /stylesheets
    acl url_static       path_end       -i .jpg .gif .png .css .js    use_backend static          if url_static
    default_backend             app#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
backend static
    balance     roundrobin
    server      static 127.0.0.1:4331 check#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend app
    balance     roundrobin
{% for server in groups['http_server'] %}
    server  app{{ 99999 | random(step=10) }} {{ server }}:80 check
{% endfor %}

At the bottom, I’ve used the for loop to add every host from the http_server host group in this file. and I’ve used a random function to randomly generate app id. This is following syntax as follows

{% for server in groups['http_server'] %}
    server  app{{ 99999 | random(step=10) }} {{ server }}:80 check{% endfor %}

Running Playbook

Before Running the playbook, we should check whether the nodes are connected or not with the Control node. For this, we need to run the following command to check

ansible all -m ping

This ad-hoc command will check the host whether it is connected or not and give the output “pong” if the nodes are connected successfully or pingable.

The following output can be seen

3.21.159.243 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "changed": false,
    "ping": "pong"
}
18.191.192.66 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "changed": false,
    "ping": "pong"
}
3.139.69.93 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "changed": false,
    "ping": "pong"
}

Great! It’s working fine…

Now, it’s time to run the playbook by using the following command

ansible-playbook Playbook/reverse.yml

in this command, the “Playbook/reverse.yml” is the directory for the reverse.yml file and the folder is Playbook and the command is ansible-playbook

The output will be

PLAY [http_server] *****************************************      ********************TASK [Gathering Facts] *************************************      ********************
ok: [3.139.69.93]
ok: [3.21.159.243]TASK [Getting Public IP] ***********************************      ********************
ok: [3.21.159.243]
ok: [3.139.69.93]TASK [Installing HTTPD] ************************************      ********************
changed: [3.21.159.243]
changed: [3.139.69.93]TASK [Installing PHP] **************************************      ********************
changed: [3.21.159.243]
changed: [3.139.69.93]TASK [Copying Webpage] *************************************      ********************
changed: [3.21.159.243]
changed: [3.139.69.93]TASK [Starting HTTPD] **************************************      ********************
changed: [3.139.69.93]
changed: [3.21.159.243]PLAY [load_balancer] ***************************************      ********************TASK [Gathering Facts] *************************************      ********************
ok: [18.191.192.66]TASK [Installing haproxy] **********************************      ********************
changed: [18.191.192.66]TASK [Changing server] *************************************      ********************
changed: [18.191.192.66]TASK [starting service] ************************************      ********************
changed: [18.191.192.66]PLAY RECAP *************************************************      ********************
18.191.192.66              : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
3.139.69.93                : ok=6    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
3.21.159.243               : ok=6    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

It’s working fine… Now, we can test it

Again, the configuration of our nodes as follows

Control Node : 3.137.168.187

Reverse Proxy Server: 18.191.192.66

Backend Server A: 3.139.69.93

Backend Server B: 3.21.159.243

& the Proxy Server default port number is: 5000

Testing the operation

In this section, and in the above steps, we’ve successfully executed the ansible-playbook, now it’s time to verify whether it is working or not

Checking the Reverse Proxy Server Configuration File

To check, we need to open the haproxy.cfg file with the following command in the Reverse Proxy Server

vi /etc/haproxy/haproxy.cfg

and the file content would be like this

#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   https://www.haproxy.org/download/1.8/doc/configuration.txt
#
#---------------------------------------------------------------------#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats    # utilize system-wide crypto-policies
    ssl-default-bind-ciphers PROFILE=SYSTEM
    ssl-default-server-ciphers PROFILE=SYSTEM#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend main
    bind *:5000
    acl url_static       path_beg       -i /static /images /javascript /stylesheets
    acl url_static       path_end       -i .jpg .gif .png .css .js    use_backend static          if url_static
    default_backend             app#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
backend static
    balance     roundrobin
    server      static 127.0.0.1:4331 check#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend app
    balance     roundrobin
    server  app80490 3.139.69.93:80 check
    server  app80490 3.21.159.243:80 check

In this file, we can see at the bottom, the two Backend servers are configured successfully.

To test this operation on the web, we need to open the Reverse Proxy Server using the following IP address and port

https://18.191.192.66:5000

Output

The output for the complete operation can be seen in the following video and in this video, you can also see that how can we also add a new backend server without touching the playbook file and the managed nodes manually, only we need to edit the inventory file and add the new backend server public IP and login info.

Adding more backend server and attaching them in HAProxy Reverse Proxy Server for Load Balancing

This can be found in this video

Great, now the conclusion is we can solve this challenge very easily using template and loop in Ansible Playbook. We can solve other future challenges or different problem statements using the template feature where we only want to update or edit the content of any configuration file.

***

Thanks for reading…

This article is written, edited, and published by Shobhit Sharma

Connect With Me On Twitter | Instagram | LinkedIn | Facebook | WhatsApp

Email Me @ shobhit0812@gmail.com

Shobhit Sharma
Scriptkkiddie
Ansible
Haproxy
Amazon Web Services

--

--

Shobhit Sharma
Shobhit Sharma

Written by Shobhit Sharma

47 Followers
105 Following

Documenting my life's experiences and learning.

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams