Difference between revisions of "Configure Apache load balancer with mod proxy"

From LogicalDOC Community Wiki
Jump to navigationJump to search
(Define Apache Load-balancer)
 
(45 intermediate revisions by the same user not shown)
Line 3: Line 3:
  
 
The layout may look something like this (we will refer to these names through the rest of the guide).
 
The layout may look something like this (we will refer to these names through the rest of the guide).
 +
[[File:Apache-load-balancer-mod-proxy-http.png|656px|center|Apache load balancer mod_proxy_http Tomcat]]
  
 +
=== Enable Apache modules in Ubuntu ===
  
 +
Use the following commands:
 +
 +
<syntaxhighlight lang="bash">
 +
$ sudo a2enmod proxy
 +
$ sudo a2enmod proxy_http
 +
$ sudo a2enmod proxy_balancer
 +
$ sudo a2enmod lbmethod_byrequests
 +
$ sudo a2enmod headers
 +
</syntaxhighlight>
 +
 +
Run all the above commands then restart Apache to obtain the effect of changes we have made.
 +
<syntaxhighlight lang="bash">
 +
$ sudo service apache2 restart
 +
</syntaxhighlight>
  
 
== Define Apache Load-balancer ==
 
== Define Apache Load-balancer ==
Line 10: Line 26:
 
This server will handle all HTTP requests from site visitors. As you might see, this means even though you run a load balanced system, using only a single load balancer means you still have a SPOF (single point of failure). It is also possible to configure an environment where yet another server will act as the fail-over load-balancer if the first one fails, but this is outside the scope of this guide.
 
This server will handle all HTTP requests from site visitors. As you might see, this means even though you run a load balanced system, using only a single load balancer means you still have a SPOF (single point of failure). It is also possible to configure an environment where yet another server will act as the fail-over load-balancer if the first one fails, but this is outside the scope of this guide.
  
To set up our load-balancer, we use the Apache web-server and its modules [https://httpd.apache.org/docs/2.4/mod/mod_proxy.html mod_proxy], [https://httpd.apache.org/docs/2.4/mod/mod_proxy_ajp.html mod_proxy_ajp] and [https://httpd.apache.org/docs/2.4/mod/mod_proxy_balancer.html mod_proxy_balancer]. These are part of most of the Apache web-server distributions.
+
To set up our load-balancer, we use the Apache web-server and its modules [https://httpd.apache.org/docs/2.4/mod/mod_proxy.html mod_proxy], [https://httpd.apache.org/docs/2.4/mod/mod_proxy_http.html mod_proxy_http] and [https://httpd.apache.org/docs/2.4/mod/mod_proxy_balancer.html mod_proxy_balancer]. These are part of most of the Apache web-server distributions.
  
First, create a virtual host handling the requests for your domain: www.yourcompany.com
+
First, create a virtual host handling the requests for your domain: ldproxy.org
  
<source lang="text">
+
<pre>
 
<VirtualHost *:80>
 
<VirtualHost *:80>
  
# value is not decisive as it is used as a last resort host regardless.
 
# However, you must set it for any further virtual host explicitly.
 
 
   ServerName ldproxy.org
 
   ServerName ldproxy.org
  
ServerAdmin webmaster@localhost
+
  ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
+
  DocumentRoot /var/www/html
  
 
   ProxyRequests Off
 
   ProxyRequests Off
Line 30: Line 44:
 
   <Proxy balancer://mycluster>
 
   <Proxy balancer://mycluster>
 
         BalancerMember "http://eva00:9080" route=1
 
         BalancerMember "http://eva00:9080" route=1
         BalancerMember "http://192.168.2.11:8080" route=2
+
         BalancerMember "http://192.168.2.15:8082" route=2
 
 
      # AllowOverride None
 
      # Order allow,deny
 
      # allow from all
 
  
 
         ProxySet lbmethod=byrequests
 
         ProxySet lbmethod=byrequests
Line 43: Line 53:
 
   ProxyPassReverse / balancer://mycluster/
 
   ProxyPassReverse / balancer://mycluster/
  
ErrorLog ${APACHE_LOG_DIR}/proxy-error.log
+
  ErrorLog ${APACHE_LOG_DIR}/proxy-error.log
CustomLog ${APACHE_LOG_DIR}/proxy-access.log combined
+
  CustomLog ${APACHE_LOG_DIR}/proxy-access.log combined
  
 
</VirtualHost>
 
</VirtualHost>
 +
</pre>
  
== Configure Tomcat Public1 / Public2 ==
+
The load balancer nodes are two LogicalDOC servers that are accessed by IP and by hostname (eva00), of course the Apache server must be able to resolve the hostname.
  
Let's look at the relevant configuration here to set up the load-balancer. Most likely you will also have an Apache web-server installed on this machines, as for accessing the author instance if located on one of this servers with a nice URL. Here we suggest to use a single Tomcat application server for hosting one public instance. Make sure the AJP Port is set correctly to what you have defined in the virtual host configuration of the load-balancer (8009 as the default value used here).
+
Furthermore we can note that the Apache server adds a Cookie ROUTEID to make sessions sticky and redirect all subsequent requests to the same LogicalDOC node of the cluster
  
If you want to change the AJP Port of your application server, this can be done here.
+
The setting ProxySet [https://httpd.apache.org/docs/2.4/mod/mod_lbmethod_byrequests.html lbmethod=byrequests] distribute the requests among the various workers to ensure that each gets their configured share of the number of requests.
 +
 
 +
=== Load balancer scheduler algorithm ===
 +
At present, there are 4 load balancer scheduler algorithms available for use: Request Counting (mod_lbmethod_byrequests), Weighted Traffic Counting (mod_lbmethod_bytraffic), Pending Request Counting (mod_lbmethod_bybusyness) and Heartbeat Traffic Counting (mod_lbmethod_heartbeat). These are controlled via the lbmethod value of the Balancer definition. See the [https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass ProxyPass directive] for more information, especially regarding how to configure the Balancer and BalancerMembers.
 +
 
 +
== Configure Tomcat ==
 +
 
 +
You need to change the configuration of the Tomcat in LogicalDOC. Generally it is a matter of modifying the file server.xml located in the /tomcat/conf folder
  
 
Tomcat config: <span style="background-color: #F2CEF2;">LOGICALDOC_HOME/tomcat/conf/server.xml</span>
 
Tomcat config: <span style="background-color: #F2CEF2;">LOGICALDOC_HOME/tomcat/conf/server.xml</span>
  
<source lang="text">
+
In this file we are going to add the proxyName and proxyPort attributes to the Connector element
<Connector port="8009" protocol="AJP/1.3" (...)>
+
 
</source>
+
<syntaxhighlight lang="xml">
 +
    <Connector port="8082" protocol="HTTP/1.1"
 +
              connectionTimeout="20000"
 +
              redirectPort="8445"
 +
              URIEncoding="UTF-8" server="Undisclosed/8.41" proxyName="ldproxy.org" proxyPort="80" />
 +
</syntaxhighlight>
 +
 
 +
The proxyName will cause servlets inside this web application to think that all proxied requests were directed to ldproxy.org on port 80
 +
 
 +
== Done ==
 +
 
 +
That's basically it. Now you can set your DNS entry of ldproxy.org to your Load-Balancer's IP address and enjoy the comfort and security of a redundant LogicalDOC installation. If one of the public LogicalDOC servers is failing, mod_proxy on your load-balancer will automatically detect this and stop serving requests to that server.
  
Now in the same file as we configure the AJP Port server.xml we need to configure the jvmRoute for sticky sessions working correctly. Use the name defined in the virtual host configuration on load-balancer, the route value here separately for the two servers.
+
You can test this by stopping Tomcat on one of the machines. The configured ErrorLog file proxy-error.log will show something like
  
<source lang="text">
+
<pre>
<Engine name="Catalina" defaultHost="localhost" jvmRoute="publicXY">
+
[Tue Jun 25 09:55:34.431671 2019] [proxy:error] [pid 12308] (113)No route to host: AH00957: HTTP: attempt to connect to 192.168.2.15:8082 (192.168.2.15) failed
</source>
+
[Tue Jun 25 09:55:34.431768 2019] [proxy:error] [pid 12308] AH00959: ap_proxy_connect_backend disabling worker for (192.168.2.15) for 60s
 +
</pre>
  
Note: on the LogicalDOC node Tomcat's config you should change publicXY with public1 or public2 depending on the node
+
== Advanced Configuration ==
 +
This below is a more advanced example. It requires a specific Apache module: [https://httpd.apache.org/docs/2.4/mod/mod_proxy_balancer.html mod_proxy_balancer] which allows you to modify some parameters of the nodes that make up the cluster in real time. For more information [https://httpd.apache.org/docs/trunk/howto/reverse_proxy.html#manager Balancer Manager]
  
== Done ==
+
Also in this example the [https://httpd.apache.org/docs/trunk/howto/reverse_proxy.html#manager status] directive was applied to the second node of the cluster. This causes all requests to be sent to the first server and keeps the second LogicalDOC server in hot-standby mode (it will only be used if no other viable workers or spares are available in the balancer set).
 +
 
 +
<pre>
 +
<VirtualHost *:80>
 +
 
 +
    ServerName ldproxy.org
 +
 
 +
    ServerAdmin webmaster@localhost
 +
    DocumentRoot /var/www/html
 +
 
 +
    ProxyRequests Off
 +
    ProxyPreserveHost On
 +
 
 +
    <Location /balancer-manager>
 +
      SetHandler balancer-manager
 +
      #Require ip 10.0.0.1 10.0.0.2
 +
    </Location>
 +
 
 +
    ProxyPass /balancer-manager !
 +
 
 +
    Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
 +
    <Proxy balancer://mycluster>
 +
        BalancerMember "http://192.168.2.15:8082" route=1
 +
        BalancerMember "http://192.168.2.5:8080" route=2
 +
        # The server below is on hot standby. It will be used only if no other members (or spares) are available in the load balancer set
 +
        BalancerMember "http://192.168.2.11:8080" route=3 status=+H
 +
 
 +
        ProxySet lbmethod=byrequests
 +
        ProxySet stickysession=ROUTEID
 +
    </Proxy>
 +
 
 +
    ProxyPass / balancer://mycluster/
 +
    ProxyPassReverse / balancer://mycluster/
 +
 
 +
    ErrorLog ${APACHE_LOG_DIR}/proxy-error.log
 +
    CustomLog ${APACHE_LOG_DIR}/proxy-access.log combined
  
That's basically it. Now you can set your DNS entry of www.yourcompany.com to your Load-Balancer's IP address and enjoy the comfort and security of a redundant LogicalDOC installation. If one of the public LogicalDOC servers is failing, mod_proxy on your load-balancer will automatically detect this and stop serving requests to that server.
+
</VirtualHost>
 +
</pre>
  
You can test this by stopping Tomcat on one of the machines. Your load-balancer Apache webserver error_log will show something like
+
{{Warning|Do not enable the balancer-manager until you have [https://httpd.apache.org/docs/trunk/mod/mod_proxy.html#access secured your server]. In particular, ensure that access to the URL is tightly restricted.}}
  
<source lang="text">
+
== Additional information ==
[Tue Jul 28 18:17:35 2009] [error] proxy: AJP: failed to make connection to backend: public1.yourcompany.com
 
[Tue Jul 28 18:17:36 2009] [error] ap_proxy_connect_backend disabling worker for (public1.yourcompany.com)
 
</source>
 
  
Also you could access the balancer-manager with http://www.yourcompany.com/balancer-manager to manually disable a worker. The balancer-manager also offers you an easy way to set different load factors for your servers.
+
* [https://tomcat.apache.org/tomcat-8.5-doc/proxy-howto.html Tomcat 8 - Proxy Support HOW-TO]
 +
* [https://examples.javacodegeeks.com/enterprise-java/tomcat/apache-tomcat-reverse-proxy-configuration-tutorial/ Tomcat Reverse Proxy Configuration Tutorial]
 +
* [https://httpd.apache.org/docs/2.4/howto/reverse_proxy.html Apache Reverse Proxy Guide]
 +
* [https://poweruphosting.com/blog/apache-reverse-proxy/ How To Use Apache Reverse Proxy with mod_proxy on Ubuntu 16.04]
 +
* [https://httpd.apache.org/docs/trunk/howto/reverse_proxy.html Apache Reverse Proxy Guide]
 +
* [http://www.techjira.com/html/loadbalancer/mod_proxy.html Techjira - Apache mod_proxy]

Latest revision as of 09:59, 27 April 2020

Server setup

The layout may look something like this (we will refer to these names through the rest of the guide).

Apache load balancer mod_proxy_http Tomcat

Enable Apache modules in Ubuntu

Use the following commands:

$ sudo a2enmod proxy
$ sudo a2enmod proxy_http
$ sudo a2enmod proxy_balancer
$ sudo a2enmod lbmethod_byrequests
$ sudo a2enmod headers

Run all the above commands then restart Apache to obtain the effect of changes we have made.

$ sudo service apache2 restart

Define Apache Load-balancer

This server will handle all HTTP requests from site visitors. As you might see, this means even though you run a load balanced system, using only a single load balancer means you still have a SPOF (single point of failure). It is also possible to configure an environment where yet another server will act as the fail-over load-balancer if the first one fails, but this is outside the scope of this guide.

To set up our load-balancer, we use the Apache web-server and its modules mod_proxy, mod_proxy_http and mod_proxy_balancer. These are part of most of the Apache web-server distributions.

First, create a virtual host handling the requests for your domain: ldproxy.org

<VirtualHost *:80>

  ServerName ldproxy.org

  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/html

  ProxyRequests Off
  ProxyPreserveHost On

  Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
  <Proxy balancer://mycluster>
        BalancerMember "http://eva00:9080" route=1
        BalancerMember "http://192.168.2.15:8082" route=2

        ProxySet lbmethod=byrequests
        ProxySet stickysession=ROUTEID
    </Proxy>

  ProxyPass / balancer://mycluster/
  ProxyPassReverse / balancer://mycluster/

  ErrorLog ${APACHE_LOG_DIR}/proxy-error.log
  CustomLog ${APACHE_LOG_DIR}/proxy-access.log combined

</VirtualHost>

The load balancer nodes are two LogicalDOC servers that are accessed by IP and by hostname (eva00), of course the Apache server must be able to resolve the hostname.

Furthermore we can note that the Apache server adds a Cookie ROUTEID to make sessions sticky and redirect all subsequent requests to the same LogicalDOC node of the cluster

The setting ProxySet lbmethod=byrequests distribute the requests among the various workers to ensure that each gets their configured share of the number of requests.

Load balancer scheduler algorithm

At present, there are 4 load balancer scheduler algorithms available for use: Request Counting (mod_lbmethod_byrequests), Weighted Traffic Counting (mod_lbmethod_bytraffic), Pending Request Counting (mod_lbmethod_bybusyness) and Heartbeat Traffic Counting (mod_lbmethod_heartbeat). These are controlled via the lbmethod value of the Balancer definition. See the ProxyPass directive for more information, especially regarding how to configure the Balancer and BalancerMembers.

Configure Tomcat

You need to change the configuration of the Tomcat in LogicalDOC. Generally it is a matter of modifying the file server.xml located in the /tomcat/conf folder

Tomcat config: LOGICALDOC_HOME/tomcat/conf/server.xml

In this file we are going to add the proxyName and proxyPort attributes to the Connector element

    <Connector port="8082" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8445"
               URIEncoding="UTF-8" server="Undisclosed/8.41" proxyName="ldproxy.org" proxyPort="80" />

The proxyName will cause servlets inside this web application to think that all proxied requests were directed to ldproxy.org on port 80

Done

That's basically it. Now you can set your DNS entry of ldproxy.org to your Load-Balancer's IP address and enjoy the comfort and security of a redundant LogicalDOC installation. If one of the public LogicalDOC servers is failing, mod_proxy on your load-balancer will automatically detect this and stop serving requests to that server.

You can test this by stopping Tomcat on one of the machines. The configured ErrorLog file proxy-error.log will show something like

[Tue Jun 25 09:55:34.431671 2019] [proxy:error] [pid 12308] (113)No route to host: AH00957: HTTP: attempt to connect to 192.168.2.15:8082 (192.168.2.15) failed
[Tue Jun 25 09:55:34.431768 2019] [proxy:error] [pid 12308] AH00959: ap_proxy_connect_backend disabling worker for (192.168.2.15) for 60s

Advanced Configuration

This below is a more advanced example. It requires a specific Apache module: mod_proxy_balancer which allows you to modify some parameters of the nodes that make up the cluster in real time. For more information Balancer Manager

Also in this example the status directive was applied to the second node of the cluster. This causes all requests to be sent to the first server and keeps the second LogicalDOC server in hot-standby mode (it will only be used if no other viable workers or spares are available in the balancer set).

<VirtualHost *:80>

    ServerName ldproxy.org

    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html

    ProxyRequests Off
    ProxyPreserveHost On

    <Location /balancer-manager>
      SetHandler balancer-manager
      #Require ip 10.0.0.1 10.0.0.2
    </Location>

    ProxyPass /balancer-manager !

    Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
    <Proxy balancer://mycluster>
        BalancerMember "http://192.168.2.15:8082" route=1
        BalancerMember "http://192.168.2.5:8080" route=2
        # The server below is on hot standby. It will be used only if no other members (or spares) are available in the load balancer set
        BalancerMember "http://192.168.2.11:8080" route=3 status=+H

        ProxySet lbmethod=byrequests
        ProxySet stickysession=ROUTEID
    </Proxy>

    ProxyPass / balancer://mycluster/
    ProxyPassReverse / balancer://mycluster/

    ErrorLog ${APACHE_LOG_DIR}/proxy-error.log
    CustomLog ${APACHE_LOG_DIR}/proxy-access.log combined

</VirtualHost>


Note warning.png Do not enable the balancer-manager until you have secured your server. In particular, ensure that access to the URL is tightly restricted.


Additional information