Nginx负载均衡及Active Health Check模块安装（年代久远，归档）

Introduction

Load balancing across multiple application instances is a commonly used technique for optimizing resource utilization, maximizing throughput, reducing latency, and ensuring fault-tolerant configurations.

It is possible to use nginx as a very efficient HTTP load balancer to distribute traffic to several application servers and to improve performance, scalability and reliability of web applications with nginx.

Load balancing methods

The following load balancing mechanisms (or methods) are supported in nginx:

round-robin — requests to the application servers are distributed in a round-robin fashion,

least-connected — next request is assigned to the server with the least number of active connections,

ip-hash — a hash-function is used to determine what server should be selected for the next request (based on the client’s IP address).

Default load balancing configuration

The simplest configuration for load balancing with nginx may look like the following:

http { upstream myapp1 { server srv1.example.com; server srv2.example.com; server srv3.example.com; } server { listen 80; location / { proxy_pass http://myapp1; } } }

In the example above, there are 3 instances of the same application running on srv1-srv3. When the load balancing method is not specifically configured, it defaults to round-robin. All requests are proxied to the server group myapp1, and nginx applies HTTP load balancing to distribute the requests.

Reverse proxy implementation in nginx includes load balancing for HTTP, HTTPS, FastCGI, uwsgi, SCGI, memcached, and gRPC.

To configure load balancing for HTTPS instead of HTTP, just use “https” as the protocol.

When setting up load balancing for FastCGI, uwsgi, SCGI, memcached, or gRPC, use fastcgi_pass, uwsgi_pass, scgi_pass, memcached_pass, and grpc_pass directives respectively.

Least connected load balancing

Another load balancing discipline is least-connected. Least-connected allows controlling the load on application instances more fairly in a situation when some of the requests take longer to complete.

With the least-connected load balancing, nginx will try not to overload a busy application server with excessive requests, distributing the new requests to a less busy server instead.

Least-connected load balancing in nginx is activated when the least_conn directive is used as part of the server group configuration:

upstream myapp1 { least_conn; server srv1.example.com; server srv2.example.com; server srv3.example.com; }

Session persistence

Please note that with round-robin or least-connected load balancing, each subsequent client’s request can be potentially distributed to a different server. There is no guarantee that the same client will be always directed to the same server.

If there is the need to tie a client to a particular application server — in other words, make the client’s session “sticky” or “persistent” in terms of always trying to select a particular server — the ip-hash load balancing mechanism can be used.

With ip-hash, the client’s IP address is used as a hashing key to determine what server in a server group should be selected for the client’s requests. This method ensures that the requests from the same client will always be directed to the same server except when this server is unavailable.

To configure ip-hash load balancing, just add the ip_hash directive to the server (upstream) group configuration:

upstream myapp1 { ip_hash; server srv1.example.com; server srv2.example.com; server srv3.example.com; }

Weighted load balancing

It is also possible to influence nginx load balancing algorithms even further by using server weights.

In the examples above, the server weights are not configured which means that all specified servers are treated as equally qualified for a particular load balancing method.

With the round-robin in particular it also means a more or less equal distribution of requests across the servers — provided there are enough requests, and when the requests are processed in a uniform manner and completed fast enough.

When the weight parameter is specified for a server, the weight is accounted as part of the load balancing decision.

upstream myapp1 { server srv1.example.com weight=3; server srv2.example.com; server srv3.example.com; }

With this configuration, every 5 new requests will be distributed across the application instances as the following: 3 requests will be directed to srv1, one request will go to srv2, and another one — to srv3.

It is similarly possible to use weights with the least-connected and ip-hash load balancing in the recent versions of nginx.

Health checks

Reverse proxy implementation in nginx includes in-band (or passive) server health checks. If the response from a particular server fails with an error, nginx will mark this server as failed, and will try to avoid selecting this server for subsequent inbound requests for a while.

The max_fails directive sets the number of consecutive unsuccessful attempts to communicate with the server that should happen during fail_timeout. By default, max_fails is set to 1. When it is set to 0, health checks are disabled for this server. The fail_timeout parameter also defines how long the server will be marked as failed. After fail_timeout interval following the server failure, nginx will start to gracefully probe the server with the live client’s requests. If the probes have been successful, the server is marked as a live one.

More details

In addition, there are more directives and parameters that control server load balancing in nginx, e.g. proxy_next_upstream, backup, down, and keepalive. For more information please check reference documentation.

Let's make a load balancing server now

1. Install Nginx on Ubuntu Linux

Install the prerequisites:

sudo apt install curl gnupg2 ca-certificates lsb-release

To set up the apt repository for stable nginx packages, run the following command:

echo "deb http://nginx.org/packages/ubuntu `lsb_release -cs` nginx" | sudo tee /etc/apt/sources.list.d/nginx.list

If you would like to use mainline nginx packages, run the following command instead:

echo "deb http://nginx.org/packages/mainline/ubuntu `lsb_release -cs` nginx" | sudo tee /etc/apt/sources.list.d/nginx.list

Next, import an official nginx signing key so apt could verify the packages authenticity:

curl -fsSL https://nginx.org/keys/nginx_signing.key | sudo apt-key add -

Verify that you now have the proper key:

sudo apt-key fingerprint ABF5BD827BD9BF62

The output should contain the full fingerprint

573B FD6B 3D8F BC64 1079 A6AB ABF5 BD82 7BD9 BF62

as follows:

pub rsa2048 2011-08-19 [SC] [expires: 2024-06-14] 573B FD6B 3D8F BC64 1079 A6AB ABF5 BD82 7BD9 BF62 uid [ unknown] nginx signing key <signing-key@nginx.com>

To install nginx, run the following commands:

sudo apt update sudo apt install nginx

2. Configure the "nginx.conf" file

sudo vim /etc/nginx/nginx.conf

upstream pos_server { server 192.168.110.65:3458; server 192.168.110.83:3458; } #static server server { root /data/www; location / { } location /nginx_status { stub_status; } } #posd load balance server server { listen 3458; location /pos_api { proxy_pass http://pos_server/location/api; } }

Please replace "server 192.168.110.65:3458" "server 192.168.110.83:3458" with yours.

3. Start nginx server

sudo service nginx start

For getting the list of all running nginx processes, in the following way:

ps -ax | grep nginx

 

 

4. Test

You can use chrome rest client plugin to test the api: 

 

It works.

Advanced Topic

The open source version nginx does not support active health check, if you want that function, you need to buy nginx plus or use third-party nginx module like nginx_upstream_check_module. But Installing the module gets a little annoying.I just introduce how to install the module on ubuntu14.04 simply.

1. Solve the denpendencies

Nginx is a program written in C, so we need to install the C compiler (GCC).

sudo apt install build-essential -y

Download the Nginx dependencies' source code and extract them:

Nginx depends on 3 libraries: PCRE, zlib and OpenSSL:

1)Install PCRE

$ wget https://ftp.pcre.org/pub/pcre/pcre-8.39.tar.gz && tar xzvf pcre-8.39.tar.gz $ cd pcre-8.40 $ ./configure $ sudo make && make install

2) Install zlib

wget http://www.zlib.net/zlib-1.2.11.tar.gz && tar xzvf zlib-1.2.11.tar.gz $ cd zlib-1.2.11 $ ./configure $ sudo make && make install

3) Install OpenSSL

# OpenSSL version 1.0.2 - 1.1.0 wget https://www.openssl.org/source/openssl-1.1.0f.tar.gz && tar xzvf openssl-1.1.0f.tar.gz

2. Compile Nginx with "nginx_upstream_check_module"

Download the latest version of the release tarball of this module from github (http://github.com/yaoweibin/nginx_upstream_check_module)

Grab the nginx source code from nginx.org (http://nginx.org/), for

```

# 下载并解压 nginx 1.14.2
wget https://nginx.org/download/nginx-1.14.2.tar.gz && tar zxvf nginx-1.14.2.tar.gz

# 下载并解压 nginx 1.0.14
wget 'http://nginx.org/download/nginx-1.0.14.tar.gz'
tar -xzvf nginx-1.0.14.tar.gz

# 进入目录并打补丁
cd nginx-1.0.14/
patch -p1 < /path/to/nginx_http_upstream_check_module/check.patch

# 配置编译参数
./configure --prefix=/usr/share/nginx \
--sbin-path=/usr/sbin/nginx \
--modules-path=/usr/lib/nginx/modules \
--conf-path=/etc/nginx/nginx.conf \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--pid-path=/run/nginx.pid \
--lock-path=/var/lock/nginx.lock \
--user=www-data \
--group=www-data \
--build=Ubuntu \
--http-client-body-temp-path=/var/lib/nginx/body \
--http-fastcgi-temp-path=/var/lib/nginx/fastcgi \
--http-proxy-temp-path=/var/lib/nginx/proxy \
--http-scgi-temp-path=/var/lib/nginx/scgi \
--http-uwsgi-temp-path=/var/lib/nginx/uwsgi \
--with-debug \
--add-module=/path/to/nginx_upstream_check_module

# 编译并安装
sudo make && make install

```

Notice: check_patch should match your nginx version, see compatibility detail.

3. Check syntax and potential errors:

sudo nginx -t # Will throw this error nginx: [emerg] mkdir() "/var/lib/nginx/body" failed (2: No such file or directory) # Just create directory mkdir -p /var/lib/nginx && sudo nginx -t

4. Configure Nginx service script and nginx.conf, then test it

You can search on Internet to find how configure them.

If you can see this page, you get it works!