lvs的安装部署

    消息系统节点服务器单机需支持25万长连接及每秒1.2万消息发送，本文档用于指导部署lvs搭建（软件版本ipvsadm-1.24.tar.gz keepalived-1.1.20.tar.gz）

一、LVS搭建

    本次搭建基于DR负载均衡模式，设置了一个VIP（Virtual IP）为192.168.3.187，用户只需要访问这个IP地址即可获得网页服务。其中，负载均衡主机为192.168.3.203，备机为192.168.3.204。Web服务器A为192.168.3.129，Web服务器B为192.168.3.127

2.1 关闭防火墙

service iptables stop   验证  service iptables status

2.2 安装keepalived依赖

 yum install -y gcc gcc-c++ openssl openssl-devel

2.3 查看系统环境

 lsb_release -a

2.4 检查linux内核是否支持lvs

 lsmod | grep ip_vs

2.5 安装keepalived

上传keepalived-1.1.20.tar.gz包

tar -zxvf keepalived-1.1.20.tar.gz 

cd keepalived-1.1.20

./configure --prefix=/usr/local/keepalived //指定安装路径，出现如下图时代表安装成功

 

注：configure: error: Popt libraries is required

出现此错误的原因：

未安装popt的开发包

解决方法：

yum install popt-devel

 

make && make install  //安装，出现如下图代表安装成功

 

将keepalived命令软连接到/usr/bin目录下

ln -s /usr/local/keepalived/sbin/keepalived /usr/bin/keepalived

添加启动脚本且方便使用service keepalived start|stop|restart 命令进行管理

cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/keepalived

chmod 755 /etc/init.d/keepalived //添加执行权限

chkconfig keepalived on //开机自启动

vi /etc/init.d/keepalived  //修改/etc/init.d/keepalived中的程序路径

# Source configuration file (we set KEEPALIVED_OPTIONS there)

. /etc/sysconfig/keepalived

改为：

# Source configuration file (we set KEEPALIVED_OPTIONS there)

. /usr/local/keepalived/etc/sysconfig/keepalived

默认情况下，keepalived 会读取 /etc/keepalived 下keepalived.conf 文件

如果没有建立这个文件，keepalived也不会报错，但是会发现，所创建的关于keepalived的相关参数根本就没有生效。

mkdir /etc/keepalived

建立配置文件的软连接

ln -s /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf 

service keepalived restart  //启动测试

 

编辑keepalived的文件，并上传keepalived.conf至 /usr/local/keepalived/etc/keepalived 目录

上传健康检查脚本check_port.pl至/root目录，并赋予运行权限

chmod 755 /root/check_port.pl 

service keepalived restart //重新启动keepalived

2.6 安装ipvsadm管理工具

上传ipvsadm-1.24.tar.gz

tar -zxvf ipvsadm-1.24.tar.gz 

cd ipvsadm-1.24

建立编译时必须的一个软链接

uname -a  //显示系统环境

Linux lvs-slave 2.6.18-274.el5 #1 SMP Fri Jul 8 17:36:59 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

ln -s /usr/src/kernels/2.6.18-274.el5/ /usr/src/linux   //其中2.6.18-274.el5 需和上一步看到一致

make && make install //安装

whereis ipvsadm //显示如下表明安装成功

 

 

注：出现libipvs.h:14:23: 错误：net/ip_vs.h：没有那个文件或目录错误时

   需先yum -y install kernel-devel，然后再重新make&install 那一步

2.7 上传节点服务器工程

  启动节点服务器

  编辑lvs.sh，修改其中的vip地址，上传lvs.sh 赋予运行权限 并运行 

chmod 755 lvs.sh

Sh lvs.sh

 

2.8 备注

Lvs通常为主备，以上对lvs设置可以为主机和备机一起设置，备机只需修改keepalived.conf配置文件中 state MASTER 为 state SLAVE 即可

 

 

 

 

附件一：lvs.sh

 

#!/bin/bash

#description : start realserver

VIP=192.168.3.187

chmod +x /etc/rc.d/init.d/functions

/etc/rc.d/init.d/functions

case "$1" in

start)

echo " start LVS of REALServer"

/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up

echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

;;

stop)

/sbin/ifconfig lo:0 down

echo "close LVS Directorserver"

echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore

echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce

echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore

echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce

;;

*)

echo "Usage: $0 {start|stop}"

exit 1

esac

 

附件二:check_port.pl

 

#!/usr/bin/perl  -w

#===============================================================================

#

#         FILE:  check_port.pl

#

#        USAGE:  check_port.pl -p <port> -h <host> (-c <critical> -w <warning> -v)

#

#  DESCRIPTION:  tests to see if the port is responding and can display timing

#

#      OPTIONS:  ---

# REQUIREMENTS:  ---

#         BUGS:  ---

#        NOTES:  ---

#       AUTHOR:  Tim Pretlove

#      VERSION:  1.3

#      CREATED:  04/12/09 13:57:23

#     REVISION:  ---

#      LICENCE:  GNU

#

#       AUTHOR:  Jim Sander [e-mail]

#      VERSION:  1.2

#     MODIFIED:  10-04-2014 16:00

#         BUGS:  Socket::pack_sockaddr_in, length is 0 error for unresolvable hostnames

#        NOTES:  Fixed; now exits with '3', status UNKNOWN, and 'host lookup failed'

#

#    This program is free software: you can redistribute it and/or modify

#    it under the terms of the GNU General Public License as published by

#    the Free Software Foundation, either version 3 of the License, or

#    (at your option) any later version.

#

#    This program is distributed in the hope that it will be useful,

#    but WITHOUT ANY WARRANTY; without even the implied warranty of

#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

#    GNU General Public License for more details.

#

#    You should have received a copy of the GNU General Public License

#    along with this program.  If not, see <http://www.gnu.org/licenses/>.

#

#===============================================================================

 

use strict;

use warnings;

use Socket;

use Getopt::Long;

use Time::HiRes qw(gettimeofday tv_interval);

 

my ($crit, $warn, $timeout, $host, $portnum, $verbose);

GetOptions(

    'crtitical=s'   => \$crit,

    'warning=s'     => \$warn,

    'timeout=s'     => \$timeout,

    'host=s'        => \$host,

    'port=s'        => \$portnum,

    'verbose'       => \$verbose) or HELP_MESSAGE();

 

sub testport {

my ($host,$port,$protocol,$timeout) = @_;

    my $startsec;

    my $elapsed = 0;

if (!defined $timeout) { $timeout = 10 }

if (!defined $protocol) { $protocol = "tcp" }

my $proto = getprotobyname($protocol);

my $iaddr = inet_aton($host);

if ( !defined $iaddr ){ return 3,$elapsed; }

my $paddr = sockaddr_in($port, $iaddr);

$startsec = [gettimeofday()];

socket(SOCKET, PF_INET, SOCK_STREAM, $proto) or die "socket: $!";

eval {

local $SIG{ALRM} = sub { die "timeout" };

alarm($timeout);

connect(SOCKET, $paddr) or error();

alarm(0);

};

if ($@) {

close SOCKET || die "close: $!";

$elapsed = tv_interval ($startsec, [gettimeofday]);

return "1",$elapsed;

} else {

close SOCKET || die "close: $!";

$elapsed = tv_interval ($startsec, [gettimeofday]);

return "0",$elapsed;

}

}

sub HELP_MESSAGE {

print "$0 -p <port> -h <host> (-c <critical> -w <warning> -v)\n"; 

print "\t -p <port> # port number to examine\n";

print "\t -h <hostname> # hostname or ip address to contact\n";

print "\t -c <seconds> # the number of seconds to wait before a going critical\n";

print "\t -w <seconds> # the number of seconds to wait before a flagging a warning\n";

print "\t -v # displays nagios performance information\n";

print "\te.g $0 -p 80 -h www.google.com -c 1.5 -w 1.0 -v\n";

    exit(4);

}

 

sub printperf {

    my ($warning,$critical,$elapsed) = @_;

if ((defined $warning) && (defined $critical)) {

print "|rta=$elapsed" . "s;$warning;$critical;0;$critical";

} else {

print "|rta=$elapsed" 

}

}

 

sub test {

    my ($critical,$warning,$host,$portnum,$timeout) = @_;

    my $proto = "tcp";

my ($rc,$elapsed) = testport($host,$portnum,$proto,$timeout); 

if ($rc == 0) {

if (defined $critical) {

if ($critical <= $elapsed) {

return 2,$elapsed;

}

}

if (defined $warning) {

if ($warning <= $elapsed) {

return 1,$elapsed;

}

}

return $rc,$elapsed;

} else {

return 2,$elapsed;

}

}

 

unless ((defined $portnum) && (defined $host)) { 

HELP_MESSAGE();

exit 1;

}

if ((defined $crit) && (defined $warn)) {

if ($crit <= $warn) {

print "Error: warning is greater than critical will never reach warning\n";

exit 4;

}

}

 

my @mess = qw(OK WARNING CRITICAL UNKNOWN);

my @mess2 = ("is responding","is slow responding","is not responding","host lookup failed");

my ($rc,$elapsed) = test($crit,$warn,$host,$portnum,$timeout);

print "PORT $portnum $mess[$rc]: $host/$portnum $mess2[$rc]";

if (defined $verbose) {

printperf($warn,$crit,$elapsed);

}

exit($rc);

 

附件三:keepalived.conf

 

! Configuration File for keepalived

 

global_defs {

   notification_email {

     [e-mail]

     [e-mail]

     [e-mail]

   }

   notification_email_from [e-mail]

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id ms1

}

 

vrrp_instance VI_1 {

    state MASTER

    interface eth0

    virtual_router_id 51

    priority 110

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.3.187

    }

}

 

virtual_server 192.168.3.187 1883{

    delay_loop 6

    lb_algo lc

    lb_kind DR

    nat_mask 255.255.255.0

    protocol TCP

    real_server 192.168.3.129 1883{

     weight 100

    MISC_CHECK {

          misc_path "/root/check_port.pl -h 192.168.3.129 -p 1883 -w 5 -c 10"

          misc_timeout 10

        }

    }

    

    real_server 192.168.3.127 1883 {

     weight 100

     MISC_CHECK {

          misc_path "/root/check_port.pl -h 192.168.3.127 -p 1883 -w 5 -c 10"

          misc_timeout 10

        }

    }

}

 

 

virtual_server 192.168.3.187 1884{

    delay_loop 6

    lb_algo rr

    lb_kind DR

    nat_mask 255.255.255.0

    persistence_timeout 1200

    protocol TCP

    real_server 192.168.3.129 1884{

     weight 100

     MISC_CHECK {

          misc_path "/root/check_port.pl -h 192.168.3.129 -p 1884 -w 5 -c 10"

          misc_timeout 10

        }

    }

    

    real_server 192.168.3.127 1884 {

     weight 100

     MISC_CHECK {

          misc_path "/root/check_port.pl -h 192.168.3.127 -p 1884 -w 5 -c 10"

          misc_timeout 10

        }

    }

}