- 论坛徽章:
- 0
|
RHEL6.2平台利用KVM虚拟机搭建高可用负载均衡WEB集群(keepalived+lvs+iscsi+gfs)
最近学习KVM虚拟化及负载均衡集群技术,参考了很多帖子,在学习的过程中顺便将练手的过程记录如下,希望能给大家些许帮助!
keepalived+lvs+iscsi+gfs搭建高可用负载均衡web集群,主要参考了这个帖子:http://www.linuxidc.com/Linux/2011-12/48409.htm
也不知是不是原作者,谨此感谢!
一、环境搭建及虚拟机部署:
1、虚拟机宿主:
主机名 IP地址
WEB 172.16.9.1
RHEL6.2基本服务器和虚拟化
KVM虚拟机采用桥接方式安装
网卡桥接:
[root@Web:/root]$cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-br0
修改为如下即可:
[root@Web:/root]$more /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE="eth0"
NM_CONTROLLED="yes"
ONBOOT=yes
HWADDR=5C:F3:FC:94:88:EC
TYPE=Ethernet
BRIDGE=br0
BOOTPROTO=none
PREFIX=27
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME="System eth0"
注:eth0不要设置IP
[root@Web:/root]$more /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE="br0"
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=none
IPADDR=172.16.9.2
GATEWAY=172.16.9.1
DNS1=219.149.6.99
2、虚拟机安装:
安装、克隆总计8台虚拟机
虚拟机 主机名 IP
LVS-MASTER LVS-MASTER 172.16.9.10
LVS-BACKUP LVS-BACKUP 172.16.9.11
Mysql-A Mysql-A 172.16.9.21
Mysql-B Mysql-B 172.16.9.22
Web-A Web-A 172.16.9.31
Web-B Web-B 172.16.9.32
Web-C Web-C 172.16.9.33
Firesystem Firesystem 172.16.9.41
VIP IP 172.16.9.254
root@Web:[/root]~$virt-install \
--name Web-A \
--ram 2048 \
--vcpus=64 \ ##虚拟机CPU个数
--disk path=/var/lib/libvirt/images/Web-A.img,size=11 \
--network bridge=br0 \
--os-variant=rhel6 \
--cdrom /var/lib/libvirt/rhel-server-6.2-x86_64-dvd.iso \
--vnclisten=172.16.9.1 \
--vncport=6901 \
--vnc
安装过程选RHEL6.2基本服务器即可
3、克隆虚拟机:
暂停虚拟机以便克隆
root@Web:[/root]~$virsh suspend Web-A
开始克隆
root@Web:[/root]~$virt-clone --connect qemu:///system --original=Web-A --name=Web-B --file=/var/lib/libvirt/images/Web-B.img
root@Web:[/root]~$virt-clone --connect qemu:///system --original=Web-A --name=Web-C --file=/var/lib/libvirt/images/Web-C.img
root@Web:[/root]~$virt-clone --connect qemu:///system --original=Web-A --name=Mysql-A--file=/var/lib/libvirt/images/Mysql-A.img
root@Web:[/root]~$virt-clone --connect qemu:///system --original=Web-A --name=Mysql-B --file=/var/lib/libvirt/images/Mysql-B.img
root@Web:[/root]~$virt-clone --connect qemu:///system --original=Web-A --name=Firesystem --file=/var/lib/libvirt/images/Firesystem.img
root@Web:[/root]~$virt-clone --connect qemu:///system --original=Web-A --name=LVS-MASTER --file=/var/lib/libvirt/images/LVS-MASTER.img
root@Web:[/root]~$virt-clone --connect qemu:///system --original=Web-A --name=LVS-BACKUP --file=/var/lib/libvirt/images/LVS-BACKUP.img
恢复虚拟机
root@Web:[/root]~$virsh resume Web-A
如需强制关闭虚拟机,如下:
root@Web:[/etc/libvirt/qemu]~$virsh destroy Web-A
虚拟机克隆完毕后,需修改vnc端口
root@Web:[/root]~$vi /etc/libvirt/qemu/Web-B.xml
<graphics type='vnc' port='6902' autoport='no' listen='172.16.9.32'>
<listen type='address' address='172.16.9.32'/>
</graphics>
所有克隆虚拟机的VNC端口修改完毕后,从启libvirtd服务
从启libvirtd服务
root@Web:[/root]~$service libvirtd restart
Stopping libvirtd daemon: [ OK ]
Starting libvirtd daemon: 15:06:04.222: 51157: info : libvirt version: 0.9.4, package: 23.el6_2.4 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2012-01-06-02:50:25, x86-007.build.bos.redhat.com)
15:06:04.222: 51157: warning : virGetHostname:1957 : getaddrinfo failed for 'Web': Name or service not known
[ OK ]
出现此警告warning : virGetHostname:1957 : getaddrinfo failed for 'Web': Name or service not known
解决方法:
root@Web:[/root]~$vi /etc/hosts
添加
172.16.9.2 Web
root@Web:[/root]~$service libvirtd restart
Stopping libvirtd daemon: [ OK ]
Starting libvirtd daemon: [ OK ]
启动全部虚拟机
root@Web:[/root]~$virsh start XXX
root@Web:[/root]~$virsh list
Id Name State
----------------------------------
1 LVS-MASTER running
2 LVS-BACKUP running
3 Filesystem running
4 Web-A running
5 Web-A running
6 Web-C running
7 Mysql-A running
8 Mysql-B running
虚拟机启动后网卡无法正常工作,解决方法如下:
kvm虚拟机克隆后网卡无法启动解决办法:
1 修改/etc/udev/rules.d/70-persistent-net.rules文件,删除第一个网卡记录,并将第二个的NAME="eth1"改为NAME="eth0"
2 如果在/etc/sysconfig/network-scripts/ifcfg-eth0中有配置网卡信息的话,如:#HWADDR="00:0C:29:C8:1A:92",将其注释。
顺便修改主机名和ip地址
修改主机名:
root@Web-A:[/root]~$vi /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=LVS-BACKUP
GATEWAY=172.16.9.1
更新系统时间
yum -y install ntp
ntpdate time.nist.gov
echo "00 01 * * * /usr/sbin/ntpdate time.nist.gov" > /etc/crontab
重启系统
全部虚拟机准备好后,开始搭建高可用负载均衡WEB集群!
二、keepalived+lvs搭建高可用负载均衡WEB集群
rhel6.2系统盘里有ipvsadm-1.25-9.el6.x86_64.rpm
keepalived-1.2.2.tar.gz需要下载http://www.keepalived.org
1、LVS-MASTER及LVS-BACKUP安装步骤:
登录LVS-MASTER
root@LVS-MASTER:[/root]~$rpm -ivh ipvsadm-1.25-9.el6.x86_64.rpm
root@LVS-MASTER:[/root]~$yum -y install kernel-devel
root@LVS-BACKUP:[/root]~$yum -y groupinstall "Development tools"
root@LVS-MASTER:[/root]~$tar xzvf keepalived-1.2.2.tar.gz
root@LVS-MASTER:[/root]~$cd keepalived-1.2.2
root@LVS-MASTER:[/root/keepalived-1.2.2]~$./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-220.el6.x86_64/
出错:
configure: error:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files. !!!
解决方法:
root@LVS-MASTER:[/root/keepalived-1.2.2]~$yum -y install openssl-devel
出错:
checking for poptGetContext in -lpopt... no
configure: error: Popt libraries is required
解决方法:
root@LVS-MASTER:[/root/keepalived-1.2.2]~$yum -y install popt-devel
再次运行
root@LVS-MASTER:[/root/keepalived-1.2.2]~$./configure -- prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-220.el6.x86_64/
完毕后出现如下信息:
############################################
Keepalived configuration
------------------------
Keepalived version : 1.2.2
Compiler : gcc
Compiler flags : -g -O2
Extra Lib : -lpopt -lssl -lcrypto
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
IPVS use libnl : No
Use VRRP Framework : Yes
Use Debug flags : No
############################################
root@LVS-MASTER:[/root/keepalived-1.2.2]~$make
root@LVS-MASTER:[/root/keepalived-1.2.2]~$make install
############################################
make -C keepalived install
make[1]: Entering directory `/root/keepalived-1.2.2/keepalived'
install -d /usr/local/keepalived/sbin
install -m 700 ../bin/keepalived /usr/local/keepalived/sbin/
install -d /usr/local/keepalived/etc/rc.d/init.d
install -m 755 etc/init.d/keepalived.init /usr/local/keepalived/etc/rc.d/init.d/keepalived
install -d /usr/local/keepalived/etc/sysconfig
install -m 755 etc/init.d/keepalived.sysconfig /usr/local/keepalived/etc/sysconfig/keepalived
install -d /usr/local/keepalived/etc/keepalived/samples
install -m 644 etc/keepalived/keepalived.conf /usr/local/keepalived/etc/keepalived/
install -m 644 ../doc/samples/* /usr/local/keepalived/etc/keepalived/samples/
install -d /usr/local/keepalived/share/man/man5
install -d /usr/local/keepalived/share/man/man8
install -m 644 ../doc/man/man5/keepalived.conf.5 /usr/local/keepalived/share/man/man5
install -m 644 ../doc/man/man8/keepalived.8 /usr/local/keepalived/share/man/man8
make[1]: Leaving directory `/root/keepalived-1.2.2/keepalived'
make -C genhash install
make[1]: Entering directory `/root/keepalived-1.2.2/genhash'
install -d /usr/local/keepalived/bin
install -m 755 ../bin/genhash /usr/local/keepalived/bin/
install -d /usr/local/keepalived/share/man/man1
install -m 644 ../doc/man/man1/genhash.1 /usr/local/keepalived/share/man/man1
make[1]: Leaving directory `/root/keepalived-1.2.2/genhash'
############################################
安装文件路径
2、LVS-MASTER及LVS-BACKUP主机配置keepalived
(1)、LVS-MASTER
root@LVS-MASTER:[/root]~$mkdir /etc/keepalived
root@LVS-MASTER:[/root]~$ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
root@LVS-MASTER:[/root]~$ln -s /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
root@LVS-MASTER:[/root]~$ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
root@LVS-MASTER:[/root]~$ln -s /usr/local/keepalived/bin/* /bin/
root@LVS-MASTER:[/root]~$ln -s /usr/local/keepalived/sbin/* /sbin/
root@LVS-BACKUP:[/root]~$modprobe ip_vs
root@LVS-MASTER:[/root]~$vi /etc/rc.local
############################################
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
modprobe ip_vs
############################################
LVS-MASTER主机keepalived.conf文件设置:
root@LVS-MASTER:[/root]~$vi /etc/keepalived/keepalived.conf
############################################
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from root@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.9.254
}
}
virtual_server 172.16.9.254 80 {
delay_loop 6
lb_algo rr
lb_kind DR
# nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 172.16.9.31 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.16.9.32 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.16.9.33 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
############################################
启动keepalived
root@LVS-MASTER:[/root]~$service keepalived restart
停止 keepalived:[失败]
正在启动 keepalived:[确定]
root@LVS-MASTER:[/root]~$chkconfig --level 2345 keepalived on
(2)、LVS-BACKUP
LVS-BACKUP主机keepalived.conf文件设置:
root@LVS-BACKUP:[/root]~$vi /etc/keepalived/keepalived.conf
############################################
root@LVS-BACKUP:[/root]~$more /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from root@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 50
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.9.254
}
}
virtual_server 172.16.9.254 80 {
delay_loop 6
lb_algo rr
lb_kind DR
# nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 172.16.9.31 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.16.9.32 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.16.9.33 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
############################################
启动keepalived
root@LVS-BACKUP:[/root]~$service keepalived restart
停止 keepalived:[失败]
正在启动 keepalived:[确定]
root@LVS-BACKUP:[/root]~$chkconfig --level 2345 keepalived on
3、REAL_SERVER(WEB-A、WEB-B、WEB-C)做如下设置:
分别在 REAL_SERVER(WEB-A、WEB-B、WEB-C)上执行以下命令:
[root@Test-Web-B ~]# yum groupremove "MySQL Database client" "MySQL Database server" ##原来安装了mysql,卸载掉
安装web服务及php
[root@Web-A ~]# yum -y groupinstall "Web Server" "PHP Support"
[root@Web-A ~]# vi /etc/sysctl.conf
################################################
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
################################################
[root@Web-A ~]# sysctl -p
[root@Web-A ~]# ifconfig eth0:0 172.16.9.254 netmask 255.255.255.255 up
[root@Web-A ~]# route add -host 172.16.9.254 dev eth0:0
[root@Web-A ~]# echo `hostname` > /var/www/html/index.html
[root@Web-C ~]# vi /etc/httpd/conf/httpd.conf
修改
ServerName
[root@Web-A ~]# service httpd start
4、测试:
(1)
把所有三个WEB服务全停掉
[root@Test-Web-A ~]service httpd stop
[root@Test-Web-B ~]service httpd stop
[root@Test-Web-C ~]service httpd stop
访问VIP
root@Web:[/root]~$lynx 172.16.9.254
Alert!: Unable to connect to remote host.
Looking up 172.16.9.254 first
Looking up 172.16.9.254
Making HTTP connection to 172.16.9.254
Alert!: Unable to connect to remote host.
lynx: Can't access startfile http://172.16.9.254/
(2)
开启Test-Web-A服务:
[root@Test-Web-A ~]service httpd start
root@Web:[/root]~$lynx 172.16.9.254
显示页面如下:
Test-Web-A
(3)
停掉Test-Web-A服务,启动Test-Web-B服务:
[root@Test-Web-A ~]service httpd stop
[root@Test-Web-B ~]service httpd start
root@Web:[/root]~$lynx 172.16.9.254
刷新几次后,显示页面如下:
Test-Web-B
(4)
停掉Test-Web-B服务,启动Test-Web-C服务:
[root@Test-Web-B ~]service httpd stop
[root@Test-Web-C ~]service httpd start
root@Web:[/root]~$lynx 172.16.9.254
刷新几次后,显示页面如下:
Test-Web-C
(5)
把所有三个WEB服务全启动
[root@Web-A ~]service httpd start
[root@Web-B ~]service httpd start
[root@Web-C ~]service httpd start
分别用不同ip的主机访问会发现显示不同的页面
今天试验到此结束,接下来是mysql双主高可用服务器的搭建及iscsi+gfs文件系统的搭建。
明天继续………… |
|