Linux Virtual Server on Gentoo for Email

me09

Linux Virtual Server on Gentoo for Email
Introduction
Setting up LVS on Gentoo is pretty simple. My setup is described below.
I use LVS-NAT instead of VS-DR (Direct routing). VS-DR is faster than LVS-NAT,
but I'm doing it this way for a few reasons.

• I want to firewall the cluster off from the rest of the world.
• The nodes comminicate with the NFS and accounts servers on a private
     network. I do this to help speed up the connections between the nodes and
     the NFS server where the mail store is located.
    
• NFS has some security issues that I want to minimize and isolating it
     to a private network.
    
• I netboot my nodes and I really don't feel like doing DHCP over a
     public network.
    

Note: This document does not describe the setup of the mail nodes.
That will be left for another document.
Note: I run a load balanced cluster for email. You should be able
to adapt these docs for many other services.
Note: I'm going to assume you have a working installation of
Gentoo on a box with two NICs. You might be able to do this on one NIC but
that is left as an excersize for the reader.
And here we go .... :-)
Add IPVS to the Kernel
The first thing we're going to do is upgrade to the 2.6.x kernel. You
don't have to do this, I suppose, but I did.
emerge -k gentoo-dev-sources
Now add IPVS to the kernel.
cd /usr/src/linux
make menuconfig
Here are the options that I set specific to LVS. Feel free to adjust as
needed.
Code maturity level options  --->

Prompt for development and/or incomplete code/drivers
Device Drivers  --->
Networking support  --->
Networking options  --->
  

Network packet filtering (replaces ipchains)  --->
  [ ]   Network packet filtering debugging
  IP: Virtual Server Configuration  --->
    IP virtual server support (EXPERIMENTAL)
   [ ]   IP virtual server debugging
   (12)  IPVS connection table size (the Nth power of 2)
   ---   IPVS transport protocol load balancing support
   

   TCP load balancing support
   

   UDP load balancing support
   [ ]   ESP load balancing support
   [ ]   AH load balancing support
   ---   IPVS scheduler
      round-robin scheduling
      weighted round-robin scheduling
      least-connection scheduling
      weighted least-connection scheduling
      locality-based least-connection scheduling
      locality-based least-connection with replication scheduling
      destination hashing scheduling
      source hashing scheduling
      shortest expected delay scheduling
      never queue scheduling
   ---   IPVS application helper
      FTP protocol helper
Now build and install your new kernel.
make && make modules_install
mount /boot
cp arch/i386/boot/bzImage /boot/kernel-version
cp System.map /boot/System.map-version
cp .config /boot/config-version
umount /boot
Boot to your new kernel and make sure everything works.
Install Needed Ports
ipvsadm
Because we're using the latest kernel we need to be sure that we have the
latest version of ipvsadm.
emerge -k >=ipvsadm-1.24
or
cd /usr/portage/sys-cluster/ipvsadm
emerge -k ipvsadm-1.24.ebuild
iptables
You may be able to skip this step. I setup NAT through iptables before
I setup the LVS-NAT. It's not a bad idea to have iptables on the
gateway/firewall, though.
emerge -k iptables
Install hearbeat
heartbeat is what will provide the high-availability features needed to
automatically remove nodes from the rotation when it goes out of service
for whatever reason. It also allows me to setup a redundant LB to take over
from the main one if it goes down.
USE='ldirectord' emerge -k heartbeat
Note: You can add ldirectord to your USE line in
/etc/make.conf.
Configuring LVS
Now the real fun begins. All of the configs are in /etc/ha.d so
we'll start there.
Configuring HA
We'll first copy the example configs to /etc/ha.d.
cd /usr/share/doc/heartbeat-version
cp ha.cf haresources /etc/ha.d
ha.cf
Here is my ha.cf with the comments removed.
logfacility     local0
bcast eth1
node    hydra cerberus
hydra is my primary LB and cerberus is the secondary. The
names need to match the output of uname -n.
haresources
Here is my haresources, again, with the comments removed.
wan_ip is the WAN address of the cluster and lan_ip is the LAN
address. The nodes will use lan_ip as their gateway address.
hydra   wan_ip/24/eth0 ldirectord
hydra   lan_ip/24/eth1 ldirectord
Note: This file should be the same on all LVS servers in this
group.
authkeys
authkeys controls access to the LVS group. My LBs are talking to
each other on a private network so security isn't a big an issue here. If
you are trying to setup VS-DR, you will want to use something a bit more
secure that this.
auth 1
1 crc
ldirectord.cf
ldirectord.cf controls the load balancer itself. This is where you
set which nodes handle which services and the weights for the nodes.
logfile="local0"
virtual = wan_ip:25
        real = node1:25 masq 1000
        real = node2:25 masq 1000
        real = node3:25 masq 1000
        real = node4:25 masq  667
        checktimeout = 10
        scheduler = sed
        protocol = tcp
virtual = wan_ip:110
        real = node1:110 masq 1000
        real = node2:110 masq 1000
        real = node3:110 masq 1000
        real = node4:110 masq  667
        scheduler = sed
        protocol = tcp
virtual = wan_ip:143
        real = node1:143 masq 1000
        real = node2:143 masq 1000
        real = node3:143 masq 1000
        real = node4:143 masq 1000
        scheduler = sed
        protocol = tcp
virtual = wan_ip:80
        real = node1:80 masq   10
        real = node2:80 masq   10
        real = node3:80 masq   10
        real = node4:80 masq   10
        real = node5:80 masq 1000
        scheduler = sed
        persistent = 300
        protocol = tcp
        request = "/testpage.html"
        receive = "This server seems to be up."
See ipvsadm(8) and ldirectord(8) for details on what these
options mean.
I want to bring the last block to your attention. That is the setup for
WebMail. The persistent option is required to keep users going to the
web server when the log in. That is needed to preserve session information.
Turn on IP Forwarding
We need to turn on IP forwarding in /etc/conf.d/iptables or the
NAT won't work.
ENABLE_FORWARDING_IPv4="yes"
Setup iptables
Speaking of NAT, I added a NAT rule to iptables. I'm still not 100% sure
that it was needed, but it doesn't seem to hurt.
iptables -A POSTROUTING -s lan_net/255.255.255.0 -j MASQUERADE
/etc/init.d/iptables save
Turning on Services
Everything is configured, let's turn things on so that we test.
/etc/init.d/iptables start
/etc/init.d/heartbeat start
/etc/init.d/ldirector start
Use ipvsadm to view the status of the load balancer.
# ipvsadm
IP Virtual Server version 1.2.0 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  wan_ip:smtp sed
  -> mail4:smtp                   Masq    667    22         99        
  -> mail3:smtp                   Masq    1000   34         130      
  -> mail2:smtp                   Masq    1000   28         193      
  -> mail1:smtp                   Masq    1000   28         104      
TCP  wan_ip:www sed persistent 300
  -> mail5:www                    Masq    1000   6          18        
  -> mail4:www                    Masq    10     0          0         
  -> mail3:www                    Masq    10     0          0         
  -> mail2:www                    Masq    10     0          0         
  -> mail1:www                    Masq    10     0          0         
TCP  wan_ip:pop3 sed
  -> mail4:pop3                   Masq    667    2          46        
  -> mail3:pop3                   Masq    1000   3          54        
  -> mail2:pop3                   Masq    1000   3          21        
  -> mail1:pop3                   Masq    1000   2          43        
TCP  wan_ip:imap2 sed
  -> mail4:imap2                  Masq    1000   2          0         
  -> mail3:imap2                  Masq    1000   1          1         
  -> mail2:imap2                  Masq    1000   1          0         
  -> mail1:imap2                  Masq    1000   0          3      
Testing
Now that everything is up, it's time to make sure it works. Use a client
or telnet to the wan_ip from outside the cluster. You should be able to see
your connection in the server logs.
Making it Permanent
At this point you should have a working system and it's now time to make
sure all the services we need will be started when the machine reboots.
rc-update add iptables default
rc-update add heartbeat default
And that, as they say, is that.
               
               
               

本文来自ChinaUnix博客，如果查看原文请点：http://blog.chinaunix.net/u/17006/showart_144975.html