lvs-dr 模式异常

greysky-zfj

lvs-dr模式对 80 端口做了集群，使用rr算法。在dr上用 ipvsadm 命令查看，一切正常，3台realserver均在其中。

[root@localhost ~]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddressort Scheduler Flags
  -> RemoteAddressort           Forward Weight ActiveConn InActConn
FWM  3 rr
  -> x. x. x. x:80                          Local   1      0          0         
  -> x. x. x. x:80                         Route   1      0          0         
  -> x. x. x. x:80                         Route   1      0          0     


从分配的连接数上可以看出有异常，但其实2台realserver并未有任何连接被分配，所有的连接都被分配到第三台，而且每次都是这样。重启keepalived服务后，恢复正常。keepalived的日志中并未有任何异常信息。


May  6 15:17:55 localhost Keepalived[19816]: Starting Keepalived v1.2.6 (01/22,2013)
May  6 15:17:55 localhost Keepalived[19817]: Starting Healthcheck child process, pid=19818
May  6 15:17:55 localhost Keepalived[19817]: Starting VRRP child process, pid=19819
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Initializing ipvs 2.6
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Interface queue is empty
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: No such interface, eth1
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Netlink reflector reports IP x. x. x. x added
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Netlink reflector reports IP fe80::2e:4fff:fec:436 added
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Registering Kernel netlink reflector
May  6 15:17:55 localhost Keepalived_vrrp[19819]: Interface queue is empty
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Registering Kernel netlink command channel
May  6 15:17:55 localhost Keepalived_vrrp[19819]: No such interface, eth1
May  6 15:17:55 localhost Keepalived_vrrp[19819]: Netlink reflector reports IP x. x. x. x added
May  6 15:17:55 localhost Keepalived_vrrp[19819]: Netlink reflector reports IP fe80::2e:4fff:fec:436 added
May  6 15:17:55 localhost Keepalived_vrrp[19819]: Registering Kernel netlink reflector
May  6 15:17:55 localhost Keepalived_vrrp[19819]: Registering Kernel netlink command channel
May  6 15:17:55 localhost Keepalived_vrrp[19819]: Registering gratuitous ARP shared channel
May  6 15:17:55 localhost Keepalived_vrrp[19819]: Initializing ipvs 2.6
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Opening file '/etc/keepalived/keepalived.conf'.
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Configuration is using : 34487 Bytes
May  6 15:17:55 localhost Keepalived_vrrp[19819]: Opening file '/etc/keepalived/keepalived.conf'.
May  6 15:17:55 localhost Keepalived_vrrp[19819]: Configuration is using : 62292 Bytes
May  6 15:17:55 localhost Keepalived_vrrp[19819]: Using LinkWatch kernel netlink reflector...
May  6 15:17:55 localhost Keepalived_vrrp[19819]: VRRP sockpool: [ifindex(2), proto(112), fd(11,12)]
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Using LinkWatch kernel netlink reflector...
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Activating healthchecker for service [x. x. x. x]:80
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Activating healthchecker for service [x. x. x. x]:80
May  6 15:17:55 localhost Keepalived_healthcheckers[19818]: Activating healthchecker for service [x. x. x. x]:80
May  6 15:17:56 localhost Keepalived_vrrp[19819]: VRRP_Instance(VI_1) Transition to MASTER STATE
May  6 15:17:57 localhost Keepalived_vrrp[19819]: VRRP_Instance(VI_1) Entering MASTER STATE
May  6 15:17:57 localhost Keepalived_vrrp[19819]: VRRP_Instance(VI_1) setting protocol VIPs.
May  6 15:17:57 localhost Keepalived_vrrp[19819]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for x. x. x. x
May  6 15:17:57 localhost Keepalived_healthcheckers[19818]: Netlink reflector reports IP x. x. x. x added
May  6 15:18:02 localhost Keepalived_vrrp[19819]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for x. x. x. x

求解

xiaodylan

"其实2台realserver并未有任何连接被分配" 这个在哪里看出来的？

greysky-zfj

回复 2# xiaodylan


    在这2台realserver上看的 80端口没有任何连接，而第三台却有大量的80端口连接

xiaodylan

在lvs上直接telnet realserver的80 试试 可否连接呢？

greysky-zfj

回复 4# xiaodylan


    可以

xiaodylan

那如果可以的话，我觉得应该是配置的问题，能否贴一下配置？

greysky-zfj

回复 6# xiaodylan


    ! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        x.x.x.250    }
}

virtual_server fwmark 3 {
    delay_loop 3
    lb_algo rr
    lb_kind DR
    #  persistence_timeout 50
    protocol TCP

    real_server x.x.x..132 80 {
      MISC_CHECK {
           misc_path "/lvs/checkHTTP.sh x.x.x.132"
           misc_timeout 6
           nb_get_retry 3
                 }
    }
    real_server x.x.x.136 80 {
        MISC_CHECK {
           misc_path "/lvs/checkHTTP.sh x.x.x.136"
           misc_timeout 2
           nb_get_retry 2
                   }
    }
    real_server x.x.x.144 80 {
        MISC_CHECK {
           misc_path "/lvs/checkHTTP.sh x.x.x.144"
           misc_timeout 2
           nb_get_retry 2
                   }
    }
}

xiaodylan

有人遇到过这样的情况"以前一直使用tcp_check，现在准备换成misc_check用于动态调整权重,在我的rs经历一次reboot后，lvs没有恢复其权重，而是把权重变成了0" 但是和你的情况不太一样，我不知道你尝试过试用tcp_check么？

greysky-zfj

回复 8# xiaodylan


    没有
不过 既然  ipvsadm
显示是realserver存在的，就应该往上面分配啊

xiaodylan

你说的重启过后就恢复正常是什么意思？