- 论坛徽章:
- 0
|
期间 感谢wysilly的无私帮助 谢谢
声明:以下实现受限于我实际中具体硬件及具体需求 仅供参考 网上apache install文章很多 本文不在赘述 本人水平尚浅 欢迎指出文中不当或错误之处
机器有限(3台机器) 架构如下 3台服务器均是 centos4.4 2.6.9 硬盘40G
node1 eth0 10.0.0.1 eth1 192.168.1.1
node2 eth0 10.0.0.2 eth1 192.168.1.2
node3 eth0 192.168.1.3
node1/2 eth1 为对内IP eth0为对外IP 虚拟出的IP为10.0.0.3
Internet
|
-----------------------------------------------------
LDirector/RealServer(node1) LinuxDirector/RealServer(node2)
| |
-----------------------------------------------------
|
Coda Server(node3)
负载均衡的部分按照http://www.ultramonkey.org/3/topologies/sl-ha-lb-eg.html
1>在node1 和 node2 上安装
rpm -ivhhttp://dev.centos.org/centos/4/t ... el4.centos.i386.rpm
yum install net-snmp-libs
rpm -ivhhttp://dev.centos.org/centos/4/t ... el4.centos.i386.rpm
yum install gnutls
rpm -ivhhttp://dev.centos.org/centos/4/t ... el4.centos.i386.rpm
yum install perl-Crypt-SSLeay
yum install perl-HTML-Parser
yum install perl-LDAP
yum install perl-Net-DNS
yum install perl-libwww-perl
yum install perl-Net-IMAP-Simple-SSL
yum install perl-Net-IMAP-Simple
yum install perl-MailTools
rpm -ivhhttp://ftp.riken.go.jp/pub/Linux ... 2.el4.rf.noarch.rpm
rpm -ivhhttp://ftp.riken.go.jp/pub/Linux ... 1.rh9.rf.noarch.rpm
rpm -ivhhttp://dev.centos.org/centos/4/t ... el4.centos.i386.rpm
rpm -ivh http://mirror.centos.org/centos/ ... adm-1.24-6.i386.rpm
rpm -ivhhttp://dev.centos.org/centos/4/t ... el4.centos.i386.rpm
yum install arptables_jf
yum install libnet
2>node1 和 node2 上均如此配置
##### /etc/ha.d/ldirectord ###########
node1 ldirectord::ldirectord.cf LVSSyncDaemonSwap::master IPaddr2::10.0.0.3/24/eth0/10.0.0.255
##### /etc/ha.d/ldirectord.cf##########
checktimeout=3
checkinterval=1
#fallback=127.0.0.1:80
autoreload=yes
logfile="/var/log/ldirectord.log"
#logfile="local0"
emailalert="**@**.com"
emailalertfreq=7200
#emailalertstatus=all
quiescent=yes
# Sample for an http virtual service
virtual=10.0.0.3:80
real=10.0.0.1:80 gate
real=10.0.0.2:80 gate
fallback=127.0.0.1:80
service=http
request="test.html"
receive="hello"
###注意: 设http主目录在/var/www/html 则需建立文件 /var/www/html/test.html 文件内容为hello
#virtualhost=www.***.com
scheduler=rr
#persistent=600
#netmask=255.255.255.255
protocol=tcp
checktype=negotiate
checkport=80
######## /ect/ha.d/ha.cf #########
keepalive 2
deadtime 30
warntime 10
initdead 120
udpport 694
ucast Internal_IP_Address
auto_failback on
node node1
node node2
#########/etc/sysconfig/network-scripts/ifcfg-lo:0#######
DEVICE=lo:0
IPADDR=10.0.0.1
NETMASK=255.255.255.255
NETWORK=10.0.0.255
BROADCAST=10.0.0.255
ONBOOT=yes
NAME=loopback
#################################################################
3>启动 (node1 与 node2 均如此启动)
/etc/init.d/heartbeat stop
service ldirectord stop
/etc/ha.d/resource.d/LVSSyncDaemonSwap master stop
/etc/init.d/arptables_jf stop
/usr/sbin/arptables-noarp-addr 10.0.0.1 start
/etc/init.d/arptables_jf save
/etc/init.d/arptables_jf start
/etc/init.d/heartbeat/start
ifup lo
4>注意
ip addr sh 检查IP分配是否正确
ipvsadm -L -n 检查服务情况
/etc/ha.d/resource.d/LVSSyncyncDaemonSwap masterstatus 检查均衡情况,正确应该是node1 master上为on, node2 Slave上为off
特别注意认真阅读文档!!!
Note that a netmask of 255.255.255.255 on the lo:0interface indicates that this interface will only accept traffic 192.168.6.240.
heartbeat启动后4个小时左右 所有访问都连接到一台机器上 如果这台机器死掉 外部访问中断; IPVSADM显示 静止的数据
猜测是连接internets与server之间路由器的问题 ARP超时 此字段表示当非活动时，ARP项在清洗之前保留于缓存中的时间长度。ARP超时的默认值为4个小时，如上面例子所示,可以通过使用ARP timeout命令调整 ARP缓存项在缓存中的时间长度。此命令格式如：
arp timeout seconds应该是路由器保存vip的 mac 4个小时,后,再发arp包,vip的server不回应,导致该路由器找寻不到vip,导致不能访问 解决办法暂时是吧active ldirectord/real server上不执行arptables_jf 等配置语句 但是在stand-by ldirectord/real server上执行
测试的时候 不能同时ping active ldirectord/real server的 eth0 和 eth0:0 你会发现两个丢包都很厉害 只ping eth0:0 测试无丢包即可。
#####################
coda分布式文件系统分为服务器和客户端，服务器又分为SCM(SystemControl Machine) server和non-SCM server,SCM服务器负责与其它服务器同步，SCM服务器只有一个，而non-SCM服务器可以有多个。
客户端在与服务器建立连接之后会在本地生成一个新的文件系统/coda，这个文件系统下的所有文件是与服务器上的一个用户数据目录保持同步的，服务器上的用户数据目录是在安装服务器软件时指定专门用来存放用户数据的，即在客户端的/coda目录下保存的所有文件，实际上都是服务器上的一个备份，只要服务器上的文件发生了变化，则/coda目录下的文件也会发生相应的变化，这就是说，只要有一个客户端操作了/coda下的某个文件，则所有客户端/coda下的对应文件就都要发生变化，同时服务器之间也要保持同步。
coda server : node3
coda client : node1 node2
配置 coda server 先重装系统 规划分区如下所示(rvm log / data可以以文件形式存在,但会导致客户端读取速度非常之慢 参考: http://www.coda.cs.cmu.edu/doc/html/manual/c1140.html )
swap 512 M
/boot 256 M
/var 5G
/ 7G
/vice 500 M
/vicepa 19G
RVMlog 15M
RVMData 600 M
接下来就是安装,相关软件下载地址:http://gd.tuwien.ac.at/utils/fs/coda/linux/FedoraCore5/
ftp://194.199.20.114/linux/fedor ... ent/i386/os/Fedora/
rpm -ivh
lwp-devel-2.3-1.i386.rpm lwp-2.3-1.i386.rpm rvm-devel-1.14-1.i386.rpmrvm-1.14-1.i386.rpm rvm-tools-1.14-1.i386.rpm rpc2-devel-2.5-1.i386.rpmcompat-libstdc++-33-3.2.3-47.3.i386.rpm coda-server-6.1.2-1.i386.rpm
各自系统不同 所需依赖软件包要靠自己一一查找 本文不在赘述
运行vice-setup
Welcome to the Coda Server Setup script!
Setting up config files for a coda server.
Do you want the file /etc/coda/server.conf created?[yes] yes
What is the root directory for your coda server(s)?[/vice] /vice
Setting up /vice.
Directories under /vice are set up.
Is this the master server, aka the SCM machine? (y/n)y（第一次安装是scm的）
Setting up tokens for authentication.
The following token must be identical on all servers.
Enter a random token for update authentication :helloword
The following token must be identical on all servers.
Enter a random token for auth2 authentication :helloword
The following token must be identical on all servers.
Enter a random token for volutil authentication : helloword
tokens done!
Setting up the file list for update client
Filelist for update ready.
/etc/services already has new services registered!Good.
/etc/services ready for Coda
Now installing files specific to the SCM...
Setting up servers file.
Enter an id for the SCM server. (hostname distfs)
The serverid is a unique number between 0 and 255.
You should avoid 0, 127, and 255.
serverid: 1（1即为hosts中的第一个记录）
done!
Initializing the VSGDB to contain the SCM as E0000100
/vice/db/VSGDB set up
Setting up ROOTVOLUME file
Enter the name of the rootvolume (< 32 chars) :codaroot
Setting up users and groups for Coda
You need to give me a uid (not 0) and username (notroot)
for a Coda System:Administrator member on this server,
(sort of a Coda super user)
Enter the uid of this user: 666
Enter the username of this user: codaroot
Going to rebuild the protection databases
moving /vice/db/prot_users.db to/vice/db/prot_users.db.old
moving /vice/db/prot_index.db to /vice/db/prot_index.db.old
An initial administrative user codaroot (id 666)
with Coda password changeme now exists.（默认的用户和口令）
A server needs a small log disk partition, preferablyon a disk by
itself. It also needs a metadata partition of approx4% of your filespace.
For trial purposes you may give ordinary files insteadof raw
partitions. Keep all size small if you do this.
Production servers will want partitions for speed.
---------------------------
WARNING: you are going to play with your partitionsnow.
verify all answers you give.
---------------------------
WARNING: these choices are not easy to change once youare up and running.
Are you ready to set up RVM? [yes/no] yes
What is your log partition?/vice/logs/log_disk_file.log(在安装前建好的日志目录)
The log size must be smaller than you logpartition.
We
recommend not more than 30M log size, and 2M is a goodchoice.
What is your log size? (enter as e.g. '2M') 15M
What is your data partition (or file)?/vice/logs/data_disk_file.log
The data size must be approx 4% of you server filespace.
We
have templates for servers of approx: 500M, 1G, 2.2G,3.3G, 8G
(you can store less, but not more on such servers).
The corresponding data sizes are
22M, 44M, 90M, 130M, 315M.
Pick one of the defaults, otherwise I will bail out
What is the size of you data partition (or file)
[22M, 44M, 90M, 130M, 200M, 315M]: 500M
-----------------------------
WARNING: DATA and LOG partitions are about to bewiped.
log area:/usr/coda/logs/log_disk_file.log, size 15M.
data area:/usr/coda/logs/data_disk_file.log, size 500M.
Proceed, and wipe out old data? [y/n] y
LOG file has been initialized!
Rdsinit will initialize data and log.
This takes a while.
rvm_initialize succeeded.
Going to initialize data file to zero, could takeawhile.
done.
rds_zap_heap completed successfully.
rvm_terminate succeeded.
RVM setup is done!
Your server directories will hold the files (notdirectories).
You can currently only have one directory per diskpartition.
Where shall we store your file data [/vicepa]? /vicepa
Shall I set up a vicetab entry for /vicepa (y/n) y
Select the maximum number of files for the server.
[256K, 1M, 2M, 16M]: 16M
Server directory /vicepa is set up!
Congratulations: your configuration is ready...and now
to get going do the following:
start theauth2 server as: auth2
startrpc2portmap as: rpc2portmap
startupdatesrv as: updatesrv
startupdateclnt as:
updateclnt -h gtest
start thefileserver: startserver &
wait until theserver is up: tail -f /vice/srv/SrvLog
create yourroot volume: createvol_rep codaroot E0000100 /vicepa
setup aclient: venus-setup gtest 20000
start venus:venus
enjoy Coda.
for moreinformation see http://www.coda.cs.cmu.edu.
安装配置 coda client 相关软件下载地址http://linux.web.psi.ch/dist/scientific/43/kernel/all/?C=S;O=A
ftp://194.199.20.114/linux/fedor ... ent/i386/os/Fedora/
rpm -ivh lwp-devel-2.3-1.i386.rpm lwp-2.3-1.i386.rpm rvm-devel-1.14-1.i386.rpmrvm-1.14-1.i386.rpm rvm-tools-1.14-1.i386.rpm rpc2-devel-2.5-1.i386.rpm rpc2-2.5-1.i386.rpm compat-libstdc++-33-3.2.3-47.3.i386.rpm coda-client-6.1.2-1.i386.rpm kernel-module-coda-2.6.9-42.0.10.ELsmp-6.4-1.sl4.psi.i686.rpm
insmod /lib/modules/2.6.9***.ELsmp/kernel/fs/coda/coda.ko
vutil shutdown
umount /coda
venus-setup coda-server-name 200000
venus -maxworkers 100 &
clog codaroot
password: changeme
cfs wr
需要注意的几个设置
*设置连接方式
cfs wr
*设置客户端处理访问/coda的venus并发线程数
venus-maxworkers 100
*设置客户端的缓存200MB
venus-setupCoda_Server 200000
*无法认证 出现Invalid login (RPC2_NOBINDING(F))错误
检查codaserver上的iptables设置
*在客户端上执行 cfs lv /coda/*** 或者 cfs la /coda/*** 显示 disconnected或 connection time out, 表示该客户端由于网络错误或文件冲突而导致和服务器连接断开 暂时解决办法(注意 是暂时解决办法) 重新连接客户端 建立检测脚本 当coda客户端呈断开状态时 自动连接 加入定时执行即可 crontab -e
*/5 * * * */ chkcoda.sh
#####chikcoda.sh##########
#!/bin/bash
result=`cfs lv /coda/***/ |grep Connection|awk '{print$4}'`
case $result in
Connected)
exit 0
;;
Disconnected)
/test.sh
exit 0
;;
WriteDisconnected)
cfs wr
esac
#######test.sh (先 yum install expect)########
#!/usr/bin/expect
spawn cfs purgeml /coda/***
expect "]"
exec sleep 1
send "y\r"
expect eof
exit
################
暂时实现的是在 10.0.0.1 和 10.0.0.2 上分别安装 分别查看结果 两台机器可以日志合并进而分析
AWStats是在Sourceforge上发展很快的一个基于Perl的WEB日志分析工具。相对于另外一个非常优秀的开放源代码的日志分析工具Webalizer，AWStats的优势在于：
1. 界面友好：可以根据浏览器直接调用相应语言界面（有简体中文版）
参考输出样例：http://www.chedong.com/cgi-bin/awstats/awstats.pl?config=chedong
2. 基于Perl：并且很好的解决了跨平台问题，系统本身可以运行在GNU/Linux上或Windows上（安装了ActivePerl后）；分析的日志直接支持Apache格式 (combined)和IIS格式(需要修改)。Webalizer虽然也有Windows平台版，但目前已经缺乏 维护；
AWStats完全可以实现用一套系统完成对自身站点不同WEB服务器：GNU/Linux/Apache和Windows/IIS服务器的统一统计。
3. 效率比较高：AWStats输出统计项目比Webalizer丰富了很多，速度仍可以达到Webalizer的1/3左右，对于一个日访问量 百万级的站点，这个速度都是足够的；
4. 配置/定制方便：系统提供了足够灵活但缺省也很合理的配置规则，需要修改的缺省配置不超过3，4项就可以开始运行，而且修改和扩展的插件还是 比较多的；
5. AWStats的设计者是面向精确的"Human visits"设计的，因此很多搜索引擎的机器人访问都被过滤掉了，因此有可能比其他日志统计工具统计的数字要低，来自公司内部的访问也可以通过IP过滤 设置过滤掉。
6. 提供了很多扩展的参数统计功能：使用ExtraXXXX系列配置生成针对具体应用的参数分析会对产品分析非常有用。
更多与其他工具：Webalizer, analog的比较请参考：
http://awstats.sourceforge.net/#COMPARISON
环境 centos4.4
1>安装所有的软件包
wget http://www.awstats.cn/files/awstats-6.6.tar.gz
rpm -ivh http://dag.wieers.com/rpm/packag ... 2-1.el4.rf.i386.rpm
rpm -ivh http://www.silfreed.net/download ... .6.2-1.el4.i386.rpm
yum install GeoIP Geo-IP
rpm -ivh http://mirrors.ircam.fr/pub/dag/ ... 2.el4.rf.noarch.rpm
2>安装
tar -zxvf awstats-6.6.tar.gz
mv awstats-6.6 /var/www/html/awstats
3>配置http log
更改httpd的log方式
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
SetEnvIf Request_URI \.gif$ gif-image
SetEnvIf Request_URI \.GIF$ gif-image
SetEnvIf Request_URI \.jpg$ gif-image
SetEnvIf Request_URI \.JPG$ gif-image
SetEnvIf Request_URI \.png$ gif-image
SetEnvIf Request_URI \.swf$ gif-image
SetEnvIf Request_URI \.SWF$ gif-image
SetEnvIf Request_URI \.css$ gif-image
SetEnvIf Request_URI \.CSS$ gif-image
SetEnvIf Request_URI \.js$ gif-image
SetEnvIf Request_URI \.JS$ gif-image
SetEnvIf Request_URI \.ico$ gif-image
SetEnvIf Remote_Addr "10\.0\.0\.1" gif-image
SetEnvIf Remote_Addr "10\.0\.0\.2" gif-image
ErrorLog "|/usr/sbin/cronolog /var/log/httpd/hehehe.%Y-%m-%d-error_log"
CustomLog "|/usr/sbin/cronolog /var/log/httpd/hehehe.%Y-%m-%d-access_log" combined env=!gif-image
###屏蔽掉1和2的http log,因为两个IP做heartbeat,产生大量垃圾log
4>配置awstats
perl /var/log/www/html/awstats/tools/awstats_configure.pl
###默认生成的配置文件放在/etc/awstats/下
mkdir /etc/awstats/data
###建立data目录放置一些必要的数据文件
5>更新GeoIP库 每月定期下载更新
wget http://www.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
wget http://www.maxmind.com/download/geoip/database/GeoIP.dat.gz
gunzip GeoLiteCity.dat.gz
gunzip GeoIP.dat.gz
mv GeoLiteCity.dat /etc/awstats/data/
mv GeoIP.dat /etc/awstats/data/
chmod 777 /etc/awstats/data/Geo*
6>编辑/etc/awstats/awstats.www.hehehe.com.conf文件
*设置httpd logfiule的位置
LogFile="/var/log/httpd/hehehe.%YYYY-4-%MM-2-%DD-2-access_log"
*设置要分析的服务日志类型 w表示为web日志
LogType=W
*设置日志格式 1表示采用 NCSA apache combined/ELF/XLF log format
LogFormat=1
*设置要分析的域名
SiteDomain="www.hehehe.com"
HostAliases="hehehe.com www.hehehe.com 127.0.0.1 localhost"
*设置awstats的数据库存放目录
DirData="/etc/awstats/data"
*设置DirCgi
DirCgi="/var/www/html/awstats/wwwroot/cgi-bin"
DirIcons="/var/www/html/awstats/wwwroot/icon"
*设置通过浏览器刷新页面
AllowToUpdateStatsFromBrowser=1
*设置使用IP与地理名解析
LoadPlugin="geoip GEOIP_STANDARD /etc/awstats/data/GeoIP.dat"
LoadPlugin="geoip_city_maxmind GEOIP_STANDARD /etc/awstats/data/GeoLiteCity.dat"
7>修改权限
chmod -R 755 /var/www/html/awstats /etc/awstas/data
chmod 777 /var/www/html/awstats/wwwroot/cgi-bin/*.pl
8>创建awstats初始化数据库并建立awstats初始化静态统计页面
perl awstats.pl -config=www.hehehe.com -update -lang=cn
perl awstats.pl -config=www.hehehe.com -output -staticlinks -lang=cn>awstats.hehehe.html
9>设置定时更新日志统计
crontab -e
*/5 * * * * /var/www/html/awstats/wwwroot/cgi-bin/awstats.pl -config=www.hehehe.com -update -lang=cn > /dev/null
10>启动apache并查看结果
/usr/local/apache/bin/apachectl start
http://*.*.*.*/awstata/awstats.pl?config=www.hehehe.com
参考:
http://blog.csdn.net/ronaldchan2 ... /03/10/1526054.aspx
http://www.chedong.com/tech/awstats.html
[ 本帖最后由 agg230 于 2007-10-18 11:30 编辑 ] |
|
免费注册
发表于 2007-08-15 15:12
