GFS(DLM)+CLVM+ISCSI和NFS性能测试

rushmeplz

GFS（DLM MODE）和NFS关于IO性能测试，涉及到的参数：write,rewrite,read,reread,测试工具IOZONE.GFS(GULM MODE)因为性能，不做比较。以下是关于各参数的定义


Write: This test measures the performance of writing a new file. When a new file is written not only does the data need to be stored but also the overhead information for keeping track of where the data is located on the storage media. This overhead is called the “metadata” It consists of the directory information, the space allocation and any other data associated with a file that is not part of the data contained in the file. It is normal for the initial write performance to be lower than the performance of rewriting a file due to this overhead information.

Re-write: This test measures the performance of writing a file that already exists. When a file is written that already exists the work required is less as the metadata already exists. It is normal for the rewrite performance to be higher than the performance of writing a new file.

Read: This test measures the performance of reading an existing file.

Re-Read: This test measures the performance of reading a file that was recently read. It is normal for the performance to be higher as the operating system generally maintains a cache of the data for files that were recently read. This cache can be used to satisfy reads and improves the performance.

GFS(DLM)安装
1.rpm 安装顺序
kernel-smp
perl-net-telnet
magma
magma-devel
ccs
ccs-devel
cman-kernel
cman-kernheaders
cman
cman-devel
cman-kernel-smp
dlm-kernel
dlm-kernheaders
dlm
dlm-devel
dlm-kernel-smp
fence
gfs
gfs-kernel
gfs-kernheaders
gfs-kernel-smp
iddev
iddev-devel
magma-plugins
rgmanager
system-config-cluster
ipvsadm

2.配置
/etc/hosts添加节点的hostname,各个节点配置应该相同

进入system-config-cluster 进行cluster xml文件的配置，添加cluster name,node name&vote
(推荐各个节点vote相同，以此vote的总和来决定这个cluster是不是quorate,quorate为vote总数/2+1,因此2节点的DLM当1个节点失效时，此cluster为inquorate,除非有quorum disk)fence domain,各个节点的fence level,fence device

同步此cluster.conf到各个节点

此测试用软ISCSI，在ISCSI TARGET 安装scsi-target(tgtd),配置ISCSI TARGET
tgtadm --lld iscsi --op new --mode target --tid 1 -T iqn.2008-12.sys.disk1.lun1
tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b /dev/sdb
tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL (ACL)

各个节点安装iscsi initiator,as 4 修改、/etc/iscsi.conf discovery address,
5 iscsiadm -m discovery -t st -p ip
  iscsiadm -m node -T iqn -l

在各节点加载cluster相关modules
modprobe lock_dlm
modeprobe cman
modprobe gfs

启动CCS(cluster config manager) ccsd
加入cluster cman_tool join
cat /proc/cluster/status 查看个节点状态

加入fence domain
fence_tool join

为iscsi disk创建lv
pvcreate /dev/sdb
vgcreate vg01 /dev/sdb
lvcreate -n lv01 -L 20G vg01
安装lvm-cluster,使得lvm可以跨越节点，启动clvmd,lvmdiskscan可以找到相关lv

建立gfs文件系统
gfs_mkfs -p lock_dlm -t clustername:gfs -j 4 /dev/vg01/lv01

在各个节点挂载gfs
mount -t gfs /dev/vg01/lv01 /gfstest

到此，gfs已经建立，下面进行测试

安装iozone，测试脚本

/opt/iozone/bin/iozone -Reb gfs.xls -i 0 -i 1 -t 5 -F
/gfstest/thread1 /gfstest/thread2 /gfstest/thread3 /gfstest/thread4 /gfstest/thread5

5线程测试

3 nfs测试

为测试硬盘清除分区表 dd if=/dev/zero of=/dev/sdb bs=512 count=1

fdisk后编辑/etc/exports   /gfstest  *(rw,sync,root_squash) 注意权限,sync和async都测试
sync 数据写入硬盘同时回应写请求
async 数据在内存cache时就回应写请求

ok,应用前面的脚本进行测试，并发读，并发写，并发读写

4 结论

GFS(DLM)的并发写更加出色，nfs的并发读更加出色，当然，这个是纯软件环境，实际项目中根据存储的不同，2者性能会有起伏。





本文来自ChinaUnix博客，如果查看原文请点：http://blog.chinaunix.net/u2/89897/showart_1773750.html