Chinaunix

标题: Redhat Enterprise Advanced Server 3.0性能问题 [打印本页]

作者: luomingxue 时间: 2004-03-11 13:46
标题: Redhat Enterprise Advanced Server 3.0性能问题
装了台rhas3.0
Intel(R) Pentium(R) 4 CPU 3.06GHz M
80G SOFTWARE RAID 1
感觉比以前的RH0慢很多。

设了一下硬盘
/sbin/hdparm -d1 -X68 -c3 -m16 /dev/hdc
/sbin/hdparm -d1 -X68 -c3 -m16 /dev/hdc

/sbin/hdparm -Tt /dev/md0
测了一下反而比以前慢了，是和RAID 有关吗？

想请教大侠怎么找出瓶颈，
比如，一段时间内哪个PROCESS占很多CPU，MEMORY？
多谢

当了个PROCINFO
不太明白OUTPUT：
[ procinfo-18]# ./procinfo
Linux 2.4.21-9.0.1.ELsmp (bhcompile@bugs.devel.redhat.com) (gcc 3.2.3 20030502 ) #1 1CPU

Memory:    Total       Used       Free    Shared    Buffers    Cached
Mem:       768688    747932    20756          0    17824    596108
Swap:    2047672    96020    1951652

Bootup: Mon Mar  8 21:54:26 2004 Load average: 2.64 2.48 2.57 1/111 459

user  :    1:11:20.06 2.3%  page in :  8859158  disk 1: 601369r20251237w
nice  :    0:00:51.68 0.0%  page out: 96698683
system:    1:07:23.13 2.2%  swap in : 28597
idle  : 1d  7:19:32.01  61.8%  swap out: 60001
uptime: 2d  2:41:47.58       context : 57427380

irq  0:  18250759 timer                irq 14:  20887853 ide0
irq  1:    8717 keyboard             irq 15:  22888805 ide1
irq  2:       0 cascade [4]          irq 16:       0 usb-uhci
irq  4:       4                      irq 17:       0 Intel ICH4
irq  6:       5                      irq 18: 4554856 usb-uhci, eth0
irq  7:       1                      irq 19:       0 usb-uhci
irq  8:       1 rtc                irq 23:       0 ehci-hcd

作者: wolfop 时间: 2004-03-11 13:52
标题: Redhat Enterprise Advanced Server 3.0性能问题
一般不用去设置hdparam，正常情况内核应该可以自动配置的。
你知道你干了什么吗？
-X68是DMA66模式，也许系统自己认的是DMA100模式。

作者: starferry 时间: 2004-03-11 13:58
标题: Redhat Enterprise Advanced Server 3.0性能问题

原帖由 "luomingxue" 发表：
   irq 16:       0 usb-uhci
irq  4:       4                      irq 17:       0 Intel ICH4
irq  6:       5                      irq 18: 4554856 usb-uhci, eth0
irq  7:       1    ..........

强啊，楼主，4个CPU的机器还是IDE硬盘(/dev/hdc)？
hdparm只用在IDE硬盘上。

作者: hmkart 时间: 2004-03-11 14:07
标题: Redhat Enterprise Advanced Server 3.0性能问题
你用ide的做raid 1我是认为一定会慢的.
当初我的2个IDE的硬盘,作为图书馆的SQL数据库.
运行在WIN2K下,非常慢,后面改为RAID 0 后,速度快来很多.
当然因为当时每天都有打量的新书入库,时刻的图书借阅信息更新.
所以 I/0 比较多.
如果可以,你可以去掉1块硬盘,或者不要使用RAID,
再来看看你的速度,相信一定会有很大提高的,你这么高的设备.
哈哈!
Gook Luck,
对了,哪个什么IDE参数不用调了.

作者: luomingxue 时间: 2004-03-11 14:15
标题: Redhat Enterprise Advanced Server 3.0性能问题
google了一下没找到。
DMA100模式参数是什么？多谢，

作者: starferry 时间: 2004-03-11 14:17
标题: Redhat Enterprise Advanced Server 3.0性能问题

原帖由 "starferry" 发表：

强啊，楼主，4个CPU的机器还是IDE硬盘(/dev/hdc)？
hdparm只用在IDE硬盘上。

SORRY,看错了，原来是P4
如果你的RAID1是由一个硬盘的不同分区建立的，那就会变慢啊。
如果在不同物理硬盘上应该变快。
还有软件RAID可靠性不高，建议不要使用。

作者: luomingxue 时间: 2004-03-11 14:26
标题: Redhat Enterprise Advanced Server 3.0性能问题
就一个CPU，装的时候不知怎么kernel识别成多cpu了
用RAID 1是想要有备份。

不过好像disk I/O 是有问题。
老RH9(也是RAID 1)：
/dev/md2:
Timing buffer-cache reads: 128 MB in  0.45 seconds =284.44 MB/sec
Timing buffered disk reads:  64 MB in  2.13 seconds = 30.05 MB/sec

新AS3系统
/sbin/hdparm -Tt /dev/md0

/dev/md0:
Timing buffer-cache reads: 1520 MB in  2.00 seconds = 760.00 MB/sec
Timing buffered disk reads: 66 MB in  8.67 seconds = 7.61 MB/sec

作者: luomingxue 时间: 2004-03-12 06:06
标题: Redhat Enterprise Advanced Server 3.0性能问题
made some progress, use top, we can see
CPU states:  cpu user nice  system irq  softirq  iowait idle
         total 0.0% 0.0% 0.2% 3.0%    0.0% 96.8% 0.0%

So it is I/O problem.

3799 mysql    15 0 12104 mysqld
   20 root    15 0 0       kjournald
1 root    15 0 500  500 init
2 root    15 0    0 0    keventd
3 root    15 0    0 0    kapmd
looks like Kjournald is a bad guy. will investigate further.
Anybody knows how to turn Kjournald?

作者: luomingxue 时间: 2004-03-12 12:06
标题: Redhat Enterprise Advanced Server 3.0性能问题
update:

1. Seem the default RHEL 3.0 kernel has some problem:
disk I/O performance with the 2.4.21 redhat distribution kernel
http://www.redhatconfig.com/msg/65280.html
"Is anybody aware of any changes in the 2.4.21 kernel(redhat
2.4.21-4.0.1.ELsmp kernel distribution) which affects disk I/O badly? We
are noticing that the cpu taken up by iowait 's is causing significant
degradation in disk performance"

I'm doing research on how to change kernel now.

2. more discussion about tools monitoring disk i/o.
http://jeremy.zawodny.com/blog/archives/000814.html

sysstat is a useful tools,
http://www.icewalkers.com/Linux/Software/59040/sysstat.html
you have to do make and make install

sample display:
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
hda 9.24 3.76 1.04 0.69 80.98 35.67 40.49 17.84 67.42 0.09 57.25 25.89 4.48

作者: luomingxue 时间: 2004-03-13 00:34
标题: Redhat Enterprise Advanced Server 3.0性能问题
change kernel:
tried RH9 kernel 2.4.20, rpm --force -i kernel-a.rpm to downgrade it. But XWindow can not start, black screen.

Try Fedora's kernel 2.4.22, still high iowait time.

I need to compile kernel myself??

Will get 1G memory this afternoon ( now is 768M), see if that helps.

作者: luomingxue 时间: 2004-03-13 01:12
标题: Redhat Enterprise Advanced Server 3.0性能问题
speed of Journaling File System ext3
You can set Journaling mode data=ordered(default)/writeback/journal/

I can do it use mount/umount

But I couldn't figure out how to do it in fstab.
in fstab
/dev/md2 /boot ext3 -o data=writeback 1 2

showed error in this entry when bootup.
Anybody can help how to add mount options in fstab?
What the correct syntas?

speed of Journaling File System ext3 from redhat
http://www.redhat.com/support/wpapers/redhat/ext3/index.html#speed

作者: luomingxue 时间: 2004-03-13 10:46
标题: Redhat Enterprise Advanced Server 3.0性能问题
i'll test system under different kernels.
Someone said
2.4.22-1.2115.nptlsmp from fedora solved his I/O probs.
I ued 2.4.22-1.2174 does not work.

run the following program
./test_write aaa 20 200

and do
/sbin/hdparm -Tt /dev/md2
while the program is running

from
http://www.redhat.com/archives/taroon-list/2003-December/msg00174.html
test_write.c
#include <stdio.h>;
#include <unistd.h>;
#include <errno.h>;
#include <stdlib.h>;
#include <string.h>;
#include <sys/types.h>;
#include <sys/stat.h>;
#include <fcntl.h>;
#include <sys/time.h>;

#define LOG_FILENAME "/tmp/error.log"
#define MAX_FILE_NAME 32
#define MILLISEC_DIFF(new, old, diff) {                            \
      (diff) = (((new).tv_sec  - (old).tv_sec)  * 1000L ) +          \
               (((new).tv_usec - (old).tv_usec) / 1000L );          \
}

int main(int argc, char* argv[]) {

char* data, log_str[1024], filename[MAX_FILE_NAME];
int frequency, chunk_len, flags, elapsed, interval, sleep_time,
fd = -1, fd2 = -1;
struct timeval t1, t2;
mode_t       mode = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH;

if(argc != 4) {
printf("usage: test_writes filename frequency(eg. 20 for 50 ms) write_size(Bytes)\n"

;
_exit(1);
}

bzero(filename, MAX_FILE_NAME);
strncpy(filename, argv[1], MAX_FILE_NAME -1);
frequency = atoi(argv[2]);
chunk_len = atoi(argv[3]);

printf("filename <%s>; frequency <%d>; chunk_len <%d>;\n", filename, frequency, chunk_len);

flags = O_CREAT | O_WRONLY | O_TRUNC;
fd2 = open(LOG_FILENAME, O_WRONLY | O_CREAT | O_APPEND, mode);
fd = open(filename, flags);

if(fd < 0) {
perror("Unable to open the input file"

;
_exit(2);
}

if(fd2 < 0) {
perror("Unable to open the log file"

;
_exit(2);
}

data = (char*) malloc(sizeof(char)*chunk_len);
if(data == NULL) {
perror("Unable to allocate the input buffer"

;
_exit(3);
}

interval = (int)(((double)1000/frequency));
while(1) {
gettimeofday(&t1, NULL);
write(fd, data, chunk_len);
gettimeofday(&t2, NULL);
MILLISEC_DIFF(t2, t1, elapsed);
if(elapsed >;= interval) {
sprintf(log_str,"<%ld>;<%ld>; write to file <%s>;<%d>;\n", t1.tv_sec, t1.tv_usec/1000L, filename, elapsed);
write(fd2, log_str, strlen(log_str));
}
sleep_time = (interval - elapsed)*1000;
if(sleep_time >; 0)
usleep(sleep_time);
}
}

作者: luomingxue 时间: 2004-03-14 02:56
标题: Redhat Enterprise Advanced Server 3.0性能问题
Final update:

Added 1G memory module,total memory 768M to 1.5G,
Change kernel to
2.4.22-1.2115

No change to kjournald.

No high iowait time anymore!
mission accomplished!

open question:
1. how to write -o data=writeback in fstab, anybody can give me an example?
2. how to check which process is using cpu, memory?
3. MYsql performance may need to be tuned, any tips?

欢迎光临 Chinaunix (http://bbs.chinaunix.net/)