perl如何处理这样的重复行？

tener

linux的log文件，类似：

Jan 27 21:40:17 linux-123 rcpowersaved[4337]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Jan 27 21:42:19 linux-123 haadmin[4623]: <err> msg_open:errorConnection refused.
Jan 27 21:44:27 linux-123 rcpowersaved[4338]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Feb 16 22:32:10 linux-123 rcpowersaved[4339]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Feb 17 17:52:54 linux-123 rcpowersaved[4340]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Mar  1 15:25:21 linux-123 rcpowersaved[4341]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Mar  1 15:37:11 linux-123 [powersave]: ERROR (MainLoop:85) Haldaemon did not appear. Aborting...
Mar  2 23:07:08 linux-123 rcpowersaved[4342]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Mar  9 16:54:55 linux-123 rcpowersaved[4343]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.


这样的日志，有7行是重复的，就日期时间不一样，请问，使用perl如何处理比较好？

兰花仙子

需求都没描述清楚。。

tener

需求都没描述清楚。。
兰花仙子 发表于 2011-03-09 12:13

    不好意思，我大意了。


是这样的， Linux系统的报错日志中，内容如下：
Jan 27 21:40:17 linux-123 rcpowersaved[4337]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Jan 27 21:42:19 linux-123 haadmin[4623]: <err> msg_open:errorConnection refused.
Jan 27 21:44:27 linux-123 rcpowersaved[4338]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Feb 16 22:32:10 linux-123 rcpowersaved[4339]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Feb 17 17:52:54 linux-123 rcpowersaved[4340]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Mar  1 15:25:21 linux-123 rcpowersaved[4341]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Mar  1 15:37:11 linux-123 [powersave]: ERROR (MainLoop:85) Haldaemon did not appear. Aborting...
Mar  2 23:07:08 linux-123 rcpowersaved[4342]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Mar  9 16:54:55 linux-123 rcpowersaved[4343]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.



我需要处理为：

Jan 27 21:40:17 linux-123 rcpowersaved[4337]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
Jan 27 21:42:19 linux-123 haadmin[4623]: <err> msg_open:errorConnection refused.
Mar  1 15:37:11 linux-123 [powersave]: ERROR (MainLoop:85) Haldaemon did not appear. Aborting...

意思为： 出现的7次重复信息中，我只要得到一行就行了。

ziyunfei

perl -ane 'if(!$a{@F[5..8]}){$a{@F[5..8]}=print}' file

tener

perl -ane 'if(!$a{@F[5..8]}){$a{@F[5..8]}=print}' file
ziyunfei 发表于 2011-03-09 13:11

    这段有点看不懂。
    是否能解析一下，谢谢！

ziyunfei

这段有点看不懂。
    是否能解析一下，谢谢！
tener 发表于 2011-03-09 17:24

能达到你的要求吗

iLRainyday

这个不知道算不算简洁，只用了你提供的样本，你使用的时候，吧__DATA__去掉，直接处理Log文件就行了。

1. #!/usr/bin/perl
   
2. 
   
3. use strict;
   
4. use warnings;
   
5. 
   
6. my %log;
   
7. 
   
8. while (<DATA>) {
   
9.     my($field_1, $field_2) = split /(?<=])(?=:)/;
   
10.     push @{ $log{$field_2} }, $field_1;
    
11. }
    
12. 
    
13. foreach (keys %log) {
    
14.     print shift ( @{ $log{$_} } ).$_."\n";
    
15. }
    
16. 
    
17. 
    
18. __DATA__
    
19. Jan 27 21:40:17 linux-123 rcpowersaved[4337]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
    
20. Jan 27 21:42:19 linux-123 haadmin[4623]: <err> msg_open:errorConnection refused.
    
21. Jan 27 21:44:27 linux-123 rcpowersaved[4338]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
    
22. Feb 16 22:32:10 linux-123 rcpowersaved[4339]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
    
23. Feb 17 17:52:54 linux-123 rcpowersaved[4340]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
    
24. Mar  1 15:25:21 linux-123 rcpowersaved[4341]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
    
25. Mar  1 15:37:11 linux-123 [powersave]: ERROR (MainLoop:85) Haldaemon did not appear. Aborting...
    
26. Mar  2 23:07:08 linux-123 rcpowersaved[4342]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
    
27. Mar  9 16:54:55 linux-123 rcpowersaved[4343]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.
    

复制代码

输出：

1. Jan 27 21:42:19 linux-123 haadmin[4623]: <err> msg_open:errorConnection refused.
   
2. Mar  1 15:37:11 linux-123 [powersave]: ERROR (MainLoop:85) Haldaemon did not appear. Aborting...
   
3. Jan 27 21:40:17 linux-123 rcpowersaved[4337]: enter 'CPUFREQ_ENABLED=no' in /etc/powersave/cpufreq to avoid this warning.

复制代码

tener

能达到你的要求吗
ziyunfei 发表于 2011-03-09 17:43

    不好意思，回复晚了。

是满足要求的。  我想弄明白这段的意思，呵呵

zhlong8

不好意思，回复晚了。

是满足要求的。  我想弄明白这段的意思，呵呵
tener 发表于 2011-03-10 10:41

    看不懂太正常了，我以前也看不懂，不过自从看了 perlrun 我就全明白了  看里面的 -a -n

tener

这个不知道算不算简洁，只用了你提供的样本，你使用的时候，吧__DATA__去掉，直接处理Log文件就行了。输出： ...
iLRainyday 发表于 2011-03-09 18:47

使用了一段时间之后，又发现新的问题了。

log日志中，错误信息不一定只有一个冒号:, 也不一定是[ID_number]: ，发现错误信息中有这样的：

May 26 18:30:42 linux-server [powersaved][129]: ERROR in Function read_line; line 49: Could not open file for reading: /sys/devices/system/cpu/cpu7/cpufreq/scaling_available_frequencies

这段如果按照 “]： ”来进行split,是不会有问题的，但是问题是还有如下信息：

May 26 18:29:51 linux-server kernel: hda: packet command error: status=0x51 { DriveReady SeekComplete Error }
May 26 18:29:51 linux-server kernel: hda: packet command error: error=0x50
May 26 18:29:53 linux-server kernel: hda: packet command error: status=0x51 { DriveReady SeekComplete Error }
May 26 18:29:53 linux-server kernel: hda: packet command error: error=0x50
May 26 18:39:51 linux-server kernel: hda: packet command error: status=0x51 { DriveReady SeekComplete Error }
May 26 18:39:51 linux-server kernel: hda: packet command error: error=0x50
May 26 18:49:51 linux-server kernel: hda: packet command error: status=0x51 { DriveReady SeekComplete Error }
May 26 18:49:51 linux-server kernel: hda: packet command error: error=0x50
May 26 19:29:51 linux-server kernel: hda: packet command error: status=0x51 { DriveReady SeekComplete Error }
May 26 19:29:51 linux-server kernel: hda: packet command error: error=0x50
May 26 18:29:51 linux-server kernel: hdb: packet command error: status=0x51 { DriveReady SeekComplete Error }
May 26 18:29:51 linux-server8 kernel: hdb: packet command error: error=0x54

冒号前面没有]中括弧了，而且有多个": "组合，不好进行单一的split /: /切割了，是否有更好的方法可以达到以上2中的筛选重复行的方法（既能有"]:" 也能有 ": ",而且，第一个冒号之后的所有字符都要打印出来？