Perl日志过滤问题

delkar

Perl语言入门刚学习到第四章，来了个急活，求助各位。
将下列日志中的date、time、cs-method、cs-username、c-ip、cs(User-Agent)几项过滤出来转换成Excel格式。

#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken
2013-04-06 00:00:00 10.10.11.22 POST /Microsoft-Server-ActiveSync/default.eas Cmd=Ping&User=qinyue%40ibm.com&DeviceId=SEC13B6E5F7CFA5D&DeviceType=SAMSUNGGTN7100&Log=V121_Sst3_LdapC0_LdapL0_RpcC18_RpcL31_Hb470_Rto1_Erq1_S1_ 443 qinyue@ibm.com 117.12.51.176 SAMSUNG-GT-N7100/100.40101 200 0 1236 483840
2013-04-06 00:00:01 10.10.11.22 GET /owa/ ae=Item&a=Open&t=IPM.Note&id=RgAAAADsHIIkyKeaQa2gOQnXgoQLBwCR%2btQjnK8pTLLGR%2bYPOyPcAB%2b89qhQAACR%2btQjnK8pTLLGR%2bYPOyPcACUHIvisAAAJ&prfltncy=49&prfrpccnt=5&prfrpcltncy=46&prfldpcnt=2&prfldpltncy=0&prfavlcnt=0&prfavlltncy=0 443 gaopan 120.128.3.96 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) 200 0 0 124
2013-04-06 00:00:01 10.10.11.22 GET /owa/8.3.83.4/themes/base/icon-flag.gif - 443 - 120.128.3.96 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) 200 0 0 218
2013-04-06 00:00:01 10.10.11.22 GET /owa/8.3.83.4/themes/base/msgdtls.gif - 443 - 120.128.3.96 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) 200 0 0 374
2013-04-06 00:00:01 10.10.11.22 GET /owa/8.3.83.4/themes/base/ctgrs.gif - 443 - 120.128.3.96 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) 200 0 0 374
2013-04-06 00:00:01 10.10.11.22 GET /owa/8.3.83.4/themes/base/prev.gif - 443 - 120.128.3.96 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) 200 0 0 359
2013-04-06 00:00:01 10.10.11.22 GET /owa/8.3.83.4/themes/base/print.gif - 443 - 120.128.3.96 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) 200 0 0 359
2013-04-06 00:00:01 10.10.11.22 GET /owa/8.3.83.4/themes/base/next.gif - 443 - 120.128.3.96 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) 200 0 0 359
2013-04-06 00:00:07 127.0.0.1 GET /pki/crl/products/microsoftrootcert.crl - 80 - 127.0.0.1 Microsoft-CryptoAPI/6.0 403 4 5 0
2013-04-06 00:00:07 127.0.0.1 GET /pki/crl/products/CodeSigPCA.crl - 80 - 127.0.0.1 Microsoft-CryptoAPI/6.0 403 4 5 0
2013-04-06 00:00:07 127.0.0.1 GET /pki/crl/products/MicrosoftTimeStampPCA.crl - 80 - 127.0.0.1 Microsoft-CryptoAPI/6.0 403 4 5 0
2013-04-06 00:00:08 10.10.11.22 POST /Microsoft-Server-ActiveSync/default.eas User=shisy&DeviceId=ApplC37GN11QDPMW&DeviceType=iPhone&Cmd=Ping&Log=V121_Sst8_LdapC0_LdapL0_RpcC24_RpcL31_Hb1005_Erq1_Pk914388648_S2_ 443 ibm\shisy 61.148.244.119 Apple-iPhone3C1/902.206 200 0 64 319546
2013-04-06 00:00:09 10.10.11.22 POST /Microsoft-Server-ActiveSync/default.eas User=shisy&DeviceId=ApplC37GN11QDPMW&DeviceType=iPhone&Cmd=Ping&Log=V121_Sst4_LdapC0_LdapL0_RpcC20_RpcL31_Hb1005_Erq1_Pk914388648_S2_ 443 ibm\shisy 61.148.244.119 Apple-iPhone3C1/902.206 200 0 0 11138
2013-04-06 00:00:10 10.10.11.22 POST /Microsoft-Server-ActiveSync/default.eas Cmd=Ping&User=ibm%5Cxiehang&DeviceId=SEC11375924568&DeviceType=SAMSUNGGTI9082&Log=V121_Sst10_LdapC0_LdapL0_RpcC26_RpcL15_Hb470_Erq1_S2_ 443 ibm\xiehang 118.186.252.130 SAMSUNG-GT-I9082/100.40102 200 0 0 108699
2013-04-06 00:00:10 10.10.11.22 POST /Microsoft-Server-ActiveSync/default.eas User=shisy&DeviceId=ApplC37GN11QDPMW&DeviceType=iPhone&Cmd=Sync&Log=V121_Fc1_Fid:12_Ty:Em_Filt3_St:S_Sk:116468500_Sst4_Srv:1a0c0d0s0e0r_LdapC0_LdapL0_RpcC23_RpcL46_Pk914388648_S1_ 443 ibm\shisy 61.148.244.119 Apple-iPhone3C1/902.206 200 0 0 249
2013-04-06 00:00:10 10.10.11.22 POST /owa/ev.owa oeh=1&ns=RecipWell&ev=GetCache&prfltncy=42&prfrpccnt=7&prfrpcltncy=46&prfldpcnt=0&prfldpltncy=0&prfavlcnt=0&prfavlltncy=0 443 gaopan 120.128.3.96 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) 200 0 0 109

mcshell

提取出来，制作成CSV不就行了{:3_185:}，还有一点。。你的文本里面有的没有cs-username  这个字段 如何解决？  不匹配 还是？

delkar

回复 2# mcshell
POST方法有这个字段


   

xfoucs

回复 3# delkar


   
楼上已经说了

1. perl -nE "say join(',',(split)[0,1,3,7,8,9]) if /^\d/" log.txt >log.csv

复制代码

excel打开csv就成了

墨迹哥

又是日志。。又是Web日志。。正则处理最快了，我对比过的。
你也可以参考下我的blog文，以前写过类似的，不过后来改版了。
给你几个建议吧，这样可以少走弯路：
1。用fork，提高效率。
2。多线程不管用，不用考虑，因为I／O问题。
3。正则越简便越好。
4。减少使用第三方库，因为会让速度变慢。宁可使用split。

http://www.cnblogs.com/xiaoCon/archive/2013/02/26/2934323.html