请问一个文本分段的问题

kakaliu

用vi 查看日志的时候发现有些行 内容很长 在VI中查看特别不方便 大致内容如下
{history={id=272707330, returnState=null, businessId=879, linkId=null, gatewayMessageId=null}, map={messageId=-8491669862039930271, phone=13750416569, returnChannelTime=2013-08-21 23:51:00.0, returnConnTime=2013-08-21 23:51:00.0, returnRspCode=0, returnRspDesc=MI:0024, returnState=0}, table=t_sms_mt_history201308},{history={id=272571083, returnState=null, businessId=879, linkId=null, gatewayMessageId=null}, map={messageId=-8491669312283684446, phone=13798830345, returnChannelTime=2013-08-21 23:51:00.0, returnConnTime=2013-08-21 23:51:00.0, returnRspCode=0, returnRspDesc=MI:0024, returnState=0}, table=t_sms_mt_history201308}, {history=xxx}

首先我想分段成每行行首为这样的格式
{history={id=272707330, returnState=null, businessId=879, linkId=null, gatewayMessageId=null}, map={messageId=-8491669862039930271, phone=13750416569, returnChannelTime=2013-08-21 23:51:00.0, returnConnTime=2013-08-21 23:51:00.0, returnRspCode=0, returnRspDesc=MI:0024, returnState=0}, table=t_sms_mt_history201308},
{history={id=272571083, returnState=null, businessId=879, linkId=null, gatewayMessageId=null}, map={messageId=-8491669312283684446, phone=13798830345, returnChannelTime=2013-08-21 23:51:00.0, returnConnTime=2013-08-21 23:51:00.0, returnRspCode=0, returnRspDesc=MI:0024, returnState=0}, table=t_sms_mt_history201308},

其次在用awk 来过滤我想要的字段 如下所示
id=272707330,businessId=879,messageId=-8491669862039930271, phone=13750416569, returnChannelTime=2013-08-21 23:51:00.0, returnConnTime=2013-08-21 23:51:00.0, returnRspCode=0
id=272571083,businessId=879,messageId=-8491669312283684446, phone=13798830345, returnChannelTime=2013-08-21 23:51:00.0, returnConnTime=2013-08-21 23:51:00.0, returnRspCode=0

请各位大神帮忙

reyleon

1. cat urfile | grep -Po '(id=|businessId=|messageId=|phone=|returnChannelTime=|returnConnTime=|returnRspCode=)[^,]+' | sed ':a;/returnRspCode/!N;s/\n/,/;ta'  

复制代码

liion631818

1. 
   
2. Q1:
   
3. awk -vRS="{history" '{if($0!="")print RS $0}' myfile
   
4. 
   
5. Q2:
   
6. awk  -vRS="{history" -vFS="{|,|}" '{for(i=1;i<NF;i++){if($i~/.+=[-|0-9].*/)printf $i ","};if(NF!=0)print "\n"}' myfile
   
7. 
   

复制代码

guogang225

直接得出最后的结果：

1. sed -nr 's/(\{history)/\n\1/gp' urfile |awk -F'[{},]+' 'NF>3{print $3,$5,$9,$10,$11,$12,$13}'

复制代码

kakaliu

guogang225 发表于 2013-08-23 10:33
直接得出最后的结果：

谢谢 这个感觉很有用

kakaliu

回复 3# liion631818


    [mis@MimmsApp3 resp]$ awk -vRS="{history" '{if($0!="")print RS $0}' resp.log.2013-08-22 >temp2.log
[mis@MimmsApp3 resp]$ more temp2.log
{history2013-08-22 00:00:00.024 startQuertz_Worker-9 [INFO ]RemoveRespHistoryQuartz : -----定时删除读取过的回执记录，本次成功删除的记录条数为：短信=118,彩信=0
2013-08-22 00:00:01.233 彩信读取线程 [INFO ]MmsReadRespThread : --彩信转发队列里长度：0
2013-08-22 00:00:01.575 短信读取线程 [INFO ]ReadRespThread : ---短信转发队里长度：0
2013-08-22 00:00:01.597 短信读取线程 [INFO ]ReadRespThread : ---短信转发队里长度：22
2013-08-22 00:00:03.513 http-8084-6 [INFO ]ConnectMtRespController : 收到异步回执状态报告 message(json)=[{"messageId":"-8439819642452688014","phone":"15282139603","returnState"
:"1","returnChannelTime":"2013-08-21 23:54","returnConnTime":"2013-08-21 23:54","returnRspCode":"1","returnRspDesc":"DELIVRD"}]
2013-08-22 00:00:03.514 http-8084-6 [INFO ]ConnectMtRespController : 接收一批回执状态报告，放入队列成功，数量为=1

没有过滤干净！好多不相关的内容都过滤进来了！

rdcwayx

1. awk  '{gsub(/[{|}]/,",",$0);l=split($0,a,",");for (i=1;i<=l;i++) if (a[i]~/(id=|businessId=|messageId=|phone=|returnChannelTime=|returnConnTime=|returnRspCode=)/) printf a[i] OFS;print ORS }' RS="{history=" OFS=, infile

复制代码

王晓兜

if(NF!=0)print "\n"   如果域的个数不是0 就换行    但不是很明白

回复 3# liion631818


   

kakaliu

[mis@MimmsApp3 resp]$ sed -nr 's/(\{history)/\n\1/gp' resp.log.2013-08-22 >temp1.txt
[mis@MimmsApp3 resp]$ more temp1.txt
2013-08-22 00:00:11.665 短信转发线程7 [INFO ]SchedulerService : 回执状态报告处理list数量为=22,list=[
{history={id=272658520, returnState=null, businessId=879, linkId=null, gatewayMessageId=null}, map={messageId=-8491688828615389155, phone=15118424475, returnChannelTime=2013-08
-21 23:50:00.0, returnConnTime=2013-08-21 23:50:00.0, returnRspCode=0, returnRspDesc=MI:0024, returnState=0}, table=t_sms_mt_history201308},
{history={id=272658402, returnState=null, businessId=879, linkId=null, gatewayMessageId=null}, map={messageId=-8491689378370931555, phone=15220785281, returnChannelTime=2013-08
-21 23:50:00.0, returnConnTime=2013-08-21 23:50:00.0, returnRspCode=0, returnRspDesc=MI:0024, returnState=0}, table=t_sms_mt_history201308},

第一条命令用sed 语法简单看着也容易懂

那请问如果用awk的话 应该咋写

kakaliu

reyleon 发表于 2013-08-23 10:29

谢谢兄弟 不过也会有很多不相关的东西都过滤出来了