文本处理求助-任务耗时计算

bikkuri

大家好！
我有一个问题向大家请教。

有下面一个文本，是一个任务列表，希望从中抓取出所有状态为正在进行的任务（<status>inProgress</status>），并显示出任务名（<taskName>和</taskName>中间的部分），任务起始时间（<startTime>和</startTime>中间的部分），并计算出任务耗时（=当前时间-任务起始时间），然后按任务耗时长短分类显示任务。

1. <SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/"><SOAP:Header><header xmlns="xmlapi_1.0"><requestID>:0:1</requestID><requestTime>Dec 20, 2017 8:59:58 PM</requestTime><responseTime>Dec 20, 2017 8:59:58 PM</responseTime></header></SOAP:Header><SOAP:Body><findResponse xmlns="xmlapi_1.0"><result><taskmgmt.TaskTracker><userName>admin</userName><stage>entry</stage><identifier>0.0.0.0</identifier><sessionId>3</sessionId><clientId>3</clientId><requestId>42013</requestId><taskName>Database Manager [Full Backup]</taskName><status>inProgress</status><startTime>1513774424757</startTime><elapsedTime>0</elapsedTime><queuedTime>2</queuedTime><cancelAreq><taskmgmt.CancelAreq><className>N/A</className><methodName>N/A</methodName><paramType></paramType><paramValue></paramValue></taskmgmt.CancelAreq></cancelAreq><activityId>824670356</activityId><deploymentState>-1</deploymentState><objectFullName>nmsSystem:task-tracker-manager:taskTracker-1510</objectFullName><name>Task</name><selfAlarmed>false</selfAlarmed></taskmgmt.TaskTracker><taskmgmt.TaskTracker><userName>admin</userName><stage>entry</stage><identifier>0.0.0.0</identifier><sessionId>3</sessionId><clientId>3</clientId><requestId>42241</requestId><taskName>Network Element R108 10.0.0.8 R105 10.0.0.103 [Multi Resync All MIBs]</taskName><status>inProgress</status><startTime>1513774791674</startTime><elapsedTime>0</elapsedTime><queuedTime>1</queuedTime><cancelAreq><taskmgmt.CancelAreq><className>N/A</className><methodName>N/A</methodName><paramType></paramType><paramValue></paramValue></taskmgmt.CancelAreq></cancelAreq><activityId>824670360</activityId><deploymentState>-1</deploymentState><objectFullName>nmsSystem:task-tracker-manager:taskTracker-1514</objectFullName><name>Task</name><selfAlarmed>false</selfAlarmed></taskmgmt.TaskTracker><taskmgmt.TaskTracker><userName>admin</userName><stage>entry</stage><identifier>0.0.0.0</identifier><sessionId>0</sessionId><clientId>:0:1</clientId><requestId>:0:1</requestId><taskName>Find [taskmgmt.TaskTracker]</taskName><status>succeeded</status><startTime>1513774798376</startTime><elapsedTime>2</elapsedTime><queuedTime>0</queuedTime><cancelAreq><taskmgmt.CancelAreq><className>N/A</className><methodName>N/A</methodName><paramType></paramType><paramValue></paramValue></taskmgmt.CancelAreq></cancelAreq><activityId>0</activityId><deploymentState>-1</deploymentState><objectFullName>nmsSystem:task-tracker-manager:taskTracker-1520</objectFullName><name>Task</name><selfAlarmed>false</selfAlarmed></taskmgmt.TaskTracker></result></findResponse></SOAP:Body></SOAP:Envelope>

复制代码

这个文本中有3个任务：
          <taskName>Database Manager [Full Backup]</taskName>
          <status>inProgress</status>
          <startTime>1513774424757</startTime>
--
          <taskName>Network Element R108 10.0.0.8 R105 10.0.0.103 [Multi Resync All MIBs]</taskName>
          <status>inProgress</status>
          <startTime>1513774791674</startTime>
--
          <taskName>Find [taskmgmt.TaskTracker]</taskName>
          <status>succeeded</status>
          <startTime>1513774798376</startTime>

首先取"Find [taskmgmt.TaskTracker]"任务（该任务有且仅有一个）的开始时间定为当前时间，即
<taskName>Find [taskmgmt.TaskTracker]</taskName><status>succeeded</status>后面的<startTime>1513774798376</startTime>为当前时间；
CurrentTime=1513774798376

然后取其他所有状态为inProgress的任务的taskName和startTime，并计算出任务耗时：
Database Manager [Full Backup]: (1513774798376-1513774424757=373619ms=6.23m)
Network Element R108 10.0.0.8 R105 10.0.0.103 [Multi Resync All MIBs]: (1513774798376-1513774791674=6702ms=0.11m)

如果任务耗时不到5分钟，则显示INFO;
如果任务耗时超过5分钟，则显示WARNING;
如果任务耗时超过10分钟，则显示MAJOR;
如果任务耗时超过30分钟，则显示CRITICAL;

以上文本经过awk处理后希望得到的结果是：


WARNING: TASK - Database Manager [Full Backup] has taken 6.23 minutes
INFO: TASK - Network Element R108 10.0.0.8 R105 10.0.0.103 [Multi Resync All MIBs] has taken 0.11 minutes
INFO: TASK - Find [taskmgmt.TaskTracker] has taken 0 minutes

bikkuri

[root@dhp9:/tmp]# cat tasktracker_20171220_205957.out.txt |awk -F "[<>]+" '{j=1;for(i=1;i<=NF;i++){if($i=="/taskName"){N[j]=$(i-1);if($(i-1)=="Find [taskmgmt.TaskTracker]")k=j;};if($i=="/status")S[j]=$(i-1);if($i=="/startTime")T[j]=$(i-1);if($i=="/taskmgmt.TaskTracker")j++}}END{C=T[k];for(m=1;m<j;m++){U[m]=(C-T[m])/60000;if(U[m]<5){E[m]="INFO"}else{if(U[m]<10){E[m]="WARNING"}else{if(U[m]<30){E[m]="MAJOR"}else{E[m]="CRITICAL"}}}};for(n=1;n<j;n++)printf("%s: TASK - %s has taken %.2f minutes\n",E[n],N[n],U[n])}'
WARNING: TASK - Database Manager [Full Backup] has taken 6.23 minutes
INFO: TASK - Network Element R108 10.0.0.8 R105 10.0.0.103 [Multi Resync All MIBs] has taken 0.11 minutes
INFO: TASK - Find [taskmgmt.TaskTracker] has taken 0.00 minutes
[root@dhp9:/tmp]#

jason680

回复 2# bikkuri

How about this way ...

$ awk -f get_task.awk FILE
WARNING:   TASK - Database Manager [Full Backup] has taken 6.23 minutes
INFO:      TASK - Network Element R108 10.0.0.8 R105 10.0.0.103 [Multi Resync All MIBs] has taken 0.11 minutes


$ cat get_task.awk

BEGIN{
  FS = "[<>]+"
  t=split("INFO 5 WARNING 10 MAJOR 30 CRITICAL",a," ");
  for(n=1;n<=t/2;++n){
    c=n*2
    aNum[n] = a[c]
    aMsg[n] = a[c-1]
  }
  aMsg[n] = a[t]
  sMsg = n
}
function show(n, c,t){
  msg = ""
  # for minutes
  t = (E[now] - E[n])/60000
  for(c=1;c < sMsg; ++c){
    if(t < aNum[c]){
      msg = aMsg[c]
      break
    }
  }
  if(msg == "") msg = aMsg[c]

  printf("%*s TASK - %s has taken %.2f minutes\n",-10,msg":",N[n],t)
}
{
  c=0;
  for(n=1;n<=NF;++n){
    if($n=="taskName"){
      N[++c]=$(n+1);
      #print "c="c,N[c]
      if($(n+1)~/taskmgmt.TaskTracker/)
       now=c;
      continue;
    }
    if($n=="status"){
      S[c]=$(n+1);
      continue;
    }
    if($n=="startTime"){
     E[c] = $(n+1);
    }
  }
}


END{
  for(n=1;n<=c;++n){
    #print "n=",n,N[n],S[n],E[n]
    if(S[n]=="inProgress" && n!=now)
      show(n);
  }
}

zxy877298415

回复 1# bikkuri

1. awk -vRS='<taskName>|</startTime>' -F ">|<" '/Find \[taskmgmt.TaskTracker]/{t=$NF;}$1{a[$1]=$NF}END{for(i in a) {s=sprintf ("%.2f",(t-a[i])/60000);h=" has taken ";k=" :TASK -";m=" minutes";if(s>5) print "WARNING"k i h s m;else if(s>10) print "MAJOR"k i h s m; else if(s>30) print "CRITICAL"k i h s m;else print "INFO"k i h s m}}' file
   

复制代码

bikkuri

谢谢两位高手指点！学习了！

q1208c

这么复杂的结果，建议用 perl 或 python 吧。 shell script 不容易维护。

wh7211

回复 1# bikkuri

1. awk -F"><|>|</" '{y="<taskName>Find \\[taskmgmt.TaskTracker]</taskName><status>succeeded</status><startTime>([0-9]+)</startTime>";match($0,y,t);for(i=1;i<=NF;i++){if($i~/inProgress|succeeded/){name=$(i-3);time=(t[1]-$(i+3))/1000/60;if(time<5){msg="INFO"}else if(time>=5&&time<10){msg="WARNING"}else if(time>=10&&time<30){msg="MAJOR"}else{msg="CRITICAL"};printf("%s: TASK - %s has taken %.2f minutes\n",msg,name,time)}}}' file

复制代码

输出：
WARNING: TASK - Database Manager [Full Backup] has taken 6.23 minutes
INFO: TASK - Network Element R108 10.0.0.8 R105 10.0.0.103 [Multi Resync All MIBs] has taken 0.11 minutes
INFO: TASK - Find [taskmgmt.TaskTracker] has taken 0.00 minutes

1cpuer

回复 5# bikkuri

彩色字符是咋回事

bikkuri

回复 9# 1cpuer

MobaXterm默认自动扫描文本中的敏感词提醒