
平台论坛博客文库

› 论坛 › 程序设计 › Shell › 过滤文本输出报表

[文本处理] 过滤文本输出报表 [复制链接]

zl624867243

丰衣足食

论坛徽章:: 8

电梯直达

1楼 [收藏(0)] [报告]

发表于 2018-05-17 00:09 |只看该作者 |倒序浏览

本帖最后由 zl624867243 于 2018-05-17 00:30 编辑

请教一个问题：文本如下：

*************************** 1. row ***************************
alertid: 4088
SUBJECT: 故障PROBLEM,服务器:ALY-HN1_Core-public-appweb-PRD-2发生: uappweb request time avg above 3000ms故障!
message: 告警主机: ALY-HN1_Core-public-appweb-PRD-2
主机IP: 10.18.25.180
告警时间: 2018.05.16 04:23:55
告警等级: Warning
告警信息: uappweb request time avg above 3000ms
告警项目: site_fang_time_avg[uappweb]
问题详情: uappweb avg time per min:6408
当前状态: PROBLEM
事件ID: 29126
*************************** 2. row ***************************
alertid: 4101
SUBJECT: 故障PROBLEM,服务器:ALY-HN1_PannelService-PRD-4发生: dtool 4xx above 40故障!
message: 告警主机: ALY-HN1_PannelService-PRD-4
主机IP: 10.18.25.160
告警时间: 2018.05.16 08:27:16
告警等级: Warning
告警信息: dtool 4xx above 40
告警项目: site_fang[dtool,40.]
问题详情: dtool 4xx per min:45
当前状态: PROBLEM
事件ID: 29131
*************************** 3. row ***************************
alertid: 4114
SUBJECT: 故障PROBLEM,服务器:ALY-HN1-EP-web-PRD-02发生: eappweb request time avg above 3000ms故障!
message: 告警主机: ALY-HN1-EP-web-PRD-02
主机IP: 10.18.25.187
告警时间: 2018.05.16 08:38:03
告警等级: Warning
告警信息: eappweb request time avg above 3000ms
告警项目: site_fang_time_avg[eappweb]
问题详情: eappweb avg time per min:3958
当前状态: PROBLEM
事件ID: 29133
*************************** 4. row ***************************
alertid: 4129
SUBJECT: 故障PROBLEM,服务器:ALY-HN1_PannelService-PRD-4发生: ap2 4xx above 20故障!
message: 告警主机: ALY-HN1_PannelService-PRD-4
主机IP: 10.18.25.160
告警时间: 2018.05.16 08:57:50
告警等级: Warning
告警信息: ap2 4xx above 20
告警项目: site_fang[ap2,40.]
问题详情: ap2 4xx per min:36
当前状态: PROBLEM
事件ID: 29135
*************************** 5. row ***************************
alertid: 4142
SUBJECT: 故障PROBLEM,服务器:ALY-HN1_PannelService-PRD-4发生: ap2 4xx above 20故障!
message: 告警主机: ALY-HN1_PannelService-PRD-4
主机IP: 10.18.25.160
告警时间: 2018.05.16 09:10:50
告警等级: Warning
告警信息: ap2 4xx above 20
告警项目: site_fang[ap2,40.]
问题详情: ap2 4xx per min:72
当前状态: PROBLEM
事件ID: 29137
*************************** 6. row ***************************
alertid: 4155
SUBJECT: 故障PROBLEM,服务器:ALY-HN1_PannelService-PRD-4发生: ap2 4xx above 20故障!
message: 告警主机: ALY-HN1_PannelService-PRD-4
主机IP: 10.18.25.160
告警时间: 2018.05.16 09:13:50
告警等级: Warning
告警信息: ap2 4xx above 20
告警项目: site_huolala[ap2,40.]
问题详情: ap2 4xx per min:36
当前状态: PROBLEM
事件ID: 29139
*************************** 7. row ***************************
alertid: 4168
SUBJECT: 故障PROBLEM,服务器:ALY-HN1-EP-web-PRD-02发生: eappweb request time avg above 3000ms故障!
message: 告警主机: ALY-HN1-EP-web-PRD-02
主机IP: 10.18.25.187
告警时间: 2018.05.16 11:47:15
告警等级: Warning
告警信息: eappweb request time avg above 3000ms
告警项目: site_fang_time_avg[eappweb]
问题详情: eappweb avg time per min:3840
当前状态: PROBLEM
事件ID: 29141
*************************** 8. row ***************************
alertid: 4183
SUBJECT: 故障PROBLEM,服务器:ALY-HN1-BaseService-PRD-2发生: Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS故障!
message: 告警主机: ALY-HN1-BaseService-PRD-2
主机IP: 10.18.1.194
告警时间: 2018.05.16 13:58:23
告警等级: Warning
告警信息: Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS
告警项目: system.cpu.util[,iowait]
问题详情: CPU iowait time:31.67 %
当前状态: PROBLEM
事件ID: 29143
*************************** 9. row ***************************
alertid: 4196
SUBJECT: 故障PROBLEM,服务器:ALY-HN1-BaseService-PRD-2发生: Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS故障!
message: 告警主机: ALY-HN1-BaseService-PRD-2
主机IP: 10.18.1.194
告警时间: 2018.05.16 16:13:23
告警等级: Warning
告警信息: Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS
告警项目: system.cpu.util[,iowait]
问题详情: CPU iowait time:8.83 %
当前状态: PROBLEM
事件ID: 29145
*************************** 10. row ***************************
alertid: 4209
SUBJECT: 故障PROBLEM,服务器:ALY-HN1-BaseService-PRD-2发生: Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS故障!
message: 告警主机: ALY-HN1-BaseService-PRD-2
主机IP: 10.18.1.194
告警时间: 2018.05.16 16:23:23
告警等级: Warning
告警信息: Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS
告警项目: system.cpu.util[,iowait]
问题详情: CPU iowait time:40.75 %
当前状态: PROBLEM
事件ID: 29147
*************************** 11. row ***************************
alertid: 4222
SUBJECT: 故障PROBLEM,服务器:ALY-HN1-EP-web-PRD-01发生: e 5xx above 20故障!
message: 告警主机: ALY-HN1-EP-web-PRD-01
主机IP: 10.18.1.252
告警时间: 2018.05.16 16:53:29
告警等级: Warning
告警信息: e 5xx above 20
告警项目: site_fang[e,50.]
问题详情: e 5xx per min:26
当前状态: PROBLEM
事件ID: 29480
*************************** 12. row ***************************
alertid: 4237
SUBJECT: 故障PROBLEM,服务器:ALY-HN1_PannelService-PRD-4发生: hpayp request time avg above 3000 ms故障!
message: 告警主机: ALY-HN1_PannelService-PRD-4
主机IP: 10.18.25.160
告警时间: 2018.05.16 17:33:59
告警等级: Warning
告警信息: hpayp request time avg above 3000 ms
告警项目: site_fang_time_avg[hpayp]
问题详情: hpayp avg time per min:5769
当前状态: PROBLEM
事件ID: 29821
*************************** 13. row ***************************
alertid: 4240
SUBJECT: 故障PROBLEM,服务器:ALY-HN1_PannelService-PRD-4发生: hpayp request time avg above 3000ms故障!
message: 告警主机: ALY-HN1_PannelService-PRD-4
主机IP: 10.18.25.160
告警时间: 2018.05.16 17:33:59
告警等级: Warning
告警信息: hpayp request time avg above 3000ms
告警项目: site_fang_time_avg[hpayp]
问题详情: hpayp avg time per min:5769
当前状态: PROBLEM
事件ID: 29822

复制代码

想要的结果：(结果格式化输出类似表格那样对齐)

告警主机 |主机IP | 告警信息 | 告警信息的次数
ALY-HN1_Core-public-appweb-PRD-2 | 10.18.25.180 | uappweb request time avg above 3000ms | 2
....

复制代码

文库|博客

cfwyy

丰衣足食

论坛徽章:: 8

2楼 [报告]

发表于 2018-05-17 10:27 |只看该作者

告警信息的次数是哪来的怎么定义的？
小白先随便写了个，把次数字段先省略了，大神们见笑了

BEGIN{
FS=":"
ORS=""
printf("%-33s%-15s%-30s\n","告警主机","| 主机IP","| 告警信息")
}
/message/{
split($3,a," ")
printf("%-35s|",a[1])
}
/IP/{
split($2,b," ")
printf(" %-14s|",b[1])
}
/\xE4\xBF\xA1\xE6\x81\xAF/{ #"信息" 的UTF8码
printf("%-35s\n",$2)
}

复制代码

gawk -f script.awk file

复制代码

结果：

告警主机 | 主机IP | 告警信息
ALY-HN1_Core-public-appweb-PRD-2 | 10.18.25.180 | uappweb request time avg above 3000ms
ALY-HN1_PannelService-PRD-4 | 10.18.25.160 | dtool 4xx above 40
ALY-HN1-EP-web-PRD-02 | 10.18.25.187 | eappweb request time avg above 3000ms
ALY-HN1_PannelService-PRD-4 | 10.18.25.160 | ap2 4xx above 20
ALY-HN1_PannelService-PRD-4 | 10.18.25.160 | ap2 4xx above 20
ALY-HN1_PannelService-PRD-4 | 10.18.25.160 | ap2 4xx above 20
ALY-HN1-EP-web-PRD-02 | 10.18.25.187 | eappweb request time avg above 3000ms
ALY-HN1-BaseService-PRD-2 | 10.18.1.194 | Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS
ALY-HN1-BaseService-PRD-2 | 10.18.1.194 | Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS
ALY-HN1-BaseService-PRD-2 | 10.18.1.194 | Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS
ALY-HN1-EP-web-PRD-01 | 10.18.1.252 | e 5xx above 20
ALY-HN1_PannelService-PRD-4 | 10.18.25.160 | hpayp request time avg above 3000 ms
ALY-HN1_PannelService-PRD-4 | 10.18.25.160 | hpayp request time avg above 3000ms

复制代码

实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？

wh7211

版主

论坛徽章:: 25

3楼 [报告]

发表于 2018-05-17 12:05 |只看该作者

本帖最后由 wh7211 于 2018-05-17 15:41 编辑

回复 1# zl624867243

awk 'BEGIN{FS=":";a1="告警主机";a2="主机IP";a3="告警信息";a4="告警信息的次数";printf("%-29s|%-14s|%-56s|%-13s\n",a1,a2,a3,a4)}$2~a1{sub(" +","",$3);b++;c1[b]=$3}$1~a2{sub(" +","",$2);c2[b]=$2}$1~a3{sub(" +","",$2);c3[b]=$2;d[$2]++}END{for(i=1;i<=length(c1);i++){printf("%-33s|%-16s|%-60s|%-20s\n",c1[i],c2[i],c3[i],d[c3[i]])}}' file
告警主机 |主机IP |告警信息 |告警信息的次数
ALY-HN1_Core-public-appweb-PRD-2 |10.18.25.180 |uappweb request time avg above 3000ms |1
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |dtool 4xx above 40 |1
ALY-HN1-EP-web-PRD-02 |10.18.25.187 |eappweb request time avg above 3000ms |2
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |ap2 4xx above 20 |3
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |ap2 4xx above 20 |3
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |ap2 4xx above 20 |3
ALY-HN1-EP-web-PRD-02 |10.18.25.187 |eappweb request time avg above 3000ms |2
ALY-HN1-BaseService-PRD-2 |10.18.1.194 |Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS |3
ALY-HN1-BaseService-PRD-2 |10.18.1.194 |Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS |3
ALY-HN1-BaseService-PRD-2 |10.18.1.194 |Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS |3
ALY-HN1-EP-web-PRD-01 |10.18.1.252 |e 5xx above 20 |1
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |hpayp request time avg above 3000 ms |1
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |hpayp request time avg above 3000ms |1

复制代码

1cpuer

腰缠万贯

论坛徽章:: 6

4楼 [报告]

发表于 2018-05-17 16:11 |只看该作者

本帖最后由 1cpuer 于 2018-05-17 16:39 编辑

回复 3# wh7211
a1="告警主机"
$2~a1{sub(" +","",$3);b++;c1=$3}
c1 是一个告警主机的第3列的数组 1，2，3 ...
for(i=1;i<=length(c1);i++)
i 取值范围
lenght(c1)
lenght 不是指的是字符串长度吗？

有一点思绪了
length(c1) 是一个数组 length[c1] ... l[1],l[2] ...length[13 #hang]
就是用一个数组的长度来引证 i
如换作其他函数例 match 或也行

wh7211

版主

论坛徽章:: 25

5楼 [报告]

发表于 2018-05-17 17:22 |只看该作者

回复 4# 1cpuer

man awk -> length

zl624867243

丰衣足食

论坛徽章:: 8

6楼 [报告]

发表于 2018-05-17 21:49 |只看该作者

回复 2# cfwyy

告警信息的次数就是告警信息去重然后统计次数

zl624867243

丰衣足食

论坛徽章:: 8

7楼 [报告]

发表于 2018-05-17 21:52 |只看该作者

回复 3# wh7211

前面的内容去重得出来，告警信息的次数就是去重统计来的

龙牙地主天

小富即安

论坛徽章:: 8

8楼 [报告]

发表于 2018-05-18 09:34 |只看该作者

回复 1# zl624867243

awk 'BEGIN{RS="message";FS="\n"; printf("%-31s|%-12s|%-66s|%-s\n","告警主机","主机IP","告警信息","告警信息的次数")}NR>1{split($1,a,":") ; split($2,b,":") ; split($5,c,":") ; cc[c[2]]=cc[c[2]]?++cc[c[2]]:1 ; printf("%-35s|%-10s|%-70s|%-s\n",a[3],b[2],c[2],cc[c[2]])}' file

复制代码

wh7211

版主

论坛徽章:: 25

9楼 [报告]

发表于 2018-05-18 09:51 |只看该作者

本帖最后由 wh7211 于 2018-05-18 10:32 编辑

回复 7# zl624867243

前面的内容去重得出来，告警信息的次数就是去重统计来的

awk 'BEGIN{FS=":";a1="告警主机";a2="主机IP";a3="告警信息";a4="告警信息的次数";printf("%-29s|%-14s|%-56s|%-13s\n",a1,a2,a3,a4)}$2~a1{sub(" +","",$3);b++;c1[b]=$3}$1~a2{sub(" +","",$2);c2[b]=$2}$1~a3{sub(" +","",$2);c3[b]=$2;d[c2[b]" "c3[b]]++}END{for(i=1;i<=length(c1);i++){printf("%-33s|%-16s|%-60s|%-20s\n",c1[i],c2[i],c3[i],d[c2[i]" "c3[i]])}}' file
告警主机 |主机IP |告警信息 |告警信息的次数
ALY-HN1_Core-public-appweb-PRD-2 |10.18.25.180 |uappweb request time avg above 3000ms |1
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |dtool 4xx above 40 |1
ALY-HN1-EP-web-PRD-02 |10.18.25.187 |eappweb request time avg above 3000ms |2
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |ap2 4xx above 20 |3
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |ap2 4xx above 20 |3
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |ap2 4xx above 20 |3
ALY-HN1-EP-web-PRD-02 |10.18.25.187 |eappweb request time avg above 3000ms |2
ALY-HN1-BaseService-PRD-2 |10.18.1.194 |Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS |3
ALY-HN1-BaseService-PRD-2 |10.18.1.194 |Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS |3
ALY-HN1-BaseService-PRD-2 |10.18.1.194 |Disk I/O is overloaded on ALY-HN1-BaseService-PRD-2-从DNS |3
ALY-HN1-EP-web-PRD-01 |10.18.1.252 |e 5xx above 20 |1
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |hpayp request time avg above 3000 ms |1
ALY-HN1_PannelService-PRD-4 |10.18.25.160 |hpayp request time avg above 3000ms |1

复制代码

返回列表

Chinaunix › 论坛 › 程序设计 › Shell › 过滤文本输出报表