有如下日志要处理:
com.wearebigmoney.aomen,219.137.118.105,117327,870,2354,26,0,1,0,-1,0.0;0.0,
com.wearebigmoney.aomen,214.147.78.105,457347,870,2354,26,0,1,0,-1,0.0;0.0,
com.wearebigmoney.aomen,216.12.118.105,317337,870,2354,26,0,1,0,-1,0.0;0.0,
com.wearebigmoney.aomen,217.10.19.105,147327,870,2354,26,0,1,0,-1,0.0;0.0,
com.wearebigmoney.aomen,217.10.19.105,147327,870,2111,26,0,1,0,-1,0.0;0.0,
要求 :日志以逗号分割,第三个字段是id ,给定若干个id(例如 :117327,317337,3424,4234,2111 ),快速筛选所有日志,凡是第三个字段与给定id相同的,输出该行到新文件,新文件的名称以 id_year_month.log 格式。给的日志文件名是每个小时一个文件,大概500M左右一个文件,合计大约处理3个T的日志,需要按月合成一个指定id的月文件,想到使用awk,不知如何处理对给定id比对和快速高效处理。请高手帮忙。
requests.log.2016-07-02-22
requests.log.2016-07-02-22
requests.log.2016-07-02-23
requests.log.2016-07-03-00
requests.log.2016-07-03-01
|