nginx log 转成csv格式

251744647

想把下面行
1.11.1.1 - - [28/Jul/2016:03:00:23 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko"

转成如下效果：

1.11.1.1,-,-,[28/Jul/2016:03:00:23 +0000],"GET / HTTP/1.1",304,0,"-","Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko"

关键就是双引号和[]之间的空格不要转换。求awk语句，谢谢！

moperyblue

1. awk 'BEGIN{split("11 10 9 8 5 3 2 1",a)}{for(i in a)$0=gensub(/ /,",",a[i])}1'

复制代码

1.11.1.1,-,-,[28/Jul/2016:03:00:23 +0000],"GET / HTTP/1.1",304,0,"-","Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko"

1. awk -vRS='\\[[^]]*|"[^"]*"' -vORS= '{RT=gensub(/ /,"@","g",RT);gsub(/ /,",");gsub(/@/," ",RT);print $0 RT}'

复制代码

1.11.1.1,-,-,[28/Jul/2016:03:00:23 +0000],"GET / HTTP/1.1",304,0,"-","Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko"

reyleon

建议修改 nginx 配置..

    log_format  main  '$remote_addr,-,$remote_user,[$time_local],"$request",'
                      '$status,$body_bytes_sent,"$http_referer",'
                      '"$http_user_agent","$http_x_forwarded_for"';

elu_ligao

awk 4 以上用

1. awk 'BEGIN{OFS=",";FPAT="([^ ]+)|(\"[^\"]+\")|(\\[[^[]+\\])"}$1=$1'

复制代码

251744647

找到需要替换的空格的具体位置，然后进行替换，不失为一种思路。谢谢啦。 不过，这样貌似不是很严谨？ 期待新思路。回复 2# moperyblue


   

vagrant_1220

nginx 配置文件中直接定义自己需要的log格式。

moperyblue

回复 5# 251744647


    这需要根据数据格式特点来处理 如果确定格式是固定的 可以这样做

其它方法:
1.正则
2.参照3楼 修改nginx 配置 直接输出需要的格式
...

1. sed -r -e{,,}'s/ /,/' -e 's/ (")/,\1/g;s/(") /\1,/;s/(",\S+) /\1,/'

复制代码

ll104567

顶三楼       -  -

sunzhiguolu

1. perl -ple '$_=join (",",grep {length} split(/\h(\[[^]]+]|"[^"]+")?/))' f

复制代码

1.11.1.1,-,-,[28/Jul/2016:03:00:23 +0000],"GET / HTTP/1.1",304,0,"-","Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko"

moperyblue

1. sed -r 's/^|"[^"]*"|\[[^]]*]/{&}/g;:a;s/(}[^{]*) /\1,/;ta;s/[{}]//g'

复制代码

1.11.1.1,-,-,[28/Jul/2016:03:00:23 +0000],"GET / HTTP/1.1",304,0,"-","Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko"