解析wget日志

liumilan2009

Spider mode enabled. Check if remote file exists.
--2015-06-21 19:21:17--  http://a.b.com/
Resolving a.b.com... 1.2.3.4
Connecting to a.b.com||1.2.3.4 failed: Connection timed out.
Giving up.

从wget中生成的日志格式如上，当碰到timed out时候，就把a.b.com给解析出来

这个具体怎么写啊

一行一行读，读到url，存起来，然后再往下读碰到timed out，解析出url?

聆雨淋夜

解析出url?  是什么意思，是过滤出来吗

liumilan2009

就是把a.b.com给保存下来

聆雨淋夜

cat file
Spider mode enabled. Check if remote file exists.
--2015-06-21 19:21:17--  http://a.b.com/
Resolving a.b.com... 1.2.3.4
Connecting to a.b.com||1.2.3.4 failed: Connection timed out.
Giving up.
Spider mode enabled. Check if remote file exists.
--2015-06-21 19:21:17--  http://c.d.com/
Resolving a.b.com... 1.2.3.4
Connecting to a.b.com||1.2.3.4 failed: Connection timed out.
Giving up.

sed -rn '/http:\/\//{s#.*http://([^/]+)/#\1#;h;b};/timed out/{g;p}' file
a.b.com
c.d.com

聆雨淋夜

awk '/http:\/\//{a=gensub(/.*http:\/\/([^/]+)\//,"\\1",1,$0)}/timed out/{print a}' file
a.b.com
c.d.com

bianlimit

1. $ awk '/timed out.$/{split($3,t,"|");print t[1]}' 222.txt
   
2. a.b.com

复制代码

liumilan2009

回复 5# 聆雨淋夜

\\1是啥意思
查了半天，没查明白


   

聆雨淋夜

awk读取字符串"\\1"，得到\1，这个你就熟悉了，跟sed中的后向引用一样

songyc_2015

回复 1# liumilan2009

1. grep -Pzo '(?<=http://)[^/]+(?=/\s(.*\s)+.*timed out)' file

复制代码

yiranhendai

sed -nr '/timed out/{s#.*([a-z]{1,3}\.[a-z]{1,3}\.[a-z]{1,3}).*#\1#p}'