shell高效提取页面信息

dongh

小弟想要用shell去提取页面 http://www.verycd.com/topics/2756367/ 的信息， 信息内容包括 中文名、英文名、地区、 语言、 IMDB得分、 导演、发行日期 等等， 以key-value的方式提取出来。

curl  http://www.verycd.com/topics/2756367/  出来的内容中大致有下面内容：

</div><div class="iptcom" id="iptcomCname"><span class="iptcom-title"><strong>中文名</strong>:&nbsp;</span>先知</div><div class=
"iptcom" id="iptcomEname"><span class="iptcom-title"><strong>英文名</strong>:&nbsp;</span>Knowing</div><div class="iptcom" id="iptcomAlias">
<span class="iptcom-title"><strong>别名</strong>:&nbsp;</span>先知/地球毁灭密码(港)/末日预言(台)/预知/预难大逃亡</div><div class="iptcom" id
="iptcomIMDb"><span class="iptcom-title"><strong>IMDb</strong>:&nbsp;</span><a href="http://www.imdb.com/title/tt448011/" target="_blank">6.
6/10&nbsp;(27,921&nbsp;votes)</a></div><div class="iptcom" id="iptcomFiletype"><span class="iptcom-title"><strong>资源格式</strong>:&nbsp;</
span>720P</div><div class="iptcom" id="iptcomVersion"><span class="iptcom-title"><strong>版本</strong>:&nbsp;</span>思路</div><div class="ip
tcom" id="iptcomTime"><span class="iptcom-title"><strong>发行日期</strong>:&nbsp;</span>2009年</div><div class="iptcom-multiline" id="iptcom
Director"><span class="iptcom-title"><strong>导演</strong>:&nbsp;</span><span class="iptcom-info"><a href="http://www.verycd.com/search/star
/%E4%BA%9A%E5%8E%86%E5%85%8B%E6%96%AF%C2%B7%E6%99%AE%E7%BD%97%E4%BA%9A%E6%96%AF+Alex+Proyas" target="_blank" onclick="pageTracker._trackPage
view('/stat/topicsSearchStar/')">亚历克斯·普罗亚斯 Alex Proyas</a></span></div><div class="iptcom-multiline" id="iptcomActor"><span class="i
ptcom-title"><strong>演员</strong>:&nbsp;</span><span class="iptcom-info"><a href="http://www.verycd.com/search/star/%E5%B0%BC%E5%8F%A4%E6%8
B%89%E6%96%AF%C2%B7%E5%87%AF%E5%A5%87+Nicolas+Cage" target="_blank" onclick="pageTracker._trackPageview('/stat/topicsSearchStar/')">尼古拉斯
·凯奇 Nicolas Cage ....Ted Myles</a><br /><a href="http://www.verycd.com/search/star/%E8%90%9D%E4%B8%9D%C2%B7%E6%8B%9C%E6%81%A9+Rose+Byrne"
target="_blank" onclick="pageTracker._trackPageview('/stat/topicsSearchStar/')">萝丝·拜恩 Rose Byrne ....Diana Whelan</a><br /><a href="http
://www.verycd.com/search/star/%E9%92%B1%E5%BE%B7%E5%8B%92%C2%B7%E5%9D%8E%E7%89%B9%E5%B8%83%E7%91%9E+Chandler+Canterbury" target="_blank" onc
lick="pageTracker._trackPageview('/stat/topicsSearchStar/')">钱德勒·坎特布瑞 Chandler Canterbury ....Caleb</a><br /><a href="http://www.very
cd.com/search/star/Tamara+Donnellan" target="_blank" onclick="pageTracker._trackPageview('/stat/topicsSearchStar/')">Tamara Donnellan ....Lu
cinda's mother</a><br /><a href="http://www.verycd.com/search/star/%E6%9C%AC%C2%B7%E9%97%A8%E5%BE%B7%E5%B0%94%E6%A3%AE+Ben+Mendelsohn" targe
t="_blank" onclick="pageTracker._trackPageview('/stat/topicsSearchStar/')">本·门德尔森 Ben Mendelsohn ....Phil Bergman</a><br /><a href="htt
p://www.verycd.com/search/star/Adrienne+Pickering" target="_blank" onclick="pageTracker._trackPageview('/stat/topicsSearchStar/')">Adrienne
Pickering ....Allison</a><br /><a href="http://www.verycd.com/search/star/Angie+Diaz" target="_blank" onclick="pageTracker._trackPageview('/
stat/topicsSearchStar/')">Angie Diaz ....Female Reporter</a><br /><a href="http://www.verycd.com/search/star/Brett+Robson" target="_blank" o
nclick="pageTracker._trackPageview('/stat/topicsSearchStar/')">Brett Robson ....University Teacher / Parent</a><br /><a href="http://www.ver
ycd.com/search/star/%E5%88%A9%E4%BA%9A%E5%A7%86%C2%B7%E6%B5%B7%E8%8E%AB%E6%96%AF%E6%B2%83%E6%96%AF+Liam+Hemsworth" target="_blank" onclick="
pageTracker._trackPageview('/stat/topicsSearchStar/')">利亚姆·海莫斯沃斯 Liam Hemsworth ....Spencer</a></span></div><div class="iptcom" id="
iptcomCountry"><span class="iptcom-title"><strong>地区</strong>:&nbsp;</span><a href="http://www.verycd.com/search/area/usa" target="_blank"
onclick="pageTracker._trackPageview('/stat/topicsSearchArea/')">美国</a></div><div class="iptcom" id="iptcomLanguage"><span class="iptcom-t
itle"><strong>语言</strong>:&nbsp;</span><a href="http://www.verycd.com/search/lang/english" target="_blank" onclick="pageTracker._trackPage
view('/stat/topicsSearchLang/')">英语</a></div><div class="iptcom" id="iptcomContents"><span class="iptcom-title"><strong>简介</strong>:&nbs
p;</span><br />        <!--Wrap-head begin-->


主要是提取   </span>    </div> 这两个标识符中间的内容。
请问各位大哥有什么好办法去做这个事情？

blackold

凑合着用

1. curl ...| awk -v RS="</?div>" '/<strong>/&&!/script/{sub(/^.*<strong>/,"");gsub(/<br *\/>/,"\n");gsub(/&nbsp;/,"");gsub(/<[a-z]*\/?[^>]*>/,"");sub(/:/,"-");print}'

复制代码

[ 本帖最后由 blackold 于 2009-7-10 12:44 编辑 ]

dongh

perl不会用啊， 现在就会用shell，呵呵

lucash

egrep -o '<\/span>(.*)<\/div' rec

dongh

貌似都不行啊，呵呵， grep肯定不行，   我要得不是匹配这一行，而是取出一行里面的某些字符

lucash

那就把4楼的圆括号改在.*上面

kwokcn

这个？

1. 
   
2. grep -Po '(?<=</span>)[^<]*(?=</div>)' URFILE
   

复制代码

lucash

grep也行：
grep -oP '(?<=<\/span>).*?(?=<\/div)' rec


呵呵 ，发现和狼大哥撞上了

[ 本帖最后由 lucash 于 2009-7-10 12:15 编辑 ]