再问一个升级版的多列排序去重的问题

哭过的天空

有如下文本：
111.7.161.9 111.7.161.5 111.7.161.3 111.7.161.7 - hebeiyd
122.141.231.231 - heilongjianglt
111.40.197.24 111.40.197.23 - heilongjiangyd
111.7.161.5 111.7.161.7 111.7.161.9 111.7.161.3 - henanyd
180.97.153.148 - hubeidx
27.221.6.137 27.221.6.131 27.221.6.133 27.221.6.135 - hubeilt
117.144.232.3 117.144.232.5 117.144.232.7 117.144.232.9 - hubeiyi
119.147.41.33 - hunandx
183.95.87.1 153.37.233.1 - hunanlt
183.232.27.131 183.232.27.135 183.232.27.133 183.232.27.137 - hunanyd
221.204.166.7 221.204.166.5 221.204.166.9 221.204.166.3 - jiangsult
117.144.232.5 117.144.232.3 117.144.232.7 117.144.232.9 - jiangsuyd
36.248.21.1 153.37.233.1 - jiangxilt
183.232.27.133 183.232.27.137 183.232.27.131 183.232.27.135 - jiangxiyd
59.47.79.226 - jilindx
122.141.231.231 122.141.231.233 122.141.231.227 122.141.231.229 - jilinlt
111.26.20.102 111.26.20.115 111.26.20.113 111.26.20.109 111.26.20.111 111.26.20.103 - jilinyd
59.47.79.226 - liaoningdx
61.189.0.7 61.189.0.9 61.189.0.3 61.189.0.5 - liaoninglt
111.13.141.5 111.13.141.9 111.13.141.3 111.13.141.7 - liaoningyd
42.81.26.2 - neimenggudx
122.141.231.229 122.141.231.231 122.141.231.227 122.141.231.233 - neimenggult
120.193.146.110 - neimengguyd
61.185.45.3 - ningxiadx
221.204.166.9 221.204.166.3 221.204.166.7 221.204.166.5 - ningxialt
111.7.161.3 111.7.161.5 111.7.161.7 111.7.161.9 - ningxiayd
61.185.45.3 - qinghaidx
27.221.6.131 27.221.6.137 27.221.6.135 27.221.6.133 - qinghailt
111.12.251.174 111.12.251.176 - qinghaiyd
42.81.26.2 - shandongdx
60.217.235.175 - shandongjn
223.99.250.155 - shandongyd
221.204.166.7 221.204.166.9 221.204.166.3 221.204.166.5 - shanghailt
117.144.106.45 - shanghaiyd
221.204.166.3 221.204.166.5 221.204.166.7 221.204.166.9 - shanxi1lt
111.7.161.7 111.7.161.3 111.7.161.5 111.7.161.9 - shanxi1yd
221.204.166.7 221.204.166.3 221.204.166.9 221.204.166.5 - shanxi3lt
111.7.161.5 111.7.161.3 111.7.161.7 111.7.161.9 - shanxi3yd
119.84.68.68 - sichuandx
117.177.240.39 117.177.240.64 117.177.240.40 117.177.240.37 117.177.240.38 - sichuanyd
42.81.26.2 - tianjingdx
111.13.141.7 111.13.141.9 111.13.141.3 111.13.141.5 - tianjingyd
61.185.45.3 - xinjiangdx
221.204.166.7 221.204.166.3 221.204.166.5 221.204.166.9 - xinjianglt
111.13.141.9 111.13.141.3 111.13.141.7 111.13.141.5 - xinjiangyd


IP的列数不固定，IP的顺序不一样，
我想将IP去重，不考虑IP行的顺序，比如下面2行就是重复的，重复的把最后一列累计并排写后面

221.204.166.7 221.204.166.9 221.204.166.3 221.204.166.5 - shanghailt
221.204.166.3 221.204.166.5 221.204.166.7 221.204.166.9 - shanxi1lt
比如上面2行重复的处理后的结果是 ：

221.204.166.7 221.204.166.9 221.204.166.3 221.204.166.5 - shanghailt shanxi1lt


谢了各位

haooooaaa

1. result = {}
   
2. 
   
3. with open("file") as f:
   
4.     for i in f:
   
5.         if "-" in i:
   
6.             result.setdefault(tuple(sorted(i.split("-")[0].split())),[]).append(i.split("-")[-1].strip())
   
7. 
   
8. for i in result:
   
9.     print "%s - %s" % (" ".join(i), " ".join(result[i]))

复制代码

mswsg

顶顶楼上，我就不会打了。
不知道楼主考虑好了？这就是你想要的格式？
我猜，这样的格式会不会更好，
一个IP一行，后边跟着不同的地点（）回复 2# haooooaaa


   

haooooaaa

1. awk -F'[ -]+' 'function x(){asorti(a,b);s="";for(i in b)s=s b[i]" ";delete a}NF{for(i=1;i<NF;i++)a[$i];x();q[s]=(s in q)?q[s]" "$NF:$NF}END{for(i in q)print i"- "q[i]}'

复制代码

jason680

回复 1# 哭过的天空

$ awk 'function x(s, t,n,r){split(s,_x,FS);t=asort(_x);for(n=1;n<=t;++n)r=r _x[n]" ";return(r)}{v=$NF;NF-=2;k=x($0);if(!d[k]++)b[++c]=k;a[k]=a[k]v" "}END{for(n=1;n<=c;++n)print b[n]"- "a[b[n]]}' FILE
111.7.161.3 111.7.161.5 111.7.161.7 111.7.161.9 - hebeiyd henanyd ningxiayd shanxi1yd shanxi3yd
122.141.231.231 - heilongjianglt
...
119.84.68.68 - sichuandx
117.177.240.37 117.177.240.38 117.177.240.39 117.177.240.40 117.177.240.64 - sichuanyd

   

lll1985911

awk -F' - ' '{split($1,a," ");asort(a,b);for(i=1;i<=length(b);i++){sep=" ";if(i==1){sep=""};key=key""sep""b[i]};c[key]=c[key]" "$2;key=""}END{for(i in c){print i" "c[i]}}' FILE

zxy877298415

1. awk '{if(NR==1) {a[$0];next} s=0;for ( j in a) {p=0;l=split(j,c);for (i=1;i<NF-1;i++) {for (k=1;k<l-1;k++) if($i==c[k]) p++} if(NF-2==p) {a[j" "$NF];delete a[j];s=1}} if(s==0) {a[$0]}}END{for(i in a) print i}' file

复制代码

回复 1# 哭过的天空


   

sunzhiguolu

1. perl -e 'BEGIN{$re=qr/(.*) - (.*)/}{map {m/$re/;$c1 eq $1?print s/$c1 -//r:do{$c1=$1;print "\n$c1 - $2"}} sort map {@a=m/$re/g;$_=join(" ",sort split " ",$a[0])." - $a[-1]"} <>}' f

复制代码

111.12.251.174 111.12.251.176 - qinghaiyd
111.13.141.3 111.13.141.5 111.13.141.7 111.13.141.9 - liaoningyd tianjingyd xinjiangyd
111.26.20.102 111.26.20.103 111.26.20.109 111.26.20.111 111.26.20.113 111.26.20.115 - jilinyd
111.40.197.23 111.40.197.24 - heilongjiangyd
111.7.161.3 111.7.161.5 111.7.161.7 111.7.161.9 - hebeiyd henanyd ningxiayd shanxi1yd shanxi3yd
117.144.106.45 - shanghaiyd
117.144.232.3 117.144.232.5 117.144.232.7 117.144.232.9 - hubeiyi jiangsuyd
117.177.240.37 117.177.240.38 117.177.240.39 117.177.240.40 117.177.240.64 - sichuanyd
119.147.41.33 - hunandx
119.84.68.68 - sichuandx
120.193.146.110 - neimengguyd
122.141.231.227 122.141.231.229 122.141.231.231 122.141.231.233 - jilinlt neimenggult
122.141.231.231 - heilongjianglt
153.37.233.1 183.95.87.1 - hunanlt
153.37.233.1 36.248.21.1 - jiangxilt
180.97.153.148 - hubeidx
183.232.27.131 183.232.27.133 183.232.27.135 183.232.27.137 - hunanyd jiangxiyd
221.204.166.3 221.204.166.5 221.204.166.7 221.204.166.9 - jiangsult ningxialt shanghailt shanxi1lt shanxi3lt xinjianglt
223.99.250.155 - shandongyd
27.221.6.131 27.221.6.133 27.221.6.135 27.221.6.137 - hubeilt qinghailt
42.81.26.2 - neimenggudx shandongdx tianjingdx
59.47.79.226 - jilindx liaoningdx
60.217.235.175 - shandongjn
61.185.45.3 - ningxiadx qinghaidx xinjiangdx
61.189.0.3 61.189.0.5 61.189.0.7 61.189.0.9 - liaoninglt

哭过的天空

非常感谢各位：） shell的，python的,perl的都全啦：）