123 / 3 页下一页

菜鸟求助 [复制链接]

sunzhiguolu

巨富豪门

论坛徽章:: 307

11楼 [报告]

发表于 2016-07-29 11:44 |只看该作者

即，第1列有的内容是"字符串_字符串”；第9列是最后一个引号内“字符串_dup*”；这里的例子是单一，很多时候也是重复的，即“chr*_hap1”第二行仍是”chr*_hap1"。第9列也是一样，可能好几个“*_dup1”。数据仍是规律排列。

你将特殊的情况一同贴出一小部分, 然后将希望的到的结果贴出来瞧瞧. 看的更直观些.

实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？

jacqueslm2001

白手起家

论坛徽章:: 0

12楼 [报告]

发表于 2016-07-29 12:25 |只看该作者

我刚刚贴的附件里面test.rar里面的txt文件，所有类型都包含了。
8-15行，是第9列“*_dup1”的例子，有、无“_dup1”各重复4次；21-28行是“*_dup(1..7)”单一不重复的例子。
最后14行，是第1列变化类型，有、无“_*_*”各重复7次；16-20行是不重复的例子。
因此，test.txt已经包含了所有的情况。

非常感谢您的及时回复。

实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？

sunzhiguolu

巨富豪门

论坛徽章:: 307

13楼 [报告]

发表于 2016-07-29 13:19 |只看该作者

本帖最后由 sunzhiguolu 于 2016-07-29 13:20 编辑

#!/usr/bin/perl
use strict;
use warnings;
my ($regex, %hData) = qr/\b(\d+)\h+(\d+)\b/;
push (@{$hData{qq(@{[split]}[-1])}}, $_) while (<>);
open (my $fhF1, '>', './1.txt');
open (my $fhF2, '>', './2.txt');
foreach (keys %hData){
chomp (my @aData = @{$hData{$_}});
my ($sum, $min, $max, $tmp) = (0) x 3;
foreach my $offset (0 .. $#aData){
my ($n1, $n2) = $aData[$offset] =~ /$regex/;
$min = $n1 if (!$offset);
$max = $tmp if (($tmp = $n1 > $n2 ? $n1 : $n2) > $max);
$sum += $offset ? ($2 - $1 + 1) : ($2 - $1);
}
if (@aData == 1){
print $fhF1 "@aData $sum\n";
next;
}
print $fhF2 join ("\n", @aData[0 .. $#aData-1], ""), "$aData[-1] ", scalar (@aData), " $sum ", ($max - $min), "\n";
}
close ($fhF1);
close ($fhF2);

复制代码

perl abc.pl file
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

cat 1.txt
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
chr4 hg19_refGene exon 9345874 9347466 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup7";  1592
chrX       hg19_refGene       exon       13336768       13338518       0       -       .       gene_id "NM_001135995"; transcript_id "NM_001135995";  1750
chrX       hg19_refGene       exon       13608411       13608465       0       +       .       gene_id "NR_106734"; transcript_id "NR_106734";  54
chr4 hg19_refGene exon 9350619 9352211 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331";  1592
chr4 hg19_refGene exon 9326891 9328483 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup3";  1592
chr4 hg19_refGene exon 9364855 9366447 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup2";  1592
chr4 hg19_refGene exon 9331637 9333229 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup4";  1592
chr4 hg19_refGene exon 9355364 9356956 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup1";  1592
chr4 hg19_refGene exon 9336384 9337976 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup5";  1592
chr4 hg19_refGene exon 9341129 9342721 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup6";  1592

cat 2.txt
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
chrX hg19_refGene exon 119206241    119206810    0    -    .    gene_id "NM_032498"; transcript_id "NM_032498";
chrX hg19_refGene exon 119209865    119209910    0    -    .    gene_id "NM_032498"; transcript_id "NM_032498";
chrX hg19_refGene exon 119210842    119211253    0    -    .    gene_id "NM_032498"; transcript_id "NM_032498";
chrX hg19_refGene exon 119211439    119211707    0    -    .    gene_id "NM_032498"; transcript_id "NM_032498";  4 1296 5466
chrX       hg19_refGene       exon       13707240       13707407       0       +       .       gene_id "NM_004251"; transcript_id "NM_004251";
chrX       hg19_refGene       exon       13721933       13722021       0       +       .       gene_id "NM_004251"; transcript_id "NM_004251";
chrX       hg19_refGene       exon       13726840       13727944       0       +       .       gene_id "NM_004251"; transcript_id "NM_004251";  3 1361 20704
chrX hg19_refGene exon 119292467    119292735    0    +    .    gene_id "NM_032498"; transcript_id "NM_032498_dup1";
chrX hg19_refGene exon 119292921    119293332    0    +    .    gene_id "NM_032498"; transcript_id "NM_032498_dup1";
chrX hg19_refGene exon 119294264    119294309    0    +    .    gene_id "NM_032498"; transcript_id "NM_032498_dup1";
chrX hg19_refGene exon 119297364    119297933    0    +    .    gene_id "NM_032498"; transcript_id "NM_032498_dup1";  4 1296 5466
chrX       hg19_refGene       exon       13707240       13707407       0       +       .       gene_id "NM_001195328"; transcript_id "NM_001195328";
chrX       hg19_refGene       exon       13726840       13727944       0       +       .       gene_id "NM_001195328"; transcript_id "NM_001195328";  2 1272 20704
chr6 hg19_refGene exon 31633657       31633985       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31634598       31634680       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31635645       31635747       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31636316       31636431       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31636874       31636949       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31637105       31637285       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31637613       31637847       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3143277 3143605 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3144218 3144300 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3145265 3145367 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3145936 3146051 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3146494 3146569 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3146725 3146905 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3147233 3147467 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";  14 2245 4190
chr6 hg19_refGene exon 31804853       31804916       0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";
chr6_apd_hap1 hg19_refGene exon 3119608 3119671 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";
chr6_cox_hap2 hg19_refGene exon 3314427 3314490 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";
chr6_dbb_hap3 hg19_refGene exon 3090439 3090502 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";
chr6_qbl_hap6 hg19_refGene exon 3098509 3098572 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";  5 319 63

实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？

jacqueslm2001

白手起家

论坛徽章:: 0

14楼 [报告]

发表于 2016-07-29 13:25 |只看该作者

sunzhiguolu 发表于 2016-07-29 13:19
perl abc.pl file
---------------------------------------------------------------------------------- ...

还是不对，第1列不一样，应该也区分开。也就是数据要用第1列先判断，再用第9列判断，用双重标准回归最后的两个文件
还需要再麻烦您

实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？

sunzhiguolu

巨富豪门

论坛徽章:: 307

15楼 [报告]

发表于 2016-07-29 13:29 |只看该作者

你看下结果是不是你想要的那样, 我只是按照你的要求将最后一对双引号内的字符串作为 ID 进行分类统计的.

实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？

sunzhiguolu

巨富豪门

论坛徽章:: 307

16楼 [报告]

发表于 2016-07-29 13:32 |只看该作者

回复 14# jacqueslm2001
你所说的双重标准不是很理解, 能否将这个示例文件预期结果贴出来看下双重标准到底是个啥样?

实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？

jacqueslm2001

白手起家

论坛徽章:: 0

17楼 [报告]

发表于 2016-07-29 13:55 |只看该作者

sunzhiguolu 发表于 2016-07-29 13:32
回复 14# jacqueslm2001
你所说的双重标准不是很理解, 能否将这个示例文件预期结果贴出来看下双重标准[/ ...

比如：
chr6 hg19_refGene exon 31633657       31633985       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31634598       31634680       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31635645       31635747       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31636316       31636431       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31636874       31636949       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31637105       31637285       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31637613       31637847       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385"; 7 1122 4190
chr6_cox_hap2 hg19_refGene exon 3143277 3143605 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3144218 3144300 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3145265 3145367 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3145936 3146051 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3146494 3146569 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3146725 3146905 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3147233 3147467 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";  7 1122 4190
还有最后5行，且应该回归到1.txt：
chr6 hg19_refGene exon 31804853       31804916       0    +    .    gene_id "NR_002742"; transcript_id "NR_002742"; 1 63 63
chr6_apd_hap1 hg19_refGene exon 3119608 3119671 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742"; 1 63 63
chr6_cox_hap2 hg19_refGene exon 3314427 3314490 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742"; 1 63 63
chr6_dbb_hap3 hg19_refGene exon 3090439 3090502 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742"; 1 63 63
chr6_qbl_hap6 hg19_refGene exon 3098509 3098572 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";  1 63 63

最初以为只是第9列会不同，结果运算完回到数据本身发现第1列也不同，因此先要比对第1列是不是一样，再比对第9列，问题变成这么复杂以后，我感觉自己能力完全不具备修改您的代码，实现最终的正确结果的输出
真的麻烦您了

实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？

sunzhiguolu

巨富豪门

论坛徽章:: 307

18楼 [报告]

发表于 2016-07-29 14:05 |只看该作者

本帖最后由 sunzhiguolu 于 2016-07-29 14:17 编辑

#!/usr/bin/perl
use strict;
use warnings;
my ($regex, %hData) = qr/\b(\d+)\h+(\d+)\b/;
push (@{$hData{qq(@{[split]}[0,-1])}}, $_) while (<>);
open (my $fhF1, '>', './1.txt');
open (my $fhF2, '>', './2.txt');
foreach (keys %hData){
chomp (my @aData = @{$hData{$_}});
my ($sum, $min, $max) = (0, 9e9, 0);
foreach my $offset (0 .. $#aData){
my ($n1, $n2) = $aData[$offset] =~ /$regex/;
my @aSort = sort {$a <=> $b} ($n1, $n2);
$min = $aSort[0] if ($aSort[0] < $min);
$max = $aSort[-1] if ($aSort[-1] > $max);
$sum += $offset ? ($2 - $1 + 1) : ($2 - $1);
}
if (@aData == 1){
print $fhF1 "@aData $sum\n";
next;
}
print $fhF2 join ("\n", @aData[0 .. $#aData-1], ""), "$aData[-1] ", scalar (@aData), " $sum ", ($max - $min), "\n";
}
close ($fhF1);
close ($fhF2);

复制代码

perl abc.pl file

cat 1.txt
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
chr4 hg19_refGene exon 9331637 9333229 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup4";  1592
chrX       hg19_refGene       exon       13608411       13608465       0       +       .       gene_id "NR_106734"; transcript_id "NR_106734";  54
chr6_cox_hap2 hg19_refGene exon 3314427 3314490 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";  63
chr4 hg19_refGene exon 9326891 9328483 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup3";  1592
chr4 hg19_refGene exon 9336384 9337976 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup5";  1592
chr6_dbb_hap3 hg19_refGene exon 3090439 3090502 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";  63
chr4 hg19_refGene exon 9364855 9366447 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup2";  1592
chr4 hg19_refGene exon 9355364 9356956 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup1";  1592
chr4 hg19_refGene exon 9350619 9352211 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331";  1592
chr6_apd_hap1 hg19_refGene exon 3119608 3119671 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";  63
chr4 hg19_refGene exon 9345874 9347466 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup7";  1592
chr6_qbl_hap6 hg19_refGene exon 3098509 3098572 0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";  63
chr4 hg19_refGene exon 9341129 9342721 0    +    .    gene_id "NM_001242331"; transcript_id "NM_001242331_dup6";  1592
chrX       hg19_refGene       exon       13336768       13338518       0       -       .       gene_id "NM_001135995"; transcript_id "NM_001135995";  1750
chr6 hg19_refGene exon 31804853       31804916       0    +    .    gene_id "NR_002742"; transcript_id "NR_002742";  63

cat 2.txt
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
chrX hg19_refGene exon 119206241    119206810    0    -    .    gene_id "NM_032498"; transcript_id "NM_032498";
chrX hg19_refGene exon 119209865    119209910    0    -    .    gene_id "NM_032498"; transcript_id "NM_032498";
chrX hg19_refGene exon 119210842    119211253    0    -    .    gene_id "NM_032498"; transcript_id "NM_032498";
chrX hg19_refGene exon 119211439    119211707    0    -    .    gene_id "NM_032498"; transcript_id "NM_032498";  4 1296 5466
chr6_cox_hap2 hg19_refGene exon 3143277 3143605 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3144218 3144300 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3145265 3145367 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3145936 3146051 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3146494 3146569 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3146725 3146905 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6_cox_hap2 hg19_refGene exon 3147233 3147467 0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";  7 1122 4190
chrX       hg19_refGene       exon       13707240       13707407       0       +       .       gene_id "NM_001195328"; transcript_id "NM_001195328";
chrX       hg19_refGene       exon       13726840       13727944       0       +       .       gene_id "NM_001195328"; transcript_id "NM_001195328";  2 1272 20704
chrX hg19_refGene exon 119292467    119292735    0    +    .    gene_id "NM_032498"; transcript_id "NM_032498_dup1";
chrX hg19_refGene exon 119292921    119293332    0    +    .    gene_id "NM_032498"; transcript_id "NM_032498_dup1";
chrX hg19_refGene exon 119294264    119294309    0    +    .    gene_id "NM_032498"; transcript_id "NM_032498_dup1";
chrX hg19_refGene exon 119297364    119297933    0    +    .    gene_id "NM_032498"; transcript_id "NM_032498_dup1";  4 1296 5466
chr6 hg19_refGene exon 31633657       31633985       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31634598       31634680       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31635645       31635747       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31636316       31636431       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31636874       31636949       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31637105       31637285       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";
chr6 hg19_refGene exon 31637613       31637847       0    +    .    gene_id "NM_001282385"; transcript_id "NM_001282385";  7 1122 4190
chrX       hg19_refGene       exon       13707240       13707407       0       +       .       gene_id "NM_004251"; transcript_id "NM_004251";
chrX       hg19_refGene       exon       13721933       13722021       0       +       .       gene_id "NM_004251"; transcript_id "NM_004251";
chrX       hg19_refGene       exon       13726840       13727944       0       +       .       gene_id "NM_004251"; transcript_id "NM_004251";  3 1361 20704