perl的多线程怀疑内存泄漏，大家帮忙看看这个小程序是不是有问题？

aweizai

有一段perl代码用到了perl 的fork 生成多个子线程，客户执行后说是有很多子线程的执行被忽略掉了，我看了客户提供的执行日志，发现最后一个子线程在主线程关闭后反而能正常执行，怀疑子线程占用的资源是不是都要等到主线程结束后才释放，在本地测试了一下，重现不了这个问题，也没发现有内存泄漏。
大家忙我看看吧，如果有应该要怎么改？

1. #!/usr/local/bin/perl -w
   
2. 
   
3. use strict;
   
4. 
   
5. use POSIX ":sys_wait_h";
   
6. use threads::shared;
   
7. use JSON;
   
8. use Data::Dumper;
   
9. use Getopt::Long qw(GetOptions);
   
10. 
    
11. my @feeds = ();
    
12. my $children_num;
    
13. my $help;
    
14. my $debug;
    
15. my $verbose;
    
16. 
    
17. 
    
18. my $main_start = time();
    
19. 
    
20. print "\nStart process_feeds_multi.pl, main process pid is $$\n" if $debug;
    
21. 
    
22. my @children;
    
23. my $curr_children_num :shared = 0;
    
24. my $feeds_index :shared = 0;
    
25. my $front_pid;
    
26. 
    
27. $children_num = 1 unless $children_num;
    
28. @feeds = split ',', join ',', @feeds;
    
29. my $feeds_to_run;
    
30. 
    
31. pipe(RD,WT) or die "Can't open pipe: $!\n";
    
32. 
    
33. my $f_pid = fork();
    
34. if ( $f_pid ) {
    
35.     $front_pid = $f_pid;
    
36.     close WT;
    
37.     $feeds_to_run = jsonToObj(<RD>);
    
38. }
    
39. elsif ( $f_pid == 0) {
    
40.     if ( $debug ) {
    
41.         my $ppid = getppid();
    
42.         print "Create child process to get active feeds, its pid is $$ and parent pid is $ppid \n";
    
43.     }
    
44. 
    
45.      $feeds_to_run = &get_active_feeds(@feeds);
    
46. 
    
47.      if ( $verbose ) {
    
48.          my $feeds_str = join (', ', map { $_->{feed_id} } @$feeds_to_run);
    
49.          print "Active feeds: [ $feeds_str ] \n";
    
50.      }
    
51. 
    
52.     my $feeds_json = objToJson($feeds_to_run);
    
53.     close RD;
    
54.     select WT;
    
55.     $| = 1;
    
56.     print $feeds_json;
    
57.     exit 0;
    
58. }
    
59. else {
    
60.     die "couldn't fork:$!";
    
61. }
    
62. if ( waitpid( $front_pid, 0 ) ) {
    
63.     print "Finish getting active feeds, done with child-process $front_pid \n" if $debug;
    
64. 
    
65.     while ( $curr_children_num < ($children_num - 1) && $feeds_index < scalar (@$feeds_to_run) ) {
    
66.         &create_child();
    
67.     }
    
68. 
    
69.     while ( $feeds_index < scalar (@$feeds_to_run) ) {
    
70.         &create_child();
    
71.         my $child = undef;
    
72.         do {
    
73.             $child = waitpid(-1, WNOHANG);
    
74.         } until $child > 0;
    
75.     }
    
76. }
    
77. 
    
78. print "Done with parent-process $$, all child-processes are created.\n" if $debug;
    
79. 
    
80. exit;
    
81. 
    
82. sub create_child() {
    
83.     my $pid = fork();
    
84.     if ( $pid ) {
    
85.         $curr_children_num++;
    
86.         $feeds_index++;
    
87.         push ( @children, $pid );
    
88.     }
    
89.     elsif ( $pid == 0 ) {
    
90.         my $child_begin_time = time();
    
91.         if ( $debug ) {
    
92.             my $ppid = getppid();
    
93.             print "Create a new child-process, its pid is $$ and parent-process is $ppid \n";
    
94.         }
    
95. 
    
96.         my $feed_id = $tmp_data->{feed_id};
    
97.         print "Start producing feed $feed_id ...\n" if $debug;
    
98. 
    
99.         process_feed( $tmp_data );
    
100. 
     
101.         my $child_end_time = time();
     
102.         if ( $verbose ) {
     
103.             my $cost_time = $child_end_time - $child_begin_time;
     
104.             print "Done with producing " if $debug;
     
105.             print "feed $feed_id, cost $cost_time s\n";
     
106.         }
     
107. 
     
108.         print "Done with child-process $$ and exit.\n" if $debug;
     
109.         exit 0;
     
110.     }
     
111.     else {
     
112.         die "couldn't fork:$!";
     
113.     }
     
114.     return 1;
     
115. }
     
116. 
     
117. sub get_active_feeds {
     
118.     my @limit = @_;
     
119.         ...
     
120.     return \@feeds;
     
121. }
     
122. 
     
123. sub process_feed {
     
124.     ...
     
125. }
     
126. 
     

复制代码

运行日志大概如此

1. Start process_feeds_multi.pl, main process pid is 5706
   
2. Create child process to get active feeds, its pid is 5708 and parent pid is 5706
   
3. Get active feeds:
   
4. Active feeds: [ 34, 44, 62, 63, 65, 66, 67, 72, 78, 79, 84, 85, 86, 87, 88, 91, 92, 97, 100, 104, 106, 110, 115, 116, 119, 122, 132, 134, 135, 137, 138, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 154, 156, 157, 158, 159, 160, 161, 162, 163, 171, 172, 175, 176, 184, 185, 187, 188, 189, 190, 196, 200, 209, 210, 211, 214, 215, 227, 231, 232, 233, 234, 235, 236, 237, 238, 241, 247, 248, 249, 250, 253, 257, 258, 260, 267, 269, 271, 272, 274, 275, 278, 279, 280, 282, 283, 284, 285, 286, 287, 288, 292, 293, 294, 295, 300, 301, 302, 305, 306, 307, 309, 310, 311, 315, 318, 319, 322, 323, 324, 325, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 342, 343, 344, 345, 346, 347, 348, 349, 350, 356, 357, 358, 359, 360, 361, 362, 365, 368, 372, 375, 376, 377, 384, 386, 387, 388, 389, 390, 392, 394, 397, 398, 400, 402, 403, 404, 405, 406, 407, 409, 410, 412, 418, 419, 420, 421, 424, 427, 429, 431, 433, 434, 435, 436, 437, 438, 439, 445, 447, 448, 449, 450, 452, 453, 454, 456 ]
   
5. Finsh getting active feeds, done with child-process 5708
   
6. Create a new child-process, its pid is 5710 and parent-process is 5706
   
7. Start producing feed 44 ...
   
8. Done with producing feed 44, cost 27 s
   
9. Done with child-process 5710 and exit.
   
10. Create a new child-process, its pid is 5973 and parent-process is 5706
    
11. Start producing feed 62 ...
    
12. Done with producing feed 62, cost 1083 s
    
13. Done with child-process 5973 and exit.
    
14. Create a new child-process, its pid is 5709 and parent-process is 5706
    
15. Start producing feed 34 ...
    
16. Done with producing feed 34, cost 1111 s
    
17. Done with child-process 5709 and exit.
    
18. Create a new child-process, its pid is 8084 and parent-process is 5706
    
19. Start producing feed 65 ...
    
20. Done with producing feed 65, cost 789 s
    
21. Done with child-process 8084 and exit.
    
22. Create a new child-process, its pid is 9448 and parent-process is 5706
    
23. Start producing feed 66 ...
    
24. Done with producing feed 66, cost 73 s
    
25. Done with child-process 9448 and exit.
    
26. Create a new child-process, its pid is 9491 and parent-process is 5706
    
27. Start producing feed 67 ...
    
28. .....
    
29. .....
    
30. Create a new child-process, its pid is 21087 and parent-process is 5706
    
31. Start producing feed 196 ...
    
32. Create a new child-process, its pid is 21088 and parent-process is 5706
    
33. Start producing feed 200 ...
    
34. Create a new child-process, its pid is 21089 and parent-process is 5706
    
35. Start producing feed 209 ...
    
36. Create a new child-process, its pid is 21090 and parent-process is 5706
    
37. Start producing feed 210 ...
    
38. Create a new child-process, its pid is 21091 and parent-process is 5706
    
39. Start producing feed 211 ...
    
40. .....
    
41. .....
    
42. Create a new child-process, its pid is 21287 and parent-process is 5706
    
43. Start producing feed 454 ...
    
44. Done with parent-process 5706, all child-processes are created.
    
45. Create a new child-process, its pid is 21288 and parent-process is 5706
    
46. Start producing feed 456 ...
    
47. Create a new child-process, its pid is 20787 and parent-process is 5706
    
48. Start producing feed 184 ...
    
49. Done with producing feed 184, cost 723 s
    
50. Done with child-process 20787 and exit.

复制代码

zhlong8

fork 怎么成多线程了，没用 threads 用 :shared 有用吗

aweizai

回复 2# zhlong8


    有引用的，代码里面贴少这个引用了。
use threads::shared;

aweizai

应该说是进程更合适，或者ithread, waitpid好像是可以释放进程资源的，代码中已经用了waitpid了还会造成内存泄漏吗？

nsnake

多进程怎么使用线程的共享方法呢

nsnake

多进程怎么使用线程的共享方法呢

aweizai

回复 6# nsnake


    是的，通过测试，我也发现变量共享实际上没起作用。当初设置这个变量共享的目的是控制进程数。现在其实不共享的变量程序也能达到控制进程数的目的（后面是每结束一个进程再新建一个进程，所以我就还没去掉这个共享的），等之后做修改的时候我会把这些不合理的地方都修改一下。

现在主要的问题是内存泄漏，据我所知，只要用了waitpid或者wait，进程结束后，resource就会自动回收，这个程序里面是用了waitpid的，但是还是有内存泄漏，不知道怎么回事。