Pid: 28045, comm: unbound Not tainted 2.6.28.8 #26

pigcrying

unbound，四个相同进程运行，出错后一个进程为<defunct>状态，其他三个能正常工作。

内存使用情况
Memory:
Mem: 8100728k total, 6562480k used, 1538248k free
Swap:  0k total, 0k used, 0k free

从日志信息中看出是跟内存有关，内存使用空间占满，对页的分配失败而产生分页错误。
但是为什么会产生unbound<defunct>状态，这个是个僵死状态。

第一次遇见这个问题，请各位高手解答！

/var/log/messages信息如下：
automount invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
Jun  1 02:34:31 root kernel:  [<ffffffff80270d1f>] __alloc_pages_internal+0x31b/0x3c0
Jun  1 02:34:31 root kernel:  [<ffffffff802724a8>] __do_page_cache_readahead+0x7c/0x161
Jun  1 02:34:31 root kernel:  [<ffffffff8026ce5e>] filemap_fault+0x15d/0x33b
Jun  1 02:34:31 root kernel:  [<ffffffff80278adb>] __do_fault+0x51/0x3b2
Jun  1 02:34:31 root kernel:  [<ffffffff8027a767>] handle_mm_fault+0x36e/0x72e
Jun  1 02:34:31 root kernel:  [<ffffffff80456cac>] do_page_fault+0x3d4/0x796
Jun  1 02:34:32 root kernel:  [<ffffffff80262433>] audit_syscall_exit+0x2e4/0x302
Jun  1 02:34:32 root kernel:  [<ffffffff80454cd9>] error_exit+0x0/0x51
Jun  1 02:34:32 root kernel: Mem-Info:
Jun  1 02:34:32 root kernel: Node 0 DMA per-cpu:
Jun  1 02:34:32 root kernel: CPU    0: hi:    0, btch:   1 usd:   0
Jun  1 02:34:32 root kernel: CPU    1: hi:    0, btch:   1 usd:   0
Jun  1 02:34:32 root kernel: CPU    2: hi:    0, btch:   1 usd:   0
Jun  1 02:34:32 root kernel: CPU    3: hi:    0, btch:   1 usd:   0
Jun  1 02:34:32 root kernel: Node 0 DMA32 per-cpu:
Jun  1 02:34:32 root kernel: CPU    0: hi:  186, btch:  31 usd: 144
Jun  1 02:34:32 root kernel: CPU    1: hi:  186, btch:  31 usd: 162
Jun  1 02:34:32 root kernel: CPU    2: hi:  186, btch:  31 usd: 156
Jun  1 02:34:32 root kernel: CPU    3: hi:  186, btch:  31 usd: 155
Jun  1 02:34:32 root kernel: Node 0 Normal per-cpu:
Jun  1 02:34:32 root kernel: CPU    0: hi:  186, btch:  31 usd: 147
Jun  1 02:34:32 root kernel: CPU    1: hi:  186, btch:  31 usd: 177
Jun  1 02:34:32 root kernel: CPU    2: hi:  186, btch:  31 usd: 156
Jun  1 02:34:32 root kernel: CPU    3: hi:  186, btch:  31 usd: 168
Jun  1 02:34:32 root kernel: Active_anon:1779068 active_file:0 inactive_anon:218835
Jun  1 02:34:32 root kernel:  inactive_file:26 unevictable:0 dirty:0 writeback:0 unstable:0
Jun  1 02:34:32 root kernel:  free:10241 slab:3165 mapped:1 pagetables:4270 bounce:0
Jun  1 02:34:32 root kernel: lowmem_reserve[]: 0 2962 7949 7949
Jun  1 02:34:32 root kernel: lowmem_reserve[]: 0 0 4986 4986
Jun  1 02:34:32 root kernel: lowmem_reserve[]: 0 0 0 0
Jun  1 02:34:32 root kernel: Swap cache stats: add 0, delete 0, find 0/0
Jun  1 02:34:32 root kernel: 56537 pages reserved
Jun  1 02:34:32 root kernel: Killed process 6452 (automount)
Jun  1 02:34:33 root kernel:  [<ffffffff8026db8d>] out_of_memory+0x1e5/0x24f
Jun  1 02:34:33 root kernel:  [<ffffffff80270d1f>] __alloc_pages_internal+0x31b/0x3c0
Jun  1 02:34:33 root kernel:  [<ffffffff802724a8>] __do_page_cache_readahead+0x7c/0x161
Jun  1 02:34:33 root kernel:  [<ffffffff8026ce5e>] filemap_fault+0x15d/0x33b
Jun  1 02:34:33 root kernel:  [<ffffffff80278adb>] __do_fault+0x51/0x3b2
Jun  1 02:34:33 root kernel:  [<ffffffff8027a767>] handle_mm_fault+0x36e/0x72e
Jun  1 02:34:33 root kernel:  [<ffffffff80456cac>] do_page_fault+0x3d4/0x796
Jun  1 02:34:33 root kernel:  [<ffffffff80454cd9>] error_exit+0x0/0x51
Jun  1 02:34:33 root kernel: Mem-Info:
Jun  1 02:34:33 root kernel: Node 0 DMA per-cpu:
Jun  1 02:34:33 root kernel: CPU    0: hi:    0, btch:   1 usd:   0
Jun  1 02:34:33 root kernel: CPU    1: hi:    0, btch:   1 usd:   0
Jun  1 02:34:33 root kernel: CPU    2: hi:    0, btch:   1 usd:   0
Jun  1 02:34:33 root kernel: CPU    3: hi:    0, btch:   1 usd:   0
Jun  1 02:34:33 root kernel: Node 0 DMA32 per-cpu:
Jun  1 02:34:33 root kernel: CPU    0: hi:  186, btch:  31 usd: 146
Jun  1 02:34:33 root kernel: CPU    1: hi:  186, btch:  31 usd: 162
Jun  1 02:34:33 root kernel: CPU    2: hi:  186, btch:  31 usd: 161
Jun  1 02:34:33 root kernel: CPU    3: hi:  186, btch:  31 usd: 170
Jun  1 02:34:33 root kernel: Node 0 Normal per-cpu:
Jun  1 02:34:33 root kernel: CPU    0: hi:  186, btch:  31 usd: 181
Jun  1 02:34:33 root kernel: CPU    1: hi:  186, btch:  31 usd: 182
Jun  1 02:34:33 root kernel: CPU    2: hi:  186, btch:  31 usd: 168
Jun  1 02:34:33 root kernel: CPU    3: hi:  186, btch:  31 usd: 144
Jun  1 02:34:33 root kernel: Active_anon:1761168 active_file:17 inactive_anon:236681
Jun  1 02:34:33 root kernel:  inactive_file:54 unevictable:0 dirty:0 writeback:0 unstable:0
Jun  1 02:34:33 root kernel:  free:10210 slab:3140 mapped:0 pagetables:4250 bounce:0
Jun  1 02:34:33 root kernel:  [<ffffffff80278adb>] __do_fault+0x51/0x3b2
Jun  1 02:34:33 root kernel:  [<ffffffff8027a767>] handle_mm_fault+0x36e/0x72e
Jun  1 02:34:33 root kernel:  [<ffffffff80456cac>] do_page_fault+0x3d4/0x796
Jun  1 02:34:33 root kernel:  [<ffffffff80454cd9>] error_exit+0x0/0x51
Jun  1 02:34:33 root kernel: Mem-Info:
Jun  1 02:34:33 root kernel: Node 0 DMA per-cpu:
Jun  1 02:34:33 root kernel: CPU    0: hi:    0, btch:   1 usd:   0
Jun  1 02:34:33 root kernel: CPU    1: hi:    0, btch:   1 usd:   0
Jun  1 02:34:33 root kernel: CPU    2: hi:    0, btch:   1 usd:   0
Jun  1 02:34:33 root kernel: CPU    3: hi:    0, btch:   1 usd:   0
Jun  1 02:34:33 root kernel: Node 0 DMA32 per-cpu:
Jun  1 02:34:33 root kernel: CPU    0: hi:  186, btch:  31 usd: 146
Jun  1 02:34:33 root kernel: CPU    1: hi:  186, btch:  31 usd: 162
Jun  1 02:34:33 root kernel: CPU    2: hi:  186, btch:  31 usd: 161
Jun  1 02:34:33 root kernel: CPU    3: hi:  186, btch:  31 usd: 170
Jun  1 02:34:33 root kernel: Node 0 Normal per-cpu:
Jun  1 02:34:33 root kernel: CPU    0: hi:  186, btch:  31 usd: 181
Jun  1 02:34:33 root kernel: CPU    1: hi:  186, btch:  31 usd: 182
Jun  1 02:34:33 root kernel: CPU    2: hi:  186, btch:  31 usd: 168
Jun  1 02:34:33 root kernel: CPU    3: hi:  186, btch:  31 usd: 144
Jun  1 02:34:33 root kernel: Active_anon:1761168 active_file:17 inactive_anon:236681
Jun  1 02:34:33 root kernel:  inactive_file:54 unevictable:0 dirty:0 writeback:0 unstable:0
Jun  1 02:34:33 root kernel:  free:10210 slab:3140 mapped:0 pagetables:4250 bounce:0
Jun  1 02:34:33 root kernel: Node 0 DMA free:9808kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:8868kB pages_scanned:0 all_unreclaimable? yes
Jun  1 02:34:33 root kernel: lowmem_reserve[]: 0 2962 7949 7949
Jun  1 02:34:33 root kernel: Node 0 DMA32 free:24000kB min:4248kB low:5308kB high:6372kB active_anon:2537760kB inactive_anon:338544kB active_file:8kB inactive_file:0kB unevictable:0kB present:3033908kB pages_scanned:720172 all_unreclaimable? no
Jun  1 02:34:33 root kernel: lowmem_reserve[]: 0 0 4986 4986
Jun  1 02:34:33 root kernel: Node 0 Normal free:7032kB min:7152kB low:8940kB high:10728kB active_anon:4506912kB inactive_anon:608180kB active_file:60kB inactive_file:216kB unevictable:0kB present:5106560kB pages_scanned:786203 all_unreclaimable? no
Jun  1 02:34:33 root kernel: lowmem_reserve[]: 0 0 0 0
Jun  1 02:34:33 root kernel: Node 0 DMA: 4*4kB 4*8kB 2*16kB 4*32kB 2*64kB 2*128kB 2*256kB 1*512kB 2*1024kB 1*2048kB 1*4096kB = 9808kB
Jun  1 02:34:33 root kernel: Node 0 DMA32: 24*4kB 4*8kB 2*16kB 27*32kB 31*64kB 36*128kB 20*256kB 6*512kB 0*1024kB 0*2048kB 2*4096kB = 24000kB
Jun  1 02:34:33 root kernel: Node 0 Normal: 8*4kB 3*8kB 5*16kB 1*32kB 2*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 1*4096kB = 7080kB
Jun  1 02:34:33 root kernel: 80 total pagecache pages
Jun  1 02:34:33 root kernel: 0 pages in swap cache
Jun  1 02:34:33 root kernel: Swap cache stats: add 0, delete 0, find 0/0
Jun  1 02:34:33 root kernel: Free swap  = 0kB
Jun  1 02:34:33 root kernel: Total swap = 0kB
Jun  1 02:34:33 root kernel: 2080768 pages RAM
Jun  1 02:34:33 root kernel: 56537 pages reserved
Jun  1 02:34:33 root kernel: 944 pages shared
Jun  1 02:34:33 root kernel: 2011946 pages non-shared
Jun  1 02:34:33 root kernel: Out of memory: kill process 2498 (python) score 483 or a child
Jun  1 02:34:33 root kernel: Killed process 2498 (python)
Jun  1 02:34:33 root kernel: unbound invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
Jun  1 02:34:33 root kernel: Pid: 28044, comm: unbound Not tainted 2.6.28.8-root #26
Jun  1 02:34:33 root kernel: Call Trace:
Jun  1 02:34:33 root kernel:  [<ffffffff8026d653>] oom_kill_process+0x57/0x1e5
Jun  1 02:34:33 root kernel:  [<ffffffff8024d88b>] getnstimeofday+0x53/0xb2
Jun  1 02:34:33 root kernel:  [<ffffffff8026d969>] badness+0x16a/0x1a9
Jun  1 02:34:33 root kernel:  [<ffffffff8026db8d>] out_of_memory+0x1e5/0x24f
Jun  1 02:34:33 root kernel:  [<ffffffff80270d1f>] __alloc_pages_internal+0x31b/0x3c0
Jun  1 02:34:33 root kernel:  [<ffffffff802724a8>] __do_page_cache_readahead+0x7c/0x161
Jun  1 02:34:33 root kernel:  [<ffffffff8026ce5e>] filemap_fault+0x15d/0x33b
Jun  1 02:34:33 root kernel:  [<ffffffff80278adb>] __do_fault+0x51/0x3b2
Jun  1 02:34:33 root kernel:  [<ffffffff8027a767>] handle_mm_fault+0x36e/0x72e
Jun  1 02:34:33 root kernel:  [<ffffffff80456cac>] do_page_fault+0x3d4/0x796
Jun  1 02:34:33 root kernel:  [<ffffffff80454cd9>] error_exit+0x0/0x51