一个应用程序导致了内核oops，请求帮助

bobozhang

我们有一个应用程序，一个线程负责接收另一个叫cu的服务器发送来的广播数据，称为A组线程；一个线程负责通过组播发送视频数据，一个线程负责通过组播发送音频数据，称为B组线程；还有一个线程通过cifs文件系统把音视频数据录入一个远程smb服务器，称为C组线程。可是最近却出现了问题。
当cu发送出来的广播数据是88bytes时，这三组线程同时跑没出现过oops；
但最近把cu发送来的广播数据改成了96bytes，如果上述三个线程组同时跑，则在几分钟之内必出oops，如果停掉三个中的任何一个，系统可以跑几天都不出oops, kernel打印出来的信息多数时间是显示系统调用recvfrom时出错了，有很少的时候显示其它地方oops了。

麻烦各位高手帮忙分析分析，到底是我们的这个应用程序的代码写得有问题还是内核本身有问题？(cpu是arm 9，内核版本2.6.14，内核裁减过，加入了几个跟音视频相关的内核模块)

Unable to handle kernel paging request at virtual address 1ee3cf32
pgd = c176c000
[1ee3cf32] *pgd=00000000
Internal error: Oops: 1 [#1]
Modules linked in: hi3511_pciv hi3511_jpege hi3511_h264d hi3511_h264e hi3511_chnl hi3511_ao hi3511_ai hi3511_sio hi3511_md hi3511_vdec hi3511_group hi3511_venc hi3511_vpp hi3511_dsu hi3511_vou hi3511_viu hi3511_sys hi3511_base rt61 bridge hiether libphy unix tde tlv_320aic31 tvp_5150 hidmac nls_iso8859_1 vfat fat nls_cp936 nls_cp437 nls_base hii2c hiwdt mmz
CPU: 0
PC is at skb_release_data+0x78/0x100
LR is at kfree_skbmem+0x14/0xcc
pc : [<c018b32c>]    lr : [<c018b080>]    Tainted: P   
sp : c1091cec  ip : c1091d04  fp : c1091d00
r10: 00000000  r9 : 00000000  r8 : c1091d48
r7 : c1091f2c  r6 : c103cac0  r5 : c33490a0  r4 : 00000000
r3 : dfe3a13e  r2 : c33ff8c0  r1 : c33ff8c0  r0 : 1ee3cf32
Flags: nzcv  IRQs on  FIQs on  Mode SVC_32  Segment user
Control: 5317F  Table: E176C000  DAC: 00000015
Process sunrise (pid: 743, stack limit = 0xc1090194)
Stack: (0xc1091cec to 0xc1092000)
1ce0:                            c33490a0 c33490a0 c1091d14 c1091d04 c018b080
1d00: c018b2c4 c3dd0160 c1091d2c c1091d18 c018b1dc c018b07c c33490a0 00000080
1d20: c1091d3c c1091d30 c018d898 c018b148 c1091d7c c1091d40 c01c9060 c018d888
1d40: 90af0608 c1091eac 00000000 00000000 00000080 c1091d8c c0231e9c 00000000
1d60: c1091f2c c0232aac 00000000 00000080 c1091dac c1091d80 c0189f80 c01c8df8
1d80: 00000000 00000000 c1091d8c 00000010 00000000 c1091f2c c104cd60 c1091e5c
1da0: c1091e9c c1091db0 c018636c c0189f44 00000000 c1091df0 c1091dc4 00000000
1dc0: 00000080 c104da40 c005a498 00000000 c1091f2c c32e50c0 60000013 000000d0
1de0: 00000000 c02245b0 00000000 00000000 000000d0 00000000 c0224934 c1091e54
1e00: c1091e0c 00000000 00000001 ffffffff 00000000 c3c7d828 00000000 00000000
1e20: 00000000 00000010 c104cd60 c104cd60 00000000 00000000 c104da58 c1030d60
1e40: 00001000 00000000 c1091e74 c1091e58 c005a1e0 c104cd60 c004ce9c c1091e5c
1e60: c1091e5c 00000345 c1091e84 c1091e78 c1091db4 00000000 00000080 4007a1b8
1e80: c104da40 c1091eac c1090000 4007a23c c1091f6c c1091ea0 c0187790 c01862b4
1ea0: 4007a238 00000000 c005ac30 00000002 00010000 00000000 00000000 c1091ec8
1ec0: c008a52c c008a02c c1090000 00000000 00001000 00000000 00000000 00001000
1ee0: c334446c c3344468 c3344464 c3344478 c3344474 c3344470 0000000d 00000258
1f00: 00000001 00000000 c1091f60 c1091f64 c008a078 c0a53000 00000000 00000004
1f20: 00000000 00000008 0000000c c1091eac 00000080 c1091ea0 00000001 00000000
1f40: 00000000 c005e240 bd7ffcb0 0000000b 00000000 00000066 c001fee4 4007a0bc
1f60: c1091fa4 c1091f70 c0187ebc c018771c 4007a23c bd7ffcb0 0000000c 4007a1b8
1f80: 00000080 00000000 4007a23c bd7ffcb0 bd7ffcb0 00000080 00000000 c1091fa8
1fa0: c001fd60 c0187d84 bd7ffcb0 c01e0ab0 0000000c bd7ffbd8 bd7ffcb0 4007a23c
1fc0: bd7ffcb0 00000080 00000000 00001c08 0000000c 00020170 4007a0bc bd7ffcc8
1fe0: 400e173c bd7ffbd0 400a81ec 400cdbc0 60000010 0000000c 00000000 00000000
Backtrace:
[<c018b2b4>] (skb_release_data+0x0/0x100) from [<c018b080>] (kfree_skbmem+0x14/0xcc)
r5 = C33490A0  r4 = C33490A0
[<c018b06c>] (kfree_skbmem+0x0/0xcc) from [<c018b1dc>] (__kfree_skb+0xa4/0x114)
r4 = C3DD0160
[<c018b138>] (__kfree_skb+0x0/0x114) from [<c018d898>] (skb_free_datagram+0x20/0x4c)
r5 = 00000080  r4 = C33490A0
[<c018d878>] (skb_free_datagram+0x0/0x4c) from [<c01c9060>] (udp_recvmsg+0x278/0x2e8)
[<c01c8de8>] (udp_recvmsg+0x0/0x2e8) from [<c0189f80>] (sock_common_recvmsg+0x4c/0x60)
[<c0189f34>] (sock_common_recvmsg+0x0/0x60) from [<c018636c>] (sock_recvmsg+0xc8/0xf8)
r7 = C1091E5C  r6 = C104CD60  r5 = C1091F2C  r4 = 00000000
[<c01862a4>] (sock_recvmsg+0x0/0xf8) from [<c0187790>] (sys_recvfrom+0x84/0xd8)
[<c018770c>] (sys_recvfrom+0x0/0xd8) from [<c0187ebc>] (sys_socketcall+0x148/0x1f0)
[<c0187d74>] (sys_socketcall+0x0/0x1f0) from [<c001fd60>] (ret_fast_syscall+0x0/0x2c)
r5 = 00000080  r4 = BD7FFCB0
Code: e1540003 2a000016 e0812184 e5920010 (e5903000)

[ 本帖最后由 bobozhang 于 2009-2-23 21:39 编辑 ]

xinglp

嵌入式啊

bobozhang

是啊，嵌入式阿，不稳定阿，麻烦啊

思一克

oops后你能重新启动程序运行？

bobozhang

多数时间可以重新运行那个应用程序，但有时候出现过一次oops后，运行其它任何程序都会立马出现oops

crazyervip

内核有问题，看看哪个内核模块崩掉的

epegasus

那就把oops贴出来...recvfrom出错是因为它执行的比较经常,
几个跟音视频相关的内核模块是否有并行访问临界区的代码,好好检查..
广播数据是什么?目的地址为同一个网段内的广播地址?A接受以后再拆分成视频和音频?

[ 本帖最后由 epegasus 于 2009-2-22 13:22 编辑 ]

bobozhang

广播数据是中心服务器的一个状态信息；目的地址为同一个网段内的广播地址；A接受的这个数据不是音视频数据，音视频数据是我们这个设备发送给别人的。
下面是Oops打印出来的信息，麻烦帮忙看看，谢谢！！！
Unable to handle kernel paging request at virtual address 1ee3cf32
pgd = c176c000
[1ee3cf32] *pgd=00000000
Internal error: Oops: 1 [#1]
Modules linked in: hi3511_pciv hi3511_jpege hi3511_h264d hi3511_h264e hi3511_chnl hi3511_ao hi3511_ai hi3511_sio hi3511_md hi3511_vdec hi3511_group hi3511_venc hi3511_vpp hi3511_dsu hi3511_vou hi3511_viu hi3511_sys hi3511_base rt61 bridge hiether libphy unix tde tlv_320aic31 tvp_5150 hidmac nls_iso8859_1 vfat fat nls_cp936 nls_cp437 nls_base hii2c hiwdt mmz
CPU: 0
PC is at skb_release_data+0x78/0x100
LR is at kfree_skbmem+0x14/0xcc
pc : [<c018b32c>]    lr : [<c018b080>]    Tainted: P   
sp : c1091cec  ip : c1091d04  fp : c1091d00
r10: 00000000  r9 : 00000000  r8 : c1091d48
r7 : c1091f2c  r6 : c103cac0  r5 : c33490a0  r4 : 00000000
r3 : dfe3a13e  r2 : c33ff8c0  r1 : c33ff8c0  r0 : 1ee3cf32
Flags: nzcv  IRQs on  FIQs on  Mode SVC_32  Segment user
Control: 5317F  Table: E176C000  DAC: 00000015
Process sunrise (pid: 743, stack limit = 0xc1090194)
Stack: (0xc1091cec to 0xc1092000)
1ce0:                            c33490a0 c33490a0 c1091d14 c1091d04 c018b080
1d00: c018b2c4 c3dd0160 c1091d2c c1091d18 c018b1dc c018b07c c33490a0 00000080
1d20: c1091d3c c1091d30 c018d898 c018b148 c1091d7c c1091d40 c01c9060 c018d888
1d40: 90af0608 c1091eac 00000000 00000000 00000080 c1091d8c c0231e9c 00000000
1d60: c1091f2c c0232aac 00000000 00000080 c1091dac c1091d80 c0189f80 c01c8df8
1d80: 00000000 00000000 c1091d8c 00000010 00000000 c1091f2c c104cd60 c1091e5c
1da0: c1091e9c c1091db0 c018636c c0189f44 00000000 c1091df0 c1091dc4 00000000
1dc0: 00000080 c104da40 c005a498 00000000 c1091f2c c32e50c0 60000013 000000d0
1de0: 00000000 c02245b0 00000000 00000000 000000d0 00000000 c0224934 c1091e54
1e00: c1091e0c 00000000 00000001 ffffffff 00000000 c3c7d828 00000000 00000000
1e20: 00000000 00000010 c104cd60 c104cd60 00000000 00000000 c104da58 c1030d60
1e40: 00001000 00000000 c1091e74 c1091e58 c005a1e0 c104cd60 c004ce9c c1091e5c
1e60: c1091e5c 00000345 c1091e84 c1091e78 c1091db4 00000000 00000080 4007a1b8
1e80: c104da40 c1091eac c1090000 4007a23c c1091f6c c1091ea0 c0187790 c01862b4
1ea0: 4007a238 00000000 c005ac30 00000002 00010000 00000000 00000000 c1091ec8
1ec0: c008a52c c008a02c c1090000 00000000 00001000 00000000 00000000 00001000
1ee0: c334446c c3344468 c3344464 c3344478 c3344474 c3344470 0000000d 00000258
1f00: 00000001 00000000 c1091f60 c1091f64 c008a078 c0a53000 00000000 00000004
1f20: 00000000 00000008 0000000c c1091eac 00000080 c1091ea0 00000001 00000000
1f40: 00000000 c005e240 bd7ffcb0 0000000b 00000000 00000066 c001fee4 4007a0bc
1f60: c1091fa4 c1091f70 c0187ebc c018771c 4007a23c bd7ffcb0 0000000c 4007a1b8
1f80: 00000080 00000000 4007a23c bd7ffcb0 bd7ffcb0 00000080 00000000 c1091fa8
1fa0: c001fd60 c0187d84 bd7ffcb0 c01e0ab0 0000000c bd7ffbd8 bd7ffcb0 4007a23c
1fc0: bd7ffcb0 00000080 00000000 00001c08 0000000c 00020170 4007a0bc bd7ffcc8
1fe0: 400e173c bd7ffbd0 400a81ec 400cdbc0 60000010 0000000c 00000000 00000000
Backtrace:
[<c018b2b4>] (skb_release_data+0x0/0x100) from [<c018b080>] (kfree_skbmem+0x14/0xcc)
r5 = C33490A0  r4 = C33490A0
[<c018b06c>] (kfree_skbmem+0x0/0xcc) from [<c018b1dc>] (__kfree_skb+0xa4/0x114)
r4 = C3DD0160
[<c018b138>] (__kfree_skb+0x0/0x114) from [<c018d898>] (skb_free_datagram+0x20/0x4c)
r5 = 00000080  r4 = C33490A0
[<c018d878>] (skb_free_datagram+0x0/0x4c) from [<c01c9060>] (udp_recvmsg+0x278/0x2e8)
[<c01c8de8>] (udp_recvmsg+0x0/0x2e8) from [<c0189f80>] (sock_common_recvmsg+0x4c/0x60)
[<c0189f34>] (sock_common_recvmsg+0x0/0x60) from [<c018636c>] (sock_recvmsg+0xc8/0xf8)
r7 = C1091E5C  r6 = C104CD60  r5 = C1091F2C  r4 = 00000000
[<c01862a4>] (sock_recvmsg+0x0/0xf8) from [<c0187790>] (sys_recvfrom+0x84/0xd8)
[<c018770c>] (sys_recvfrom+0x0/0xd8) from [<c0187ebc>] (sys_socketcall+0x148/0x1f0)
[<c0187d74>] (sys_socketcall+0x0/0x1f0) from [<c001fd60>] (ret_fast_syscall+0x0/0x2c)
r5 = 00000080  r4 = BD7FFCB0
Code: e1540003 2a000016 e0812184 e5920010 (e5903000)

[ 本帖最后由 bobozhang 于 2009-2-23 21:38 编辑 ]