如何分析kernel call trace？

a_jige

写了一个简单的netlink 通信的模块，非常容易的就 OOPS了。
想从Log 里分析，但是不太会看，请问 kernel call trace 怎么看呢？ 还有内核的函数 哪里可以看到用法呢？

下面是我的oops的log，求大侠指点

Dec 18 21:11:01 localhost kernel: BUG: unable to handle kernel NULL pointer dereference at 00000004
Dec 18 21:11:01 localhost kernel: IP: [<c063978f>] skb_dequeue+0x39/0x4c
Dec 18 21:11:01 localhost kernel: *pde = 00000000
Dec 18 21:11:01 localhost kernel: Oops: 0002 [#1] SMP
Dec 18 21:11:01 localhost kernel: last sysfs file: /sys/devices/pci0000:00/0000:00:11.0/0000:02:01.0
/irq
Dec 18 21:11:01 localhost kernel: Dumping ftrace buffer:
Dec 18 21:11:01 localhost kernel:   (ftrace buffer empty)
Dec 18 21:11:01 localhost kernel: Modules linked in: nlkern ipt_MASQUERADE iptable_nat nf_nat sco br
idge stp bnep l2cap bluetooth sunrpc ipv6 lp fuse vmhgfs vsock vmmemctl dm_multipath uinput ppdev fl
oppy snd_ens1371 pcspkr gameport snd_rawmidi snd_ac97_codec ac97_bus snd_seq_dummy snd_seq_oss snd_s
eq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm pcnet32 parport_pc snd_timer
parport snd soundcore mii snd_page_alloc vmxnet i2c_piix4 i2c_core vmci pvscsi vmxnet3 mptspi mptscs
ih mptbase scsi_transport_spi [last unloaded: microcode]
Dec 18 21:11:01 localhost kernel:
Dec 18 21:11:01 localhost kernel: Pid: 4827, comm: a.out Not tainted (2.6.28.5 #3) VMware Virtual Pl
atform
Dec 18 21:11:01 localhost kernel: EIP: 0060:[<c063978f>] EFLAGS: 00210046 CPU: 1
Dec 18 21:11:01 localhost kernel: EIP is at skb_dequeue+0x39/0x4c
Dec 18 21:11:01 localhost kernel: EAX: f3bc906c EBX: e742e300 ECX: 00200246 EDX: 00000000
Dec 18 21:11:01 localhost kernel: ESI: f3bc906c EDI: f3bc9078 EBP: e7483ef0 ESP: e7483ee4
Dec 18 21:11:01 localhost kernel: DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Dec 18 21:11:01 localhost kernel: Process a.out (pid: 4827, ti=e7483000 task=e7408cc0 task.ti=e74830
00)
Dec 18 21:11:01 localhost kernel: Stack:
Dec 18 21:11:01 localhost kernel: f3bc906c f3bc9000 00000000 e7483efc c0639dc9 f3bc9000 e7483f08 c06
5396e
Dec 18 21:11:01 localhost kernel: f3bc9000 e7483f1c c063756c f3bc90d4 f3bc9000 00000000 e7483f24 c06
53742
Dec 18 21:11:01 localhost kernel: e7483f44 c06538f8 c09c3180 000012db 00000014 00000000 ec232c00 e74
2e9c0
Dec 18 21:11:01 localhost kernel: Call Trace:
Dec 18 21:11:01 localhost kernel: [<c0639dc9>] ? skb_queue_purge+0x14/0x1b
Dec 18 21:11:01 localhost kernel: [<c065396e>] ? netlink_sock_destruct+0x2c/0x93
Dec 18 21:11:01 localhost kernel: [<c063756c>] ? sk_free+0x14/0xbf
Dec 18 21:11:01 localhost kernel: [<c0653742>] ? sock_put+0x17/0x19
Dec 18 21:11:01 localhost kernel: [<c06538f8>] ? netlink_release+0x1b4/0x1be
Dec 18 21:11:01 localhost kernel: [<c063520d>] ? sock_release+0x14/0x60
Dec 18 21:11:01 localhost kernel: [<c0635275>] ? sock_close+0x1c/0x20
Dec 18 21:11:01 localhost kernel: [<c04948c1>] ? __fput+0xc9/0x159
Dec 18 21:11:01 localhost kernel: [<c0494968>] ? fput+0x17/0x19
Dec 18 21:11:01 localhost kernel: [<c0492204>] ? filp_close+0x50/0x5a
Dec 18 21:11:01 localhost kernel: [<c0492280>] ? sys_close+0x72/0xb1
Dec 18 21:11:01 localhost kernel: [<c04039bb>] ? sysenter_do_call+0x12/0x34
Dec 18 21:11:01 localhost kernel: Code: 1b f7 07 00 8b 1e 39 f3 89 c1 75 04 31 db eb 1e 85 db 74 1a
ff 4e 08 8b 13 8b 43 04 c7 03 00 00 00 00 c7 43 04 00 00 00 00 89 10 <89> 42 04 89 f8 89 ca e8 d6 f5
07 00 89 d8 5b 5e 5f 5d c3 55 89
Dec 18 21:11:01 localhost kernel: EIP: [<c063978f>] skb_dequeue+0x39/0x4c SS:ESP 0068:e7483ee4
Dec 18 21:11:01 localhost kernel: ---[ end trace 658d79fdbb2a05c9 ]---

我的应用层的进程叫 a.out， 内核的模块名叫 nlusr。 怎么分析这个呢？

bluempire

想从Log 里分析，但是不太会看，请问 kernel call trace 怎么看呢？
---  从底往上看。从log上看你的程序最终在 skb_dequeue+0x39/0x4c  挂掉，可以反汇编看看skb_dequeue 的0x39处的指令。当然到底是谁导致的oops，需要进一步分析，可能是SKB使用不当。

还有内核的函数 哪里可以看到用法呢？
--- linux code