如何单步源码级调试Linux内核一直困扰着linux驱动开发人员，内核有其代码量大、逻辑复杂、与硬件交互的特性。因此，有着不同于应用程序的调试方法，据统计Linux内核开发者使用最普遍的调试方法是printk方法，而其他的方法大多是在特殊情况下才会使用。本文介绍的Kgdb调试方法是一种源码级的Linux内核调试器。使用Kgdb调试内核时，需要结合gdb一起使用，使用他们可以对内核进行单步调试，设置断点，观察变量、寄存器的值等。
首先安装ctyunos image，mirrors.ctyun.cn/ctyunos/ctyunos-2.0.1/ISO/x86_64/。安装之后，download ctyunos kernel代码，路径是mirrors.ctyun.cn/ctyunos/ctyunos-2.0.1/update_1022/sources/Packages/的kernel-4.19.90-2102.2.0.0062.ctl2.src.rpm。然后试用安装好的ctyunsos的/boot/config-4.19.90-2102.2.0.0062.ctl2.x86_64作为config基础文件，其中以下配置已经开启，CONFIG_KGDB_KDB(KGDB: console based KGDB support), CONFIG_KGDB_SERIAL_CONSOLE(Kernel hacking -> KGDB console support), CONFIG_DEBUG_KERNEL (Kernel hacking -> Kernel Debugging), CONFIG_DEBUG_INFO，make rpm-pkg开始编译rpm包。完成后将rpm包安装，修改/boot/efi/EFI/ctyunos/grub.cfg，对新装的kernel添加参数"console=ttyS0,115200 kgdboc=ttyS0,115200 nokaslr"，并选择新Kernel版本启动。启动后检查参数cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.19.95-2025novergw root=/dev/mapper/ctyunos-root ro rd.lvm.lv=ctyunos/root crashkernel=512M console=ttyS0,115200 kgdboc=ttyS0,115200 nokaslr
然后用串口线把ctyunos的机器(target机)和另一台linux机器(host机)连起来。在targe机上执行echo 1 > /proc/sys/kernel/sysrq和echo 1 > /sys/module/kdb/parameters/cmd_enable后，再执行echo g > /proc/sysrq-trigger。在host机上执行以下命令：
[root@localhost /root]#gdb vmlinux
(gdb) set serial baud 115200
(gdb) set debug remote 1
(gdb) target remote /dev/ttyUSB0
host的gdb和target的kgdb将进行交互，交互成功后便可以使用gdb调试。
对于没有串口线环境的，可以使用kgdboe。
download kgdboe代码并编译。
git clone github.com/sysprogs/kgdboe.git
cd kgdboe
make -C /lib/modules/$(uname -r)/build M=$(pwd)
在target上执行操作
cat /proc/kallsyms | grep kallsyms_lookup_name的到ffffffff8115b8b0 T kallsyms_lookup_name
sudo insmod kgdboe.ko kallsyms_lookup_name_address=0xffffffff8115b8b0 device_name=eno1 local_ip=192.168.1.111 udp_port=1248
得到dmesg消息
[ 217.141245] kgdboe: module verification failed: signature and/or required key missing - tainting kernel
[ 217.166529] netpoll: kgdboe: local IP 10.255.251.111
[ 217.178691] kgdboe: single-core mode enabled. Shutting down all cores except #0. This is slower, but safer.
[ 217.188423] kgdboe: you can try using multi-core mode by specifying the following argument:
[ 217.196769] insmod kgdboe.ko force_single_core = 0
......
[ 237.481742] KGDB: Registered I/O driver kgdboe
[ 237.491853] kgdboe: Successfully initialized. Use the following gdb command to attach:
[ 237.499769] target remote udp:192.168.1.111:1248
在host上执行以下操作
[root@localhost /root]#gdb vmlinux
(gdb) set debug remote 1
(gdb) target remote udp:192.168.1.111:1248
host的gdb和target的kgdb将进行交互，交互成功后便可以使用gdb调试。
#0 kgdb_breakpoint () at kernel/debug/debug_core.c:1077
#1 0xffffffffc07532ea in ?? ()
#2 0xffffffff810baebb in tasklet_action_common ([remote] Sending packet: $mffffc90001a43ec0,40#b2
[remote] Received Ack
[remote] Packet received: e0792482fffffffff83ca00100c9ffffc013c0078188ffff805b8a058188fffff6b40b81ffffffff25ce0d81ffffffff000000000000000060cd0d81ffffffff
tl_head=0xffff88848fe166f0, softirq_nr=6, a=<optimized out>) at kernel/softirq.c:522
#3 0xffffffff81c000e8 in __do_softirq () at kernel/softirq.c:292
#4 0xffffffff810bb4f6 in run_ksoftirqd (cpu=<optimized out>) at kernel/softirq.c:653
#5 run_ksoftirqd ([remote] Sending packet: $mffffc90001a43f00,40#80
[remote] Received Ack
[remote] Packet received: 804a88058188ffffc015d4078188ffff37920d81ffffffffb84a88058188ffff20910d81ffffffff0000000000000000c015d4078188ffff0000000000000000
cpu=<optimized out>) at kernel/softirq.c:645
#6 0xffffffff810dce25 in smpboot_thread_fn ([remote] Sending packet: $mffffc90001a43e80,40#87
[remote] Received Ack
[remote] Packet received: 400000000700000000512082ffffffffb0658a058188ffff2cc6010001000000805b8a058188ffff0a00000040802004805b8a058188ffffc013c0078188ffff
[remote] Sending packet: $mffffc90001a43f40,40#84
[remote] Received Ack
[remote] Packet received: 00000000000000000000000000000000ff01a081ffffffff00000000000000000000000000000000000000000000000000000000000000000000000000000000
data=0xffff888107c013c0) at kernel/smpboot.c:164
#7 0xffffffff810d9237 in kthread ([remote] Sending packet: $mffffffff81a001c0,40#1b
[remote] Received Ack
[remote] Packet received: 000100009d415f415e415d415c5b5de97cd761ff66662e0f1f840000000000904889c7e8786c6eff4885db750d4889e7e8fb3f60ffe93a0800004c89e7ffd390
_create=0xffff888107d415c0) at kernel/kthread.c:256
#8 0xffffffff81a001ff in ret_from_fork () at arch/x86/entry/entry_64.S:415
#9 0x0000000000000000 in ?? ()
