案例: debuginfo对gdb调试core文件的影响

背景 #

在解析进程的core文件时,需保证符号表等调试信息的完整

若该进程调用其他so文件,则需同时保证对应so文件的符号信息完整,才能正常解析出backtrace

通过file命令查看一个so文件,会见到以下信息:

libMathFunctions.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=4b65b2df11  
140ee1ab9534794afb29e40f859dc8, with debug_info, not stripped

其中with debug_infonot stripped会直接影响解析的backtrace完整度

本文将给出案例修改这两个维度信息,观察对解析backtrace的影响效果

结论: 仅with debug info且not stripped的so文件可解析全部的backtrace

案例 #

基于CMake编译环境,生成一个so文件与可执行文件

在源码中触发一个Segment Fault,生成相应的core文件

通过修改so文件的编译选项,控制debug info与stripped的效与否,查看能否正常解析backtrace

调试环境搭建 #

案例代码: https://github.com/wonderzyp/demo_for_blog/tree/main/core_demo

借用CMake Tutorial源码,做部分修改以控制debug info与stripped行为

同时,在MathFunctions.cxx添加下述操作,触发coredump

  std::cout<<"Enter here"<<std::endl;
  int *p = nullptr;
  *p = 855;

编译源码,生成可执行文件与so文件

mkdir build
cd build
cmake ..
make

设置core文件生成的名称规则

sudo su
# ulimit -c unlimited
# echo "/home/zyp/workplace/core_demo/build/core-%e-%p-%t" > /proc/sys/kernel/core_pattern

此后,执行生成的可执行文件,将在源码解引用空指针处crash,并在目录下生成core文件:

hp# ./Tutorial 23 
Enter here 
zsh: segmentation fault (core dumped)  ./Tutorial 23

# 生成 core-Tutorial-292294-1718625982 文件

可通过gdb直接解析此core文件:

hp# gdb ./Tutorial ./core-Tutorial-292294-1718625982  

GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2 
Copyright (C) 2020 Free Software Foundation, Inc.
For help, type "help". 
Type "apropos word" to search for commands related to "word"... 
Reading symbols from ./Tutorial... 
[New LWP 292294] 
Core was generated by `./Tutorial 23'. 
Program terminated with signal SIGSEGV, Segmentation fault. 
#0  0x00007f82bf0b3201 in mathfunctions::sqrt (x=23) at /home/zyp/workplace/Step11/MathFunctions/MathFunctions.cxx:17 
17      /home/zyp/workplace/Step11/MathFunctions/MathFunctions.cxx: No such file or directory. 



(gdb) bt 
#0  0x00007f82bf0b3201 in mathfunctions::sqrt (x=23) at /home/zyp/workplace/Step11/MathFunctions/MathFunctions.cxx:17 
#1  0x000055928eeed499 in main (argc=2, argv=0x7ffffe3db8a8) at /home/zyp/workplace/Step11/tutorial.cxx:22


 
(gdb) info sharedlibrary  
From                To                  Syms Read   Shared Object Library 
0x00007f82bf0b3100  0x00007f82bf0b3456  Yes         libMathFunctions.so 
0x00007f82bef52120  0x00007f82bf03a332  Yes (*)     /lib/x86_64-linux-gnu/libstdc++.so.6 
0x00007f82bee9c5e0  0x00007f82beead055  Yes (*)     /lib/x86_64-linux-gnu/libgcc_s.so.1 
0x00007f82becc9630  0x00007f82bee3e4bd  Yes         /lib/x86_64-linux-gnu/libc.so.6 
0x00007f82beb653c0  0x00007f82bec0bfa8  Yes         /lib/x86_64-linux-gnu/libm.so.6 
0x00007f82bf0ba100  0x00007f82bf0dc684  Yes         /lib64/ld-linux-x86-64.so.2 
(*): Shared library is missing debugging information. 
(gdb)

此时的Tutorial与libMathFunctions.so均为with debug info, not stripped

➜ file Tutorial
Tutorial: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=7335df8f67f51a038a08c0b9044de7b6a8b04627, for GNU/Linux 3.2.0, with debug_info, not stripped


➜ file libMathFunctions.so 
libMathFunctions.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=491ef9425b325d4dcd1e9a44c56fab52315de629, with debug_info, not stripped

改变debug info与stripped的 #

debug info 的作用 #

删除根目录CMakeLists.txt此处的-g参数:

  "$<${gcc_like_cxx}:$<BUILD_INTERFACE:-Wall;-Wextra;-Wshadow;-Wformat=2;-Wunused;-g>>"

重新cmake及make,即生成不含debug info的可执行文件与so文件

➜  build file Tutorial 
Tutorial: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Bui
ldID[sha1]=679fb4df889b9a7b041d86365b0886b0457215df, for GNU/Linux 3.2.0, not stripped 


➜  build file libMathFunctions.so  
libMathFunctions.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=99be6d0041f59097223
c549435d4003d2364e374, not stripped

重新执行上述流程,可见gdb解析出的backtrace变少,且提示缺少debugging symbols

hp# gdb ./Tutorial core-Tutorial-359953-1718626571 

GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2 
Copyright (C) 2020 Free Software Foundation, Inc.                                                                                  
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> 
This is free software: you are free to change and redistribute it. 
There is NO WARRANTY, to the extent permitted by law. 
Type "show copying" and "show warranty" for details. 
This GDB was configured as "x86_64-linux-gnu". 
Type "show configuration" for configuration details. 
For bug reporting instructions, please see: 
<http://www.gnu.org/software/gdb/bugs/>. 
Find the GDB manual and other documentation resources online at: 
    <http://www.gnu.org/software/gdb/documentation/>. 

For help, type "help". 
Type "apropos word" to search for commands related to "word"... 
Reading symbols from ./Tutorial... 
(No debugging symbols found in ./Tutorial) 
[New LWP 359953] 
Core was generated by `./Tutorial 99'. 
Program terminated with signal SIGSEGV, Segmentation fault. 
#0  0x00007fc1003b2201 in mathfunctions::sqrt(double) () from /home/zyp/workplace/core_demo/build/libMathFunctions.so
 
 
 
 (gdb) info sharedlibrary  
From                To                  Syms Read   Shared Object Library 
0x00007fc1003b2100  0x00007fc1003b2456  Yes (*)     /home/zyp/workplace/core_demo/build/libMathFunctions.so 
0x00007fc100251120  0x00007fc100339332  Yes (*)     /lib/x86_64-linux-gnu/libstdc++.so.6 
0x00007fc10019b5e0  0x00007fc1001ac055  Yes (*)     /lib/x86_64-linux-gnu/libgcc_s.so.1 
0x00007fc0fffc8630  0x00007fc10013d4bd  Yes         /lib/x86_64-linux-gnu/libc.so.6 
0x00007fc0ffe643c0  0x00007fc0fff0afa8  Yes         /lib/x86_64-linux-gnu/libm.so.6 
0x00007fc1003b9100  0x00007fc1003db684  Yes         /lib64/ld-linux-x86-64.so.2 
(*): Shared library is missing debugging information.

结论:with debug info直接影响解析backtrace的信息量

stripped 的作用 #

stripped会剥离文件的部分信息,将MathFunctions目录下的CMakeLists下述注释打开即可:

#add_custom_command(TARGET MathFunctions POST_BUILD
#  COMMAND ${CMAKE_STRIP} --strip-all $<TARGET_FILE:MathFunctions>
#)

重新编译,可见libMathFunctions.so已被stripped

hp# file libMathFunctions.so 
libMathFunctions.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=99be6d0041f59097223
c549435d4003d2364e374, stripped

运行gdb效果与无debug info一致,不再赘述。

引申:剥离debug info但保留链接 #

在编译过程中,可将so文件的debug_info保存为单个文件.sym,剥离实际运行so文件的debug_info

如此,实际运行的单个so文件容量更小,需要调试时连接对应的.sym即可

案例 #

libMathFunctions.so为例:

# 保留debuginfo至sym文件
cp libMathFunctions.so libMathFunctions.so.sym

# 剥离debug info
objcopy --strip-all ./libMathFunctions.so

# 添加link
objcopy --add-gnu-debuglink=libMathFunctions.so.sym ./libMathFunctions.so

可通过readelf查看链接情况:

root@zyp:/home/zyp/demo_for_blog/core_demo/build# readelf -x .gnu_debuglink ./libMathFunctions.so

Hex dump of section '.gnu_debuglink':
  0x00000000 6c69624d 61746846 756e6374 696f6e73 libMathFunctions
  0x00000010 2e736f2e 73796d00 22af787f          .so.sym.".x.

可见,该so文件的.gnu_debuglink指向libMathFunctions.so.sym

保证两者在同一目录下,即可正常解析backtrace