scripts: Enable inline records in breakpad symbols
Inline records allow breakpad to show the sequence of inlined frames
within each machine code frame. Consider a function that has the same
call site inlined multiple times (think of large functions with many
calls to templated //base accessors) e.g. crash/9a4654c665d1d07c if
there had been multiple callbacks invoked in that function. Currently,
the stack trace will show the crash in the machine code frame, and will
show a source line from the inlined frame, but will not distinguish
which inlined call is crashing. If inlined frames are enabled, the precise call site is shown.
An additional concern is that the presence of inlined frames, and the differences in their being inlined across varying builds causes magic signatures to shift unexpectedly, which makes it more difficult to
discern real, new crashes from changes in signatures. rnk@ has suggested
that Chrome is likely to increase usage of inlining over time.
Relatedly, broadly enabling inlined frames can also cause sudden,
painful shifts in magic signatures, which we observed when enabling for
Chrome on Windows and Linux. In anticipation of this event, we have
configured our magic signature analyzer to ignore inlined frames for
platforms not yet using them (including cros). We will instead seek to intentionally manage changes to magic signatures due to the appearance
of new inlined frames after these changes are landed and stable.
Inlined frames enabled for Chrome Windows and Chrome Linux led to a 2x
increase in breakpad symbol file size.
Dump performance on my machine for a recent build of Chrome Linux:
$ /usr/bin/time ~/breakpad/src/src/tools/linux/dump_syms/dump_syms chrome.debug > chrome.sym
139.06user 5.51system 2:24.66elapsed 99%CPU (0avgtext+0avgdata 13390680maxresident)k
96inputs+1290224outputs (8major+2457466minor)pagefaults 0swaps
$ /usr/bin/time ~/breakpad/src/src/tools/linux/dump_syms/dump_syms -d chrome.debug > chrome.sym
172.59user 12.61system 3:05.34elapsed 99%CPU (0avgtext+0avgdata 15389440maxresident)k
0inputs+2737864outputs (0major+4303021minor)pagefaults 0swaps
BUG=b:224839505
TEST=./run_tests
Change-Id: I9580ca6ef52379623d61b4f9ee78cb745bff4405
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/chromite/+/3655112
Commit-Queue: Joshua Peraza <jperaza@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Joshua Peraza <jperaza@chromium.org>
diff --git a/scripts/cros_generate_breakpad_symbols_unittest.py b/scripts/cros_generate_breakpad_symbols_unittest.py
index 3c025ea..5b589b4 100644
--- a/scripts/cros_generate_breakpad_symbols_unittest.py
+++ b/scripts/cros_generate_breakpad_symbols_unittest.py
@@ -253,7 +253,7 @@
self.elf_file, self.debug_file, self.breakpad_dir)
self.assertEqual(ret, self.sym_file)
self.assertEqual(self.rc.call_count, 1)
- self.assertCommandArgs(0, ['dump_syms', '-v', self.elf_file,
+ self.assertCommandArgs(0, ['dump_syms', '-v', '-d', self.elf_file,
self.debug_dir])
self.assertExists(self.sym_file)
@@ -290,13 +290,14 @@
def testLargeDebugFail(self):
"""Running w/large .debug failed, but retry worked"""
- self.rc.AddCmdResult(['dump_syms', '-v', self.elf_file, self.debug_dir],
+ self.rc.AddCmdResult(['dump_syms', '-v', '-d', self.elf_file,
+ self.debug_dir],
returncode=1)
ret = cros_generate_breakpad_symbols.GenerateBreakpadSymbol(
self.elf_file, self.debug_file, self.breakpad_dir)
self.assertEqual(ret, self.sym_file)
self.assertEqual(self.rc.call_count, 2)
- self.assertCommandArgs(0, ['dump_syms', '-v', self.elf_file,
+ self.assertCommandArgs(0, ['dump_syms', '-v', '-d', self.elf_file,
self.debug_dir])
self.assertCommandArgs(
1, ['dump_syms', '-v', '-c', '-r', self.elf_file, self.debug_dir])
@@ -304,7 +305,8 @@
def testDebugFail(self):
"""Running w/.debug always failed, but works w/out"""
- self.rc.AddCmdResult(['dump_syms', '-v', self.elf_file, self.debug_dir],
+ self.rc.AddCmdResult(['dump_syms', '-v', '-d', self.elf_file,
+ self.debug_dir],
returncode=1)
self.rc.AddCmdResult(['dump_syms', '-v', '-c', '-r', self.elf_file,
self.debug_dir],
@@ -313,7 +315,7 @@
self.elf_file, self.debug_file, self.breakpad_dir)
self.assertEqual(ret, self.sym_file)
self.assertEqual(self.rc.call_count, 3)
- self.assertCommandArgs(0, ['dump_syms', '-v', self.elf_file,
+ self.assertCommandArgs(0, ['dump_syms', '-v', '-d', self.elf_file,
self.debug_dir])
self.assertCommandArgs(
1, ['dump_syms', '-v', '-c', '-r', self.elf_file, self.debug_dir])