cbuildbot: Report build stage failures to cidb.
Build stage failures should be reported to cidb when applicable. In particular,
this CL
(1) Changes the cidb.InsertFailure api to accept exception details rather than
the exception object. This is needed so that CompoundFailures can be
reported.
(2) Adds cidb.GetHasBuildStageFailed to query whether a builder stage failure
has been reported yet and uses that to report runaway failures.
(2) Adds a helper function to failures_lib and uses to report failures.
BUG=chromium:443260
TEST=(1) cbuildbot/run_tests
(2) lib/cidb_integration_test
(3) Run a vanilla sanity trybot.
(3) Run trybots with injected errors of different types and verify that
cidb entries manually.
(4) Run a trybot with an exception that escapes the parallel stage runner
and verify it is attributed to all stages.
Change-Id: I7cccde4e6582f394366cfe880f055cdcf313ad86
Reviewed-on: https://chromium-review.googlesource.com/236521
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
diff --git a/scripts/cbuildbot.py b/scripts/cbuildbot.py
index 81fefe3..d38eebe 100644
--- a/scripts/cbuildbot.py
+++ b/scripts/cbuildbot.py
@@ -241,6 +241,16 @@
if not results_lib.Results.StageHasResults(name):
results_lib.Results.Record(name, ex, str(ex))
+ if cidb.CIDBConnectionFactory.IsCIDBSetup():
+ db = cidb.CIDBConnectionFactory.GetCIDBConnectionForBuilder()
+ if db:
+ for stage in stage_objs:
+ for build_stage_id in stage.GetBuildStageIDs():
+ if not db.HasBuildStageFailed(build_stage_id):
+ failures_lib.ReportStageFailureToCIDB(db,
+ build_stage_id,
+ ex)
+
raise
def _RunSyncStage(self, sync_instance):