bisect-kit: support telemetry functional regression bisecting
This CL only makes telemetry tests run with chrome HEAD. So the bisect
limitation is:
- It is unable to find culprit inside telemetry itself yet because
chrome version is fixed during chromeos and android bisection.
- Performance numbers may be still different to release build due to
lacks of AFDO optimization. So performance bisection may or may not
work.
BUG=chromium:909584
TEST=diagnose_cros_autotest.py crbug/909627
Change-Id: Iae9a42e73fa07e38b7711d7f9b61d4626415031d
Reviewed-on: https://chromium-review.googlesource.com/1354842
Commit-Ready: Kuang-che Wu <kcwu@chromium.org>
Tested-by: Kuang-che Wu <kcwu@chromium.org>
Reviewed-by: Chung-yih Wang <cywang@chromium.org>
diff --git a/eval_cros_autotest.py b/eval_cros_autotest.py
index c930100..8fa00db 100755
--- a/eval_cros_autotest.py
+++ b/eval_cros_autotest.py
@@ -56,6 +56,12 @@
default=configure.get('CHROMEOS_ROOT', ''),
help='ChromeOS tree root')
parser.add_argument(
+ '--chrome_root',
+ metavar='CHROME_ROOT',
+ type=cli.argtype_dir_path,
+ default=configure.get('CHROME_ROOT'),
+ help='Chrome tree root; necessary for telemetry tests')
+ parser.add_argument(
'--test_name',
required=True,
help='Test name, like "video_VideoDecodeAccelerator.h264"')
@@ -194,7 +200,8 @@
cmd += ['--args', args]
try:
- output = cros_util.cros_sdk(opts.chromeos_root, *cmd)
+ output = cros_util.cros_sdk(
+ opts.chromeos_root, *cmd, chrome_root=opts.chrome_root)
except subprocess.CalledProcessError as e:
output = e.output
@@ -249,6 +256,10 @@
if opts.new_value is not None:
logger.error('--new_value is provided but --metric is not')
return FATAL
+ if opts.test_name.startswith('telemetry_'):
+ if not opts.chrome_root:
+ logger.error('--chrome_root is mandatory for telemetry tests')
+ return FATAL
# Some versions of ChromeOS SDK is broken and ship bad 'ssh' executable. This
# works around the issue before we fixed the issue.