Support e.g. --workers=2x to use two workers per core.
This is mostly useful for tests performing a lot of I/O and sleeping,
when you don't know on which architecture they end up running.
The syntax can also be used to reduce CPU load (e.g. --workers=0.5x).
Bug: webrtc:9717
Change-Id: I26b4552576b1dd56a69c2223da39f4bb1115bbf6
Reviewed-on: https://webrtc-review.googlesource.com/101643
Commit-Queue: Yves Gerey <yvesg@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24830}diff --git a/tools_webrtc/gtest-parallel-wrapper.py b/tools_webrtc/gtest-parallel-wrapper.py
index 875e088..7be4b2b 100755
--- a/tools_webrtc/gtest-parallel-wrapper.py
+++ b/tools_webrtc/gtest-parallel-wrapper.py
@@ -15,8 +15,9 @@
flags. Developers should execute gtest-parallel directly.
In particular, this translates the GTEST_SHARD_INDEX and GTEST_TOTAL_SHARDS
-environment variables to the --shard_index and --shard_count flags, and renames
-the --isolated-script-test-output flag to --dump_json_test_results.
+environment variables to the --shard_index and --shard_count flags, renames
+the --isolated-script-test-output flag to --dump_json_test_results,
+and interprets e.g. --workers=2x as 2 workers per core.
Flags before '--' will be attempted to be understood as arguments to
gtest-parallel. If gtest-parallel doesn't recognize the flag or the flag is
@@ -63,6 +64,7 @@
import argparse
import collections
+import multiprocessing
import os
import shutil
import subprocess
@@ -81,6 +83,15 @@
output_file.write(input_file.read())
os.remove(filename)
+def _ParseWorkersOption(workers):
+ """Interpret Nx syntax as N * cpu_count. Int value is left as is."""
+ base = float(workers.rstrip('x'))
+ if workers.endswith('x'):
+ result = int(base * multiprocessing.cpu_count())
+ else:
+ result = int(base)
+ return max(result, 1) # Sanitize when using e.g. '0.5x'.
+
class ReconstructibleArgumentGroup(object):
"""An argument group that can be converted back into a command line.
@@ -117,13 +128,15 @@
gtest_group.AddArgument('-d', '--output_dir')
gtest_group.AddArgument('-r', '--repeat')
gtest_group.AddArgument('--retry_failed')
- gtest_group.AddArgument('-w', '--workers')
gtest_group.AddArgument('--gtest_color')
gtest_group.AddArgument('--gtest_filter')
gtest_group.AddArgument('--gtest_also_run_disabled_tests',
action='store_true', default=None)
gtest_group.AddArgument('--timeout')
+ # Syntax 'Nx' will be interpreted as N * number of cpu cores.
+ gtest_group.AddArgument('-w', '--workers', type=_ParseWorkersOption)
+
# --isolated-script-test-output is used to upload results to the flakiness
# dashboard. This translation is made because gtest-parallel expects the flag
# to be called --dump_json_test_results instead.