scripts: Cap the number of nodes pytest uses.
Cap the number of nodes pytest uses at 64. On my cloudtop with 128
cores, run_tests takes more than 10 minutes to run without specifying
-j, and only about 2 minutes and 50 seconds for both -j32 and -j64.
BUG=None
TEST=run_tests
Change-Id: I010877e0a356d8308c961b56c59b9d7fc5d7fe25
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/chromite/+/4908696
Tested-by: Alex Klein <saklein@chromium.org>
Auto-Submit: Alex Klein <saklein@chromium.org>
Commit-Queue: Lee Presland <zland@google.com>
Reviewed-by: Lee Presland <zland@google.com>
diff --git a/scripts/run_tests.py b/scripts/run_tests.py
index 88755b2..8555cdf 100644
--- a/scripts/run_tests.py
+++ b/scripts/run_tests.py
@@ -69,8 +69,9 @@
jobs = opts.jobs
if jobs is None:
# Default to running in a single process under --quickstart. User args
- # can still override this.
- jobs = 0 if opts.quick else os.cpu_count()
+ # can still override this. Cap it at 64 by default to prevent the
+ # overhead from spawning too many nodes.
+ jobs = 0 if opts.quick else min(os.cpu_count(), 64)
pytest_args = ["-n", str(jobs)] + pytest_args
# Check the environment. https://crbug.com/1015450