repository: Create PreLoad helper.
Create a new helper method that attempts to populate a new repository
checkout with information from the current repository checkout.
Use this helper in cbuildbot_launch to help with local tryjobs, and
with workspaces. In both cases, this should speed up an expensive cold
checkout (and reduce GoB quota usage).
This PreLoad looks like something that should be use as part of Sync,
but must instead be done before we clean the target tree, since the
source repository may not be in a safe state when copied.
The intial copy currently takes about 7 minutes on my workstation. It
might be possible to speed this up by using native copy tools or
links. I'm hesitate to use any form of link, since that might allow
change to the new repositgory to propogate back to the original
repository.
BUG=chromium:855291
TEST=sudo rm -rf ~/sand/tryjob && cros tryjob --local success-build.
Change-Id: I901a7bc9f5349f408562b1c2a632d760a301cebc
Reviewed-on: https://chromium-review.googlesource.com/1164558
Commit-Ready: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Lann Martin <lannm@chromium.org>
diff --git a/scripts/cbuildbot_launch.py b/scripts/cbuildbot_launch.py
index 119f70c..597ba27 100644
--- a/scripts/cbuildbot_launch.py
+++ b/scripts/cbuildbot_launch.py
@@ -275,17 +275,18 @@
osutils.RmDir(os.path.join(repo.directory, '.cache', 'distfiles'),
ignore_missing=True, sudo=True)
- try:
- # If there is any failure doing the cleanup, wipe everything.
- # The previous run might have been killed in the middle leaving stale git
- # locks. Clean those up, first.
- repo.CleanStaleLocks()
- repo.BuildRootGitCleanup(prune_all=True)
- except Exception:
- logging.info('Checkout cleanup failed, wiping buildroot:', exc_info=True)
- metrics.Counter(METRIC_CLOBBER).increment(
- field(metrics_fields, reason='repo_cleanup_failure'))
- repository.ClearBuildRoot(repo.directory)
+ try:
+ # If there is any failure doing the cleanup, wipe everything.
+ # The previous run might have been killed in the middle leaving stale git
+ # locks. Clean those up, first.
+ repo.PreLoad()
+ repo.CleanStaleLocks()
+ repo.BuildRootGitCleanup(prune_all=True)
+ except Exception:
+ logging.info('Checkout cleanup failed, wiping buildroot:', exc_info=True)
+ metrics.Counter(METRIC_CLOBBER).increment(
+ field(metrics_fields, reason='repo_cleanup_failure'))
+ repository.ClearBuildRoot(repo.directory)
# Ensure buildroot exists. Save the state we are prepped for.
osutils.SafeMakedirs(repo.directory)