buildbot: switch buildbot & cros_sdk to xz
Previously submitted in https://gerrit.chromium.org/gerrit/23967
Requires https://gerrit.chromium.org/gerrit/35098 to work properly.
This makes cros_sdk prefer .tar.xz archives over .tbz2, while
changing the chroot builder to produce and upload .tar.xz on the next
iteration.
Simple benchmarks:
tar: 1.847.654.400 cros-sdk-29.05.12.103036.tar
xz: 431.625.428 cros-sdk-29.05.12.103036.tar.xz
Enc: real 13m41.632s user 13m37.130s sys 0m01.470s
Dec: real 0m52.440s user 0m34.510s sys 0m01.390s
xz -9 (max): 350.064.736 cros-sdk-29.05.12.103036.tar.xz
Enc: real 16m01.348s user 15m56.610s sys 0m01.680s
Dec: real 0m28.328s user 0m26.670s sys 0m01.530s
pixz: 457.186.584 cros-sdk-29.05.12.103036.tpxz
Enc: real 1m00.774s user 21m14.320s sys 0m03.700s
Dec: real 0m02.898s user 0m46.030s sys 0m02.230s
pixz -9: 376.264.212 cros-sdk-29.05.12.103036.tpxz
Enc: real 1m30.333s user 23m37.600s sys 0m14.440s
Dec: real 0m03.781s user 0m36.260s sys 0m03.370s
pbzip2: 575.609.262 cros-sdk-29.05.12.103036.tbz2
Enc: real 0m18.216s user 7m01.470s sys 0m02.940s
Dec: real 0m06.980s user 2m17.220s sys 0m03.100s
bzip2: 574.026.150 cros-sdk-29.05.12.103036.tbz2
Enc: real 3m49.258s user 3m43.500s sys 0m1.470s
Dec: real 1m21.931s user 1m13.000s sys 0m2.360s
Current modes are:
pbzip2 for encoding
bzip2 for decoding, or pbzip2 on machines that have it (most don't)
New are:
xz -9 for encoding
xz for decoding
This causes a slowdown on the side of cros-sdk bot, but speedup
for decoding on most dev machines.
BUG=chromium-os:19287
TEST=trybot chromiumos-sdk
TEST=run unit tests
TEST=pylint
CQ-DEPEND=I2df2ee76eaf9d31f6fcc7aa54a3e6a1820224e5d
Change-Id: Ie15c2e327955e61c652299d9a503695cd5eb912b
Reviewed-on: https://gerrit.chromium.org/gerrit/35149
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Zdenek Behan <zbehan@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
diff --git a/scripts/cros_sdk.py b/scripts/cros_sdk.py
index 6d6d00e..c97032d 100644
--- a/scripts/cros_sdk.py
+++ b/scripts/cros_sdk.py
@@ -22,7 +22,7 @@
DEFAULT_URL = 'https://commondatastorage.googleapis.com/chromiumos-sdk'
-COMPRESSION_PREFERENCE = ('bz2', 'xz')
+COMPRESSION_PREFERENCE = ('xz', 'bz2')
SRC_ROOT = os.path.realpath(constants.SOURCE_ROOT)
OVERLAY_DIR = os.path.join(SRC_ROOT, 'src/third_party/chromiumos-overlay')
@@ -34,7 +34,7 @@
ENTER_CHROOT = [os.path.join(SRC_ROOT, 'src/scripts/sdk_lib/enter_chroot.sh')]
# We need these tools to run. Very common tools (tar,..) are ommited.
-NEEDED_TOOLS = ('curl',)
+NEEDED_TOOLS = ('curl', 'xz')
def CheckPrerequisites(needed_tools):