Replace all uses of the word 'master' with 'builder_group' in //tools/mb
This removes every reference but the "--master/-m" cmd-line arg and the "masters" mb_config.pyl key, which will be removed in a follow-up once all users of mb.py (ie: recipes) have switched over. "builder_group" is also the term we're using when replacing "master" in recipe code: crbug.com/1109276. So we should conform on using that term going forward. Bug: 1117773 Change-Id: I10a3c7a4bc5fd5146bc88d670564a745cb3fce2a Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2605225 Commit-Queue: Ben Pastene <bpastene@chromium.org> Reviewed-by: Dirk Pranke <dpranke@google.com> Cr-Commit-Position: refs/heads/master@{#840641}
This commit is contained in:
Ben Pastene
committed by
Chromium LUCI CQ
Chromium LUCI CQ
parent
ce98df46f7
commit
7450249321
@ -49,8 +49,8 @@ If no matching bot config file is found, `mb` looks in the
|
||||
(`gn args`) to use.
|
||||
|
||||
A config can either be specified directly (useful for testing) or by specifying
|
||||
the master name and builder name (useful on the bots so that they do not need
|
||||
to specify a config directly and can be hidden from the details).
|
||||
the builder_group name and builder name (useful on the bots so that they do not
|
||||
need to specify a config directly and can be hidden from the details).
|
||||
|
||||
See the [user guide](user_guide.md#mb_config.pyl) for details.
|
||||
|
||||
|
||||
@ -27,7 +27,7 @@ a list of files (e.g., the list of files in a patch on a trybot):
|
||||
mb analyze -c chromium_linux_rel //out/Release input.json output.json
|
||||
```
|
||||
|
||||
Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags
|
||||
Either the `-c/--config` flag or the `--builder-group` and `-b/--builder` flags
|
||||
must be specified so that `mb` can figure out which config to use.
|
||||
|
||||
The first positional argument must be a GN-style "source-absolute" path
|
||||
@ -90,7 +90,7 @@ differences can be subtle. We won't even go into how the `targets` and
|
||||
`build_targets` differ from each other or from `compile_targets` and
|
||||
`test_targets`.
|
||||
|
||||
The `-b/--builder`, `-c/--config`, `-f/--config-file`, `-m/--master`,
|
||||
The `-b/--builder`, `-c/--config`, `-f/--config-file`, `--builder-group`,
|
||||
`-q/--quiet`, and `-v/--verbose` flags work as documented for `mb gen`.
|
||||
|
||||
### mb gen
|
||||
@ -104,7 +104,7 @@ specify a build config and a directory, then runs GN as appropriate:
|
||||
% mb gen -c linux_rel_trybot //out/Release
|
||||
```
|
||||
|
||||
Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags
|
||||
Either the `-c/--config` flag or the `--builder-group` and `-b/--builder` flags
|
||||
must be specified so that `mb` can figure out which config to use. The
|
||||
`--phase` flag must also be used with builders that have multiple
|
||||
build/compile steps (and only with those builders).
|
||||
@ -144,7 +144,7 @@ swarming. See below for more information on isolates and swarming.
|
||||
Prints what command will be run by `mb gen` (like `mb gen -n` but does
|
||||
not require you to specify a path).
|
||||
|
||||
The `-b/--builder`, `-c/--config`, `-f/--config-file`, `-m/--master`,
|
||||
The `-b/--builder`, `-c/--config`, `-f/--config-file`, `--builder-group`,
|
||||
`--phase`, `-q/--quiet`, and `-v/--verbose` flags work as documented for
|
||||
`mb gen`.
|
||||
|
||||
@ -176,8 +176,9 @@ like one you could trigger via `git cl try` or via CQ dry runs. Basic usage is
|
||||
Your change must be uploaded to Gerrit. Local changes will not be uploaded for
|
||||
you. It uses the gerrit CL associated with your given git branch.
|
||||
|
||||
You still have to specify the mastername (`-m`) and buildername (`-b`) arguments.
|
||||
See [trybots.py](https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/trybots.py)
|
||||
You still have to specify the builder group(`--builder-group`) and buildername
|
||||
(`-b`) arguments. See
|
||||
[trybots.py](https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/trybots.py)
|
||||
for a mapping of which bots are on which tryservers, and what those bots mirror.
|
||||
Any trybot in `trybots.py` is supported; you can test your code on windows, for
|
||||
example. The tryjob will compile and run your code on windows.
|
||||
@ -236,11 +237,11 @@ listed here, and so by using the configs in this file you can avoid
|
||||
having to juggle long lists of gn args by hand.
|
||||
|
||||
`mb_config.pyl` is structured as a file containing a single PYthon Literal
|
||||
expression: a dictionary with three main keys, `masters`, `configs` and
|
||||
expression: a dictionary with three main keys, `builder_groups`, `configs` and
|
||||
`mixins`.
|
||||
|
||||
The `masters` key contains a nested series of dicts containing mappings
|
||||
of master -> builder -> config . This allows us to isolate the buildbot
|
||||
The `builder_groups` key contains a nested series of dicts containing mappings
|
||||
of builder_group -> builder -> config . This allows us to isolate the builder
|
||||
recipes from the actual details of the configs. The config should either
|
||||
be a single string value representing a key in the `configs` dictionary,
|
||||
or a list of strings, each of which is a key in the `configs` dictionary;
|
||||
|
||||
@ -10,17 +10,17 @@ import os
|
||||
import re
|
||||
|
||||
|
||||
def GetAllConfigs(masters):
|
||||
def GetAllConfigs(builder_groups):
|
||||
"""Build a list of all of the configs referenced by builders.
|
||||
"""
|
||||
all_configs = {}
|
||||
for master in masters:
|
||||
for config in masters[master].values():
|
||||
for builder_group in builder_groups:
|
||||
for config in builder_groups[builder_group].values():
|
||||
if isinstance(config, dict):
|
||||
for c in config.values():
|
||||
all_configs[c] = master
|
||||
all_configs[c] = builder_group
|
||||
else:
|
||||
all_configs[config] = master
|
||||
all_configs[config] = builder_group
|
||||
return all_configs
|
||||
|
||||
|
||||
@ -55,13 +55,13 @@ def CheckAllConfigsAndMixinsReferenced(errs, all_configs, configs, mixins):
|
||||
return errs
|
||||
|
||||
|
||||
def EnsureNoProprietaryMixins(errs, masters, configs, mixins):
|
||||
def EnsureNoProprietaryMixins(errs, builder_groups, configs, mixins):
|
||||
"""If we're checking the Chromium config, check that the 'chromium' bots
|
||||
which build public artifacts do not include the chrome_with_codecs mixin.
|
||||
"""
|
||||
if 'chromium' in masters:
|
||||
for builder in masters['chromium']:
|
||||
config = masters['chromium'][builder]
|
||||
if 'chromium' in builder_groups:
|
||||
for builder in builder_groups['chromium']:
|
||||
config = builder_groups['chromium'][builder]
|
||||
|
||||
def RecurseMixins(current_mixin):
|
||||
if current_mixin == 'chrome_with_codecs':
|
||||
@ -76,21 +76,21 @@ def EnsureNoProprietaryMixins(errs, masters, configs, mixins):
|
||||
for mixin in configs[config]:
|
||||
RecurseMixins(mixin)
|
||||
else:
|
||||
errs.append('Missing "chromium" master. Please update this '
|
||||
'proprietary codecs check with the name of the master '
|
||||
errs.append('Missing "chromium" builder_group. Please update this '
|
||||
'proprietary codecs check with the name of the builder_group '
|
||||
'responsible for public build artifacts.')
|
||||
|
||||
|
||||
def _GetConfigsByBuilder(masters):
|
||||
def _GetConfigsByBuilder(builder_groups):
|
||||
"""Builds a mapping from buildername -> [config]
|
||||
|
||||
Args
|
||||
masters: the master's dict from mb_config.pyl
|
||||
builder_groups: the builder_group's dict from mb_config.pyl
|
||||
"""
|
||||
|
||||
result = collections.defaultdict(list)
|
||||
for master in masters.values():
|
||||
for buildername, builder in master.items():
|
||||
for builder_group in builder_groups.values():
|
||||
for buildername, builder in builder_group.items():
|
||||
result[buildername].append(builder)
|
||||
|
||||
return result
|
||||
@ -142,13 +142,14 @@ def CheckExpectations(mbw, jsonish_blob, expectations_dir):
|
||||
|
||||
Returns: True if expectations are up-to-date. False otherwise.
|
||||
"""
|
||||
# Assert number of masters == number of expectation files.
|
||||
# Assert number of builder_groups == number of expectation files.
|
||||
if len(mbw.ListDir(expectations_dir)) != len(jsonish_blob):
|
||||
return False
|
||||
for master, builders in jsonish_blob.items():
|
||||
if not mbw.Exists(os.path.join(expectations_dir, master + '.json')):
|
||||
return False # No expecation file for the master.
|
||||
expectation = mbw.ReadFile(os.path.join(expectations_dir, master + '.json'))
|
||||
for builder_group, builders in jsonish_blob.items():
|
||||
if not mbw.Exists(os.path.join(expectations_dir, builder_group + '.json')):
|
||||
return False # No expecation file for the builder_group.
|
||||
expectation = mbw.ReadFile(os.path.join(expectations_dir,
|
||||
builder_group + '.json'))
|
||||
builders_json = json.dumps(builders,
|
||||
indent=2,
|
||||
sort_keys=True,
|
||||
|
||||
@ -93,7 +93,7 @@ class MetaBuildWrapper(object):
|
||||
self.args = argparse.Namespace()
|
||||
self.configs = {}
|
||||
self.public_artifact_builders = None
|
||||
self.masters = {}
|
||||
self.builder_groups = {}
|
||||
self.mixins = {}
|
||||
self.isolate_exe = 'isolate.exe' if self.platform.startswith(
|
||||
'win') else 'isolate'
|
||||
@ -120,7 +120,10 @@ class MetaBuildWrapper(object):
|
||||
def AddCommonOptions(subp):
|
||||
group = subp.add_mutually_exclusive_group()
|
||||
group.add_argument(
|
||||
'-m', '--master', help='master name to look up config from')
|
||||
'-m', '--builder-group',
|
||||
# TODO(crbug.com/1117773): Remove the 'master' args.
|
||||
'--master',
|
||||
help='builder group name to look up config from')
|
||||
subp.add_argument('-b', '--builder',
|
||||
help='builder name to look up config from')
|
||||
subp.add_argument('-c', '--config',
|
||||
@ -435,8 +438,8 @@ class MetaBuildWrapper(object):
|
||||
for f in self.ListDir(expectations_dir):
|
||||
self.RemoveFile(os.path.join(expectations_dir, f))
|
||||
obj = self._ToJsonish()
|
||||
for master, builder in sorted(obj.items()):
|
||||
expectation_file = os.path.join(expectations_dir, master + '.json')
|
||||
for builder_group, builder in sorted(obj.items()):
|
||||
expectation_file = os.path.join(expectations_dir, builder_group + '.json')
|
||||
json_s = json.dumps(builder,
|
||||
indent=2,
|
||||
sort_keys=True,
|
||||
@ -751,14 +754,14 @@ class MetaBuildWrapper(object):
|
||||
"""Dumps the config file into a json-friendly expanded dict.
|
||||
|
||||
Returns:
|
||||
A dict with master -> builder -> all GN args mapping.
|
||||
A dict with builder group -> builder -> all GN args mapping.
|
||||
"""
|
||||
self.ReadConfigFile(self.args.config_file)
|
||||
obj = {}
|
||||
for master, builders in self.masters.items():
|
||||
obj[master] = {}
|
||||
for builder_group, builders in self.builder_groups.items():
|
||||
obj[builder_group] = {}
|
||||
for builder in builders:
|
||||
config = self.masters[master][builder]
|
||||
config = self.builder_groups[builder_group][builder]
|
||||
if not config:
|
||||
continue
|
||||
if isinstance(config, dict):
|
||||
@ -777,7 +780,7 @@ class MetaBuildWrapper(object):
|
||||
args = {'gn_args': gn_helpers.FromGNArgs(flattened_config['gn_args'])}
|
||||
if flattened_config.get('args_file'):
|
||||
args['args_file'] = flattened_config['args_file']
|
||||
obj[master][builder] = args
|
||||
obj[builder_group][builder] = args
|
||||
|
||||
return obj
|
||||
|
||||
@ -787,7 +790,7 @@ class MetaBuildWrapper(object):
|
||||
self.ReadConfigFile(self.args.config_file)
|
||||
|
||||
# Build a list of all of the configs referenced by builders.
|
||||
all_configs = validation.GetAllConfigs(self.masters)
|
||||
all_configs = validation.GetAllConfigs(self.builder_groups)
|
||||
|
||||
# Check that every referenced args file or config actually exists.
|
||||
for config, loc in all_configs.items():
|
||||
@ -804,11 +807,11 @@ class MetaBuildWrapper(object):
|
||||
self.configs, self.mixins)
|
||||
|
||||
if self.args.config_file == self.default_config:
|
||||
validation.EnsureNoProprietaryMixins(errs, self.masters, self.configs,
|
||||
self.mixins)
|
||||
validation.EnsureNoProprietaryMixins(errs, self.builder_groups,
|
||||
self.configs, self.mixins)
|
||||
|
||||
validation.CheckDuplicateConfigs(errs, self.configs, self.mixins,
|
||||
self.masters, FlattenConfig)
|
||||
self.builder_groups, FlattenConfig)
|
||||
|
||||
if errs:
|
||||
raise MBErr(('mb config file %s has problems:\n ' %
|
||||
@ -830,7 +833,7 @@ class MetaBuildWrapper(object):
|
||||
build_dir = self.args.path
|
||||
|
||||
vals = DefaultVals()
|
||||
if self.args.builder or self.args.master or self.args.config:
|
||||
if self.args.builder or self.args.builder_group or self.args.config:
|
||||
vals = self.Lookup()
|
||||
# Re-run gn gen in order to ensure the config is consistent with the
|
||||
# build dir.
|
||||
@ -922,10 +925,10 @@ class MetaBuildWrapper(object):
|
||||
return vals
|
||||
|
||||
def ReadIOSBotConfig(self):
|
||||
if not self.args.master or not self.args.builder:
|
||||
if not self.args.builder_group or not self.args.builder:
|
||||
return {}
|
||||
path = self.PathJoin(self.chromium_src_dir, 'ios', 'build', 'bots',
|
||||
self.args.master, self.args.builder + '.json')
|
||||
self.args.builder_group, self.args.builder + '.json')
|
||||
if not self.Exists(path):
|
||||
return {}
|
||||
|
||||
@ -947,7 +950,9 @@ class MetaBuildWrapper(object):
|
||||
|
||||
self.configs = contents['configs']
|
||||
self.mixins = contents['mixins']
|
||||
self.masters = contents.get('masters')
|
||||
# TODO(crbug.com/1117773): Remove 'masters' below.
|
||||
self.builder_groups = (
|
||||
contents.get('builder_groups') or contents.get('masters'))
|
||||
self.public_artifact_builders = contents.get('public_artifact_builders')
|
||||
|
||||
def ReadIsolateMap(self):
|
||||
@ -974,38 +979,39 @@ class MetaBuildWrapper(object):
|
||||
|
||||
def ConfigFromArgs(self):
|
||||
if self.args.config:
|
||||
if self.args.master or self.args.builder:
|
||||
raise MBErr('Can not specific both -c/--config and -m/--master or '
|
||||
'-b/--builder')
|
||||
if self.args.builder_group or self.args.builder:
|
||||
raise MBErr('Can not specific both -c/--config and --group '
|
||||
'or -b/--builder')
|
||||
|
||||
return self.args.config
|
||||
|
||||
if not self.args.master or not self.args.builder:
|
||||
if not self.args.builder_group or not self.args.builder:
|
||||
raise MBErr('Must specify either -c/--config or '
|
||||
'(-m/--master and -b/--builder)')
|
||||
'(--group and -b/--builder)')
|
||||
|
||||
if not self.args.master in self.masters:
|
||||
raise MBErr('Master name "%s" not found in "%s"' %
|
||||
(self.args.master, self.args.config_file))
|
||||
if not self.args.builder_group in self.builder_groups:
|
||||
raise MBErr('Builder group name "%s" not found in "%s"' %
|
||||
(self.args.builder_group, self.args.config_file))
|
||||
|
||||
if not self.args.builder in self.masters[self.args.master]:
|
||||
raise MBErr('Builder name "%s" not found under masters[%s] in "%s"' %
|
||||
(self.args.builder, self.args.master, self.args.config_file))
|
||||
if not self.args.builder in self.builder_groups[self.args.builder_group]:
|
||||
raise MBErr('Builder name "%s" not found under groups[%s] in "%s"' %
|
||||
(self.args.builder, self.args.builder_group,
|
||||
self.args.config_file))
|
||||
|
||||
config = self.masters[self.args.master][self.args.builder]
|
||||
config = self.builder_groups[self.args.builder_group][self.args.builder]
|
||||
if isinstance(config, dict):
|
||||
if self.args.phase is None:
|
||||
raise MBErr('Must specify a build --phase for %s on %s' %
|
||||
(self.args.builder, self.args.master))
|
||||
(self.args.builder, self.args.builder_group))
|
||||
phase = str(self.args.phase)
|
||||
if phase not in config:
|
||||
raise MBErr('Phase %s doesn\'t exist for %s on %s' %
|
||||
(phase, self.args.builder, self.args.master))
|
||||
(phase, self.args.builder, self.args.builder_group))
|
||||
return config[phase]
|
||||
|
||||
if self.args.phase is not None:
|
||||
raise MBErr('Must not specify a build --phase for %s on %s' %
|
||||
(self.args.builder, self.args.master))
|
||||
(self.args.builder, self.args.builder_group))
|
||||
return config
|
||||
|
||||
def RunGNGen(self, vals, compute_inputs_for_analyze=False, check=True):
|
||||
|
||||
@ -9,10 +9,10 @@
|
||||
# * trailing commas are allowed.
|
||||
|
||||
{
|
||||
# This is a map of buildbot master names -> buildbot builder names ->
|
||||
# config names (where each config name is a key in the 'configs' dict,
|
||||
# below). MB uses this dict to look up which config to use for a given bot.
|
||||
'masters': {
|
||||
# This is a map of builder group names -> builder names -> config names
|
||||
# (where each config name is a key in the 'configs' dict, below). MB uses
|
||||
# this dict to look up which config to use for a given bot.
|
||||
'builder_groups': {
|
||||
'chrome': {
|
||||
'chromeos-arm-generic-beta': 'official_chromeos_arm-generic',
|
||||
'chromeos-arm-generic-ltc': 'official_chromeos_arm-generic',
|
||||
@ -682,7 +682,7 @@
|
||||
'win32-updater-builder-rel': 'updater_release_bot_x86',
|
||||
},
|
||||
|
||||
# TODO(crbug.com/818301): This master is going away.
|
||||
# TODO(crbug.com/818301): This group is going away.
|
||||
'chromium.webkit': {
|
||||
'WebKit Linux Trusty ASAN': 'asan_lsan_release_bot',
|
||||
'WebKit Linux Trusty Leak': 'release_bot',
|
||||
|
||||
@ -151,12 +151,12 @@ class FakeFile(object):
|
||||
|
||||
TEST_CONFIG = """\
|
||||
{
|
||||
'masters': {
|
||||
'builder_groups': {
|
||||
'chromium': {},
|
||||
'fake_master': {
|
||||
'fake_builder_group': {
|
||||
'fake_builder': 'rel_bot',
|
||||
'fake_debug_builder': 'debug_goma',
|
||||
'fake_args_bot': '//build/args/bots/fake_master/fake_args_bot.gn',
|
||||
'fake_args_bot': '//build/args/bots/fake_builder_group/fake_args_bot.gn',
|
||||
'fake_multi_phase': { 'phase_1': 'phase_1', 'phase_2': 'phase_2'},
|
||||
'fake_args_file': 'args_file_goma',
|
||||
'fake_ios_error': 'ios_error',
|
||||
@ -205,7 +205,7 @@ TEST_BAD_CONFIG = """\
|
||||
'rel_bot_1': ['rel', 'chrome_with_codecs'],
|
||||
'rel_bot_2': ['rel', 'bad_nested_config'],
|
||||
},
|
||||
'masters': {
|
||||
'builder_groups': {
|
||||
'chromium': {
|
||||
'a': 'rel_bot_1',
|
||||
'b': 'rel_bot_2',
|
||||
@ -229,9 +229,9 @@ TEST_BAD_CONFIG = """\
|
||||
|
||||
TEST_ARGS_FILE_TWICE_CONFIG = """\
|
||||
{
|
||||
'masters': {
|
||||
'builder_groups': {
|
||||
'chromium': {},
|
||||
'fake_master': {
|
||||
'fake_builder_group': {
|
||||
'fake_args_file_twice': 'args_file_twice',
|
||||
},
|
||||
},
|
||||
@ -249,9 +249,9 @@ TEST_ARGS_FILE_TWICE_CONFIG = """\
|
||||
|
||||
TEST_DUP_CONFIG = """\
|
||||
{
|
||||
'masters': {
|
||||
'builder_groups': {
|
||||
'chromium': {},
|
||||
'fake_master': {
|
||||
'fake_builder_group': {
|
||||
'fake_builder': 'some_config',
|
||||
'other_builder': 'some_other_config',
|
||||
},
|
||||
@ -270,7 +270,7 @@ TEST_DUP_CONFIG = """\
|
||||
|
||||
TRYSERVER_CONFIG = """\
|
||||
{
|
||||
'masters': {
|
||||
'builder_groups': {
|
||||
'not_a_tryserver': {
|
||||
'fake_builder': 'fake_config',
|
||||
},
|
||||
@ -301,7 +301,7 @@ class UnitTest(unittest.TestCase):
|
||||
},
|
||||
}''')
|
||||
mbw.files.setdefault(
|
||||
mbw.ToAbsPath('//build/args/bots/fake_master/fake_args_bot.gn'),
|
||||
mbw.ToAbsPath('//build/args/bots/fake_builder_group/fake_args_bot.gn'),
|
||||
'is_debug = false\n')
|
||||
if files:
|
||||
for path, contents in files.items():
|
||||
@ -453,17 +453,17 @@ class UnitTest(unittest.TestCase):
|
||||
'--check', mbw.out)
|
||||
|
||||
mbw = self.fake_mbw()
|
||||
self.check(['gen', '-m', 'fake_master', '-b', 'fake_args_bot',
|
||||
self.check(['gen', '-m', 'fake_builder_group', '-b', 'fake_args_bot',
|
||||
'//out/Debug'],
|
||||
mbw=mbw, ret=0)
|
||||
# TODO(https://crbug.com/1093038): This assert is inappropriately failing.
|
||||
# self.assertEqual(
|
||||
# mbw.files['/fake_src/out/Debug/args.gn'],
|
||||
# 'import("//build/args/bots/fake_master/fake_args_bot.gn")\n')
|
||||
# 'import("//build/args/bots/fake_builder_group/fake_args_bot.gn")\n')
|
||||
|
||||
def test_gen_args_file_mixins(self):
|
||||
mbw = self.fake_mbw()
|
||||
self.check(['gen', '-m', 'fake_master', '-b', 'fake_args_file',
|
||||
self.check(['gen', '-m', 'fake_builder_group', '-b', 'fake_args_file',
|
||||
'//out/Debug'], mbw=mbw, ret=0)
|
||||
|
||||
self.assertEqual(
|
||||
@ -474,7 +474,7 @@ class UnitTest(unittest.TestCase):
|
||||
def test_gen_args_file_twice(self):
|
||||
mbw = self.fake_mbw()
|
||||
mbw.files[mbw.default_config] = TEST_ARGS_FILE_TWICE_CONFIG
|
||||
self.check(['gen', '-m', 'fake_master', '-b', 'fake_args_file_twice',
|
||||
self.check(['gen', '-m', 'fake_builder_group', '-b', 'fake_args_file_twice',
|
||||
'//out/Debug'], mbw=mbw, ret=1)
|
||||
|
||||
def test_gen_fails(self):
|
||||
@ -781,27 +781,27 @@ class UnitTest(unittest.TestCase):
|
||||
|
||||
def test_multiple_phases(self):
|
||||
# Check that not passing a --phase to a multi-phase builder fails.
|
||||
mbw = self.check(['lookup', '-m', 'fake_master', '-b', 'fake_multi_phase'],
|
||||
ret=1)
|
||||
mbw = self.check(['lookup', '-m', 'fake_builder_group', '-b',
|
||||
'fake_multi_phase'], ret=1)
|
||||
self.assertIn('Must specify a build --phase', mbw.out)
|
||||
|
||||
# Check that passing a --phase to a single-phase builder fails.
|
||||
mbw = self.check(['lookup', '-m', 'fake_master', '-b', 'fake_builder',
|
||||
'--phase', 'phase_1'], ret=1)
|
||||
mbw = self.check(['lookup', '-m', 'fake_builder_group', '-b',
|
||||
'fake_builder', '--phase', 'phase_1'], ret=1)
|
||||
self.assertIn('Must not specify a build --phase', mbw.out)
|
||||
|
||||
# Check that passing a wrong phase key to a multi-phase builder fails.
|
||||
mbw = self.check(['lookup', '-m', 'fake_master', '-b', 'fake_multi_phase',
|
||||
'--phase', 'wrong_phase'], ret=1)
|
||||
mbw = self.check(['lookup', '-m', 'fake_builder_group', '-b',
|
||||
'fake_multi_phase', '--phase', 'wrong_phase'], ret=1)
|
||||
self.assertIn('Phase wrong_phase doesn\'t exist', mbw.out)
|
||||
|
||||
# Check that passing a correct phase key to a multi-phase builder passes.
|
||||
mbw = self.check(['lookup', '-m', 'fake_master', '-b', 'fake_multi_phase',
|
||||
'--phase', 'phase_1'], ret=0)
|
||||
mbw = self.check(['lookup', '-m', 'fake_builder_group', '-b',
|
||||
'fake_multi_phase', '--phase', 'phase_1'], ret=0)
|
||||
self.assertIn('phase = 1', mbw.out)
|
||||
|
||||
mbw = self.check(['lookup', '-m', 'fake_master', '-b', 'fake_multi_phase',
|
||||
'--phase', 'phase_2'], ret=0)
|
||||
mbw = self.check(['lookup', '-m', 'fake_builder_group', '-b',
|
||||
'fake_multi_phase', '--phase', 'phase_2'], ret=0)
|
||||
self.assertIn('phase = 2', mbw.out)
|
||||
|
||||
def test_recursive_lookup(self):
|
||||
@ -811,7 +811,7 @@ class UnitTest(unittest.TestCase):
|
||||
'enable_antidoom_banana = true\n'
|
||||
)
|
||||
}
|
||||
self.check(['lookup', '-m', 'fake_master', '-b', 'fake_args_file',
|
||||
self.check(['lookup', '-m', 'fake_builder_group', '-b', 'fake_args_file',
|
||||
'--recursive'], files=files, ret=0,
|
||||
out=('enable_antidoom_banana = true\n'
|
||||
'enable_doom_melon = true\n'
|
||||
@ -821,7 +821,7 @@ class UnitTest(unittest.TestCase):
|
||||
mbw = self.fake_mbw()
|
||||
temp_dir = mbw.TempDir()
|
||||
self.check(['train', '--expectations-dir', temp_dir], mbw=mbw, ret=0)
|
||||
self.assertIn(os.path.join(temp_dir, 'fake_master.json'), mbw.files)
|
||||
self.assertIn(os.path.join(temp_dir, 'fake_builder_group.json'), mbw.files)
|
||||
|
||||
def test_validate(self):
|
||||
mbw = self.fake_mbw()
|
||||
@ -855,7 +855,7 @@ class UnitTest(unittest.TestCase):
|
||||
temp_dir = mbw.TempDir()
|
||||
self.check(['train', '--expectations-dir', temp_dir], mbw=mbw, ret=0)
|
||||
# Remove one of the expectation files.
|
||||
mbw.files.pop(os.path.join(temp_dir, 'fake_master.json'))
|
||||
mbw.files.pop(os.path.join(temp_dir, 'fake_builder_group.json'))
|
||||
# Now validating should fail.
|
||||
self.check(['validate', '--expectations-dir', temp_dir], mbw=mbw, ret=1)
|
||||
self.assertIn('Expectations out of date', mbw.out)
|
||||
@ -897,11 +897,11 @@ class UnitTest(unittest.TestCase):
|
||||
def test_ios_error_config_with_ios_json(self):
|
||||
"""Ensures that ios_error config finds the correct iOS JSON file for args"""
|
||||
files = {
|
||||
'/fake_src/ios/build/bots/fake_master/fake_ios_error.json':
|
||||
'/fake_src/ios/build/bots/fake_builder_group/fake_ios_error.json':
|
||||
('{"gn_args": ["is_debug=true"]}\n')
|
||||
}
|
||||
mbw = self.fake_mbw(files)
|
||||
self.check(['lookup', '-m', 'fake_master', '-b', 'fake_ios_error'],
|
||||
self.check(['lookup', '-m', 'fake_builder_group', '-b', 'fake_ios_error'],
|
||||
mbw=mbw,
|
||||
ret=0,
|
||||
out=('\n'
|
||||
@ -917,11 +917,11 @@ class UnitTest(unittest.TestCase):
|
||||
checked.
|
||||
"""
|
||||
files = {
|
||||
'/fake_src/ios/build/bots/fake_master/fake_ios_bot.json':
|
||||
'/fake_src/ios/build/bots/fake_builder_group/fake_ios_bot.json':
|
||||
('{"gn_args": ["is_debug=true"]}\n')
|
||||
}
|
||||
mbw = self.fake_mbw(files)
|
||||
self.check(['lookup', '-m', 'fake_master', '-b', 'fake_ios_bot'],
|
||||
self.check(['lookup', '-m', 'fake_builder_group', '-b', 'fake_ios_bot'],
|
||||
mbw=mbw,
|
||||
ret=0,
|
||||
out=('\n'
|
||||
@ -937,7 +937,7 @@ class UnitTest(unittest.TestCase):
|
||||
is ios_error, but there is no iOS JSON definition for it.
|
||||
"""
|
||||
mbw = self.fake_mbw()
|
||||
self.check(['lookup', '-m', 'fake_master', '-b', 'fake_ios_error'],
|
||||
self.check(['lookup', '-m', 'fake_builder_group', '-b', 'fake_ios_error'],
|
||||
mbw=mbw,
|
||||
ret=1)
|
||||
self.assertIn('MBErr: No iOS definition was found.', mbw.out)
|
||||
@ -949,10 +949,10 @@ class UnitTest(unittest.TestCase):
|
||||
doesn't exist at all.
|
||||
"""
|
||||
mbw = self.fake_mbw()
|
||||
self.check(['lookup', '-m', 'fake_master', '-b', 'random_bot'],
|
||||
self.check(['lookup', '-m', 'fake_builder_group', '-b', 'random_bot'],
|
||||
mbw=mbw,
|
||||
ret=1)
|
||||
self.assertIn('MBErr: Builder name "random_bot" not found under masters',
|
||||
self.assertIn('MBErr: Builder name "random_bot" not found under groups',
|
||||
mbw.out)
|
||||
|
||||
|
||||
|
||||
@ -26,8 +26,8 @@ TEST_UNREFERENCED_MIXIN_CONFIG = """\
|
||||
'rel_bot_1': ['rel'],
|
||||
'rel_bot_2': ['rel'],
|
||||
},
|
||||
'masters': {
|
||||
'fake_master_a': {
|
||||
'builder_groups': {
|
||||
'fake_builder_group_a': {
|
||||
'fake_builder_a': 'rel_bot_1',
|
||||
'fake_builder_b': 'rel_bot_2',
|
||||
},
|
||||
@ -50,8 +50,8 @@ TEST_UNKNOWNMIXIN_CONFIG = """\
|
||||
'rel_bot_1': ['rel'],
|
||||
'rel_bot_2': ['rel', 'unknown_mixin'],
|
||||
},
|
||||
'masters': {
|
||||
'fake_master_a': {
|
||||
'builder_groups': {
|
||||
'fake_builder_group_a': {
|
||||
'fake_builder_a': 'rel_bot_1',
|
||||
'fake_builder_b': 'rel_bot_2',
|
||||
},
|
||||
@ -71,8 +71,8 @@ TEST_UNKNOWN_NESTED_MIXIN_CONFIG = """\
|
||||
'rel_bot_1': ['rel', 'nested_mixin'],
|
||||
'rel_bot_2': ['rel'],
|
||||
},
|
||||
'masters': {
|
||||
'fake_master_a': {
|
||||
'builder_groups': {
|
||||
'fake_builder_group_a': {
|
||||
'fake_builder_a': 'rel_bot_1',
|
||||
'fake_builder_b': 'rel_bot_2',
|
||||
},
|
||||
@ -96,14 +96,14 @@ TEST_UNKNOWN_NESTED_MIXIN_CONFIG = """\
|
||||
class UnitTest(unittest.TestCase):
|
||||
def test_GetAllConfigs(self):
|
||||
configs = ast.literal_eval(mb_unittest.TEST_CONFIG)
|
||||
all_configs = validation.GetAllConfigs(configs['masters'])
|
||||
self.assertEqual(all_configs['rel_bot'], 'fake_master')
|
||||
self.assertEqual(all_configs['debug_goma'], 'fake_master')
|
||||
all_configs = validation.GetAllConfigs(configs['builder_groups'])
|
||||
self.assertEqual(all_configs['rel_bot'], 'fake_builder_group')
|
||||
self.assertEqual(all_configs['debug_goma'], 'fake_builder_group')
|
||||
|
||||
def test_CheckAllConfigsAndMixinsReferenced_ok(self):
|
||||
configs = ast.literal_eval(mb_unittest.TEST_CONFIG)
|
||||
errs = []
|
||||
all_configs = validation.GetAllConfigs(configs['masters'])
|
||||
all_configs = validation.GetAllConfigs(configs['builder_groups'])
|
||||
config_configs = configs['configs']
|
||||
mixins = configs['mixins']
|
||||
|
||||
@ -115,7 +115,7 @@ class UnitTest(unittest.TestCase):
|
||||
def test_CheckAllConfigsAndMixinsReferenced_unreferenced(self):
|
||||
configs = ast.literal_eval(TEST_UNREFERENCED_MIXIN_CONFIG)
|
||||
errs = []
|
||||
all_configs = validation.GetAllConfigs(configs['masters'])
|
||||
all_configs = validation.GetAllConfigs(configs['builder_groups'])
|
||||
config_configs = configs['configs']
|
||||
mixins = configs['mixins']
|
||||
|
||||
@ -127,7 +127,7 @@ class UnitTest(unittest.TestCase):
|
||||
def test_CheckAllConfigsAndMixinsReferenced_unknown(self):
|
||||
configs = ast.literal_eval(TEST_UNKNOWNMIXIN_CONFIG)
|
||||
errs = []
|
||||
all_configs = validation.GetAllConfigs(configs['masters'])
|
||||
all_configs = validation.GetAllConfigs(configs['builder_groups'])
|
||||
config_configs = configs['configs']
|
||||
mixins = configs['mixins']
|
||||
|
||||
@ -140,7 +140,7 @@ class UnitTest(unittest.TestCase):
|
||||
def test_CheckAllConfigsAndMixinsReferenced_unknown_nested(self):
|
||||
configs = ast.literal_eval(TEST_UNKNOWN_NESTED_MIXIN_CONFIG)
|
||||
errs = []
|
||||
all_configs = validation.GetAllConfigs(configs['masters'])
|
||||
all_configs = validation.GetAllConfigs(configs['builder_groups'])
|
||||
config_configs = configs['configs']
|
||||
mixins = configs['mixins']
|
||||
|
||||
@ -154,7 +154,7 @@ class UnitTest(unittest.TestCase):
|
||||
def test_CheckAllConfigsAndMixinsReferenced_unused(self):
|
||||
configs = ast.literal_eval(TEST_UNKNOWN_NESTED_MIXIN_CONFIG)
|
||||
errs = []
|
||||
all_configs = validation.GetAllConfigs(configs['masters'])
|
||||
all_configs = validation.GetAllConfigs(configs['builder_groups'])
|
||||
config_configs = configs['configs']
|
||||
mixins = configs['mixins']
|
||||
|
||||
@ -168,11 +168,12 @@ class UnitTest(unittest.TestCase):
|
||||
def test_EnsureNoProprietaryMixins(self):
|
||||
bad_configs = ast.literal_eval(mb_unittest.TEST_BAD_CONFIG)
|
||||
errs = []
|
||||
masters = bad_configs['masters']
|
||||
builder_groups = bad_configs['builder_groups']
|
||||
mixins = bad_configs['mixins']
|
||||
config_configs = bad_configs['configs']
|
||||
|
||||
validation.EnsureNoProprietaryMixins(errs, masters, config_configs, mixins)
|
||||
validation.EnsureNoProprietaryMixins(errs, builder_groups, config_configs,
|
||||
mixins)
|
||||
|
||||
self.assertIn(
|
||||
'Public artifact builder "a" '
|
||||
@ -186,7 +187,7 @@ class UnitTest(unittest.TestCase):
|
||||
configs = ast.literal_eval(mb_unittest.TEST_CONFIG)
|
||||
config_configs = configs['configs']
|
||||
mixins = configs['mixins']
|
||||
grouping = configs['masters']
|
||||
grouping = configs['builder_groups']
|
||||
errs = []
|
||||
|
||||
validation.CheckDuplicateConfigs(errs, config_configs, mixins, grouping,
|
||||
@ -198,7 +199,7 @@ class UnitTest(unittest.TestCase):
|
||||
configs = ast.literal_eval(mb_unittest.TEST_DUP_CONFIG)
|
||||
config_configs = configs['configs']
|
||||
mixins = configs['mixins']
|
||||
grouping = configs['masters']
|
||||
grouping = configs['builder_groups']
|
||||
errs = []
|
||||
|
||||
validation.CheckDuplicateConfigs(errs, config_configs, mixins, grouping,
|
||||
|
||||
Reference in New Issue
Block a user