Reland "Add new ANGLE Linux and Mac tester configs."
This is a reland of fab66bae04
Includes necessary change to luci-scheduler.cfg and updated docs.
Original change's description:
> Add new ANGLE Linux and Mac tester configs.
>
> These configs will replace linux_angle_rel_ng and mac_angle_rel_ng.
> They initially mirror the configuration but will soon be updated to
> run only the subsets of tests that are most relevant to ANGLE. This
> should reduce flakiness in ANGLE and reduce load for Chrome.
>
> Also updates related configuration files to include linux-angle-rel
> and mac-angle-rel.
>
> Bug: 822310
> Change-Id: I2b40ed45f5d9e105c088ab166bd2207c96280151
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1551463
> Reviewed-by: John Budorick <jbudorick@chromium.org>
> Reviewed-by: Kenneth Russell <kbr@chromium.org>
> Commit-Queue: Jamie Madill <jmadill@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#647854}
Bug: 822310
Tbr: jbudorick@chromium.org
Change-Id: Iaeeffc7b089b05d379bb353d627d13b39fd43704
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1554296
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#648727}
This commit is contained in:
docs/gpu
infra/config
testing/buildbot
tools/mb
@ -535,63 +535,55 @@ chrome-infra team if this doesn't work as expected.)
|
||||
[go/chromecals]: http://go/chromecals
|
||||
|
||||
|
||||
### How to add a new "optional" try bot
|
||||
### How to add a new try bot that runs a subset of tests or extra tests
|
||||
|
||||
TODO(kbr): the naming of the "optional" try bots is confusing and
|
||||
unfortunate. They should probably be renamed to something like "extratests" or
|
||||
"extra_tests", so perhaps a new naming convention of "gpu_win_extratests_rel" or
|
||||
"win_gpu_extratests_rel". Unfortunately making this change at this point
|
||||
requires touching tons of files across many workspaces and is unlikely to happen
|
||||
unless someone highly motivated wants to pick up the task.
|
||||
Several projects (ANGLE, Dawn) run custom tests using the Chromium recipes. They
|
||||
use try bot bot configs that run subsets of Chromium or additional slower tests
|
||||
that can't be run on the main CQ.
|
||||
|
||||
The "optional" GPU try bots are a concession to the reality that there are some
|
||||
long-running GPU test suites that simply can not run against every Chromium CL.
|
||||
They run some additional tests that are usually run only on the
|
||||
chromium.gpu.fyi waterfall. Some of these tests, like the WebGL 2.0 conformance
|
||||
suite, are intended to be run on the normal try bots once hardware capacity is
|
||||
available. Some are not intended to ever run on the normal try bots.
|
||||
These try bots are a little different because they mirror waterfall bots that
|
||||
don't actually exist. The waterfall bots' specifications exist only to tell
|
||||
these try bots which tests to run.
|
||||
|
||||
The optional try bots are a little different because they mirror waterfall bots
|
||||
that don't actually exist. The waterfall bots' specifications exist only to
|
||||
tell the optional try bots which tests to run.
|
||||
|
||||
Let's say that you intended to add a new such optional try bot on Windows. Call
|
||||
it `win_new_optional_tests_rel` for example. Now, if you wanted to just add
|
||||
this GPU type to the existing `win_optional_gpu_tests_rel` try bot, you'd
|
||||
just follow the instructions above
|
||||
([How to start running tests on a new GPU type on an existing try bot](#How-to-start-running-tests-on-a-new-GPU-type-on-an-existing-try-bot)). The steps below describe how to spin up
|
||||
an entire new optional try bot.
|
||||
Let's say that you intended to add a new such custom try bot on Windows. Call it
|
||||
`win-myproject-rel` for example. You will need to add a "fake" mirror bot for
|
||||
each GPU family the tests you will need to run. For a GPU type of
|
||||
"CoolNewGPUType" in this example you could add a "fake" bot named "MyProject GPU
|
||||
Win10 Release (CoolNewGPUType)".
|
||||
|
||||
1. Allocate new virtual machines for the bots as described in [How to set up
|
||||
new virtual machine
|
||||
instances](#How-to-set-up-new-virtual-machine-instances).
|
||||
1. Make sure that you have some swarming capacity for the new GPU type. Since
|
||||
it's not running against all Chromium CLs you don't need the recommended 30
|
||||
minimum bots, though ~10 would be good.
|
||||
1. Create a CL in the Chromium workspace:
|
||||
1. Add your new bot (for example, "Optional Win7 Release
|
||||
1. Create a CL in the Chromium workspace the does the following. Here's an
|
||||
[example CL](https://crrev.com/c/1554296).
|
||||
1. Add your new bot (for example, "MyProject GPU Win10 Release
|
||||
(CoolNewGPUType)") to the chromium.gpu.fyi waterfall in
|
||||
[waterfalls.pyl]. (Note, this is a bad example: the
|
||||
"optional" bots have special semantics in this script. You'd probably
|
||||
want to define some new category of bot if you didn't intend to add
|
||||
this to `win_optional_gpu_tests_rel`.)
|
||||
1. Re-run the script to regenerate the JSON files.
|
||||
1. Land the above CL.
|
||||
1. Create a CL in the tools/build workspace:
|
||||
1. Modify `masters/master.tryserver.chromium.win`'s [master.cfg] and
|
||||
[slaves.cfg] to add the new tryserver. Follow the pattern for the
|
||||
existing `win_optional_gpu_tests_rel` tryserver. Namely, add the new
|
||||
entry to master.cfg, and add the new tryserver to the
|
||||
`optional_builders` list in `slaves.cfg`.
|
||||
1. Modify [`chromium_gpu_fyi.py`][chromium_gpu_fyi.py] to add the new
|
||||
"Optional Win7 Release (CoolNewGPUType)" entry.
|
||||
1. Modify [`trybots.py`][trybots.py] to add
|
||||
the new `win_new_optional_tests_rel` try bot, mirroring "Optional
|
||||
Win7 Release (CoolNewGPUType)".
|
||||
1. Land the above CL and request an off-hours restart of the
|
||||
tryserver.chromium.win waterfall.
|
||||
1. Now you can send CLs to the new bot with:
|
||||
`git cl try -m tryserver.chromium.win -b win_new_optional_tests_rel`
|
||||
|
||||
[master.cfg]: https://chromium.googlesource.com/chromium/tools/build/+/master/masters/master.tryserver.chromium.win/master.cfg
|
||||
[slaves.cfg]: https://chromium.googlesource.com/chromium/tools/build/+/master/masters/master.tryserver.chromium.win/slaves.cfg
|
||||
[waterfalls.pyl].
|
||||
1. Re-run [`src/testing/buildbot/generate_buildbot_json.py`][generate_buildbot_json.py] to regenerate the JSON files.
|
||||
1. Update [`cr-buildbucket.cfg`][cr-buildbucket.cfg] to add `win-myproject-rel`.
|
||||
1. Update [`luci-milo.cfg`][luci-milo.cfg] to include `win-myproject-rel`.
|
||||
1. Update [`luci-scheduler.cfg`][luci-scheduler.cfg] to include "MyProject GPU Win10 Release
|
||||
(CoolNewGPUType)".
|
||||
1. Update [`src/tools/mb/mb_config.pyl`][mb_config.pyl] to include `win-myproject-rel`.
|
||||
1. Also add your fake bot to [`src/testing/buildbot/generate_buildbot_json.py`][generate_buildbot_json.py] in the list of `get_bots_that_do_not_actually_exist` section.
|
||||
1. *After* the Chromium-side CL lands and the bot is on the console, create a CL
|
||||
in the [`tools/build`][tools/build] workspace which does the
|
||||
following. Here's an [example CL](https://crrev.com/c/1554272).
|
||||
1. Adds "MyProject GPU Win10 Release
|
||||
(CoolNewGPUType)" to [`chromium_gpu_fyi.py`][chromium_gpu_fyi.py] in
|
||||
`scripts/slave/recipe_modules/chromium_tests/`. You can copy a similar
|
||||
step.
|
||||
1. Adds `win-myproject-rel` to [`trybots.py`][trybots.py] in the same folder.
|
||||
This is where you associate "MyProject GPU Win10 Release
|
||||
(CoolNewGPUType)" with `win-myproject-rel`. See the sample CL for an example.
|
||||
1. Get this reviewed and landed. This step tells the Chromium recipe about
|
||||
the newly-deployed waterfall bot, so it knows which JSON file to load
|
||||
out of src/testing/buildbot and which entry to look at.
|
||||
1. After your CLs land you should be able to find and run `win-myproject-rel` on CLs
|
||||
using Choose Trybots in Gerrit.
|
||||
|
||||
### How to test and deploy a driver update
|
||||
|
||||
|
@ -3506,6 +3506,7 @@ buckets {
|
||||
name: "gpu-manual-try-linux-nvidia-tsn"
|
||||
}
|
||||
builders { mixins: "linux-try" name: "leak_detection_linux" }
|
||||
builders { mixins: "linux-angle-try" name: "linux-angle-rel" }
|
||||
builders { mixins: "linux-angle-try" name: "linux_angle_compile_dbg_ng" }
|
||||
builders { mixins: "linux-angle-try" name: "linux_angle_dbg_ng" }
|
||||
builders { mixins: "linux-angle-try" name: "linux_angle_deqp_rel_ng" }
|
||||
@ -3625,6 +3626,7 @@ buckets {
|
||||
builders { mixins: "ios-try" name: "ios-simulator-eg" }
|
||||
builders { mixins: "ios-try" name: "ios-simulator-xcode-clang" }
|
||||
builders { mixins: "ios-try" name: "ios-slimnav" }
|
||||
builders { mixins: "mac-angle-try" name: "mac-angle-rel" }
|
||||
builders { mixins: "mac-angle-try" name: "mac_angle_compile_dbg_ng" }
|
||||
builders { mixins: "mac-angle-try" name: "mac_angle_dbg_ng" }
|
||||
builders { mixins: "mac-angle-try" name: "mac_angle_rel_ng" }
|
||||
|
@ -4477,6 +4477,9 @@ consoles {
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/android_angle_vk64_rel_ng"
|
||||
}
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/linux-angle-rel"
|
||||
}
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/linux_angle_dbg_ng"
|
||||
}
|
||||
@ -4489,6 +4492,9 @@ consoles {
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/linux_angle_rel_ng"
|
||||
}
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/mac-angle-rel"
|
||||
}
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/mac_angle_dbg_ng"
|
||||
}
|
||||
@ -4718,6 +4724,9 @@ consoles {
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/gpu-manual-try-win10-nvidia-rel"
|
||||
}
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/linux-angle-rel"
|
||||
}
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/linux-blink-heap-concurrent-marking-tsan-rel"
|
||||
}
|
||||
@ -4857,6 +4866,9 @@ consoles {
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/ios12-sdk-simulator"
|
||||
}
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/mac-angle-rel"
|
||||
}
|
||||
builders {
|
||||
name: "buildbucket/luci.chromium.try/mac-jumbo-rel"
|
||||
}
|
||||
|
@ -1311,6 +1311,22 @@ job {
|
||||
# builder-Builders.
|
||||
################################################################################
|
||||
|
||||
job {
|
||||
id: "ANGLE GPU Linux Release (Intel HD 630)"
|
||||
# Triggered by "GPU FYI Linux Builder".
|
||||
acl_sets: "triggered-by-parent-builders"
|
||||
# This bot doesn't actually exist, so it's noop'ed out. crbug.com/819899
|
||||
noop: {}
|
||||
}
|
||||
|
||||
job {
|
||||
id: "ANGLE GPU Linux Release (NVIDIA)"
|
||||
# Triggered by "GPU FYI Linux Builder".
|
||||
acl_sets: "triggered-by-parent-builders"
|
||||
# This bot doesn't actually exist, so it's noop'ed out. crbug.com/819899
|
||||
noop: {}
|
||||
}
|
||||
|
||||
job {
|
||||
id: "Cast Audio Linux"
|
||||
acl_sets: "default"
|
||||
@ -1984,6 +2000,30 @@ job {
|
||||
# Mac Builders. Sorted alphabetically.
|
||||
################################################################################
|
||||
|
||||
job {
|
||||
id: "ANGLE GPU Mac Release (Intel)"
|
||||
# Triggered by "GPU FYI Mac Builder".
|
||||
acl_sets: "triggered-by-parent-builders"
|
||||
# This bot doesn't actually exist, so it's noop'ed out. crbug.com/819899
|
||||
noop: {}
|
||||
}
|
||||
|
||||
job {
|
||||
id: "ANGLE GPU Mac Retina Release (NVIDIA)"
|
||||
# Triggered by "GPU FYI Mac Builder".
|
||||
acl_sets: "triggered-by-parent-builders"
|
||||
# This bot doesn't actually exist, so it's noop'ed out. crbug.com/819899
|
||||
noop: {}
|
||||
}
|
||||
|
||||
job {
|
||||
id: "ANGLE GPU Mac Retina Release (AMD)"
|
||||
# Triggered by "GPU FYI Mac Builder".
|
||||
acl_sets: "triggered-by-parent-builders"
|
||||
# This bot doesn't actually exist, so it's noop'ed out. crbug.com/819899
|
||||
noop: {}
|
||||
}
|
||||
|
||||
job {
|
||||
id: "Dawn GPU Mac Release (Intel)"
|
||||
# Triggered by "GPU FYI Mac Builder".
|
||||
|
File diff suppressed because it is too large
Load Diff
@ -920,6 +920,11 @@ class BBJSONGenerator(object):
|
||||
# are defined only to be mirrored into trybots, and don't actually
|
||||
# exist on any of the waterfalls or consoles.
|
||||
return [
|
||||
'ANGLE GPU Linux Release (Intel HD 630)',
|
||||
'ANGLE GPU Linux Release (NVIDIA)',
|
||||
'ANGLE GPU Mac Release (Intel)',
|
||||
'ANGLE GPU Mac Retina Release (AMD)',
|
||||
'ANGLE GPU Mac Retina Release (NVIDIA)',
|
||||
'ANGLE GPU Win10 Release (Intel HD 630)',
|
||||
'ANGLE GPU Win10 Release (NVIDIA)',
|
||||
'Dawn GPU Linux Release (Intel HD 630)',
|
||||
|
@ -2070,6 +2070,67 @@
|
||||
'name': 'chromium.gpu.fyi',
|
||||
'machines': {
|
||||
# BEGIN Fake builder used as mirror targets for ANGLE's GPU tryservers
|
||||
'ANGLE GPU Linux Release (Intel HD 630)': {
|
||||
'os_type': 'linux',
|
||||
'browser_config': 'release',
|
||||
'mixins': [
|
||||
'linux_intel_hd_630',
|
||||
],
|
||||
'test_suites': {
|
||||
# TODO(jmadill): Use custom test lists. crbug.com/822310
|
||||
'gtest_tests': 'gpu_fyi_linux_release_gtests',
|
||||
'gpu_telemetry_tests': 'gpu_fyi_linux_intel_and_nvidia_release_telemetry_tests',
|
||||
},
|
||||
},
|
||||
'ANGLE GPU Linux Release (NVIDIA)': {
|
||||
'os_type': 'linux',
|
||||
'browser_config': 'release',
|
||||
'mixins': [
|
||||
'linux_nvidia_quadro_p400',
|
||||
],
|
||||
'test_suites': {
|
||||
# TODO(jmadill): Use custom test lists. crbug.com/822310
|
||||
'gtest_tests': 'gpu_fyi_linux_release_gtests',
|
||||
'isolated_scripts': 'gpu_angle_perftests',
|
||||
'gpu_telemetry_tests': 'gpu_fyi_linux_intel_and_nvidia_release_telemetry_tests',
|
||||
},
|
||||
},
|
||||
'ANGLE GPU Mac Release (Intel)': {
|
||||
'os_type': 'mac',
|
||||
'browser_config': 'release',
|
||||
'mixins': [
|
||||
'mac_mini_intel_gpu',
|
||||
],
|
||||
'test_suites': {
|
||||
# TODO(jmadill): Use custom test lists. crbug.com/822310
|
||||
'gtest_tests': 'gpu_fyi_mac_release_gtests',
|
||||
'gpu_telemetry_tests': 'gpu_fyi_mac_release_telemetry_tests',
|
||||
},
|
||||
},
|
||||
'ANGLE GPU Mac Retina Release (AMD)': {
|
||||
'os_type': 'mac',
|
||||
'browser_config': 'release',
|
||||
'mixins': [
|
||||
'mac_retina_amd_gpu',
|
||||
],
|
||||
'test_suites': {
|
||||
# TODO(jmadill): Use custom test lists. crbug.com/822310
|
||||
'gtest_tests': 'gpu_fyi_mac_release_gtests',
|
||||
'gpu_telemetry_tests': 'gpu_fyi_mac_release_telemetry_tests',
|
||||
},
|
||||
},
|
||||
'ANGLE GPU Mac Retina Release (NVIDIA)': {
|
||||
'os_type': 'mac',
|
||||
'browser_config': 'release',
|
||||
'mixins': [
|
||||
'mac_retina_nvidia_gpu',
|
||||
],
|
||||
'test_suites': {
|
||||
# TODO(jmadill): Use custom test lists. crbug.com/822310
|
||||
'gtest_tests': 'gpu_fyi_mac_release_gtests',
|
||||
'gpu_telemetry_tests': 'gpu_fyi_mac_release_telemetry_tests',
|
||||
},
|
||||
},
|
||||
'ANGLE GPU Win10 Release (Intel HD 630)': {
|
||||
'os_type': 'win',
|
||||
'browser_config': 'release',
|
||||
|
@ -669,10 +669,12 @@
|
||||
'android_angle_deqp_rel_ng': 'deqp_android_release_trybot_arm64',
|
||||
'android_angle_vk32_deqp_rel_ng': 'deqp_android_vulkan_release_trybot',
|
||||
'android_angle_vk64_deqp_rel_ng': 'deqp_android_vulkan_release_trybot_arm64',
|
||||
'linux-angle-rel': 'gpu_fyi_tests_release_trybot',
|
||||
'linux_angle_ozone_rel_ng': 'gpu_fyi_tests_ozone_linux_system_gbm_libdrm_release_trybot',
|
||||
'linux_angle_dbg_ng': 'gpu_fyi_tests_debug_trybot',
|
||||
'linux_angle_deqp_rel_ng': 'deqp_release_trybot',
|
||||
'linux_angle_rel_ng': 'gpu_fyi_tests_release_trybot',
|
||||
'mac-angle-rel': 'gpu_fyi_tests_release_trybot',
|
||||
'mac_angle_dbg_ng': 'gpu_fyi_tests_debug_trybot',
|
||||
'mac_angle_rel_ng': 'gpu_fyi_tests_release_trybot',
|
||||
'win-angle-rel': 'gpu_fyi_tests_release_trybot_x86',
|
||||
|
Reference in New Issue
Block a user