Makes some minor updates to the GPU testing documentation to clarify
some confusion that a first-time user recently encountered:
* Updates the webgl_conformance suite to correctly reflect that there is
now a webgl1_conformance and webgl2_conformance suite.
* Makes it clearer that `vpython3` needs to be manually included when
running the script on Windows.
Change-Id: Iefbef22db9234f0d7d3413433366a9e186fc7eee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4443043
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1132443}
The builder was renamed to "win-rel" as part of crbug.com/1381274. It
should have the exact same behavior. So any doc or comment that
mentions it has been updated to the new name here.
Although web_tests' references were unchanged. Filed crbug.com/1383534
to cover those since they seem to have some effect on behavior.
Bug: 1381274
Change-Id: I9204dee6f58dd34826de3a0b3a8372ec5f7e5e43
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4022168
Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Joshua Pawlicki <waffles@chromium.org>
Reviewed-by: Xianzhu Wang <wangxianzhu@chromium.org>
Reviewed-by: Kevin McNee <mcnee@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1071119}
Remove some test suite exceptions that were missed when deleting the
gl_renderer_* test suites. Also remove some filters for deleted
viz_unittests.
Bug: 1247756
Change-Id: I03f562b728fecb6437b78e14deddf5a30fe17289
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3696523
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Commit-Queue: Kyle Charbonneau <kylechar@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1012540}
Legacy SwiftShader GL is deprecated, so remove it from existing
documentation, so that only SwANGLE related documentation remains.
Bug: chromium:1060139
Change-Id: I1a59d7b4a57f80cbe627df0a77430450296df607
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3452836
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Alexis Hétu <sugoi@google.com>
Cr-Commit-Position: refs/heads/main@{#969936}
Updates the Markdown documentation about building/running GPU
integration tests to refer to the correct targets and include some
additional details that may be helpful to people that are unfamiliar
with the tests.
Bug: 1270361
Change-Id: I6eb787b7abbc9e9d24672b626c93dacd7ac42bff
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3283629
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/main@{#941916}
Updates the following in the GPU documentation:
1. Links to the Skia Gold instance. There were still references to the
old chrome-gpu-gold.skia.org instance that has been gone for a long
time.
2. Links to the pixel_wrangling.md documentation, which has been moved
to various internal documentation pages.
Bug: 1250846
Change-Id: Ib72b870c91a8048bbe4568256ff1845c2d72d7a0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3173927
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/main@{#924044}
Updates //docs/gpu/gpu_testing.md in a few ways:
1. Clarifies that vpython should be used instead of python
2. Explicitly lists all Telemetry-based test suites and the names they
can show up as on the bots. This list was generated by parsing the
//testing/buildbot JSON files.
3. Updates the instructions on how to obtain the command used to run
a test on swarming and how to download an isolate.
Bug: 1199154
Change-Id: I21bbbe7341d8788ef9e0715e5c090b8cc37469ec
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2828908
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/master@{#873436}
Refactors the Skia Gold-related code in //content/test/gpu/gpu_tests
to be the same as //build/android/pylib. All the code that interacts
with Gold is encapsulated in its own class, and users simply run one
method and check its output to perform an image comparison.
Also adds a bunch of unittests (also taken from //build/android/pylib)
since the split makes the code unittest-able.
Drive-by fixes a related TODO in //build/android since the same TODO
was fixed in the GPU code in this CL.
Bug: 1093994
Change-Id: Id2fe1506796695f258069334149550cfaea49f71
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2243718
Reviewed-by: Tibor Goldschwendt <tiborg@chromium.org>
Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#778514}
Adds an '"ignore": "1"' pair to the JSON for GPU pixel tests if a test
has a grace period active. This is so that newly added noisy tests do
not affect unrelated CLs by causing Gold to comment on them because they
happened to produce a new image due to bad luck.
Bug: 1081942
Change-Id: Ife8dbe1baa349c203a4e95d7a152cb42d29d384f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2199908
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#768520}
Switches all tests that use Skia Gold to use a mixin instead of
individually specifying arguments in each test definition. As a
prerequisite, also ensures that all test types that use Skia Gold use
the same naming for Gold-related arguments.
This is requisite work for adding a universal Gold kill switch in the
unlikely event of an outage that can't be fixed quickly, as the switch
can just be added to the mixin.
Bug: 1057848
Change-Id: Ib57480f374eab088ac3a389637bfbff1ee41e8b4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2121419
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Sven Zheng <svenzheng@chromium.org>
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#755521}
Supports outputting the closest image known to Gold and its diff with
the produced image when running the GPU pixel tests locally. This should
improve the usability of the tests when debugging locally, as before
they only output the produced image and a link to all images known to
Gold for the test.
Bug: 1030349
Change-Id: I55cd724541f86edf8d6de534bd921d70d03d6eda
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1949727
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#721626}
Reland keeping a group forwarding //gpu:gl_tests to //gpu:gpu_tests
ReReland keeping gl_tests in the isolate map but making it point to
gpu_tests.
This is to make the name reflect that there aren't just OpenGL tests in
this executable now, there are WebGPU / Dawn tests too.
TBR=zmo@chromium.org
Bug: 1017035
Change-Id: Ibcf866f3b91dd10a8825d534d160b6252de8c7e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1897927
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Cr-Commit-Position: refs/heads/master@{#713002}
Simplifies the process of running the GPU pixel tests locally by making
them attempt to automatically detect if they are being run on a
workstation instead of on a bot, as well as automatically determining
the build revision. This removes the need to pass either --local-run
or --build-revision when running locally.
Bug: 1013358
Change-Id: Ie3fd2171d94fa4452b7f334c7bcdd5ab42705414
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1860677
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#705704}
Adds the option for GPU pixel tests to define a grace period, during
which failures will not be surfaced on the waterfall bots. This is to
help prevent newly added tests from turning the bots red due to a
bunch of untriaged images showing up after a test is first added. The
grace period gives the test author a bit of time to triage any new
images before the test starts failing.
Bug: 1008467
Change-Id: I14c219f86ababe96db4d95f18a4f7ff9bf5b469f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1829465
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/master@{#700892}
Merges various parts of the GPU Gold documentation from
gpu_pixel_testing_with_gold.md into the other parts of the GPU
documentation. Also removes references to the legacy pixel tests and
their triage process since the code for them has been removed.
Bug: 985007
Change-Id: I19d5480de77f1066046ef8fe0f076264cff01ee7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1721523
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#681868}
Change links pointing to old expectation's files and folders to links pointing
to new expectation's files and folders.
Bug: chromium:698902
No-Try: True
Change-Id: Icbf5220c12b303c838a3ae2a60de95b2b63a4015
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1606966
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#659271}
These bots were renamed already in the CQ, but docs weren't updated.
This CL updates everything I could find in chromium/src.
TBR=mef, kbr, odejesush, miu, tkent
Bug: 888810
Change-Id: I4c1d5ecb829c192a6f0f91160e2b4f8162f114a6
Reviewed-on: https://chromium-review.googlesource.com/c/1464135
Reviewed-by: Misha Efimov <mef@chromium.org>
Reviewed-by: Kent Tamura <tkent@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Cr-Commit-Position: refs/heads/master@{#631055}
Redefine the chromium.gpu, chromium.gpu.fyi and client.v8.fyi
waterfalls and test suites in waterfalls.pyl and test_suites.pyl,
respectively.
The goal of this conversion was to generate these three waterfalls'
JSON files almost exactly as they currently are, which has been
achieved. There are however many cleanups that should be done to
simplify the test suites and the configurations of the GPU bots, which
will be done in follow-on CLs and bugs.
Added a few needed features to the testing/buildbot version of the
generator script. Covered these in the unit tests; the coverage test
continues to pass. Deleted the old GPU script and updated docs that
referred to it.
Tbr: dpranke@chromium.org
Tbr: jbudorick@chromium.org
Tbr: ynovikov@chromium.org
Tbr: geofflang@chromium.org
Tbr: jmadill@chromium.org
Bug: 844815
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I4e472d0e55a700fe3cec5043dcd260d31de8fd41
Reviewed-on: https://chromium-review.googlesource.com/1067132
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#563939}
This was a difficult bug to track down and is worth mentioning as a
reason to prioritize stamping out flakiness on the waterfall.
Bug: 840988
Change-Id: If2c56a80b02178ddc28efd44d73ec440a2aa2e0c
Tbr: rjkroege@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/1054585
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#557758}
Also fixup some documentation links for GPU optional bots.
Also change a PRESUBMIT.py's master name for win_angle_rel_ng.
TBR=kbr@chromium.orgTBR=sky@chromium.org
BUG=chromium:815092
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_vr
Change-Id: Ifda505e121f67f922f19da466a37b7f6266cea05
Reviewed-on: https://chromium-review.googlesource.com/1008182
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Cr-Commit-Position: refs/heads/master@{#550188}
