Updates the GPU pixel wrangling documentation to point out
//content/test/gpu/trim_culprit_cls.py as a potentially useful tool and
updates the script's documentation to include example usages.
Bug: 1120104
Change-Id: I2026e994408097bfee8b94984dc89e8110afba0e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2373391
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Zhenyao Mo <zmo@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#801232}
Refactors the Skia Gold-related code in //content/test/gpu/gpu_tests
to be the same as //build/android/pylib. All the code that interacts
with Gold is encapsulated in its own class, and users simply run one
method and check its output to perform an image comparison.
Also adds a bunch of unittests (also taken from //build/android/pylib)
since the split makes the code unittest-able.
Drive-by fixes a related TODO in //build/android since the same TODO
was fixed in the GPU code in this CL.
Bug: 1093994
Change-Id: Id2fe1506796695f258069334149550cfaea49f71
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2243718
Reviewed-by: Tibor Goldschwendt <tiborg@chromium.org>
Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#778514}
Adds an '"ignore": "1"' pair to the JSON for GPU pixel tests if a test
has a grace period active. This is so that newly added noisy tests do
not affect unrelated CLs by causing Gold to comment on them because they
happened to produce a new image due to bad luck.
Bug: 1081942
Change-Id: Ife8dbe1baa349c203a4e95d7a152cb42d29d384f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2199908
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#768520}
Updates gpu_testing_bot_details.md with the following:
1. An updated reference CL for adding a manual-only trybot.
2. Updated locations for *.star files.
3. Console and builder definitions taking place in the same file.
Change-Id: I7930c32807f590a9d8000fa8e06e95774889744d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2195326
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/master@{#767607}
Adds a script to find optimal values for inexact matching in the GPU
pixel tests and enables inexact matching on several problematic tests
that could benefit from it.
Bug: 1074130
Change-Id: If4597bbda1a94fdc4dd28f87ca59ea86489c2ac1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2173415
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/master@{#764424}
Switches all tests that use Skia Gold to use a mixin instead of
individually specifying arguments in each test definition. As a
prerequisite, also ensures that all test types that use Skia Gold use
the same naming for Gold-related arguments.
This is requisite work for adding a universal Gold kill switch in the
unlikely event of an outage that can't be fixed quickly, as the switch
can just be added to the mixin.
Bug: 1057848
Change-Id: Ib57480f374eab088ac3a389637bfbff1ee41e8b4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2121419
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Sven Zheng <svenzheng@chromium.org>
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#755521}
Updating the Pixel Wrangler docs section on extending the rotation. Making them
more general for modifying the rotation. With a portion about swapping.
Change-Id: Ief4c3ffc86f87fe924e94900c16b017f38e9ed80
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2037327
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Jonathan Ross <jonross@chromium.org>
Cr-Commit-Position: refs/heads/master@{#738363}
Supports outputting the closest image known to Gold and its diff with
the produced image when running the GPU pixel tests locally. This should
improve the usability of the tests when debugging locally, as before
they only output the produced image and a link to all images known to
Gold for the test.
Bug: 1030349
Change-Id: I55cd724541f86edf8d6de534bd921d70d03d6eda
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1949727
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#721626}
Reland keeping a group forwarding //gpu:gl_tests to //gpu:gpu_tests
ReReland keeping gl_tests in the isolate map but making it point to
gpu_tests.
This is to make the name reflect that there aren't just OpenGL tests in
this executable now, there are WebGPU / Dawn tests too.
TBR=zmo@chromium.org
Bug: 1017035
Change-Id: Ibcf866f3b91dd10a8825d534d160b6252de8c7e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1897927
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Cr-Commit-Position: refs/heads/master@{#713002}
Simplifies the process of running the GPU pixel tests locally by making
them attempt to automatically detect if they are being run on a
workstation instead of on a bot, as well as automatically determining
the build revision. This removes the need to pass either --local-run
or --build-revision when running locally.
Bug: 1013358
Change-Id: Ie3fd2171d94fa4452b7f334c7bcdd5ab42705414
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1860677
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#705704}
Adds the option for GPU pixel tests to define a grace period, during
which failures will not be surfaced on the waterfall bots. This is to
help prevent newly added tests from turning the bots red due to a
bunch of untriaged images showing up after a test is first added. The
grace period gives the test author a bit of time to triage any new
images before the test starts failing.
Bug: 1008467
Change-Id: I14c219f86ababe96db4d95f18a4f7ff9bf5b469f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1829465
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/master@{#700892}
Makes GPU gold pixel tests output links to the triage page for the
entire CL instead of each individual image when running on the trybots.
This makes it so that users only need to open one link regardless of
how many failures there are in order to triage everything.
Bug: 1008524
Change-Id: I94891663fe09bb59ba386e7bd24bc72a4ffe1468
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1825907
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/master@{#700441}
Adds some sample CLs to the GPU documentation for upgrading driver/OS
versions. Also updates the step that previously mentioned suppressing
the pixel tests if the baselines change due to the upgrade since Gold
handles this case better than the legacy pixel tests.
TBR=kbr@chromium.org
Bug: 916832
Change-Id: I674e7839b2582587363b4443bd33fe0a02753a02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1749564
Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#686067}
Merges various parts of the GPU Gold documentation from
gpu_pixel_testing_with_gold.md into the other parts of the GPU
documentation. Also removes references to the legacy pixel tests and
their triage process since the code for them has been removed.
Bug: 985007
Change-Id: I19d5480de77f1066046ef8fe0f076264cff01ee7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1721523
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#681868}
Removes all the code that was only used by the legacy (cloud
storage-based) GPU pixel tests since Gold is now used everywhere.
Additionally, moves much of the remaining code out of
cloud_storage_integration_test_base and into specific subclasses since
it was only actually being used by one subclass.
This is pretty much cut/paste work - refactoring work to make the code
more readable will come in a follow-up CL.
Bug: 985007
Change-Id: Ifd42142629f641e737675b5b4c0b1091b98b9757
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1715626
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#680655}
Adds a --local-run flag to the GPU pixel tests. This causes goldctl to
run in --dryrun mode (i.e. no uploads) and makes the test runner output
file:// URLs to the produced images. This makes it much more suitable
for local runs, since Gold's normal usage mode is designed to be used on
the bots.
Bug: 985418
Change-Id: I70021679e725d619c9dfc38598655c881b2dcb8b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1713588
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#679778}
Adds a small section to the GPU Skia Gold Markdown documentation
explaining how to find a failed swarming task for a bad image in the
rare case that a bad image is found without a link to the failed build.
Bug: 965234
Change-Id: I257c95ed686f8b6902bd0739b9a1bb93054906f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1700293
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#677127}
Updates some portions of the GPU Skia Gold markdown documentation, such
as the new triage process using the links reported in Milo.
Bug: 965234
Change-Id: If22ee0f6e7af1054df98630f2f9ebb616a8fdbf9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1668092
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#675816}
Adds Markdown documentation on the GPU team's Skia Gold-enabled pixel
tests, such as how to triage images, how they work, and how to modify
Gold if necessary.
Bug: 965234
Change-Id: Ie77c9679e458d7196af1457edbf2b06f45799f8a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1638739
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#668976}
The section on pixel tests lists an output link format. However for the
pixel_skia_gold_tests the link format is different.
This updates the docs to include a note about the differences in links, with an
example.
Change-Id: I444c5391a3c05561b4177a76a7d3db70d4f03361
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1638548
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Commit-Queue: Jonathan Ross <jonross@chromium.org>
Cr-Commit-Position: refs/heads/master@{#665247}
The documentation for creating a new VM for the GPU bots says to run
gen.star. That file no longer exists. It is now main.star.
This CL updates the documentation to reflect this change.
TBR=kbr@chromium.org
Bug: 963242
Change-Id: Ie8cb26cf92bb59c38af12dca01934a536c299314
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1612346
Reviewed-by: Chris Blume <cblume@chromium.org>
Commit-Queue: Chris Blume <cblume@chromium.org>
Cr-Commit-Position: refs/heads/master@{#659772}
Change links pointing to old expectation's files and folders to links pointing
to new expectation's files and folders.
Bug: chromium:698902
No-Try: True
Change-Id: Icbf5220c12b303c838a3ae2a60de95b2b63a4015
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1606966
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#659271}
