Refactors the Skia Gold-related code in //content/test/gpu/gpu_tests
to be the same as //build/android/pylib. All the code that interacts
with Gold is encapsulated in its own class, and users simply run one
method and check its output to perform an image comparison.
Also adds a bunch of unittests (also taken from //build/android/pylib)
since the split makes the code unittest-able.
Drive-by fixes a related TODO in //build/android since the same TODO
was fixed in the GPU code in this CL.
Bug: 1093994
Change-Id: Id2fe1506796695f258069334149550cfaea49f71
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2243718
Reviewed-by: Tibor Goldschwendt <tiborg@chromium.org>
Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#778514}
Adds an '"ignore": "1"' pair to the JSON for GPU pixel tests if a test
has a grace period active. This is so that newly added noisy tests do
not affect unrelated CLs by causing Gold to comment on them because they
happened to produce a new image due to bad luck.
Bug: 1081942
Change-Id: Ife8dbe1baa349c203a4e95d7a152cb42d29d384f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2199908
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#768520}
Switches all tests that use Skia Gold to use a mixin instead of
individually specifying arguments in each test definition. As a
prerequisite, also ensures that all test types that use Skia Gold use
the same naming for Gold-related arguments.
This is requisite work for adding a universal Gold kill switch in the
unlikely event of an outage that can't be fixed quickly, as the switch
can just be added to the mixin.
Bug: 1057848
Change-Id: Ib57480f374eab088ac3a389637bfbff1ee41e8b4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2121419
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Sven Zheng <svenzheng@chromium.org>
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#755521}
Supports outputting the closest image known to Gold and its diff with
the produced image when running the GPU pixel tests locally. This should
improve the usability of the tests when debugging locally, as before
they only output the produced image and a link to all images known to
Gold for the test.
Bug: 1030349
Change-Id: I55cd724541f86edf8d6de534bd921d70d03d6eda
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1949727
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#721626}
Reland keeping a group forwarding //gpu:gl_tests to //gpu:gpu_tests
ReReland keeping gl_tests in the isolate map but making it point to
gpu_tests.
This is to make the name reflect that there aren't just OpenGL tests in
this executable now, there are WebGPU / Dawn tests too.
TBR=zmo@chromium.org
Bug: 1017035
Change-Id: Ibcf866f3b91dd10a8825d534d160b6252de8c7e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1897927
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Cr-Commit-Position: refs/heads/master@{#713002}
Simplifies the process of running the GPU pixel tests locally by making
them attempt to automatically detect if they are being run on a
workstation instead of on a bot, as well as automatically determining
the build revision. This removes the need to pass either --local-run
or --build-revision when running locally.
Bug: 1013358
Change-Id: Ie3fd2171d94fa4452b7f334c7bcdd5ab42705414
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1860677
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#705704}
Adds the option for GPU pixel tests to define a grace period, during
which failures will not be surfaced on the waterfall bots. This is to
help prevent newly added tests from turning the bots red due to a
bunch of untriaged images showing up after a test is first added. The
grace period gives the test author a bit of time to triage any new
images before the test starts failing.
Bug: 1008467
Change-Id: I14c219f86ababe96db4d95f18a4f7ff9bf5b469f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1829465
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/master@{#700892}
Merges various parts of the GPU Gold documentation from
gpu_pixel_testing_with_gold.md into the other parts of the GPU
documentation. Also removes references to the legacy pixel tests and
their triage process since the code for them has been removed.
Bug: 985007
Change-Id: I19d5480de77f1066046ef8fe0f076264cff01ee7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1721523
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#681868}
Change links pointing to old expectation's files and folders to links pointing
to new expectation's files and folders.
Bug: chromium:698902
No-Try: True
Change-Id: Icbf5220c12b303c838a3ae2a60de95b2b63a4015
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1606966
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#659271}
These bots were renamed already in the CQ, but docs weren't updated.
This CL updates everything I could find in chromium/src.
TBR=mef, kbr, odejesush, miu, tkent
Bug: 888810
Change-Id: I4c1d5ecb829c192a6f0f91160e2b4f8162f114a6
Reviewed-on: https://chromium-review.googlesource.com/c/1464135
Reviewed-by: Misha Efimov <mef@chromium.org>
Reviewed-by: Kent Tamura <tkent@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Cr-Commit-Position: refs/heads/master@{#631055}
Redefine the chromium.gpu, chromium.gpu.fyi and client.v8.fyi
waterfalls and test suites in waterfalls.pyl and test_suites.pyl,
respectively.
The goal of this conversion was to generate these three waterfalls'
JSON files almost exactly as they currently are, which has been
achieved. There are however many cleanups that should be done to
simplify the test suites and the configurations of the GPU bots, which
will be done in follow-on CLs and bugs.
Added a few needed features to the testing/buildbot version of the
generator script. Covered these in the unit tests; the coverage test
continues to pass. Deleted the old GPU script and updated docs that
referred to it.
Tbr: dpranke@chromium.org
Tbr: jbudorick@chromium.org
Tbr: ynovikov@chromium.org
Tbr: geofflang@chromium.org
Tbr: jmadill@chromium.org
Bug: 844815
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I4e472d0e55a700fe3cec5043dcd260d31de8fd41
Reviewed-on: https://chromium-review.googlesource.com/1067132
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#563939}
This was a difficult bug to track down and is worth mentioning as a
reason to prioritize stamping out flakiness on the waterfall.
Bug: 840988
Change-Id: If2c56a80b02178ddc28efd44d73ec440a2aa2e0c
Tbr: rjkroege@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/1054585
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#557758}
Also fixup some documentation links for GPU optional bots.
Also change a PRESUBMIT.py's master name for win_angle_rel_ng.
TBR=kbr@chromium.orgTBR=sky@chromium.org
BUG=chromium:815092
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_vr
Change-Id: Ifda505e121f67f922f19da466a37b7f6266cea05
Reviewed-on: https://chromium-review.googlesource.com/1008182
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Cr-Commit-Position: refs/heads/master@{#550188}
