A fresh WPT import checkout is very likely to fail when no GitHub
credentials are used due to the number of API requests we make vs.
GitHub's rate limits for unauthenticated requests.
Make things more obvious by:
* Allowing the credentials to be specified as environment variables in
addition to a JSON file (when both are set, the JSON file specified in
the command-line has precedence).
* Printing a warning in test_importer.py when no credentials are set and
pointing to the documentation in //docs/testing.
* Expanding the error message in test_exporter.py to point to the same
location.
Bug: 816390
Change-Id: I5508e024e8a13e6f597f027cd157d7abbe9b2fe1
Reviewed-on: https://chromium-review.googlesource.com/937462
Reviewed-by: Philip Jägenstedt <foolip@chromium.org>
Commit-Queue: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com>
Cr-Commit-Position: refs/heads/master@{#539164}
--allow-local-commits was removed in 4a301c5 ("Remove the
--allow-local-commits flag from wpt-import") so stop recommending its usage.
Additionally, given the amount of export commits we have in
web-platform-tests it is likely that unauthenticated runs will fail, so
explain how to use the --credentials-json option in both wpt-export and
wpt-import.
Bug: 816390
Change-Id: I72d7bc6dcb80d0db372e9dac1176f68d1b812bcb
Reviewed-on: https://chromium-review.googlesource.com/937241
Reviewed-by: Philip Jägenstedt <foolip@chromium.org>
Commit-Queue: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com>
Cr-Commit-Position: refs/heads/master@{#539116}
Replaces mentions of --interactive with --test-launcher-interactive and
one stale --dialog with --ui.
Bug: None
Change-Id: I6d2f80aae8b4282fdb1bd57bee2dd52d86b43250
Reviewed-on: https://chromium-review.googlesource.com/929342
Commit-Queue: Peter Kasting <pkasting@chromium.org>
Reviewed-by: Peter Kasting <pkasting@chromium.org>
Cr-Commit-Position: refs/heads/master@{#538221}
This introduces a new base class, TestBrowserUI, and reorganizes the existing
test code into a series of virtual functions so tests can provide the necessary
implementation for non-dialog cases.
This also adds a single consumer of this new base class, InfoBarUITest, to
verify it works.
Bug: 686285
Change-Id: I2b5a7c86347f5042ac5218c9fa90e94f68adf577
Reviewed-on: https://chromium-review.googlesource.com/804953
Commit-Queue: Peter Kasting <pkasting@chromium.org>
Reviewed-by: Scott Violet <sky@chromium.org>
Reviewed-by: Trent Apted <tapted@chromium.org>
Cr-Commit-Position: refs/heads/master@{#524855}
This clarifies the meaning of forward slashes in artifact file paths,
and also clarifies the directory these paths are relative to, when no
artifact permanent location is set.
Bug: 775619
Change-Id: I1f41d86273455123823350a39b239cc04b6a0216
Reviewed-on: https://chromium-review.googlesource.com/802092
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#521525}
which is a more logical and intuitive place than Rebaseline.
Besides, we add support for removing redundant all-PASS testharness.js
baselines that are not at the root. This may happen when a platfrom
result becomes all-PASS, when its fallback platforms are already
all-PASS, in which case we only download a new all-PASS baseline to the
platform. The previous optimization only looked at the root.
Bug: 768525
Change-Id: Icf4fa98850f5a72541a20d54217a64daef8cb389
Reviewed-on: https://chromium-review.googlesource.com/803795
Commit-Queue: Robert Ma <robertma@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#521173}
If --additional-driver-flag is specified, it was weird to still let
--reset-results reset the existing baselines. There seems no use case
for that.
Now let --reset-results reset flag-specific baselines if --additional-
driver-flag is specified.
Bug: 660231
Change-Id: I6ee5a5f563a7f972fcece763256d7f99f3e97cbe
Reviewed-on: https://chromium-review.googlesource.com/740841
Reviewed-by: Chris Harrelson <chrishtr@chromium.org>
Commit-Queue: Xianzhu Wang <wangxianzhu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#512061}
--copy-baselines:
If the actual result is different from the current baseline,
copy the current baseline into the *most-specific-platform*
directory, or the flag-specific generic-platform directory if
--additional-driver-flag is specified. See --reset-results.
--reset-results:
Reset expectations to the generated results in their existing location.
If --copy-baselines is specified, the copied baselines will be reset.
--new-flag-specific-baseline:
Deprecated. Replaced by --copy-baselines --reset-results
--add-platform-exceptions:
Deprecated.
The separation of --new-flag-specific-baseline into --copy-baselines and
--reset-results (existing) is to ease code review of flag-specific
rebaseline CLs. The developer can use --copy-baselines to generate the
first patch set, then use --reset-results to generate the second patch
set. The reviewer compares the first and the second patch sets to see
the difference of the new baselines.
Bug: 660231
Change-Id: Ia91a92ddb9e260b20566e49705f551df13ccacc3
Reviewed-on: https://chromium-review.googlesource.com/713760
Commit-Queue: Xianzhu Wang <wangxianzhu@chromium.org>
Reviewed-by: Steve Kobes <skobes@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#509077}
This removes mention of NeedsRebaseline/rebaseline-o-matic, which have
now been removed.
Bug: 692811
Change-Id: I9893eef07c7380186398a89a7cfdbcd524627193
Reviewed-on: https://chromium-review.googlesource.com/584941
Reviewed-by: Jeff Carpenter <jeffcarp@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#489350}
- Update instructions for new Gerrit provisional PR flow
- Split out necessary information from additional information
- Add note that CLs with >1000 files will not be exported
- Rephrase a lot of things
Bug:
Change-Id: Ie40bdd4ca39b5efaf63fa7341d9df861d16e84dc
Reviewed-on: https://chromium-review.googlesource.com/567592
Commit-Queue: Jeff Carpenter <jeffcarp@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#486904}
Purpose: The purpose of this commit is to try to offer
clearer info/suggestions to sheriffs when import CLs
cause failures.
Change-Id: I90711d6bfe2f04ff0ec2a9d8c2f9caa92637f688
Bug: 726572
Reviewed-on: https://chromium-review.googlesource.com/516203
Reviewed-by: Matt Giuca <mgiuca@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#477522}
The new option will create new flag-specific baselines when
--additional-driver-flag is specified. For example, we can create
new baselines under LayoutTests/flag-specific/enable-slimming-paint-v2
for --additional-driver-flag=--enable-slimming-paint-v2.
Change-Id: I159f59b70538807154cd2ecd7faa324091246478
Reviewed-on: https://chromium-review.googlesource.com/522916
Commit-Queue: Xianzhu Wang <wangxianzhu@chromium.org>
Reviewed-by: Walter Korman <wkorman@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#477101}
Use --reset-results instead of --new-test-results.
Use "webkit-patch rebaseline-cl" or "--reset-results --add-platform-exceptions"
instead of --new-baseline.
Still keep "--add-platform-exceptions" to create platform-version-specific
baselines. May remove it in the future if no one wants it.
BUG=660231
Change-Id: I9671c69a806e5cba1ccd838944e757a364a947a2
Reviewed-on: https://chromium-review.googlesource.com/523386
Commit-Queue: Xianzhu Wang <wangxianzhu@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Walter Korman <wkorman@chromium.org>
Cr-Commit-Position: refs/heads/master@{#477075}
After moving testharness.js to be part of web-platform-tests its
documentation is no longer available at the old location.
BUG=None
Review-Url: https://codereview.chromium.org/2858043002
Cr-Commit-Position: refs/heads/master@{#469086}
This solves 2 problems:
1. Regenerating MANIFEST.json is failing on ToT (bug 697207)
- Now that MANIFEST.json is in .gitignore, regeneration works
2. Regenerating adds MANIFEST.json to MANIFEST.json (bug 683485)
- MANIFEST.json no longer shows up in MANIFEST.json
BUG=697207,666957,683485
Change-Id: I09987b66027e1f94055888c8ead58e7e40896d62
Reviewed-on: https://chromium-review.googlesource.com/447959
Commit-Queue: Jeff Carpenter <jeffcarp@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#454454}
Many pages of testthewebforward.org now redirect, so replace links with
something appropriate from the new documentation.
Since the new documentation consistently says "web-platform-tests"
instead of "Web Platform Tests" and this is what folks like jgraham tend
to write as well, align our own usage with that.
BUG=657117
Review-Url: https://codereview.chromium.org/2696323003
Cr-Commit-Position: refs/heads/master@{#451045}
This CL also adds a section about what kinds of changes
will be upstreamed, and the contributing guidelines for
web-platform-tests.
Review-Url: https://codereview.chromium.org/2681293004
Cr-Commit-Position: refs/heads/master@{#449798}
TestBrowserDialog landed in r440240 and is ready for wider use. Excerpt:
`TestBrowserDialog` provides a way to register an `InProcessBrowserTest`
testing harness with a framework that invokes Chrome browser dialogs in
a consistent way.
The contents are taken from the public doc on chromium.org at https://goo.gl/EFz4r2
BUG=654151
Review-Url: https://codereview.chromium.org/2684513002
Cr-Commit-Position: refs/heads/master@{#449200}
