The @DoNotBatch is likely to be copied-pasted into tests so
make it conform to the formatting standards to avoid having
and extra `git cl format` loop after fixing the test.
Bug: None
Change-Id: Id49142c27e808acfcfefffae073df1d108b4ed91
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5511384
Auto-Submit: Finnur Thorarinsson <finnur@chromium.org>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1295628}
The canonical bug format is TODO(crbug.com/<id>). TODOs of the
following forms will all be migrated to the new format:
- TODO(crbug.com/<old id>)
- TODO(https://crbug.com/<old id>)
- TODO(crbug/<old id>)
- TODO(crbug/monorail/<old id>)
- TODO(<old id>)
- TODO(issues.chromium.org/<old id>)
- TODO(https://issues.chromium.org/<old id>)
- TODO(https://issues.chromium.org/u/1/issues/<old id>)
- TODO(bugs.chromium.org/<old id>)
Bug id mapping is sourced from go/chrome-on-buganizer-prod-issues.
See go/crbug-todo-migration for details.
#crbug-todo-migration
Bug: b/321899722
Change-Id: Iee14d10d544e9f0ec046117cc4ec8a55c427adc0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5469947
Reviewed-by: Darryl James <dljames@chromium.org>
Owners-Override: Alison Gale <agale@chromium.org>
Commit-Queue: Alison Gale <agale@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1290838}
This CL implements a flag called `--inspector-protocol-log` that accepts
a path to a Chrome DevTools Protocol message log. If specified, the test
runner would replay the log mocking the actual browser. The purpose of
this flag is to allow reproducing test flakiness locally and it is not
meant to be used on the bots for now.
Bug: 327140253
Change-Id: I871442d568878b3a1a0d71a18eb2eb721b457e76
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5331573
Reviewed-by: danakj <danakj@chromium.org>
Commit-Queue: Alex Rudenko <alexrudenko@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1274969}
As we are planning to run the headful mode on CI to get performance
data, running in headful mode does not necessary mean debugging.
Add '--no-xvfb' to control if we do not want to start xvfb.
Bug: 1517496
Change-Id: Ia00b2ee0de380c2c8a835d9bce5c9eb71993f57b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5252105
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Commit-Queue: Weizhong Xia <weizhong@google.com>
Cr-Commit-Position: refs/heads/main@{#1257627}
Monorail is read-only now. Activate the Buganizer code path, update the
WPT notification instructions, and add `buganizer_public.component_id`
to `external/DIR_METADATA` to collect all bugs for now. According to the
FAQ [0], the remaining `DIR_METADATA` will receive Buganizer component
IDs "soon after migration".
[0]: go/chrome-on-buganizer-faq
Bug: b/40257571
Change-Id: I9b0b89be29964ec46c7e09d7d39eadbf5ee23030
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5270234
Reviewed-by: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1256359}
This is a reland of commit 0265879ffa
The original change was reverted because this enabled the continuous
mode on PGO bots, causing them to fail. The continuous mode was enabled
everywhere by the change made in the original CL to test_env.py. It is
believed that the continuous mode is incompatible with pgo builds
because PGO builds make manual calls to compiler-rt to flush the
counters. To address this, this reland wil omit the changes made to
test_env.py. Instead, continuous mode will be enabled on the bots later
by updating the recipe, which is in a separate repository.
Original change's description:
> Enable continuous mode/runtime counter relocation for coverage builds
>
> Enabling the continuous mode for coverage builds allows us to recover
> coverage data for test processes run in sandboxes while avoiding the CQ
> failures that we see with the noncontinuous mode that flushes all the
> counters to disk at process shutdown. This should also be expected to
> recover coverage data for tests that exit abnormally.
>
> Enabling the continuous mode is also the first step in landing several
> changes that will potentially improve instrumented test performance.
>
>
> Cq-Include-Trybots: luci.chromium.try:chromeos-amd64-generic-siso-rel
> Cq-Include-Trybots: luci.chromium.try:ios-simulator-siso
> Cq-Include-Trybots: luci.chromium.try:linux_chromium_asan_siso_rel_ng
> Cq-Include-Trybots: luci.chromium.try:mac-siso-rel
> Change-Id: Iab1177b2e8f5a24887a33417d1556c90f1002363
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4953786
> Reviewed-by: Junji Watanabe <jwata@google.com>
> Reviewed-by: danakj <danakj@chromium.org>
> Reviewed-by: Ben Pastene <bpastene@chromium.org>
> Reviewed-by: Hans Wennborg <hans@chromium.org>
> Commit-Queue: Alan Zhao <ayzhao@google.com>
> Reviewed-by: Dominic Farolino <dom@chromium.org>
> Reviewed-by: Prakhar Asthana <pasthana@google.com>
> Cr-Commit-Position: refs/heads/main@{#1235959}
Bug: 1462187,1468343,1510916
Change-Id: I8a065f0a5886a53f5509491796501dd5b288c99a
Cq-Include-Trybots: luci.chromium.try:chromeos-amd64-generic-siso-rel
Cq-Include-Trybots: luci.chromium.try:ios-simulator-siso
Cq-Include-Trybots: luci.chromium.try:linux_chromium_asan_siso_rel_ng
Cq-Include-Trybots: luci.chromium.try:mac-siso-rel
Cq-Include-Trybots: chrome/try:linux-pgo,mac-pgo,win64-pgo
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5122344
Commit-Queue: Prakhar Asthana <pasthana@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Prakhar Asthana <pasthana@google.com>
Reviewed-by: Dominic Farolino <dom@chromium.org>
Reviewed-by: danakj <danakj@chromium.org>
Reviewed-by: Hans Wennborg <hans@chromium.org>
Commit-Queue: Alan Zhao <ayzhao@google.com>
Cr-Commit-Position: refs/heads/main@{#1245683}
Adds documentation on the expectation file format used by GPU and Blink
tests. Some aspects of the format, particularly various edge cases,
do not seem to currently have documentation anywhere.
Bug: 1499971
Change-Id: I0942e3afd8b935bf813cfb4acb9fb162aa49191d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5014876
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1244250}
1. Describe `ChromeTestExpectations`.
2. Update outdated information about flag-specific rebaselining.
3. Remove outdated information about `WontFix`, `Rebaseline`, and
`Missing`; no expectation files use these anymore.
4. Remove references to the old flakiness dashboard.
5. Describe the virtual -> base TestExpectations fallback behavior.
6. Repair some dead links.
Bug: 1474702, 1429690
Change-Id: Id25d92a0200954f586edf11e2932675a53df0b1f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5063123
Reviewed-by: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1229574}
This allows us to easily override test expectations from
TestExpectations. Otherwise we will have to split tags to
avoid conflicts.
Bug: 1499775
Cq-Include-Trybots: luci.chromium.try:linux-wpt-fyi-rel
Change-Id: Ib3c559a6b3b95b72178b8947c1749394fb963b90
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5021829
Commit-Queue: Weizhong Xia <weizhong@google.com>
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1224078}
No-op renames of `webdriver_{tests_suite -> wpt_tests}` and
`wpt_tests_suite` (for Chrome) to `chrome_wpt_tests`.
All WPT suite names now follow the pattern `*_wpt_tests`. Also, `wpt` in
`webdriver_wpt_tests` distinguishes that suite from chromedriver's
non-WPT suites.
Bug: 1474702
Change-Id: Ic075372693bf3c270c4fb00c17b2905c3f1cb932
Cq-Include-Trybots: luci.chromium.try:linux-blink-rel
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5014293
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Jonathan Lee <jonathanjlee@google.com>
Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Weizhong Xia <weizhong@google.com>
Cr-Commit-Position: refs/heads/main@{#1221982}
The previous wording relied too much on the screenshot and did not point
out exactly what the user should look for (background color of line
number). Spelling it out to save future readers time.
Bug: None
Change-Id: Id447caae7116124f2ac5a31fb5346afa6a07f7c9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5013947
Reviewed-by: Tibor Goldschwendt <tiborg@chromium.org>
Commit-Queue: Demetrios Papadopoulos <dpapad@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1221797}
It is agreed that wpt metadata are too complicated to handle, and we
have decided to continue use test expectations and baselines for
run_wpt_tests.py, and convert that back to metadata when running
the test. Update the document to explain how to run and rebaseline
webdriver tests.
Bug: 1474702
Change-Id: I46a9077800c55d7a8cf6ad208b6886b7799ce6db
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4998687
Commit-Queue: Weizhong Xia <weizhong@google.com>
Reviewed-by: Thiago Perrotta <tperrotta@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1219149}
Add support for interactively debugging WPTs with upstream's `wpt run`.
`run_wpt_tests.py --verbose` ensures browser logs are dumped to stdout
so that `debug_renderer` can find the PID to attach to.
Bug: 1440021
Test: ./debug_renderer ./run_wpt_tests.py -t Default
Test: external/wpt/badging/badge-success.https.html
Change-Id: I58323aea67f6232bc28b198b8f18d3d7bb7d8a7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4893148
Reviewed-by: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Reviewed-by: Robert Flack <flackr@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1201810}
Also remove some docs. The last changes will be the gn which requires
different owners so those will be in separate CLs
Bug: 1469110
Change-Id: Ifa255da0742ff543e289b481af7c80f8f7dd3103
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4857198
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Cr-Commit-Position: refs/heads/main@{#1195423}
Currently, in the rewriting rule of the test runners, "~NOTFOUND" is
used as an invalid URL. However, '~' is NOT a forbidden character
according to the URL standard.
Chromium is addressing [1] this non-compliant behavior and will permit
'~' character in URL hosts. Thus, we can no longer use "~NOTFOUND" as
it will become a valid URL.
As an alternative, any invalid character is acceptable. Let's use '^'
because '^' is a forbidden character both before and after the CL [1].
The PR for WPT Chrome runner is:
https://github.com/web-platform-tests/wpt/pull/41840/
[1]: https://crrev.com/c/4823237
Bug: 1416013
Change-Id: Id8767ab49b54e778da53113aa4995e098bdbee59
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4845589
Reviewed-by: Yoav Weiss <yoavweiss@chromium.org>
Commit-Queue: Hayato Ito <hayato@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Reviewed-by: Adam Rice <ricea@chromium.org>
Reviewed-by: Chidera Olibie <colibie@google.com>
Reviewed-by: danakj <danakj@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1194658}
In the iOS Blink port, running web tests is essential
to assess the level of support for various web features
and functionalities. This change includes necessary
additions to enable the execution of web tests on the
iOS Blink port.
co-author: mkim@ solved the timeout issue by adding
a custom UIApplication that overrides IsRunningTests.
Bug: 1421239
Change-Id: I8835097f628280c76d1213a16a29d6cdddd19ecb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4712370
Reviewed-by: Stephen McGruer <smcgruer@chromium.org>
Reviewed-by: Dave Tapuska <dtapuska@chromium.org>
Reviewed-by: Rohit Rao <rohitrao@chromium.org>
Commit-Queue: Gyuyoung Kim <gyuyoung@igalia.com>
Reviewed-by: Peter Beverloo <peter@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1191108}
Based on discussion in crbug.com/1475247, `rebaseline-cl --fill-missing`
still seems useful for downloading platform-agnostic expectations
without the round-trip of retrying infrastructure failures, but it's not
a feature users actively seek out. Therefore:
* Hide the `--{no-,}fill-missing` options with a deprecation message
(i.e., the feature is only available for infrastructure failures).
There's no automation that uses these options.
* Remove a redundant user prompt. It's no longer possible to continue
with incomplete results.
Bug: 1383284
Change-Id: Icdedba8e406252041da4fa7ccaf4c6f60ef1ce8f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4824713
Reviewed-by: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1190222}