0
Commit Graph

352 Commits

Author SHA1 Message Date
Ashley Newson
a0b1f81d6e Document how to fix coverage warnings for test-only files
Bug: None
Change-Id: Id76038cf631a6c9a28671c43d36cded1938807e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5621691
Commit-Queue: Ashley Newson <ashleynewson@chromium.org>
Reviewed-by: Zhaoyang Li <zhaoyangli@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1313908}
2024-06-12 10:22:54 +00:00
Weizhong Xia
55d9df527b Update document following change in crrev.com/c/5551171
Make it clear in the document that the exporter will only create a PR
after the CL is submittable.

Bug: 338231967
Change-Id: I0c0671434a6ee24a5bb89ebc69934e4ebcb15f5a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5593573
Commit-Queue: Weizhong Xia <weizhong@google.com>
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1309374}
2024-06-03 16:35:00 +00:00
Takuto Ikuta
7478af7280 docs: remove goma references
Bug: 41489832
Change-Id: I1dbbb744be749f314dea32d9cf01da0e4a9390ad
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5569801
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Richard Wang <richardwa@google.com>
Commit-Queue: Richard Wang <richardwa@google.com>
Cr-Commit-Position: refs/heads/main@{#1306295}
2024-05-27 07:23:19 +00:00
Alex Bergman
9c5fc83c80 crosier: update Crosier test metadata tags to match new tag format.
Bug: b:294097177
Change-Id: I1cf0663f1b1e16b7f2df5374dd2a81bab9e1fe5e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5547819
Reviewed-by: Sven Zheng <svenzheng@chromium.org>
Commit-Queue: Alex Bergman <abergman@google.com>
Reviewed-by: James Cook <jamescook@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1303579}
2024-05-21 03:03:53 +00:00
Alex Bergman
098d616775 crosier: add debugging section to the Crosier development guide.
Bug: b:294097177
Change-Id: Iaa65c99a25b3dce50ebeac91f8b65efaf08bd6ee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5515234
Commit-Queue: Alex Bergman <abergman@google.com>
Reviewed-by: James Cook <jamescook@chromium.org>
Reviewed-by: Sven Zheng <svenzheng@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1300993}
2024-05-15 00:08:09 +00:00
Finnur Thorarinsson
06b01566f2 [Documentation]: Make @DoNotBatch line conform to formatting.
The @DoNotBatch is likely to be copied-pasted into tests so
make it conform to the formatting standards to avoid having
and extra `git cl format` loop after fixing the test.

Bug: None
Change-Id: Id49142c27e808acfcfefffae073df1d108b4ed91
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5511384
Auto-Submit: Finnur Thorarinsson <finnur@chromium.org>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1295628}
2024-05-02 18:30:33 +00:00
Alison Gale
81f4f2c793 Migrate TODOs referencing old crbug IDs to the new issue tracker IDs
The canonical bug format is TODO(crbug.com/<id>). TODOs of the
following forms will all be migrated to the new format:

- TODO(crbug.com/<old id>)
- TODO(https://crbug.com/<old id>)
- TODO(crbug/<old id>)
- TODO(crbug/monorail/<old id>)
- TODO(<old id>)
- TODO(issues.chromium.org/<old id>)
- TODO(https://issues.chromium.org/<old id>)
- TODO(https://issues.chromium.org/u/1/issues/<old id>)
- TODO(bugs.chromium.org/<old id>)

Bug id mapping is sourced from go/chrome-on-buganizer-prod-issues.
See go/crbug-todo-migration for details.

#crbug-todo-migration

Bug: b/321899722
Change-Id: Iee14d10d544e9f0ec046117cc4ec8a55c427adc0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5469947
Reviewed-by: Darryl James <dljames@chromium.org>
Owners-Override: Alison Gale <agale@chromium.org>
Commit-Queue: Alison Gale <agale@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1290838}
2024-04-22 19:33:31 +00:00
Alex Bergman
13fb47b224 crosier: add Crosier test metadata documentation.
Bug: b:294097177
Change-Id: Ib11aa0677001a44f5bba3ac7a518d85ef68d7a07
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5447172
Reviewed-by: Sven Zheng <svenzheng@chromium.org>
Reviewed-by: James Cook <jamescook@chromium.org>
Commit-Queue: Sven Zheng <svenzheng@chromium.org>
Auto-Submit: Alex Bergman <abergman@google.com>
Cr-Commit-Position: refs/heads/main@{#1287796}
2024-04-16 02:05:48 +00:00
Gyuyoung Kim
11c70a26be [IOS Blink] Bump the default SDK version up to 17.4 for the web tests
As iOS Blink port started to support 17.4 version, this CL also
sets the default version of the SDK to 17.4.

Bug: 40259174
Change-Id: I61788b018458ef195db821bed1192488d4249e3c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5400862
Commit-Queue: Gyuyoung Kim <gyuyoung@igalia.com>
Reviewed-by: Dave Tapuska <dtapuska@chromium.org>
Reviewed-by: Stephen McGruer <smcgruer@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1281185}
2024-04-02 15:21:28 +00:00
Alex Rudenko
ddedf8e944 Allow running inspector protocol tests using protocol logs
This CL implements a flag called `--inspector-protocol-log` that accepts
a path to a Chrome DevTools Protocol message log. If specified, the test
runner would replay the log mocking the actual browser. The purpose of
this flag is to allow reproducing test flakiness locally and it is not
meant to be used on the bots for now.

Bug: 327140253
Change-Id: I871442d568878b3a1a0d71a18eb2eb721b457e76
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5331573
Reviewed-by: danakj <danakj@chromium.org>
Commit-Queue: Alex Rudenko <alexrudenko@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1274969}
2024-03-19 16:16:58 +00:00
Victor Vianna
d7fe65c4aa [docs] Fix docs on Batch.PER_CLASS
Docs-only change. See here
https://source.chromium.org/chromium/chromium/src/+/main:base/test/android/javatests/src/org/chromium/base/test/util/Batch.java;l=39;drc=e4622aaeccea84652488d1822c28c78b7115684f

Change-Id: I1832b8899d8453a3c08d2a6eef31ea23c88d9689
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5366924
Reviewed-by: Henrique Nakashima <hnakashima@chromium.org>
Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
Auto-Submit: Victor Vianna <victorvianna@google.com>
Cr-Commit-Position: refs/heads/main@{#1271705}
2024-03-12 18:32:21 +00:00
An Sung
f33f852294 [Blink] Register and update expectation for mac14/arm64 builders
There are some flaky tests that might be still failing.
The added rebaseline/expectations should improve the result.

Bug: 327500692
Change-Id: I6f91b42c32948faea9a0a0162355b3c7ca7ce00b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5332802
Commit-Queue: An Sung <ansung@google.com>
Reviewed-by: Weizhong Xia <weizhong@google.com>
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Reviewed-by: Zequan Wu <zequanwu@google.com>
Cr-Commit-Position: refs/heads/main@{#1270325}
2024-03-08 19:12:33 +00:00
Alex Rudenko
16b8ecf184 Update web_tests docs
Change-Id: I6065bfc40691995c463b6e30bc4b4b95ea174e63
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5321533
Commit-Queue: Alex Rudenko <alexrudenko@chromium.org>
Auto-Submit: Alex Rudenko <alexrudenko@chromium.org>
Commit-Queue: Philip Pfaffe <pfaffe@chromium.org>
Reviewed-by: Philip Pfaffe <pfaffe@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1265704}
2024-02-27 10:23:15 +00:00
Weizhong Xia
aabc89f5d0 Document that tests requires internal APIs should be skipped on Chrome
Bug: 1474702
Change-Id: If7a161ad647aa87306b3c53368cbd0610f81913f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5233002
Commit-Queue: Weizhong Xia <weizhong@google.com>
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1265048}
2024-02-26 05:05:15 +00:00
Joel Hockey
f5f032d434 Update instructions for symbolizing crash with local build
Change-Id: I68083d83c4d7dd94a9277214462763cb815711a2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5298134
Commit-Queue: Joel Hockey <joelhockey@chromium.org>
Auto-Submit: Joel Hockey <joelhockey@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1261410}
2024-02-16 00:03:39 +00:00
Jeff Yoon
f75905058e [buganizer] Migrate remaining DIR_METADATA in chromium/src
Migrate all remaining DIR_METADATA in chromium/src.

Verify components with http://b/components/{id} or at
go/chrome-on-buganizer-prod-components.

Bug: chromium:1518875
Change-Id: I025c51f2bbe35f95fbb62f0d308f1ff1d0b438c8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5288611
Reviewed-by: Henrique Nakashima <hnakashima@chromium.org>
Reviewed-by: Avi Drissman <avi@chromium.org>
Reviewed-by: Arthur Eubanks <aeubanks@google.com>
Commit-Queue: Jeff Yoon <jeffyoon@google.com>
Cr-Commit-Position: refs/heads/main@{#1259938}
2024-02-13 18:38:14 +00:00
Weizhong Xia
a53c19f563 De-associate the headful mode with debugging
As we are planning to run the headful mode on CI to get performance
data, running in headful mode does not necessary mean debugging.

Add '--no-xvfb' to control if we do not want to start xvfb.

Bug: 1517496
Change-Id: Ia00b2ee0de380c2c8a835d9bce5c9eb71993f57b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5252105
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Commit-Queue: Weizhong Xia <weizhong@google.com>
Cr-Commit-Position: refs/heads/main@{#1257627}
2024-02-07 22:47:33 +00:00
Jonathan Lee
61fc72853a [wpt-importer] File bugs against Buganizer, not Monorail
Monorail is read-only now. Activate the Buganizer code path, update the
WPT notification instructions, and add `buganizer_public.component_id`
to `external/DIR_METADATA` to collect all bugs for now. According to the
FAQ [0], the remaining `DIR_METADATA` will receive Buganizer component
IDs "soon after migration".

[0]: go/chrome-on-buganizer-faq

Bug: b/40257571
Change-Id: I9b0b89be29964ec46c7e09d7d39eadbf5ee23030
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5270234
Reviewed-by: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1256359}
2024-02-05 19:55:49 +00:00
Prakhar
28ffd9b124 remove wrong info from coverage doc
Change-Id: I9ba39e287c30f2c7ec3c0f02ce77089276fbd0f2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5254964
Commit-Queue: Zhaoyang Li <zhaoyangli@chromium.org>
Reviewed-by: Zhaoyang Li <zhaoyangli@chromium.org>
Auto-Submit: Prakhar Asthana <pasthana@google.com>
Cr-Commit-Position: refs/heads/main@{#1255164}
2024-02-01 18:49:12 +00:00
John Abd-El-Malek
33524c2a68 Update Tast test note
Change-Id: I0f120c9c89d64da90fc737fa293f5d73426e0d00
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5246128
Commit-Queue: Rick Byers <rbyers@chromium.org>
Auto-Submit: John Abd-El-Malek <jam@chromium.org>
Reviewed-by: Rick Byers <rbyers@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1253438}
2024-01-29 19:41:23 +00:00
Jonathan Lee
9b39e4225a [wptrunner][docs] Add --headless discrepancy to known issues list
Bug: 1485918
Change-Id: I243f97c1b4090e85309958668c651ad54809dfd1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5239328
Reviewed-by: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1252445}
2024-01-26 06:39:40 +00:00
Prakhar
d4ac7f640c Improve Low Coverage Reason documentation
Bug: 1517896
Change-Id: I6e2d848e29c578b8265d68161e0be9dc2d584f47
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5201852
Auto-Submit: Prakhar Asthana <pasthana@google.com>
Commit-Queue: Zhaoyang Li <zhaoyangli@chromium.org>
Reviewed-by: Zhaoyang Li <zhaoyangli@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1247733}
2024-01-16 21:19:12 +00:00
Alan Zhao
f8fa313044 Reland "Enable continuous mode/runtime counter relocation for coverage builds"
This is a reland of commit 0265879ffa

The original change was reverted because this enabled the continuous
mode on PGO bots, causing them to fail. The continuous mode was enabled
everywhere by the change made in the original CL to test_env.py. It is
believed that the continuous mode is incompatible with pgo builds
because PGO builds make manual calls to compiler-rt to flush the
counters. To address this, this reland wil omit the changes made to
test_env.py. Instead, continuous mode will be enabled on the bots later
by updating the recipe, which is in a separate repository.

Original change's description:
> Enable continuous mode/runtime counter relocation for coverage builds
>
> Enabling the continuous mode for coverage builds allows us to recover
> coverage data for test processes run in sandboxes while avoiding the CQ
> failures that we see with the noncontinuous mode that flushes all the
> counters to disk at process shutdown. This should also be expected to
> recover coverage data for tests that exit abnormally.
>
> Enabling the continuous mode is also the first step in landing several
> changes that will potentially improve instrumented test performance.
>
>
> Cq-Include-Trybots: luci.chromium.try:chromeos-amd64-generic-siso-rel
> Cq-Include-Trybots: luci.chromium.try:ios-simulator-siso
> Cq-Include-Trybots: luci.chromium.try:linux_chromium_asan_siso_rel_ng
> Cq-Include-Trybots: luci.chromium.try:mac-siso-rel
> Change-Id: Iab1177b2e8f5a24887a33417d1556c90f1002363
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4953786
> Reviewed-by: Junji Watanabe <jwata@google.com>
> Reviewed-by: danakj <danakj@chromium.org>
> Reviewed-by: Ben Pastene <bpastene@chromium.org>
> Reviewed-by: Hans Wennborg <hans@chromium.org>
> Commit-Queue: Alan Zhao <ayzhao@google.com>
> Reviewed-by: Dominic Farolino <dom@chromium.org>
> Reviewed-by: Prakhar Asthana <pasthana@google.com>
> Cr-Commit-Position: refs/heads/main@{#1235959}

Bug: 1462187,1468343,1510916
Change-Id: I8a065f0a5886a53f5509491796501dd5b288c99a
Cq-Include-Trybots: luci.chromium.try:chromeos-amd64-generic-siso-rel
Cq-Include-Trybots: luci.chromium.try:ios-simulator-siso
Cq-Include-Trybots: luci.chromium.try:linux_chromium_asan_siso_rel_ng
Cq-Include-Trybots: luci.chromium.try:mac-siso-rel
Cq-Include-Trybots: chrome/try:linux-pgo,mac-pgo,win64-pgo
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5122344
Commit-Queue: Prakhar Asthana <pasthana@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Prakhar Asthana <pasthana@google.com>
Reviewed-by: Dominic Farolino <dom@chromium.org>
Reviewed-by: danakj <danakj@chromium.org>
Reviewed-by: Hans Wennborg <hans@chromium.org>
Commit-Queue: Alan Zhao <ayzhao@google.com>
Cr-Commit-Position: refs/heads/main@{#1245683}
2024-01-11 03:35:39 +00:00
Jonathan Lee
a89106ce40 [wptrunner] Warn that non-Linux platforms are not yet supported
Both in `//docs/testing/` and at runtime.

Bug: 1512219
Change-Id: Ib6acb38054e21c2a0b8739c4de710e766ba5a6c2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5181725
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Reviewed-by: Weizhong Xia <weizhong@google.com>
Cr-Commit-Position: refs/heads/main@{#1244995}
2024-01-10 00:44:31 +00:00
Gaston Rodriguez Lopez
b36aa7c179 Fix typo in webtests doc.
Found a small typo in the markdown document for web tests.

Change-Id: I1fac6b24514104a7053586c2e0fcce20adfddaab
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5178611
Reviewed-by: Yaroslav Shalivskyy <yshalivskyy@microsoft.com>
Commit-Queue: Gaston Rodriguez <gastonr@microsoft.com>
Cr-Commit-Position: refs/heads/main@{#1244330}
2024-01-08 23:02:48 +00:00
Brian Sheedy
3e2d85a95b Add expectation file documentation
Adds documentation on the expectation file format used by GPU and Blink
tests. Some aspects of the format, particularly various edge cases,
do not seem to currently have documentation anywhere.

Bug: 1499971
Change-Id: I0942e3afd8b935bf813cfb4acb9fb162aa49191d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5014876
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1244250}
2024-01-08 20:51:50 +00:00
Andrew Grieve
a971215fa4 android_instrumentation_tests.md: Delete obsolete info
* Removes mention of JUnit 3 and non-androidx size annotations
* Makes more heading appear in the table of contents.

Change-Id: I9d296ff81da57166accc474d05f587a64d62ad27
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5115110
Reviewed-by: Haiyang Pan <hypan@google.com>
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Auto-Submit: Andrew Grieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1236486}
2023-12-12 19:22:49 +00:00
Weizhong Xia
b731e2728f Rename linux-wpt-fyi-rel
Rename this to linux-wpt-chromium-rel. And next step is to move this to
main waterfall.

Bug: 1339410, 1382931
Cq-Include-Trybots: luci.chromium.try:linux-wpt-chromium-rel
Change-Id: Iadcaa7d6b77371ef65e181ce7d96af2e180572a8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5068651
Commit-Queue: Weizhong Xia <weizhong@google.com>
Auto-Submit: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1230316}
2023-11-28 23:56:18 +00:00
Jonathan Lee
80280d27ae [docs][web-tests] Update web test expectation documentation
1. Describe `ChromeTestExpectations`.
2. Update outdated information about flag-specific rebaselining.
3. Remove outdated information about `WontFix`, `Rebaseline`, and
   `Missing`; no expectation files use these anymore.
4. Remove references to the old flakiness dashboard.
5. Describe the virtual -> base TestExpectations fallback behavior.
6. Repair some dead links.

Bug: 1474702, 1429690
Change-Id: Id25d92a0200954f586edf11e2932675a53df0b1f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5063123
Reviewed-by: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1229574}
2023-11-27 22:40:56 +00:00
Weizhong Xia
c994d2c766 Fix broken document links
Due to the remover of web_platform_tests_wptrunner.md, some links are
broken now. Fix those links with a new one.

Bug: 1474702
Change-Id: Ic622d409bc014c5925f5612897c4bd0d82c3322b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5037915
Auto-Submit: Weizhong Xia <weizhong@google.com>
Commit-Queue: Weizhong Xia <weizhong@google.com>
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1226140}
2023-11-17 16:13:13 +00:00
Weizhong Xia
08b2af1a06 Move Chrome specific test expectations to a separate file
This allows us to easily override test expectations from
TestExpectations. Otherwise we will have to split tags to
avoid conflicts.

Bug: 1499775
Cq-Include-Trybots: luci.chromium.try:linux-wpt-fyi-rel
Change-Id: Ib3c559a6b3b95b72178b8947c1749394fb963b90
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5021829
Commit-Queue: Weizhong Xia <weizhong@google.com>
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1224078}
2023-11-14 04:09:00 +00:00
Jonathan Lee
f07495eff6 [infra][wptrunner] Set up final Chrome on Linux try builder (1/n)
No-op renames of `webdriver_{tests_suite -> wpt_tests}` and
`wpt_tests_suite` (for Chrome) to `chrome_wpt_tests`.

All WPT suite names now follow the pattern `*_wpt_tests`. Also, `wpt` in
`webdriver_wpt_tests` distinguishes that suite from chromedriver's
non-WPT suites.

Bug: 1474702
Change-Id: Ic075372693bf3c270c4fb00c17b2905c3f1cb932
Cq-Include-Trybots: luci.chromium.try:linux-blink-rel
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5014293
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Jonathan Lee <jonathanjlee@google.com>
Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Weizhong Xia <weizhong@google.com>
Cr-Commit-Position: refs/heads/main@{#1221982}
2023-11-09 00:21:39 +00:00
dpapad
3eef948b98 Docs: Be explicit about how to tell covered/non-covered lines in Gerrit.
The previous wording relied too much on the screenshot and did not point
out exactly what the user should look for (background color of line
number). Spelling it out to save future readers time.

Bug: None
Change-Id: Id447caae7116124f2ac5a31fb5346afa6a07f7c9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5013947
Reviewed-by: Tibor Goldschwendt <tiborg@chromium.org>
Commit-Queue: Demetrios Papadopoulos <dpapad@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1221797}
2023-11-08 19:13:41 +00:00
Weizhong Xia
c326c31797 Document how to run and rebaseline webdriver tests
It is agreed that wpt metadata are too complicated to handle, and we
have decided to continue use test expectations and baselines for
run_wpt_tests.py, and convert that back to metadata when running
the test. Update the document to explain how to run and rebaseline
webdriver tests.

Bug: 1474702
Change-Id: I46a9077800c55d7a8cf6ad208b6886b7799ce6db
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4998687
Commit-Queue: Weizhong Xia <weizhong@google.com>
Reviewed-by: Thiago Perrotta <tperrotta@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1219149}
2023-11-02 22:10:42 +00:00
Weizhong Xia
be863d725a Update document for run_wpt_tests.py
Update document for run_wpt_tests.py and make chrome the default
target.

Bug: 1474702
Change-Id: I17c050d14a1385c150cb06b11e43bc66049a16d7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4995989
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Weizhong Xia <weizhong@google.com>
Cr-Commit-Position: refs/heads/main@{#1218505}
2023-11-01 22:36:15 +00:00
Rebekah Potter
6f5c0035c5 WebUI: Remove outdated test documentation
This test file is outdated and has been replaced by the up to date
docs/testing_webui.md

Bug: 1457360
Change-Id: Iac6b6501f62238796acc69126b5350129928e2f9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4994753
Reviewed-by: Demetrios Papadopoulos <dpapad@chromium.org>
Commit-Queue: Rebekah Potter <rbpotter@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1217893}
2023-10-31 21:44:40 +00:00
Keren Zhu
1f7b99dab8 Add interactive_ui_tests/BrowserInteractiveUiTest.Invoke
This new test is the equivalent of browser_tests/BrowserUiTest.Invoke
for interactive_ui_tests. It is useful for running interactive_ui_tests
interactively locally.

Bug: None
Change-Id: I1488a4d72414f114d3b9b3e8aa8500d0715a3eae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4909917
Code-Coverage: findit-for-me@appspot.gserviceaccount.com <findit-for-me@appspot.gserviceaccount.com>
Reviewed-by: Peter Kasting <pkasting@chromium.org>
Reviewed-by: Stefan Kuhne <skuhne@chromium.org>
Commit-Queue: Keren Zhu <kerenzhu@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1205558}
2023-10-04 23:47:32 +00:00
Andrew Grieve
b5b3bb3d3e Consolidate Android testing docs /testing/android/docs -> /docs/testing
This should hopefully make them more discoverable.

Bug: None
Change-Id: I65885c50b655a8b867cdc9224bd1be59822dcb14
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4908056
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Owners-Override: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Haiyang Pan <hypan@google.com>
Reviewed-by: James Shen <zhiyuans@google.com>
Reviewed-by: Henrique Nakashima <hnakashima@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1205217}
2023-10-04 13:55:51 +00:00
nihardamar
3afa556646 add instructions for buganizer components
Give instructions for owners to add their individual buganizer
components to be automatically notified by the notifier.

link to proto:
[`dir_metadata` proto](https://source.chromium.org/chromium/infra/infra/+/HEAD:go/src/infra/tools/dirmd/proto/dir_metadata.proto)

Bug: 1418166
Change-Id: I61c26b7b9cdffefe672e4c9882290a045e28b56c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4881260
Reviewed-by: Weizhong Xia <weizhong@google.com>
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Commit-Queue: Nihar Damarasingu <nihardamar@google.com>
Cr-Commit-Position: refs/heads/main@{#1202857}
2023-09-28 21:41:21 +00:00
Jonathan Lee
e95877fe72 [wptrunner] Add run_wpt_tests.py support to debug_renderer
Add support for interactively debugging WPTs with upstream's `wpt run`.
`run_wpt_tests.py --verbose` ensures browser logs are dumped to stdout
so that `debug_renderer` can find the PID to attach to.

Bug: 1440021
Test: ./debug_renderer ./run_wpt_tests.py -t Default
Test: external/wpt/badging/badge-success.https.html
Change-Id: I58323aea67f6232bc28b198b8f18d3d7bb7d8a7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4893148
Reviewed-by: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Reviewed-by: Robert Flack <flackr@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1201810}
2023-09-26 21:16:30 +00:00
Prakhar
c7ae92e08e Update documentation on blocking low coverage CLs.
Bug: 1412897
Change-Id: I3d22e56bb8706cb06a446a5b40ae1a7c054a7980
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4875201
Auto-Submit: Prakhar Asthana <pasthana@google.com>
Reviewed-by: Zhaoyang Li <zhaoyangli@chromium.org>
Commit-Queue: Zhaoyang Li <zhaoyangli@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1198889}
2023-09-20 09:34:52 +00:00
Jonathan Lee
5241efd87f [wptrunner][docs] Document run_wpt_tests.py --reset-results
Also, summarize the behavior of `--reset-results` (crrev.com/c/4860008)
in `run_wpt_tests.py --help`.

Fixed: 1480061
Bug: 1479953
Change-Id: I25308211cbe1fa610abcacf4e8df6d1daacd5cff
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4867136
Reviewed-by: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1196862}
2023-09-14 22:36:36 +00:00
Struan Shrimpton
ecbd92bd62 [infra] Remove RTS from builder configs and mb
Also remove some docs. The last changes will be the gn which requires
different owners so those will be in separate CLs

Bug: 1469110
Change-Id: Ifa255da0742ff543e289b481af7c80f8f7dd3103
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4857198
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Cr-Commit-Position: refs/heads/main@{#1195423}
2023-09-12 15:46:27 +00:00
Henrique Nakashima
b34188674b [Android] Update batching_instrumentation_tests.md
Fix a couple of nits and add more information.

Change-Id: I6d32231cff8a29759bd691ae1ecc109ed324e457
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4854299
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1194956}
2023-09-11 18:26:07 +00:00
Weizhong Xia
711d3e6001 Use chrome_wpt_tests as build target for chrome
Renamed chrome_wpt to chrome_wpt_tests to be consistent, and
removed webdriver_wpt_tests which is essentially running web
driver tests only with chrome.

Updated configurations and documents as necessary.

Bug: 1474702
Change-Id: If362c39c5929b00658a2a2dbbf0d7d3089e0633a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4847474
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Reviewed-by: Vladimir Nechaev <nechaev@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Haiyang Pan <hypan@google.com>
Commit-Queue: Weizhong Xia <weizhong@google.com>
Cr-Commit-Position: refs/heads/main@{#1194942}
2023-09-11 18:04:35 +00:00
Hayato Ito
33c9a1f7a5 Use the standard-compliant forbidden character '^' as an invalid URL
Currently, in the rewriting rule of the test runners, "~NOTFOUND" is
used as an invalid URL. However, '~' is NOT a forbidden character
according to the URL standard.

Chromium is addressing [1] this non-compliant behavior and will permit
'~' character in URL hosts. Thus, we can no longer use "~NOTFOUND" as
it will become a valid URL.

As an alternative, any invalid character is acceptable. Let's use '^'
because '^' is a forbidden character both before and after the CL [1].

The PR for WPT Chrome runner is:
https://github.com/web-platform-tests/wpt/pull/41840/

[1]: https://crrev.com/c/4823237

Bug: 1416013
Change-Id: Id8767ab49b54e778da53113aa4995e098bdbee59
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4845589
Reviewed-by: Yoav Weiss <yoavweiss@chromium.org>
Commit-Queue: Hayato Ito <hayato@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Reviewed-by: Adam Rice <ricea@chromium.org>
Reviewed-by: Chidera Olibie <colibie@google.com>
Reviewed-by: danakj <danakj@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1194658}
2023-09-11 02:38:41 +00:00
Andrew Grieve
43a5cf8508 Tweak @Batch PRESUBMIT message and docs
* Makes them a bit more concise.
* Mentions @DoNotBatch in the markdown
* Mentions batching as separate changes in presubmit warning.

Bug: None
Change-Id: I4e12a7530d7f7e56d3fd1b0e0af2ecc27efe77ee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4851187
Reviewed-by: Michael Thiessen <mthiesse@chromium.org>
Reviewed-by: Nate Fischer <ntfschr@chromium.org>
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1194114}
2023-09-08 15:09:46 +00:00
Weizhong Xia
ed62b32ffa Create Wpt builder for mac11 arm64
This is needed for creating metadata files for mac11 arm64.

Resource request is approved in crbug.com/1382931

Bug: 1474702, 1382931
Change-Id: Iaf10cca025be476ceaa552ecc2834311b37eef92
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4836655
Commit-Queue: Weizhong Xia <weizhong@google.com>
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Reviewed-by: Haiyang Pan <hypan@google.com>
Cr-Commit-Position: refs/heads/main@{#1192622}
2023-09-05 19:34:15 +00:00
Gyuyoung Kim
13603829cd [iOS] Add Support for Running Web Tests on iOS Blink Port
In the iOS Blink port, running web tests is essential
to assess the level of support for various web features
and functionalities. This change includes necessary
additions to enable the execution of web tests on the
iOS Blink port.

co-author: mkim@ solved the timeout issue by adding
a custom UIApplication that overrides IsRunningTests.

Bug: 1421239
Change-Id: I8835097f628280c76d1213a16a29d6cdddd19ecb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4712370
Reviewed-by: Stephen McGruer <smcgruer@chromium.org>
Reviewed-by: Dave Tapuska <dtapuska@chromium.org>
Reviewed-by: Rohit Rao <rohitrao@chromium.org>
Commit-Queue: Gyuyoung Kim <gyuyoung@igalia.com>
Reviewed-by: Peter Beverloo <peter@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1191108}
2023-09-01 01:32:34 +00:00
Jonathan Lee
ab2a97b2d3 [rebaseline-cl] Simplify fill-missing UX
Based on discussion in crbug.com/1475247, `rebaseline-cl --fill-missing`
still seems useful for downloading platform-agnostic expectations
without the round-trip of retrying infrastructure failures, but it's not
a feature users actively seek out. Therefore:
* Hide the `--{no-,}fill-missing` options with a deprecation message
  (i.e., the feature is only available for infrastructure failures).
  There's no automation that uses these options.
* Remove a redundant user prompt. It's no longer possible to continue
  with incomplete results.

Bug: 1383284
Change-Id: Icdedba8e406252041da4fa7ccaf4c6f60ef1ce8f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4824713
Reviewed-by: Weizhong Xia <weizhong@google.com>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1190222}
2023-08-30 17:57:41 +00:00