chromium/src
0
Fork 0
Code Activity
Files
5577d7fe4b233f6d14f4eae27e6e9bd6d58d3a1f
src/docs/testing/run_web_platform_tests_on_android.md
Weizhong Xia 5b6338c45b Add explanation on when/how to request resources for additional tests 
As there would be such request often, add this to document.

Bug: None
Change-Id: Ife2b0705448526b3fccd4f445325bbd3f4c1fe82
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6333678
Commit-Queue: Weizhong Xia <weizhong@google.com>
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1429243}
2025-03-06 16:24:27 -08:00

159 lines
6.6 KiB
Markdown
Raw Blame History

# Running Web Platform Tests on Android
## Overview
This document provides a guide to running Web Platform Tests on Android.
For general instruction for running the Web Platform Tests, you should read
[Running Web Platform Tests with run_wpt_tests.py](./run_web_platform_tests.md).
[TOC]
## Initial Setup
Please follow the steps at [Checking out and building Chromium for Android
](/docs/android_build_instructions.md) to
setup your local environment. Once that is done, you need to build one of the
following targets:
```bash
autoninja -C out/Default chrome_public_wpt # For testing with Chrome Android
autoninja -C out/Default trichrome_webview_wpt_64 # For testing with WebView
```
## Running the Tests
Once you have Chrome Android/WebView and `chromedriver` built, you can launch
`run_wpt_tests.py` to run WPTs. You can either run the tests on an Android
emulator or a real Android device.
Note: You can enable [Incremental Install](/build/android/incremental_install/README.md)
for Chrome Android by setting `incremental_install = True` in your `args.gn`
file.
This will cause the test harness to use the build directory's incremental
installation script instead of the default browser APK.
### Running WPTs on an Android emulator
You will need to follow the steps in
[Using an Android Emulator](/docs/android_emulator.md) to be ready to run the
Android Emulator. Passing the `--avd-config` option to `run_wpt_tests.py` will
launch an emulator that will be shut down after running the tests.
The example below runs `external/wpt/badging/badge-success.https.html` in Chrome
Android on Android 13:
```bash
third_party/blink/tools/run_wpt_tests.py \
-t Default \
-p clank \
--avd-config=tools/android/avd/proto/android_33_google_apis_x64.textpb \
external/wpt/badging/badge-success.https.html
```
* `-t Default`: Use the build in `//out/Default/`
* `-p clank`: Runs the tests using Chrome for Android (clank).
* `--avd-config=tools/.../android_33_google_apis_x64.textpb`: Runs the tests on
Android 13 emulator (Google API 33).
To run the example in WebView:
```bash
$ third_party/blink/tools/run_wpt_tests.py \
-t Default \
-p android_webview \
--webview-provider out/Default/apks/TrichromeWebView64.apk \
--additional-apk out/Default/apks/TrichromeLibrary64.apk \
--avd-config=tools/android/avd/proto/android_33_google_apis_x64.textpb \
external/wpt/badging/badge-success.https.html
```
* `-p webview`: Runs the tests using WebView.
* `--webview-provider out/.../TrichromeWebView64.apk`: Specify
TrichromeWebView64 as WebView APK.
* `--additional-apk out/.../TrichromeLibrary64.apk`: Install TrichromeLibrary64
needed for the WebView APK after Android 10 (see [WebView Channels](/android_webview/docs/channels.md)).
Alternatively, you can launch the emulator yourself and `run_wpt_tests.py` will
detect and connect to the emulator and run WPTs with it. This can save you the
time to repeatedly launch the emulator. The commands below show how this works.
```bash
$ tools/android/avd/avd.py start \
--avd-config=tools/android/avd/proto/android_33_google_apis_x64.textpb
$ third_party/blink/tools/run_wpt_tests.py \
-t Default \
-p clank \
external/wpt/badging/badge-success.https.html
```
### Running WPTs on a real Android device
`run_wpt_tests.py` should be able to work with a real device as long as the
device can be found by `adb devices`. You will need to make sure the ABI matches
and these [steps
](/docs/android_build_instructions.md#installing-and-running-chromium-on-a-device)
are followed.
### Running Tests in CQ/CI
The builder `android-chrome-13-x64-wpt-android-specific` and
`android-webview-13-x64-wpt-android-specific` builders run tests specified by
the [`android.filter`](/third_party/blink/web_tests/TestLists/android.filter)
file, which tests Android-specific behaviors. Developers can add additional
tests to the list when necessary. A resource request might be
needed depending on the case. See [below](#Run-more-tests-with-Chrome-Android-or-WebView)
for details.
To satisfy different testing requirements, WPT coverage in CQ/CI is partitioned
between suites that target different `//content` embedders:
Suite Name | Browser Under Test | Harness | Tests Run
--- | --- | --- | ---
`android_chrome_wpt_tests` | `chrome_android` | `run_wpt_tests.py` | Tests listed in [`android.filter`](#running-tests-in-cqci).
`android_webview_wpt_tests` | `android_webview` | `run_wpt_tests.py` | Tests listed in [`android.filter`](#running-tests-in-cqci).
## Test expectations and Baselines
The
[MobileTestExpectations](../../third_party/blink/web_tests/MobileTestExpectations)
file contains the list of all known Chrome Android and Chrome WebView specific
test failures, and it inherits or overrides test expectations from the default
[TestExpectations](../../third_party/blink/web_tests/TestExpectations) file.
For baselines:
* Chrome Android specific baselines reside at
`third_party/blink/web_tests/platform/android`, and fall back to
`third_party/blink/web_tests/platform/linux`.
* WebView specific baselines reside at
`third_party/blink/web_tests/platform/webview`, and fall back to
`third_party/blink/web_tests/platform/linux`.
To update baselines:
1. Trigger tryjob(s)
* For Chrome Androids: trigger `android-chrome-13-x64-wpt-android-specific`
* For WebView: trigger `android-webview-13-x64-wpt-android-specific`
2. Run [the rebaseline tool](./web_test_expectations.md#How-to-rebaseline) after
the results are ready.
## Run more tests with Chrome Android or WebView
A resource request would be needed if running the additional tests increases the
total test time by more than one minute. If this is the case, developers can use
[the bot estimator](https://data.corp.google.com/sites/datasite_browser_infra/bot_estimator?p=task_type:%22test%22)
to estimate the bots needed, then follow the steps at [go/i-need-hw](https://g3doc.corp.google.com/company/teams/chrome/ops/business/resources/resource-request-program.md?cl=head&polyglot=physical-hw#i-need-new-resources)
to get the resources. The tests can be added to the filter file after the resource
is deployed.
Please make sure there is no unexpected failures on the related steps before using
the bot estimator, as the bot will retry those tests and that will increase the
total test time. For tests not intended to run with a target, e.g. WebView, you
can skip the tests by adding "Skip" test expectations to [MobileTestExpectations](/third_party/blink/web_tests/MobileTestExpectations).
## Known Issues
For runner bugs and feature requests, please file [an issue against
`Blink>Infra`
](https://issues.chromium.org/issues/new?component=1456928&template=1923166).
View Git Blame Copy Permalink