
The canonical bug format is TODO(crbug.com/<id>). TODOs of the following forms will all be migrated to the new format: - TODO(crbug.com/<old id>) - TODO(https://crbug.com/<old id>) - TODO(crbug/<old id>) - TODO(crbug/monorail/<old id>) - TODO(<old id>) - TODO(issues.chromium.org/<old id>) - TODO(https://issues.chromium.org/<old id>) - TODO(https://issues.chromium.org/u/1/issues/<old id>) - TODO(bugs.chromium.org/<old id>) Bug id mapping is sourced from go/chrome-on-buganizer-prod-issues. See go/crbug-todo-migration for details. #crbug-todo-migration Bug: b/321899722 Change-Id: Ib028de8bb63c99e5a81d90e24e422cf88061ad05 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5469583 Owners-Override: Alison Gale <agale@chromium.org> Reviewed-by: Darryl James <dljames@chromium.org> Commit-Queue: Alison Gale <agale@chromium.org> Cr-Commit-Position: refs/heads/main@{#1290952}
229 lines
9.7 KiB
Markdown
229 lines
9.7 KiB
Markdown
# Deploying and running gtests on Fuchsia
|
|
|
|
[TOC]
|
|
|
|
Fuchsia gtest binaries are deployed and executed via scripts that are
|
|
automatically generated by the `test()` GN target. For each test, a wrapper
|
|
scripts is created named `run_<test_target_name>` for running the Fuchsia
|
|
package on the device. These executables are found in `${OUTPUT_DIR}/bin`.
|
|
|
|
The aforementioned devices can be either emulators started by the scripts
|
|
themselves, an existing emulator instance, or a physical device. To build a
|
|
gtest binary, check this [documentation](build_instructions.md).
|
|
|
|
For the sake of this example, we will be using `base_unittests` as the package
|
|
we wish to install and/or execute.
|
|
|
|
If using anything other than the default Fuchsia boot image, see
|
|
[Supported Fuchsia Product configurations](#supported-fuchsia-product-configurations).
|
|
|
|
For details of the implementation of `run_<test_target_name>` scripts, see
|
|
[here](test_scripts.md).
|
|
|
|
## Hermetic emulation
|
|
|
|
The test script brings up an emulator, runs the tests on it, and
|
|
shuts the emulator down when finished.
|
|
```bash
|
|
$ out/fuchsia/bin/run_base_unittests
|
|
```
|
|
|
|
The flag `--custom-image` can be used to specify the Fuchsia boot image used
|
|
by the emulator. For instance, setting
|
|
`--custom-image=workstation.qemu-x64-release` would run the test on a
|
|
workstation image.
|
|
|
|
## Run on a persistent emulator
|
|
|
|
You can also run tests on physical devices or start a persistent emulator
|
|
from the Chromium tree. To start a persistent emulator, run:
|
|
|
|
```bash
|
|
$ ./build/fuchsia/test/start_emulator.py
|
|
```
|
|
|
|
Part of the output should be like the following:
|
|
|
|
```
|
|
INFO:root:Emulator successfully started. You can now run Chrome Fuchsia tests with --target-id=fuchsia-emulator-198 to target this emulator.
|
|
```
|
|
|
|
You can run tests on persistent devices by adding the command line
|
|
arguments indicated above. For example, to run `base_unittests` you
|
|
would run the following command:
|
|
|
|
```bash
|
|
$ out/fuchsia/bin/run_base_unittests --target-id fuchsia-emulator-198
|
|
```
|
|
|
|
If only one Fuchsia device is connected, you can use the following command:
|
|
|
|
```bash
|
|
$ out/fuchsia/bin/run_base_unittests -d
|
|
```
|
|
|
|
## Run on a device paved with Fuchsia built from source
|
|
|
|
Make sure that the CPU architecture of your Chromium output directory matches
|
|
the architecture of the Fuchsia output directory (x64==x64, arm64==arm64, etc.).
|
|
|
|
```bash
|
|
$ out/fuchsia/bin/run_base_unittests -d --repo [FUCHSIA_OUT_DIR]/amber-files --no-repo-init
|
|
```
|
|
|
|
Note that you should not have `fx serve` running as Chromium will handle serving
|
|
the repository.
|
|
|
|
## Run on a device the host is connected to remotely via ssh
|
|
|
|
```bash
|
|
$ out/fuchsia/bin/run_base_unittests --target-id=[::1]:8022
|
|
```
|
|
|
|
## Troubleshooting a test
|
|
|
|
To troubleshoot a specific test, consider a combination of the following
|
|
arguments to the test runner script:
|
|
|
|
* `--gtest_filter="[TestSuite.TestName]"` to only run a specific test, or a
|
|
comma-separated list to run a set of tests. Wildcards can also be used to run
|
|
a set of tests or an entire test suite, e.g. `--gtest_filter="[TestSuite.*]"`.
|
|
* `--test-launcher-jobs=1` to only have one batch of tests running at a time.
|
|
This bypasses the test launcher buffering of test log output, making it
|
|
possible to access the log output from successful test runs.
|
|
* `--single-process-tests` to run all the tests in the same process. Unlike the
|
|
above option, this will run the tests directly in the test launcher process,
|
|
making it easier to attach a debugger.
|
|
* "--logs-dir=[/path/to/log/directory]` to specify the directory to write logs
|
|
to. By default, Chromium logs are written to the "system_log" file in that
|
|
directory.
|
|
* `--gtest_repeat=[number] --gtest_break_on_failure` to run a test or test suite
|
|
a certain number of times until it fails. This is useful to investigate flaky
|
|
tests.
|
|
|
|
## Supported Fuchsia Product configurations
|
|
|
|
The prebuilt Terminal and Workstation images downloaded by `gclient sync` are
|
|
specifically configured to support running Chromium tests. It is also possible
|
|
to run Chromium tests on other Product configurations, such as Core, using a
|
|
custom build.
|
|
|
|
**Use the default unless you have a specific reason to run on a different
|
|
Product.** For example, if you need to reproduce an issue occurring on a bot
|
|
using that Product.
|
|
|
|
### Terminal
|
|
Chromium gtests are primarily supported (i.e., pass all tests) on the Terminal
|
|
Fuchsia Product configuration. This is the default image used for
|
|
[hermetic emulation](#hermetic-emulation) and when starting a
|
|
[persistent emulator](#run-on-a-persistent-emulator).
|
|
|
|
### Workstation
|
|
Workstation configurations also generally support most tests.
|
|
|
|
### Core
|
|
|
|
By default, Fuchsia Core Product configurations do **not** support running
|
|
Chromium tests. Similarly, the Core images provided with the Fuchsia SDK are
|
|
unable to run Chromium tests. (In the future, it may be possible to run them
|
|
on such images [with a corresponding TUF repo](https://crbug.com/1033794).)
|
|
|
|
When working with a local Fuchsia Core build, there are have two options for
|
|
running Chromium tests.
|
|
1. **Combined repo**: Follow the instructions in
|
|
[Run on a device paved with Fuchsia built from source](#run-on-a-device-paved-with-fuchsia-built-from-source).
|
|
2. **Packages in base**: Add the
|
|
[additional required packages](#additional-required-packages) to the
|
|
[base packages](https://fuchsia.dev/fuchsia-src/concepts/packages/package?hl=en#base-packages).
|
|
<!-- Resolving needed packages before running the tests does not work. -->
|
|
|
|
[Using method (1)](#combined-repo) with the default configuration for
|
|
`core.qemu-x64`, all `base_unittests` pass.
|
|
The same tests pass using method (2) with
|
|
[modifications](#chromium-only-repo-packages-in-base) to `core.qemu-x64`'s
|
|
configuration.
|
|
|
|
Most other test suites will require additional packages used (indirectly) by
|
|
those tests. Some of these packages may not even be in Core's default set of
|
|
[universe packages](https://fuchsia.dev/fuchsia-src/concepts/packages/package?hl=en#universe-packages)
|
|
and need to be explicitly added as described in
|
|
[Adding packages](#adding-packages).
|
|
|
|
#### Additional required packages
|
|
At a minimum, `test_manager` is required in order to run tests. This is already
|
|
available when using a combined repo as in option (1).
|
|
|
|
<!-- TODO(crbug.com/42050571): Remove this paragraph when the packages for fakes
|
|
are subpackaged with the tests.
|
|
-->
|
|
Another required package is
|
|
`//src/developer/build_info/testing:fake-build-info`, which is not available in
|
|
Core by default.
|
|
|
|
Other useful packages include:
|
|
* `intl_property_manager` - avoids many warnings when the test shard tries
|
|
to launch it.
|
|
* `audio_core` - avoids warnings related to `fuchsia.media.ProfileProvider`.
|
|
<!-- TODO(crbug.com/42050523): Replace `audio_core` with the appropriate
|
|
Package name and path here and below, respectively, when switching to
|
|
`fuchsia.scheduler.ProfileProvider`.
|
|
-->
|
|
|
|
This is sufficient to pass `base_unittests` and eliminate most warnings.
|
|
|
|
#### Adding packages
|
|
To add packages, find the path to the package's build target in the Fuchsia code
|
|
and add `--with[-base]` followed by `//path_from_fuchsia_root:target_name` to
|
|
the `fx set` command for the Fuchsia build.
|
|
|
|
##### Combined repo
|
|
|
|
When using a combined repo as in option (1), packages that are already part of
|
|
the Core configuration (including
|
|
[cached](https://fuchsia.dev/fuchsia-src/concepts/packages/package?hl=en#cached-packages)
|
|
and
|
|
[universe](https://fuchsia.dev/fuchsia-src/concepts/packages/package?hl=en#universe-packages)
|
|
packages), are available. Thus, only packages _not_ included in Core's
|
|
[base, cached, or universe packages](https://fuchsia.dev/fuchsia-src/concepts/packages/package?hl=en#types_of_packages),
|
|
need to be added. Similarly, `--with`, `--with-cache`, and `--with-base` are all
|
|
acceptable ways to add such packages.
|
|
|
|
The following makes all
|
|
[additional required packages](#additional-required-packages) available to the
|
|
Chromium tests.
|
|
```bash
|
|
$ fx set core.qemu-x64 --auto-dir --release --with //src/developer/build_info/testing:fake-build-info --with //src/testing/fidl/intl_property_manager --with //src/media/audio/audio_core
|
|
```
|
|
|
|
##### Chromium-only repo: packages in base
|
|
By default - and specifically when not using option (1) - Chromium's
|
|
[test scripts](test_scripts.md) only serve packages from the Chromium workspace,
|
|
meaning that Fuchsia provided packages not in Core's
|
|
[base packages](https://fuchsia.dev/fuchsia-src/concepts/packages/package?hl=en#base-packages)
|
|
cannot be resolved. Thus, all Fuchsia packages needed during the tests must be
|
|
added to base.
|
|
|
|
In this case `--with-base` is the only acceptable method of specifying packages,
|
|
and `test_manager` must be explicitly added to base packages by including
|
|
`--with-base //src/sys/test_manager`. The following is the minimum `fx set`
|
|
command line for this case.
|
|
```bash
|
|
$ fx set core.qemu-x64 --auto-dir --release --with-base //src/sys/test_manager
|
|
```
|
|
|
|
For the purposes of successfully running Chromium tests, the following is
|
|
equivalent to the command line in [Combined repo](#combined-repo). It adds all
|
|
[additional required packages](#additional-required-packages) to
|
|
[base packages](https://fuchsia.dev/fuchsia-src/concepts/packages/package?hl=en#base-packages).
|
|
It also adds `//sdk/lib/sys/cpp` to base packages to enable `base_unittests`'s
|
|
`TestComponentContextForProcessTest.ProvideSystemService` test to pass.
|
|
```bash
|
|
$ fx set core.qemu-x64 --auto-dir --release --with-base //src/sys/test_manager --with-base //src/developer/build_info/testing:fake-build-info --with-base //src/testing/fidl/intl_property_manager --with-base //src/media/audio/audio_core --with-base //sdk/lib/sys/cpp
|
|
```
|
|
|
|
### Other Products
|
|
Other Fuchsia Product configurations may not support running Chromium tests by
|
|
default and require some subset of the
|
|
[additional required packages](#additional-required-packages) needed for
|
|
[Core](#core) as well as others for specific test suites.
|