0

docs: replace python swarming client usage with go client

Removed run_isolated.py usage as there are no alternatives.

Bug: 984869, 1163817
Change-Id: Ieb72ebb2356d982ac5e90fde0799713daf8aa2c3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2936424
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Cr-Commit-Position: refs/heads/master@{#889080}
This commit is contained in:
Takuto Ikuta
2021-06-04 00:28:58 +00:00
committed by Chromium LUCI CQ
parent 229dbd7f10
commit 2d01a49ebf
4 changed files with 13 additions and 35 deletions

@ -368,7 +368,7 @@ machine loosely matches the architecture and OS of the bot.
The easiest way to do this is to find the ID of the swarming task and use
"swarming.py reproduce" to re-run it:
* `./src/tools/swarming_client/swarming.py reproduce -S https://chromium-swarm.appspot.com [task ID]`
* `./src/tools/luci-go/swarming reproduce -S https://chromium-swarm.appspot.com [task ID]`
The task ID can be found in the stdio for the "trigger" step for the test. For
example, look at a recent build from the [Mac Release (Intel)] bot, and
@ -405,8 +405,7 @@ Before attempting to download an isolate, you must ensure you have permission
to access the isolate server. Full instructions can be [found
here][isolate-server-credentials]. For most cases, you can simply run:
* `./src/tools/swarming_client/auth.py login
--service=https://isolateserver.appspot.com`
* `./src/tools/luci-go/isolate login`
The above link requires that you log in with your @google.com credentials. It's
not known at the present time whether this works with @chromium.org accounts.
@ -420,7 +419,7 @@ See the [Swarming documentation] for instructions on how to upload your binaries
Be sure to use the correct swarming dimensions for your desired GPU e.g. "1002:6613" instead of "AMD Radeon R7 240 (1002:6613)" which is how it appears on swarming task page. You can query bots in the chromium.tests.gpu pool to find the correct dimensions:
* `vpython tools\swarming_client\swarming.py bots -S chromium-swarm.appspot.com -d pool chromium.tests.gpu`
* `tools\luci-go\swarming bots -S chromium-swarm.appspot.com -d pool=chromium.tests.gpu`
[Swarming documentation]: https://www.chromium.org/developers/testing/isolated-testing/for-swes#TOC-Run-a-test-built-locally-on-Swarming
@ -492,11 +491,7 @@ all you need to do is make sure that your new test runs correctly via isolates.
See the documentation from the GPU bot details on [adding new isolated
tests][new-isolates] for the gn args and authentication needed to upload
isolates to the isolate server. Most likely the new test will be Telemetry
based, and included in the `telemetry_gpu_test_run` isolate. You can then
invoke it via:
* `./src/tools/swarming_client/run_isolated.py -s [HASH]
-I https://isolateserver.appspot.com -- [TEST_NAME] [TEST_ARGUMENTS]`
based, and included in the `telemetry_gpu_test_run` isolate.
[new-isolates]: gpu_testing_bot_details.md#Adding-a-new-isolated-test-to-the-bots

@ -46,7 +46,7 @@ to target the pool of Windows 10-like NVIDIA machines. (There are a few Windows
7-like NVIDIA bots in the pool, which necessitates the OS specifier.)
Details about the bots can be found on [chromium-swarm.appspot.com] and by
using `src/tools/swarming_client/swarming.py`, for example `swarming.py bots`.
using `src/tools/luci-go/swarming`, for example `swarming bots`.
If you are authenticated with @google.com credentials you will be able to make
queries of the bots and see, for example, which GPUs are available.
@ -120,11 +120,8 @@ If `cd`'d into `src/`:
1. `./tools/mb/mb.py isolate //out/Release [target name]`
* For example: `./tools/mb/mb.py isolate //out/Release angle_end2end_tests`
1. `python tools/swarming_client/isolate.py batcharchive -I https://isolateserver.appspot.com out/Release/[target name].isolated.gen.json`
* For example: `python tools/swarming_client/isolate.py batcharchive -I https://isolateserver.appspot.com out/Release/angle_end2end_tests.isolated.gen.json`
1. This will write a hash to stdout. You can run it via:
`python tools/swarming_client/run_isolated.py -I https://isolateserver.appspot.com -s [HASH] -- [any additional args for the isolate]`
1. `./tools/luci-go/isolate batcharchive -I https://isolateserver.appspot.com out/Release/[target name].isolated.gen.json`
* For example: `./tools/luci-go/isolate batcharchive -I https://isolateserver.appspot.com out/Release/angle_end2end_tests.isolated.gen.json`
See the section below on [isolate server credentials](#Isolate-server-credentials).
### Adding your new isolate to the tests that are run on the bots
@ -212,7 +209,7 @@ In the [`chromium/src`][chromium/src] workspace:
* Definitions of how bots are organized on the waterfall,
how builds are triggered, which VMs or machines are used for the
builder itself, i.e. for compilation and scheduling swarmed tasks
on GPU hardware. See
on GPU hardware. See
[README.md](https://chromium.googlesource.com/chromium/src/+/main/infra/config/README.md)
in this directory for up to date information.
@ -331,7 +328,7 @@ The process is:
GCEs from this small pool.
1. Run [`main.star`][main.star] to regenerate
`configs/chromium-swarm/bots.cfg` and `configs/gce-provider/vms.cfg`.
Double-check your work there.
Double-check your work there.
Note that previously [`vms.cfg`][vms.cfg] had to be edited manually.
Part of the difficulty was in choosing a zone. This should soon no
longer be necessary per [crbug.com/942301](http://crbug.com/942301),
@ -341,7 +338,7 @@ The process is:
[dashboard](https://viceroy.corp.google.com/chrome_infra/Quota/chrome?duration=7d).
1. Get this reviewed and landed. This step associates the VM or pool of VMs
with the bot's name on the waterfall for "builderful" bots or increases
swarmed pool capacity for "builderless" bots.
swarmed pool capacity for "builderless" bots.
Note: CR+1 is not sticky in this repo, so you'll have to ping for
re-review after every change, like rebase.
@ -369,7 +366,7 @@ Builder].
1. Work with the Chrome Infrastructure Labs team to get the (minimum 4)
physical machines added to the Swarming pool. Use
[chromium-swarm.appspot.com] or `src/tools/swarming_client/swarming.py bots`
[chromium-swarm.appspot.com] or `src/tools/luci-go/swarming bots`
to determine the PCI IDs of the GPUs in the bots. (These instructions will
need to be updated for Android bots which don't have PCI buses.)
@ -732,8 +729,7 @@ server, the swarming server, and cloud storage.
To upload and download isolates you must first authenticate to the isolate
server. From a Chromium checkout, run:
* `./src/tools/swarming_client/auth.py login
--service=https://isolateserver.appspot.com`
* `./src/tools/luci-go/isolate login`
This will open a web browser to complete the authentication flow. A @google.com
email address is required in order to properly authenticate.
@ -744,14 +740,6 @@ from a target like `gl_tests`. Then run the following:
[Running Binaries from the Bots Locally]: https://www.chromium.org/developers/testing/gpu-testing#TOC-Running-Binaries-from-the-Bots-Locally
If authentication succeeded, this will silently download a file called
`delete_me` into the current working directory. If it failed, the script will
report multiple authentication errors. In this case, use the following command
to log out and then try again:
* `./src/tools/swarming_client/auth.py logout
--service=https://isolateserver.appspot.com`
### Swarming server credentials
The swarming server uses the same `auth.py` script as the isolate server. You

@ -73,7 +73,7 @@ task locally* section on the task page. For example, to download them for
into a tmp directory and run:
```
$CHROME_DIR/tools/luci-go/isolated download -I https://chrome-isolated.appspot.com --namespace default-gzip -isolated 64919fee8b02d826df2401544a9dc0f7dfa2172d -output-dir input
python $CHROME_DIR/tools/swarming_client/swarming.py collect -S chrome-swarming.appspot.com 506a01dd12c8a610 --task-output-dir output
$CHROME_DIR/tools/luci-go/swarming collect -S chrome-swarming.appspot.com -output-dir output 506a01dd12c8a610
```
Once both isolates have been fetched you must then generate the breakpad

@ -153,14 +153,9 @@ for a platform you can't build for locally, does not yet exist.
## Authenticating
You may need to log in to `https://chromium-swarm.appspot.com` to do this
(for now you need to authenticate with python too,
TODO(https://crbug.com/984869): remove this):
```
$ tools/luci-go/isolate login
$ python tools/swarming_client/auth.py login \
--service=https://chromium-swarm.appspot.com
```
Use your google.com account for this.