The doc includes a number of isolate links, but items in isolate expire
after 90 days. So this updates these to a set of new isolate links.
Hopefully by the time these new links expire, we'll be on RDB which will
have a slightly different debugging flow.
This also adds some tips on running the tests locally.
Bug: None
Change-Id: Ib6244f787d2a914b7c3a79aaba2a33b688d346c7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2522409
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#824965}
Android unit tests don't work on Android R+, but this isn't mentioned
in the docs, and the error message is pretty arcane and hard to
understand. This change updates the docs to mention this problem.
Bug: 1094062
Change-Id: I77b196febde1009eb13b195be937243e96ace1a3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2495968
Commit-Queue: David Van Cleve <davidvc@chromium.org>
Auto-Submit: David Van Cleve <davidvc@chromium.org>
Reviewed-by: Nate Fischer <ntfschr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#820493}
This lays out a skeleton for the document, plus fills in some details of
how to go from a build results page to the test that flaked.
Bug: 1127369
Change-Id: Icea0ef102d694a61cf8f390c563ce108afee0553
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2405418
Commit-Queue: Stephen McGruer <smcgruer@chromium.org>
Reviewed-by: Nicolás Peña Moreno <npm@chromium.org>
Cr-Commit-Position: refs/heads/master@{#815262}
In retrospect, my prior attempt at documenting ways of
disabling tests was too ambiguous. This rewrite collapses
a couple of cases and provides clearer examples of the
various mechanisms. Hopefully this will be more useful.
Change-Id: I024ef5398c9a1fe9024e923a367a1b2ad1e23daa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2443632
Reviewed-by: Chan Li <chanli@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Cr-Commit-Position: refs/heads/master@{#814771}
In an email to a Google-internal group a while ago I talked
about the various mechanisms we potentially have for disabling
tests and their advantages and disadvantages. This seems like
something worth preserving, so I'm moving it into a doc checked
into the repo.
Change-Id: If8288eac8704354c92ffe12b9e79c930e0738c72
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2419612
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Cr-Commit-Position: refs/heads/master@{#811004}
The existing documentation for where to put any artifacts produced by a
test wasn't completely clear, so this adds some additional text to
hopefully make it so. Namely, paths should be specified relative to the
location of the results file, which normally is in $ISOLATED_OUTDIR,
meaning that the paths will also be relative to $ISOLATED_OUTDIR).
Change-Id: I7f03b9f31e7141051a4c783c383547e9dd27d286
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2426975
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Cr-Commit-Position: refs/heads/master@{#810021}
This adds an optional --isolated=outdir=PATH flag to the
test executable API; this gives the test runners a stable path
to the output directory from a swarming task, so that we can key
other files off of it without needing to pass them all explicitly
via the command line.
This also adds the arg to the isolated_script wrapper, so that we
those types of tests can handle the flag (by ignoring it for now).
We may need to work through the other cases (like gtest tests)
individually, which will be done in subsequent CLs.
Bug: 816629
Change-Id: Ibff6a47544bf94af5d7693a8446dfb669351cc07
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2358604
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Erik Staab <estaab@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#799378}
This reverts commit b679821 (change #784739) and
re-lands commit 956c4dc (change #784653) (without the addition
of //docs/testing/test_executable_api.md, which
landed separately already). Other changes landed that made
the re-land possible.
Original change's description:
> Add a first pass of the desktop test wrapper scripts.
>
> As described in the linked bug and documented in
> https://docs.google.com/document/d/19qpDeR2z-8j-WQJC-5waSd134kz94UXgYU_e_N-VplU/edit?usp=sharing
> we want to change how tests are launched under swarming so that
> they all have a consistent API and usage.
>
> We need to do at least some of this work now in order to unblock
> work going on in Swarming to change how a task's command line and
> list of files are specified, but doing all of the work in the linked
> doc and bug will significantly restructure how we configure and launch
> tests, and should bring a lot more consistency to our continuous
> integration infrastructure.
>
> This CL is a first step: a proof of concept that converts
> url_unittests on desktop to the new API in a backwards-compatible
> way (i.e., this will work with the existing chromium recipes).
TBR=tikuta@chromium.org, liaoyuke@chromium.org
Bug: 816629
Change-Id: Ia4d20501783f844fdbc1728ee986724c5301d638
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2280322
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Cr-Commit-Position: refs/heads/master@{#788424}
Contrary to crrev.com/c/2217275 it is possible to specify the directory
which the script uses to serve generated content.
Point users towards --enable-blink-features which is required for some
tests.
Change-Id: I4b9cb454dce3fabdc62c778c810342047f18b539
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2241199
Auto-Submit: Johann Koenig <johannkoenig@google.com>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#777844}
Updates:
- Guides added for adding experimental attribute to a story and when to
do so.
- How to run perf tests locally is added to the document
Bug: none
Change-Id: Ie2c3f34906d185cd3992e4a614c49f43e4c3ad4f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2210632
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Commit-Queue: Behdad Bakhshinategh <behdadb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#771156}
This followups crrev.com/c/2173656 which let virtual tests inherit
base tests' expectations except for [ Skip ]. This CL also allows
inheritance of [ Skip ].
In less common cases, we intentionally skip some base tests which is
only meaningful to run with specific flags in virtual tests. In these
cases we need to override the [ Skip ] with [ Pass ], like:
(In NeverFixTests)
# This test run under virtual suites only.
crbug.com/981970 external/wpt/fetch/http-cache/split-cache.tentative.html [ Skip ]
virtual/not-split-http-cache/external/wpt/fetch/http-cache/split-cache.tentative.html [ Pass ]
virtual/split-http-cache/external/wpt/fetch/http-cache/split-cache.tentative.html [ Pass ]
Bug: 1072015
Change-Id: Ie6c5b5472203bd81120dd2c3589a89d9df6fa45b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2178173
Commit-Queue: Xianzhu Wang <wangxianzhu@chromium.org>
Reviewed-by: Rakib Hasan <rmhasan@google.com>
Reviewed-by: Robert Ma <robertma@chromium.org>
Reviewed-by: Philip Rogers <pdr@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#766598}
This can avoid duplicated entries for virtual tests. This allows
a virtual test without specific expectations inherit the base test
expectations. If a virtual test has specific expectations, they will
override the base test expectations, except that [ Slow ] will be
merged.
This requires flag-specific expectations to use the same override rule.
Previously they always inherit the base test expectations, causing us
unable to override the expectations to e.g. [ Pass ].
Requires https://chromium-review.googlesource.com/c/catapult/+/2173206.
Bug: 1072015
Change-Id: Ie3d1911a96f70799a996b6abbd78f9a5107d7eff
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2173656
Commit-Queue: Xianzhu Wang <wangxianzhu@chromium.org>
Reviewed-by: Rakib Hasan <rmhasan@google.com>
Reviewed-by: Robert Ma <robertma@chromium.org>
Cr-Commit-Position: refs/heads/master@{#765359}
This reverts commit 0b8c0e6a09.
Reason for revert: Fixed runtime deps
Original change's description:
> Revert "Make java_deobfuscate not require being built first"
>
> This reverts commit 962e70305e.
>
> Reason for revert: Failing on swarming
> https://ci.chromium.org/p/chromium/builders/ci/android-pie-x86-fyi-rel/747
>
> Original change's description:
> > Make java_deobfuscate not require being built first
> >
> > Makes the tool easier to use, and easier to document how to use by not
> > requiring it to be built first. We have only a single non-prebuilt .java
> > file, so it's easy enough to just check in the .class file for it.
Bug: 995290
Change-Id: Ib1f396e07d6b0737133f03f1bded1a2d312ebfc8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2106972
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Sam Maier <smaier@chromium.org>
Cr-Commit-Position: refs/heads/master@{#751322}
Makes the tool easier to use, and easier to document how to use by not
requiring it to be built first. We have only a single non-prebuilt .java
file, so it's easy enough to just check in the .class file for it.
Bug: 995290
Change-Id: I7b934b3337eccb044a470dd92270ce7338b1fd20
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2106257
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Sam Maier <smaier@chromium.org>
Cr-Commit-Position: refs/heads/master@{#750855}
This involves
* Moving non-MD-related code out of Initialize into LinearAnimation
* Moving other Initialize() code into the constructor
* Removing all Initialize() calls
* Replacing other static methods with instance methods
* Adding GetInstance() calls to callers to get the instance
* Converting the TestAPI class to a scoping object
* Other assorted #include removal etc. in the process
Bug: 940773
Change-Id: Ieb8d44eabae63b014a90ff452010eb4c067e5e90
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2090877
Commit-Queue: Peter Kasting <pkasting@chromium.org>
Reviewed-by: Scott Violet <sky@chromium.org>
Cr-Commit-Position: refs/heads/master@{#748895}
The guides for investigating the representative perf tests are added:
- Common failures listed
- accessing the traces and benchmark logs are documented
Bug: none
Change-Id: Ic087fb97ee362e04d839414f1396165f111b9934
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2087220
Commit-Queue: Behdad Bakhshinategh <behdadb@chromium.org>
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Cr-Commit-Position: refs/heads/master@{#747732}
