Added the option --flag-specific=<foo>.
Currently supporting only 'highdpi' for flag-specific.
New experimental highdpi builder is added which gets triggered with this option.
Results from the new builder are fetched and rebaselined under
third_party/blink/web_tests/flag-specific/<foo>/. Other builders are ignored with this option.
Rebaselined the highdpi tests using these changes.
Bug: 1068400
Change-Id: I5638eb225398159caa06963c9ef5f3991b1a7a3b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2545863
Reviewed-by: Xianzhu Wang <wangxianzhu@chromium.org>
Reviewed-by: Stephen McGruer <smcgruer@chromium.org>
Commit-Queue: Preethi Mohan <preethim@google.com>
Cr-Commit-Position: refs/heads/master@{#859222}
commit 2389045 (Move content/shell/$process/web_test
to content/web_test/$process) has updated the
path to test_info_extractor.h
The current patch fixes the link to "protocol mode"
Bug: 1174454
Change-Id: Ib22507db10f75d0e5cdabb53aa13bd38ef52bb53
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2673426
Reviewed-by: Kentaro Hara <haraken@chromium.org>
Commit-Queue: Arnaud Mandy <arnaud.mandy@intel.com>
Cr-Commit-Position: refs/heads/master@{#850567}
This reverts commit 8da79b7118.
Reason for revert: We're looking into other methods of test selection because our filtering of source files conflicts with state sharing between test files.
Original change's description:
> Proof of concept for test selection via GN
>
> This CL implements gn based unittest filtering. The files in the rts_exclude_file are manually selected for this example, but will later be populated using the smart logic in rts-chromium.
> These two cases (source_set and test template) cover most of the test files. I haven't seen any other templates with unittests, but even if there are some this would be a great start. This also doesn't cover webtests, but I think there's enough exclusion power here to save a lot of compute.
> This also adds a linux-rts builder to the mb config to prepare the experimental builder.
>
> Bug: 1145216
> Change-Id: I7343249f3df0739e39cfe351e5714d7978059d86
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2551276
> Commit-Queue: Gregory Guterman <guterman@google.com>
> Reviewed-by: Dirk Pranke <dpranke@google.com>
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#841248}
TBR=dpranke@google.com,nodir@chromium.org,chromium-scoped@luci-project-accounts.iam.gserviceaccount.com,guterman@google.com
Bug: 1145216
Change-Id: I96ec160c5f2e08e8be7faed7c6126aff3f82c053
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2667128
Reviewed-by: Gregory Guterman <guterman@google.com>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Gregory Guterman <guterman@google.com>
Cr-Commit-Position: refs/heads/master@{#849942}
Feature flags in instrumentation tests only change when Chrome is
restarted. To allow to run tests in Batches that have methods with
@Features annotation, split tests into groups based on enabled/disabled
Features by adding a Batch.SplitByFeature annotation.
Bug: 1170735
Change-Id: Id90d704bcd949eb8630172ed1051cb3fdfc5435f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2645136
Reviewed-by: Yaron Friedman <yfriedman@chromium.org>
Reviewed-by: Michael Thiessen <mthiesse@chromium.org>
Reviewed-by: benjamin joyce <bjoyce@chromium.org>
Reviewed-by: Rohit Agarwal <roagarwal@chromium.org>
Commit-Queue: Christian Dullweber <dullweber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#848181}
This CL implements gn based unittest filtering. The files in the rts_exclude_file are manually selected for this example, but will later be populated using the smart logic in rts-chromium.
These two cases (source_set and test template) cover most of the test files. I haven't seen any other templates with unittests, but even if there are some this would be a great start. This also doesn't cover webtests, but I think there's enough exclusion power here to save a lot of compute.
This also adds a linux-rts builder to the mb config to prepare the experimental builder.
Bug: 1145216
Change-Id: I7343249f3df0739e39cfe351e5714d7978059d86
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2551276
Commit-Queue: Gregory Guterman <guterman@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Cr-Commit-Position: refs/heads/master@{#841248}
These instructions should let folks manually symbolize any Chrome
crash dumps generated during Tast tests on bots, which will make
test debugging a bit easier.
Bug: 1155738
Change-Id: I5cd9f11d53721b82fd23003d1f89c6e921a0a122
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2587583
Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#836342}
This patch lists some common causes of flakes in GTests, and links to
examples that might be helpful. Credit for the examples goes to the
security architecture team plus an example of my own.
Bug: 1127370
Change-Id: Ied695425ebab5935ef54094972fdbc1bfa9f187d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2559271
Reviewed-by: Stephen McGruer <smcgruer@chromium.org>
Commit-Queue: Nick Burris <nburris@chromium.org>
Cr-Commit-Position: refs/heads/master@{#831889}
The doc includes a number of isolate links, but items in isolate expire
after 90 days. So this updates these to a set of new isolate links.
Hopefully by the time these new links expire, we'll be on RDB which will
have a slightly different debugging flow.
This also adds some tips on running the tests locally.
Bug: None
Change-Id: Ib6244f787d2a914b7c3a79aaba2a33b688d346c7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2522409
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#824965}
Android unit tests don't work on Android R+, but this isn't mentioned
in the docs, and the error message is pretty arcane and hard to
understand. This change updates the docs to mention this problem.
Bug: 1094062
Change-Id: I77b196febde1009eb13b195be937243e96ace1a3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2495968
Commit-Queue: David Van Cleve <davidvc@chromium.org>
Auto-Submit: David Van Cleve <davidvc@chromium.org>
Reviewed-by: Nate Fischer <ntfschr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#820493}
This lays out a skeleton for the document, plus fills in some details of
how to go from a build results page to the test that flaked.
Bug: 1127369
Change-Id: Icea0ef102d694a61cf8f390c563ce108afee0553
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2405418
Commit-Queue: Stephen McGruer <smcgruer@chromium.org>
Reviewed-by: Nicolás Peña Moreno <npm@chromium.org>
Cr-Commit-Position: refs/heads/master@{#815262}
In retrospect, my prior attempt at documenting ways of
disabling tests was too ambiguous. This rewrite collapses
a couple of cases and provides clearer examples of the
various mechanisms. Hopefully this will be more useful.
Change-Id: I024ef5398c9a1fe9024e923a367a1b2ad1e23daa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2443632
Reviewed-by: Chan Li <chanli@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Cr-Commit-Position: refs/heads/master@{#814771}
In an email to a Google-internal group a while ago I talked
about the various mechanisms we potentially have for disabling
tests and their advantages and disadvantages. This seems like
something worth preserving, so I'm moving it into a doc checked
into the repo.
Change-Id: If8288eac8704354c92ffe12b9e79c930e0738c72
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2419612
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Cr-Commit-Position: refs/heads/master@{#811004}
The existing documentation for where to put any artifacts produced by a
test wasn't completely clear, so this adds some additional text to
hopefully make it so. Namely, paths should be specified relative to the
location of the results file, which normally is in $ISOLATED_OUTDIR,
meaning that the paths will also be relative to $ISOLATED_OUTDIR).
Change-Id: I7f03b9f31e7141051a4c783c383547e9dd27d286
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2426975
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Cr-Commit-Position: refs/heads/master@{#810021}
This adds an optional --isolated=outdir=PATH flag to the
test executable API; this gives the test runners a stable path
to the output directory from a swarming task, so that we can key
other files off of it without needing to pass them all explicitly
via the command line.
This also adds the arg to the isolated_script wrapper, so that we
those types of tests can handle the flag (by ignoring it for now).
We may need to work through the other cases (like gtest tests)
individually, which will be done in subsequent CLs.
Bug: 816629
Change-Id: Ibff6a47544bf94af5d7693a8446dfb669351cc07
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2358604
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Erik Staab <estaab@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#799378}
This reverts commit b679821 (change #784739) and
re-lands commit 956c4dc (change #784653) (without the addition
of //docs/testing/test_executable_api.md, which
landed separately already). Other changes landed that made
the re-land possible.
Original change's description:
> Add a first pass of the desktop test wrapper scripts.
>
> As described in the linked bug and documented in
> https://docs.google.com/document/d/19qpDeR2z-8j-WQJC-5waSd134kz94UXgYU_e_N-VplU/edit?usp=sharing
> we want to change how tests are launched under swarming so that
> they all have a consistent API and usage.
>
> We need to do at least some of this work now in order to unblock
> work going on in Swarming to change how a task's command line and
> list of files are specified, but doing all of the work in the linked
> doc and bug will significantly restructure how we configure and launch
> tests, and should bring a lot more consistency to our continuous
> integration infrastructure.
>
> This CL is a first step: a proof of concept that converts
> url_unittests on desktop to the new API in a backwards-compatible
> way (i.e., this will work with the existing chromium recipes).
TBR=tikuta@chromium.org, liaoyuke@chromium.org
Bug: 816629
Change-Id: Ia4d20501783f844fdbc1728ee986724c5301d638
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2280322
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Cr-Commit-Position: refs/heads/master@{#788424}
