chromium/src
0
Fork 0
Code Activity

Rename webkit_layout_tests to blink_web_tests

Browse Source 
Rename test webkit_layout_tests to blink_web_tests, and update
related tools and docs.

Bug: 914213
Change-Id: I13031dee55409da093f8450db834c3c4595f579e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2040050
Commit-Queue: John Chen <johnchen@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Robert Ma <robertma@chromium.org>
Cr-Commit-Position: refs/heads/master@{#746262}
This commit is contained in:
John Chen
2020-03-03 06:35:45 +00:00
committed by Commit Bot
parent 4944fbd5c7
commit aab16fc028
29 changed files with 1031 additions and 1031 deletions

10
docs/parsing_test_results.md

@ -60,10 +60,10 @@ Most builds follow a similar pattern. The key *steps* are listed here.
suites are affected by the patch. suites are affected by the patch.
* **compile (with patch)** Builds test suites and associated artifacts. * **compile (with patch)** Builds test suites and associated artifacts.
* **isolate tests** Archives test suite binaries and artifacts. * **isolate tests** Archives test suite binaries and artifacts.
* **test_pre_run.[trigger] webkit_layout_tests (with patch)** Triggers a test * **test_pre_run.[trigger] blink_web_tests (with patch)** Triggers a test
suite on swarming [remote execution framework] -- in this case, suite on swarming [remote execution framework] -- in this case,
webkit_layout_tests. blink_web_tests.
* **webkit_layout_tests (with patch)** Collects the results from swarming for a * **blink_web_tests (with patch)** Collects the results from swarming for a
test suite. test suite.
If all test suites pass, then the *build* is marked as a success and no further If all test suites pass, then the *build* is marked as a success and no further
@ -74,9 +74,9 @@ test failure is due to the CL or due to a problem with tip of tree.
* **bot_update [without patch]** Deapplies the CL patch. * **bot_update [without patch]** Deapplies the CL patch.
* **compile [without patch]** Compiles test suites. * **compile [without patch]** Compiles test suites.
* **isolate tests (2)** Archives test suite binaries and artifacts. * **isolate tests (2)** Archives test suite binaries and artifacts.
* **test_pre_run.[trigger] webkit_layout_tests (without patch)** Triggers test * **test_pre_run.[trigger] blink_web_tests (without patch)** Triggers test
suite on swarming. Only failing tests are rerun. suite on swarming. Only failing tests are rerun.
* **webkit_layout_tests (without patch)** Collects results from swarming. * **blink_web_tests (without patch)** Collects results from swarming.
**Important safety notice**. When test suites are run with the patch applied, **Important safety notice**. When test suites are run with the patch applied,
each test is run up to N times -- any success will mark the test as a success. each test is run up to N times -- any success will mark the test as a success.

76
testing/buildbot/chromium.clang.json

@ -22022,6 +22022,25 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -22187,25 +22206,6 @@
"can_use_on_swarming_builders": true "can_use_on_swarming_builders": true
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },
@ -24204,6 +24204,25 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -24369,25 +24388,6 @@
"can_use_on_swarming_builders": true "can_use_on_swarming_builders": true
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },

48
testing/buildbot/chromium.devtools-frontend.json

@ -25,6 +25,30 @@
} }
], ],
"isolated_scripts": [ "isolated_scripts": [
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--num-retries=3", "--num-retries=3",
@ -50,30 +74,6 @@
"shards": 4 "shards": 4
}, },
"test_target": "//:devtools_web_tests" "test_target": "//:devtools_web_tests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
] ]
} }

284
testing/buildbot/chromium.fyi.json

@ -501,7 +501,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -554,7 +554,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -3382,6 +3382,31 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"cpu": "x86-64",
"os": "Windows-10-17134"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -3639,31 +3664,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"cpu": "x86-64",
"os": "Windows-10-17134"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },
@ -12021,7 +12021,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -12168,7 +12168,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -12895,7 +12895,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -13068,7 +13068,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -14618,7 +14618,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -14635,6 +14635,30 @@
}, },
"linux-blink-rel-dummy": { "linux-blink-rel-dummy": {
"isolated_scripts": [ "isolated_scripts": [
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"isolate_name": "webdriver_wpt_tests", "isolate_name": "webdriver_wpt_tests",
"merge": { "merge": {
@ -14654,30 +14678,6 @@
"shards": 4 "shards": 4
}, },
"test_target": "//:webdriver_wpt_tests" "test_target": "//:webdriver_wpt_tests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },
@ -17700,6 +17700,30 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -17847,7 +17871,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "not_site_per_process_webkit_layout_tests", "name": "not_site_per_process_blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -18000,30 +18024,6 @@
"shards": 4 "shards": 4
}, },
"test_target": "//:webdriver_wpt_tests" "test_target": "//:webdriver_wpt_tests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },
@ -19542,6 +19542,32 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3",
"--additional-env-var=LLVM_PROFILE_FILE=${ISOLATED_OUTDIR}/profraw/default-%2m.profraw"
],
"isolate_coverage_data": true,
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 20
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -19698,7 +19724,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "not_site_per_process_webkit_layout_tests", "name": "not_site_per_process_blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -19853,32 +19879,6 @@
"shards": 4 "shards": 4
}, },
"test_target": "//:webdriver_wpt_tests" "test_target": "//:webdriver_wpt_tests"
},
{
"args": [
"--num-retries=3",
"--additional-env-var=LLVM_PROFILE_FILE=${ISOLATED_OUTDIR}/profraw/default-%2m.profraw"
],
"isolate_coverage_data": true,
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 20
},
"test_target": "//:blink_web_tests"
} }
], ],
"scripts": [ "scripts": [
@ -22873,6 +22873,32 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "8086:0a2e",
"os": "Mac-10.15"
}
],
"expiration": 21600,
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -23122,32 +23148,6 @@
"expiration": 21600 "expiration": 21600
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "8086:0a2e",
"os": "Mac-10.15"
}
],
"expiration": 21600,
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },
@ -23164,7 +23164,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -23193,7 +23193,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -23222,7 +23222,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -23252,7 +23252,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -23283,7 +23283,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -23314,7 +23314,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -23388,7 +23388,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -24631,7 +24631,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,

156
testing/buildbot/chromium.linux.json

@ -2375,7 +2375,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -4086,6 +4086,32 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3",
"--additional-env-var=LLVM_PROFILE_FILE=${ISOLATED_OUTDIR}/profraw/default-%2m.profraw"
],
"isolate_coverage_data": true,
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -4242,7 +4268,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "not_site_per_process_webkit_layout_tests", "name": "not_site_per_process_blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -4380,32 +4406,6 @@
] ]
}, },
"test_target": "//:blink_web_tests" "test_target": "//:blink_web_tests"
},
{
"args": [
"--num-retries=3",
"--additional-env-var=LLVM_PROFILE_FILE=${ISOLATED_OUTDIR}/profraw/default-%2m.profraw"
],
"isolate_coverage_data": true,
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
], ],
"scripts": [ "scripts": [
@ -5884,6 +5884,31 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3",
"--debug"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 20
},
"test_target": "//:blink_web_tests"
},
{ {
"isolate_name": "content_shell_crash_test", "isolate_name": "content_shell_crash_test",
"merge": { "merge": {
@ -6057,31 +6082,6 @@
"shards": 4 "shards": 4
}, },
"test_target": "//:webdriver_wpt_tests" "test_target": "//:webdriver_wpt_tests"
},
{
"args": [
"--num-retries=3",
"--debug"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 20
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },
@ -7750,6 +7750,30 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-14.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -7897,7 +7921,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "not_site_per_process_webkit_layout_tests", "name": "not_site_per_process_blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -8050,30 +8074,6 @@
"shards": 4 "shards": 4
}, },
"test_target": "//:webdriver_wpt_tests" "test_target": "//:webdriver_wpt_tests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-14.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
], ],
"scripts": [ "scripts": [

356
testing/buildbot/chromium.mac.json

@ -1371,6 +1371,31 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "none",
"os": "Mac-10.10.5"
}
],
"shards": 25
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -1564,31 +1589,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "none",
"os": "Mac-10.10.5"
}
],
"shards": 25
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },
@ -2965,6 +2965,31 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "none",
"os": "Mac-10.11.6"
}
],
"shards": 25
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -3158,31 +3183,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "none",
"os": "Mac-10.11.6"
}
],
"shards": 25
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },
@ -4559,6 +4559,31 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "8086:0a2e",
"os": "Mac-10.12.6"
}
],
"shards": 25
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -4799,31 +4824,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "8086:0a2e",
"os": "Mac-10.12.6"
}
],
"shards": 25
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },
@ -6182,6 +6182,31 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "8086:0a2e",
"os": "Mac-10.13.6"
}
],
"shards": 25
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -6399,31 +6424,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "8086:0a2e",
"os": "Mac-10.13.6"
}
],
"shards": 25
},
"test_target": "//:blink_web_tests"
} }
], ],
"scripts": [ "scripts": [
@ -7812,6 +7812,32 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3",
"--debug"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "none",
"os": "Mac-10.13.6"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"isolate_name": "content_shell_crash_test", "isolate_name": "content_shell_crash_test",
"merge": { "merge": {
@ -7976,32 +8002,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3",
"--debug"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "none",
"os": "Mac-10.13.6"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
] ]
}, },
@ -9458,6 +9458,32 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_coverage_data": true,
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "none",
"os": "Mac-10.13.6"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -9707,32 +9733,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_coverage_data": true,
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "none",
"os": "Mac-10.13.6"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
], ],
"scripts": [ "scripts": [
@ -11123,6 +11123,31 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "none",
"os": "Mac-10.14.6"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -11360,31 +11385,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "none",
"os": "Mac-10.14.6"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
], ],
"scripts": [ "scripts": [
@ -11413,7 +11413,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,

6
testing/buildbot/chromium.memory.json

@ -14204,7 +14204,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -14237,7 +14237,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -14270,7 +14270,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,

206
testing/buildbot/chromium.win.json

@ -14,7 +14,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -3098,6 +3098,33 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3",
"--target",
"Release_x64"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"cpu": "x86-64",
"os": "Windows-10-15063"
}
],
"shards": 28
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -3311,33 +3338,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3",
"--target",
"Release_x64"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"cpu": "x86-64",
"os": "Windows-10-15063"
}
],
"shards": 28
},
"test_target": "//:blink_web_tests"
} }
], ],
"scripts": [ "scripts": [
@ -7084,6 +7084,34 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3",
"--target",
"Release_x64"
],
"isolate_coverage_data": true,
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"cpu": "x86-64",
"os": "Windows-10-15063"
}
],
"shards": 28
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -7308,34 +7336,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3",
"--target",
"Release_x64"
],
"isolate_coverage_data": true,
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"cpu": "x86-64",
"os": "Windows-10-15063"
}
],
"shards": 28
},
"test_target": "//:blink_web_tests"
} }
], ],
"scripts": [ "scripts": [
@ -8534,6 +8534,25 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -8717,25 +8736,6 @@
"can_use_on_swarming_builders": true "can_use_on_swarming_builders": true
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
], ],
"scripts": [ "scripts": [
@ -9854,6 +9854,34 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3",
"--debug",
"--time-out-ms",
"60000"
],
"experiment_percentage": 100,
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Windows-7-SP1"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"isolate_name": "content_shell_crash_test", "isolate_name": "content_shell_crash_test",
"merge": { "merge": {
@ -9988,34 +10016,6 @@
"can_use_on_swarming_builders": true "can_use_on_swarming_builders": true
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3",
"--debug",
"--time-out-ms",
"60000"
],
"experiment_percentage": 100,
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Windows-7-SP1"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
] ]
} }

48
testing/buildbot/client.devtools-frontend.integration.json

@ -25,6 +25,30 @@
} }
], ],
"isolated_scripts": [ "isolated_scripts": [
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--num-retries=3", "--num-retries=3",
@ -50,30 +74,6 @@
"shards": 4 "shards": 4
}, },
"test_target": "//:devtools_web_tests" "test_target": "//:devtools_web_tests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 12
},
"test_target": "//:blink_web_tests"
} }
] ]
} }

12
testing/buildbot/client.v8.fyi.json

@ -1312,7 +1312,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -1363,7 +1363,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -1414,7 +1414,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -1465,7 +1465,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -1516,7 +1516,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -1544,7 +1544,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,

590
testing/buildbot/test_suite_exceptions.pyl

@ -83,6 +83,300 @@
'Linux FYI GPU TSAN Release', 'Linux FYI GPU TSAN Release',
], ],
}, },
'blink_web_tests': {
'remove_from': [
'Win 7 Tests x64 (1)', # 64-bit win is unsupported by the layout tests.
'Win10 Tests x64 (dbg)',
],
'modifications': {
'Fuchsia x64': {
'args': [
'--platform=fuchsia',
],
'swarming': {
'shards': 1,
},
},
'Linux Tests': {
"args": [
"--additional-env-var=LLVM_PROFILE_FILE=${ISOLATED_OUTDIR}/profraw/default-%2m.profraw",
],
},
'Linux Tests (dbg)(1)': {
'args': [
'--debug',
],
'swarming': {
'shards': 20,
},
},
'Mac10.10 Tests': {
# TODO(kbr): this is essentially redundant.
'swarming': {
'dimension_sets': [
{
'gpu': 'none',
'os': 'Mac-10.10.5',
},
],
'shards': 25,
},
},
'Mac10.11 Tests': {
# TODO(kbr): this is essentially redundant.
'swarming': {
'dimension_sets': [
{
'gpu': 'none',
'os': 'Mac-10.11.6',
},
],
'shards': 25,
},
},
'Mac10.12 Tests': {
'swarming': {
'shards': 25,
},
},
'Mac10.13 Tests': {
'swarming': {
'dimension_sets': [
{
# Run the layout tests on a consistent set of hardware
# (Mac Minis).
'gpu': '8086:0a2e',
},
],
'shards': 25,
},
},
'Mac10.13 Tests (dbg)': {
'args': [
'--debug',
],
},
'V8 Blink Linux Debug': {
'args': [
'--debug',
],
},
'V8 Blink Linux Future': {
'args': [
'--additional-driver-flag=--js-flags=--future',
],
},
'WebKit Linux ASAN': {
'args': [
'--additional-expectations',
'../../third_party/blink/web_tests/ASANExpectations',
'--time-out-ms',
'48000',
'--enable-sanitizer',
],
'swarming': {
'shards': 20,
},
},
'WebKit Linux Leak': {
'args': [
'--additional-expectations',
'../../third_party/blink/web_tests/LeakExpectations',
'--time-out-ms',
'48000',
'--enable-leak-detection',
],
'swarming': {
'shards': 10,
},
},
'WebKit Linux MSAN': {
'args': [
'--additional-expectations',
'../../third_party/blink/web_tests/MSANExpectations',
'--time-out-ms',
'66000',
'--enable-sanitizer',
],
'swarming': {
'expiration': 36000,
'hard_timeout': 10800,
'io_timeout': 3600,
'shards': 20,
},
},
'WebKit Linux composite_after_paint Dummy Builder': {
'args': [
'--additional-driver-flag=--enable-blink-features=CompositeAfterPaint',
],
},
'WebKit Linux layout_ng_disabled Builder': {
'args': [
'--additional-driver-flag=--disable-blink-features=LayoutNG',
],
},
'Win10 Tests x64': {
'args': [
'--target',
'Release_x64',
],
'swarming': {
"shards": 28
},
},
'Win10 Tests x64 (dbg)': {
'args': [
'--debug',
'--target',
'Debug_x64',
],
},
'Win10 Tests x64 Code Coverage': {
'args': [
'--target',
'Release_x64',
],
'swarming': {
"shards": 28
},
},
'Win7 Tests (dbg)(1)': {
'args': [
'--debug',
'--time-out-ms',
'60000'
],
'experiment_percentage': 100,
# TODO(kbr): the "Windows-7-SP1" Swarming dimension set is implicit,
# from prefered_os_dimension in
# tools/build/scripts/slave/recipe_modules/swarming/api.py. Ideally,
# Swarming dimensions would be either specified for all testers on the
# waterfall (and therefore, explicitly specified for all tests in the
# generated JSON), or this would be removed, and the implicit one used.
'swarming': {
'dimension_sets': [
{
'os': 'Windows-7-SP1',
}
],
},
},
# tryserver.chromium.android
'android_blink_rel': {
'args': [
'--release',
'--android',
# TODO(crbug.com/912120): Breakpad was entirely disabled on
# Android. Remove this when it no longer affects Android crash
# log handling.
'--disable-breakpad',
# TODO(crbug.com/875172): run on the real GPU since the
# software fallback, SwiftShader, isn't yet for 32-bit ARM.
'--additional-driver-flag=--use-gpu-in-tests',
],
},
'devtools_frontend_linux_blink_light_rel': {
'args': [
'http/tests/devtools',
],
'swarming': {
'shards': 4,
},
},
'fuchsia-fyi-x64-rel': {
'args': [
'--platform=fuchsia',
],
'swarming': {
'shards': 1,
},
},
'linux-bfcache-debug': {
'args': [
'--debug',
'--additional-driver-flag=--enable-features=BackForwardCache',
'--additional-expectations=../../third_party/blink/web_tests/FlagExpectations/enable-features=BackForwardCache',
],
},
'linux-blink-animation-use-time-delta': {
'args': [
'--debug',
],
},
'linux-blink-cors-rel': {
'args': [
'--additional-driver-flag=--disable-features=OutOfBlinkCors',
'--additional-expectations=../../third_party/blink/web_tests/FlagExpectations/disable-features=OutOfBlinkCors',
],
},
'linux-blink-heap-concurrent-marking-tsan-rel': {
'args': [
'--release',
'--additional-driver-flag=--enable-features=BlinkHeapConcurrentMarking',
],
},
'linux-code-coverage': {
'args': [
'--additional-env-var=LLVM_PROFILE_FILE=${ISOLATED_OUTDIR}/profraw/default-%2m.profraw',
],
'swarming': {
'shards': 20,
},
},
'linux-layout-tests-fragment-item': {
'args': [
'--additional-driver-flag=--enable-blink-features=LayoutNGFragmentItem',
],
},
'linux-layout-tests-fragment-paint': {
'args': [
'--additional-driver-flag=--enable-blink-features=LayoutNGFragmentPaint',
],
},
'mac-rel': {
'swarming': {
'dimension_sets': [
{
# Run the layout tests on a consistent set of hardware
# (Mac Minis).
'gpu': '8086:0a2e',
},
],
'shards': 25,
},
},
'mac10.10-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
'mac10.11-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
'mac10.12-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
'mac10.13-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
'mac10.13_retina-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
'mac10.14-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
},
},
'boringssl_crypto_tests': { 'boringssl_crypto_tests': {
'remove_from': [ 'remove_from': [
# https://crbug.com/1054545 # https://crbug.com/1054545
@ -1402,7 +1696,7 @@
'android-code-coverage-native', # https://crbug.com/1018431 'android-code-coverage-native', # https://crbug.com/1018431
], ],
}, },
'not_site_per_process_webkit_layout_tests': { 'not_site_per_process_blink_web_tests': {
'remove_from': [ 'remove_from': [
# TODO(dpranke): Should we be running this step on Linux Tests (dbg)(1)? # TODO(dpranke): Should we be running this step on Linux Tests (dbg)(1)?
'Linux Tests (dbg)(1)', 'Linux Tests (dbg)(1)',
@ -2002,300 +2296,6 @@
'Win7 FYI Debug (AMD)', 'Win7 FYI Debug (AMD)',
], ],
}, },
'webkit_layout_tests': {
'remove_from': [
'Win 7 Tests x64 (1)', # 64-bit win is unsupported by the layout tests.
'Win10 Tests x64 (dbg)',
],
'modifications': {
'Fuchsia x64': {
'args': [
'--platform=fuchsia',
],
'swarming': {
'shards': 1,
},
},
'Linux Tests': {
"args": [
"--additional-env-var=LLVM_PROFILE_FILE=${ISOLATED_OUTDIR}/profraw/default-%2m.profraw",
],
},
'Linux Tests (dbg)(1)': {
'args': [
'--debug',
],
'swarming': {
'shards': 20,
},
},
'Mac10.10 Tests': {
# TODO(kbr): this is essentially redundant.
'swarming': {
'dimension_sets': [
{
'gpu': 'none',
'os': 'Mac-10.10.5',
},
],
'shards': 25,
},
},
'Mac10.11 Tests': {
# TODO(kbr): this is essentially redundant.
'swarming': {
'dimension_sets': [
{
'gpu': 'none',
'os': 'Mac-10.11.6',
},
],
'shards': 25,
},
},
'Mac10.12 Tests': {
'swarming': {
'shards': 25,
},
},
'Mac10.13 Tests': {
'swarming': {
'dimension_sets': [
{
# Run the layout tests on a consistent set of hardware
# (Mac Minis).
'gpu': '8086:0a2e',
},
],
'shards': 25,
},
},
'Mac10.13 Tests (dbg)': {
'args': [
'--debug',
],
},
'V8 Blink Linux Debug': {
'args': [
'--debug',
],
},
'V8 Blink Linux Future': {
'args': [
'--additional-driver-flag=--js-flags=--future',
],
},
'WebKit Linux ASAN': {
'args': [
'--additional-expectations',
'../../third_party/blink/web_tests/ASANExpectations',
'--time-out-ms',
'48000',
'--enable-sanitizer',
],
'swarming': {
'shards': 20,
},
},
'WebKit Linux Leak': {
'args': [
'--additional-expectations',
'../../third_party/blink/web_tests/LeakExpectations',
'--time-out-ms',
'48000',
'--enable-leak-detection',
],
'swarming': {
'shards': 10,
},
},
'WebKit Linux MSAN': {
'args': [
'--additional-expectations',
'../../third_party/blink/web_tests/MSANExpectations',
'--time-out-ms',
'66000',
'--enable-sanitizer',
],
'swarming': {
'expiration': 36000,
'hard_timeout': 10800,
'io_timeout': 3600,
'shards': 20,
},
},
'WebKit Linux composite_after_paint Dummy Builder': {
'args': [
'--additional-driver-flag=--enable-blink-features=CompositeAfterPaint',
],
},
'WebKit Linux layout_ng_disabled Builder': {
'args': [
'--additional-driver-flag=--disable-blink-features=LayoutNG',
],
},
'Win10 Tests x64': {
'args': [
'--target',
'Release_x64',
],
'swarming': {
"shards": 28
},
},
'Win10 Tests x64 (dbg)': {
'args': [
'--debug',
'--target',
'Debug_x64',
],
},
'Win10 Tests x64 Code Coverage': {
'args': [
'--target',
'Release_x64',
],
'swarming': {
"shards": 28
},
},
'Win7 Tests (dbg)(1)': {
'args': [
'--debug',
'--time-out-ms',
'60000'
],
'experiment_percentage': 100,
# TODO(kbr): the "Windows-7-SP1" Swarming dimension set is implicit,
# from prefered_os_dimension in
# tools/build/scripts/slave/recipe_modules/swarming/api.py. Ideally,
# Swarming dimensions would be either specified for all testers on the
# waterfall (and therefore, explicitly specified for all tests in the
# generated JSON), or this would be removed, and the implicit one used.
'swarming': {
'dimension_sets': [
{
'os': 'Windows-7-SP1',
}
],
},
},
# tryserver.chromium.android
'android_blink_rel': {
'args': [
'--release',
'--android',
# TODO(crbug.com/912120): Breakpad was entirely disabled on
# Android. Remove this when it no longer affects Android crash
# log handling.
'--disable-breakpad',
# TODO(crbug.com/875172): run on the real GPU since the
# software fallback, SwiftShader, isn't yet for 32-bit ARM.
'--additional-driver-flag=--use-gpu-in-tests',
],
},
'devtools_frontend_linux_blink_light_rel': {
'args': [
'http/tests/devtools',
],
'swarming': {
'shards': 4,
},
},
'fuchsia-fyi-x64-rel': {
'args': [
'--platform=fuchsia',
],
'swarming': {
'shards': 1,
},
},
'linux-bfcache-debug': {
'args': [
'--debug',
'--additional-driver-flag=--enable-features=BackForwardCache',
'--additional-expectations=../../third_party/blink/web_tests/FlagExpectations/enable-features=BackForwardCache',
],
},
'linux-blink-animation-use-time-delta': {
'args': [
'--debug',
],
},
'linux-blink-cors-rel': {
'args': [
'--additional-driver-flag=--disable-features=OutOfBlinkCors',
'--additional-expectations=../../third_party/blink/web_tests/FlagExpectations/disable-features=OutOfBlinkCors',
],
},
'linux-blink-heap-concurrent-marking-tsan-rel': {
'args': [
'--release',
'--additional-driver-flag=--enable-features=BlinkHeapConcurrentMarking',
],
},
'linux-code-coverage': {
'args': [
'--additional-env-var=LLVM_PROFILE_FILE=${ISOLATED_OUTDIR}/profraw/default-%2m.profraw',
],
'swarming': {
'shards': 20,
},
},
'linux-layout-tests-fragment-item': {
'args': [
'--additional-driver-flag=--enable-blink-features=LayoutNGFragmentItem',
],
},
'linux-layout-tests-fragment-paint': {
'args': [
'--additional-driver-flag=--enable-blink-features=LayoutNGFragmentPaint',
],
},
'mac-rel': {
'swarming': {
'dimension_sets': [
{
# Run the layout tests on a consistent set of hardware
# (Mac Minis).
'gpu': '8086:0a2e',
},
],
'shards': 25,
},
},
'mac10.10-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
'mac10.11-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
'mac10.12-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
'mac10.13-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
'mac10.13_retina-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
'mac10.14-blink-rel-dummy': {
'swarming': {
'shards': 20,
},
},
},
},
'webkit_unit_tests': { 'webkit_unit_tests': {
'remove_from': [ 'remove_from': [
# Flakily fails on Win7. https://crbug.com/943372. # Flakily fails on Win7. https://crbug.com/943372.

78
testing/buildbot/test_suites.pyl

@ -923,19 +923,7 @@
}, },
'chromium_web_tests_and_wpt_webdriver_isolated_scripts': { 'chromium_web_tests_and_wpt_webdriver_isolated_scripts': {
'webdriver_tests_suite': { 'blink_web_tests': {
'isolate_name': 'webdriver_wpt_tests',
'merge': {
'args': [
'--verbose',
],
'script': '//third_party/blink/tools/merge_web_test_results.py',
},
'swarming': {
'shards': 4,
},
},
'webkit_layout_tests': {
# layout test failures are retried 3 times when '--test-list' is not # layout test failures are retried 3 times when '--test-list' is not
# passed, but 0 times when '--test-list' is passed. We want to always # passed, but 0 times when '--test-list' is passed. We want to always
# retry 3 times, so we explicitly specify it. # retry 3 times, so we explicitly specify it.
@ -954,10 +942,22 @@
'shards': 12, 'shards': 12,
} }
}, },
'webdriver_tests_suite': {
'isolate_name': 'webdriver_wpt_tests',
'merge': {
'args': [
'--verbose',
],
'script': '//third_party/blink/tools/merge_web_test_results.py',
},
'swarming': {
'shards': 4,
},
},
}, },
'chromium_webkit_isolated_scripts': { 'chromium_webkit_isolated_scripts': {
'webkit_layout_tests': { 'blink_web_tests': {
# layout test failures are retried 3 times when '--test-list' is not # layout test failures are retried 3 times when '--test-list' is not
# passed, but 0 times when '--test-list' is passed. We want to always # passed, but 0 times when '--test-list' is passed. We want to always
# retry 3 times, so we explicitly specify it. # retry 3 times, so we explicitly specify it.
@ -1110,6 +1110,25 @@
'desktop_chromium_isolated_scripts': { 'desktop_chromium_isolated_scripts': {
'blink_python_tests': {}, 'blink_python_tests': {},
'blink_web_tests': {
# layout test failures are retried 3 times when '--test-list' is not
# passed, but 0 times when '--test-list' is passed. We want to always
# retry 3 times, so we explicitly specify it.
'args': [
'--num-retries=3',
],
'isolate_name': 'blink_web_tests',
'merge': {
'args': [
'--verbose',
],
'script': '//third_party/blink/tools/merge_web_test_results.py',
},
'results_handler': 'layout tests',
'swarming': {
'shards': 12,
}
},
'content_shell_crash_test': {}, 'content_shell_crash_test': {},
'flatbuffers_unittests': {}, 'flatbuffers_unittests': {},
'grit_python_unittests': {}, 'grit_python_unittests': {},
@ -1142,7 +1161,10 @@
'--gtest-benchmark-name=views_perftests', '--gtest-benchmark-name=views_perftests',
], ],
}, },
'webkit_layout_tests': { },
'devtools_webkit_isolated_scripts': {
'blink_web_tests': {
# layout test failures are retried 3 times when '--test-list' is not # layout test failures are retried 3 times when '--test-list' is not
# passed, but 0 times when '--test-list' is passed. We want to always # passed, but 0 times when '--test-list' is passed. We want to always
# retry 3 times, so we explicitly specify it. # retry 3 times, so we explicitly specify it.
@ -1161,9 +1183,6 @@
'shards': 12, 'shards': 12,
} }
}, },
},
'devtools_webkit_isolated_scripts': {
'webkit_layout_from_devtools': { 'webkit_layout_from_devtools': {
# layout test failures are retried 3 times when '--test-list' is not # layout test failures are retried 3 times when '--test-list' is not
# passed, but 0 times when '--test-list' is passed. We want to always # passed, but 0 times when '--test-list' is passed. We want to always
@ -1185,25 +1204,6 @@
'shards': 4, 'shards': 4,
} }
}, },
'webkit_layout_tests': {
# layout test failures are retried 3 times when '--test-list' is not
# passed, but 0 times when '--test-list' is passed. We want to always
# retry 3 times, so we explicitly specify it.
'args': [
'--num-retries=3',
],
'isolate_name': 'blink_web_tests',
'merge': {
'args': [
'--verbose',
],
'script': '//third_party/blink/tools/merge_web_test_results.py',
},
'results_handler': 'layout tests',
'swarming': {
'shards': 12,
}
},
}, },
'fieldtrial_browser_tests': { 'fieldtrial_browser_tests': {
@ -2742,8 +2742,8 @@
}, },
'linux_specific_chromium_isolated_scripts': { 'linux_specific_chromium_isolated_scripts': {
'not_site_per_process_webkit_layout_tests': { 'not_site_per_process_blink_web_tests': {
# not_site_per_process_webkit_layout_tests provides coverage for # not_site_per_process_blink_web_tests provides coverage for
# running Layout Tests without site-per-process. This is the mode used # running Layout Tests without site-per-process. This is the mode used
# on Android and Android bots currently do not run the full set of # on Android and Android bots currently do not run the full set of
# layout tests. Running in this mode on linux compensates for lack of # layout tests. Running in this mode on linux compensates for lack of

2
testing/buildbot/tryserver.chromium.android.json

@ -162,7 +162,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,

4
testing/buildbot/tryserver.chromium.linux.json

@ -18,7 +18,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -50,7 +50,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,

50
testing/buildbot/tryserver.chromium.mac.json

@ -1356,6 +1356,31 @@
}, },
"test_target": "//:blink_python_tests" "test_target": "//:blink_python_tests"
}, },
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "blink_web_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "8086:0a2e",
"os": "Mac-10.13.6"
}
],
"shards": 25
},
"test_target": "//:blink_web_tests"
},
{ {
"args": [ "args": [
"--test-type=integration" "--test-type=integration"
@ -1573,31 +1598,6 @@
] ]
}, },
"test_target": "//ui/views:views_perftests" "test_target": "//ui/views:views_perftests"
},
{
"args": [
"--num-retries=3"
],
"isolate_name": "blink_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_tests",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"gpu": "8086:0a2e",
"os": "Mac-10.13.6"
}
],
"shards": 25
},
"test_target": "//:blink_web_tests"
} }
], ],
"scripts": [ "scripts": [

68
testing/buildbot/tryserver.devtools-frontend.json

@ -3,32 +3,6 @@
"AAAAA2 See generate_buildbot_json.py to make changes": {}, "AAAAA2 See generate_buildbot_json.py to make changes": {},
"devtools_frontend_linux_blink_light_rel": { "devtools_frontend_linux_blink_light_rel": {
"isolated_scripts": [ "isolated_scripts": [
{
"args": [
"--num-retries=3",
"--layout-tests-directory=../../third_party/devtools-frontend/src/test/webtests/",
"http/tests/devtools"
],
"isolate_name": "devtools_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_from_devtools",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 4
},
"test_target": "//:devtools_web_tests"
},
{ {
"args": [ "args": [
"--num-retries=3", "--num-retries=3",
@ -41,7 +15,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -53,11 +27,7 @@
"shards": 4 "shards": 4
}, },
"test_target": "//:blink_web_tests" "test_target": "//:blink_web_tests"
} },
]
},
"devtools_frontend_linux_blink_rel": {
"isolated_scripts": [
{ {
"args": [ "args": [
"--num-retries=3", "--num-retries=3",
@ -83,7 +53,11 @@
"shards": 4 "shards": 4
}, },
"test_target": "//:devtools_web_tests" "test_target": "//:devtools_web_tests"
}, }
]
},
"devtools_frontend_linux_blink_rel": {
"isolated_scripts": [
{ {
"args": [ "args": [
"--num-retries=3" "--num-retries=3"
@ -95,7 +69,7 @@
], ],
"script": "//third_party/blink/tools/merge_web_test_results.py" "script": "//third_party/blink/tools/merge_web_test_results.py"
}, },
"name": "webkit_layout_tests", "name": "blink_web_tests",
"results_handler": "layout tests", "results_handler": "layout tests",
"swarming": { "swarming": {
"can_use_on_swarming_builders": true, "can_use_on_swarming_builders": true,
@ -107,6 +81,32 @@
"shards": 12 "shards": 12
}, },
"test_target": "//:blink_web_tests" "test_target": "//:blink_web_tests"
},
{
"args": [
"--num-retries=3",
"--layout-tests-directory=../../third_party/devtools-frontend/src/test/webtests/",
"http/tests/devtools"
],
"isolate_name": "devtools_web_tests",
"merge": {
"args": [
"--verbose"
],
"script": "//third_party/blink/tools/merge_web_test_results.py"
},
"name": "webkit_layout_from_devtools",
"results_handler": "layout tests",
"swarming": {
"can_use_on_swarming_builders": true,
"dimension_sets": [
{
"os": "Ubuntu-16.04"
}
],
"shards": 4
},
"test_target": "//:devtools_web_tests"
} }
] ]
} }

6
third_party/blink/tools/blinkpy/common/net/results_fetcher.py vendored

@ -145,11 +145,11 @@ class TestResultsFetcher(object):
data = json.loads(json_results_generator.strip_json_wrapper(data)) data = json.loads(json_results_generator.strip_json_wrapper(data))
suites = [ suites = [
entry['TestType'] for entry in data entry['TestType'] for entry in data
# Some suite names are like 'webkit_layout_tests on Intel GPU (with # Some suite names are like 'blink_web_tests on Intel GPU (with
# patch)'. Only make sure it starts with webkit_layout_tests and # patch)'. Only make sure it starts with blink_web_tests and
# runs with a patch. This should be changed eventually to use actual # runs with a patch. This should be changed eventually to use actual
# structured data from the test results server. # structured data from the test results server.
if (entry['TestType'].startswith('webkit_layout_tests') and if (entry['TestType'].startswith('blink_web_tests') and
entry['TestType'].endswith('(with patch)')) entry['TestType'].endswith('(with patch)'))
] ]
# In manual testing, I sometimes saw results where the same suite was # In manual testing, I sometimes saw results where the same suite was

2
third_party/blink/tools/blinkpy/common/net/results_fetcher_mock.py vendored

@ -40,7 +40,7 @@ class MockTestResultsFetcher(TestResultsFetcher):
self._webdriver_results = {} self._webdriver_results = {}
self.fetched_builds = [] self.fetched_builds = []
self.fetched_webdriver_builds = [] self.fetched_webdriver_builds = []
self._layout_test_step_name = 'webkit_layout_tests (with patch)' self._layout_test_step_name = 'blink_web_tests (with patch)'
def set_results(self, build, results): def set_results(self, build, results):
self._canned_results[build] = results self._canned_results[build] = results

16
third_party/blink/tools/blinkpy/common/net/results_fetcher_test.py vendored

@ -53,9 +53,9 @@ class BuilderTest(LoggingTestCase):
def test_results_url_with_build_number_step_name(self): def test_results_url_with_build_number_step_name(self):
self.assertEqual( self.assertEqual(
TestResultsFetcher().results_url('Test Builder', 10, TestResultsFetcher().results_url('Test Builder', 10,
'webkit_layout_tests (with patch)'), 'blink_web_tests (with patch)'),
'https://test-results.appspot.com/data/layout_results/Test_Builder' 'https://test-results.appspot.com/data/layout_results/Test_Builder'
'/10/webkit_layout_tests%20%28with%20patch%29/layout-test-results') '/10/blink_web_tests%20%28with%20patch%29/layout-test-results')
def test_results_url_with_non_numeric_build_number(self): def test_results_url_with_non_numeric_build_number(self):
with self.assertRaisesRegexp(AssertionError, 'expected numeric build number'): with self.assertRaisesRegexp(AssertionError, 'expected numeric build number'):
@ -87,10 +87,10 @@ class BuilderTest(LoggingTestCase):
'https://test-results.appspot.com/testfile?buildnumber=123&' 'https://test-results.appspot.com/testfile?buildnumber=123&'
'callback=ADD_RESULTS&builder=builder&name=full_results.json': 'callback=ADD_RESULTS&builder=builder&name=full_results.json':
'ADD_RESULTS(%s);' % (json.dumps( 'ADD_RESULTS(%s);' % (json.dumps(
[{"TestType": "webkit_layout_tests on Intel GPU (with patch)"}, [{"TestType": "blink_web_tests on Intel GPU (with patch)"},
{"TestType": "base_unittests (with patch)"}])), {"TestType": "base_unittests (with patch)"}])),
'https://test-results.appspot.com/data/layout_results/builder/123/' 'https://test-results.appspot.com/data/layout_results/builder/123/'
'webkit_layout_tests%20on%20Intel%20GPU%20%28with%20patch%29/' 'blink_web_tests%20on%20Intel%20GPU%20%28with%20patch%29/'
'layout-test-results/failing_results.json': 'layout-test-results/failing_results.json':
json.dumps({'passed': True}), json.dumps({'passed': True}),
}) })
@ -110,13 +110,13 @@ class BuilderTest(LoggingTestCase):
'https://test-results.appspot.com/testfile?buildnumber=5&' 'https://test-results.appspot.com/testfile?buildnumber=5&'
'callback=ADD_RESULTS&builder=foo&name=full_results.json': 'callback=ADD_RESULTS&builder=foo&name=full_results.json':
'ADD_RESULTS(%s);' % (json.dumps( 'ADD_RESULTS(%s);' % (json.dumps(
[{"TestType": "webkit_layout_tests (with patch)"}, [{"TestType": "blink_web_tests (with patch)"},
{"TestType": "not_site_per_process_webkit_layout_tests (with patch)"}, {"TestType": "not_site_per_process_blink_web_tests (with patch)"},
{"TestType": "webkit_layout_tests (retry with patch)"}, {"TestType": "blink_web_tests (retry with patch)"},
{"TestType": "base_unittests (with patch)"}])) {"TestType": "base_unittests (with patch)"}]))
}) })
step_name = fetcher.get_layout_test_step_name(Build('foo', 5)) step_name = fetcher.get_layout_test_step_name(Build('foo', 5))
self.assertEqual(step_name, 'webkit_layout_tests (with patch)') self.assertEqual(step_name, 'blink_web_tests (with patch)')
self.assertLog([]) self.assertLog([])
def test_get_step_name_without_build_number(self): def test_get_step_name_without_build_number(self):

2
third_party/blink/tools/blinkpy/tool/commands/flaky_tests.py vendored

@ -43,7 +43,7 @@ class FlakyTests(Command):
FLAKINESS_DASHBOARD_URL = ( FLAKINESS_DASHBOARD_URL = (
'https://test-results.appspot.com/dashboards/flakiness_dashboard.html' 'https://test-results.appspot.com/dashboards/flakiness_dashboard.html'
'#testType=webkit_layout_tests&tests=%s') '#testType=blink_web_tests&tests=%s')
BUG_TEMPLATE = ( BUG_TEMPLATE = (
'https://code.google.com/p/chromium/issues/entry?owner=FILL_ME_IN&status=Assigned&' 'https://code.google.com/p/chromium/issues/entry?owner=FILL_ME_IN&status=Assigned&'

6
third_party/blink/tools/blinkpy/tool/commands/rebaseline_cl_unittest.py vendored

@ -400,7 +400,7 @@ class RebaselineCLTest(BaseTestCase, LoggingTestCase):
'--port-name', 'test-win-win7', '--port-name', 'test-win-win7',
'--builder', 'MOCK Try Win', '--builder', 'MOCK Try Win',
'--build-number', '5000', '--build-number', '5000',
'--step-name', 'webkit_layout_tests (with patch)', '--step-name', 'blink_web_tests (with patch)',
]], ]],
[[ [[
'python', 'echo', 'optimize-baselines', 'python', 'echo', 'optimize-baselines',
@ -434,7 +434,7 @@ class RebaselineCLTest(BaseTestCase, LoggingTestCase):
'INFO: Finished try jobs found for all try bots.\n', 'INFO: Finished try jobs found for all try bots.\n',
'INFO: Failed to fetch results for "MOCK Try Win".\n', 'INFO: Failed to fetch results for "MOCK Try Win".\n',
('INFO: Results URL: https://test-results.appspot.com/data/layout_results/' ('INFO: Results URL: https://test-results.appspot.com/data/layout_results/'
'MOCK_Try_Win/5000/webkit_layout_tests%20%28with%20patch%29/layout-test-results/results.html\n'), 'MOCK_Try_Win/5000/blink_web_tests%20%28with%20patch%29/layout-test-results/results.html\n'),
'INFO: There are some builders with no results:\n', 'INFO: There are some builders with no results:\n',
'INFO: MOCK Try Win\n', 'INFO: MOCK Try Win\n',
'INFO: Would you like to continue?\n', 'INFO: Would you like to continue?\n',
@ -451,7 +451,7 @@ class RebaselineCLTest(BaseTestCase, LoggingTestCase):
'INFO: Finished try jobs found for all try bots.\n', 'INFO: Finished try jobs found for all try bots.\n',
'INFO: Failed to fetch results for "MOCK Try Win".\n', 'INFO: Failed to fetch results for "MOCK Try Win".\n',
('INFO: Results URL: https://test-results.appspot.com/data/layout_results/' ('INFO: Results URL: https://test-results.appspot.com/data/layout_results/'
'MOCK_Try_Win/5000/webkit_layout_tests%20%28with%20patch%29/layout-test-results/results.html\n'), 'MOCK_Try_Win/5000/blink_web_tests%20%28with%20patch%29/layout-test-results/results.html\n'),
'INFO: There are some builders with no results:\n', 'INFO: There are some builders with no results:\n',
'INFO: MOCK Try Win\n', 'INFO: MOCK Try Win\n',
'INFO: For one/flaky-fail.html:\n', 'INFO: For one/flaky-fail.html:\n',

16
third_party/blink/tools/blinkpy/tool/commands/rebaseline_unittest.py vendored

@ -230,7 +230,7 @@ class TestRebaseline(BaseTestCase):
'--suffixes', 'txt,png', '--suffixes', 'txt,png',
'--port-name', 'test-win-win7', '--port-name', 'test-win-win7',
'--builder', 'MOCK Win7', '--builder', 'MOCK Win7',
'--step-name', 'webkit_layout_tests (with patch)', '--step-name', 'blink_web_tests (with patch)',
]], ]],
[[ [[
'python', 'echo', 'optimize-baselines', 'python', 'echo', 'optimize-baselines',
@ -264,7 +264,7 @@ class TestRebaseline(BaseTestCase):
'--suffixes', 'txt,png', '--suffixes', 'txt,png',
'--port-name', 'test-win-win7', '--port-name', 'test-win-win7',
'--builder', 'MOCK Win7 (dbg)', '--builder', 'MOCK Win7 (dbg)',
'--step-name', 'webkit_layout_tests (with patch)', '--step-name', 'blink_web_tests (with patch)',
]], ]],
[[ [[
'python', 'echo', 'optimize-baselines', 'python', 'echo', 'optimize-baselines',
@ -298,7 +298,7 @@ class TestRebaseline(BaseTestCase):
'--suffixes', 'txt,png', '--suffixes', 'txt,png',
'--port-name', 'test-win-win7', '--port-name', 'test-win-win7',
'--builder', 'MOCK Win7', '--builder', 'MOCK Win7',
'--step-name', 'webkit_layout_tests (with patch)', '--step-name', 'blink_web_tests (with patch)',
]] ]]
]) ])
@ -325,7 +325,7 @@ class TestRebaseline(BaseTestCase):
'--port-name', 'test-win-win7', '--port-name', 'test-win-win7',
'--builder', 'MOCK Win7', '--builder', 'MOCK Win7',
'--results-directory', '/tmp', '--results-directory', '/tmp',
'--step-name', 'webkit_layout_tests (with patch)', '--step-name', 'blink_web_tests (with patch)',
]], ]],
]) ])
@ -352,7 +352,7 @@ class TestRebaseline(BaseTestCase):
'--suffixes', 'txt,png', '--suffixes', 'txt,png',
'--port-name', 'test-win-win10', '--port-name', 'test-win-win10',
'--builder', 'MOCK Win7', '--builder', 'MOCK Win7',
'--step-name', 'webkit_layout_tests (with patch)', '--step-name', 'blink_web_tests (with patch)',
]], ]],
[[ [[
'python', 'echo', 'optimize-baselines', 'python', 'echo', 'optimize-baselines',
@ -728,7 +728,7 @@ class TestRebaselineExecute(BaseTestCase):
'--suffixes', 'txt,png', '--suffixes', 'txt,png',
'--port-name', 'test-win-win7', '--port-name', 'test-win-win7',
'--builder', 'MOCK Win7', '--builder', 'MOCK Win7',
'--step-name', 'webkit_layout_tests (with patch)', '--step-name', 'blink_web_tests (with patch)',
]] ]]
]) ])
@ -766,7 +766,7 @@ class TestRebaselineExecute(BaseTestCase):
'--suffixes', 'txt,png', '--suffixes', 'txt,png',
'--port-name', 'test-win-win7', '--port-name', 'test-win-win7',
'--builder', 'MOCK Win7', '--builder', 'MOCK Win7',
'--step-name', 'webkit_layout_tests (with patch)', '--step-name', 'blink_web_tests (with patch)',
], ],
[ [
'python', 'echo', 'rebaseline-test-internal', 'python', 'echo', 'rebaseline-test-internal',
@ -775,7 +775,7 @@ class TestRebaselineExecute(BaseTestCase):
'--suffixes', 'wav,png', '--suffixes', 'wav,png',
'--port-name', 'test-win-win7', '--port-name', 'test-win-win7',
'--builder', 'MOCK Win7', '--builder', 'MOCK Win7',
'--step-name', 'webkit_layout_tests (with patch)', '--step-name', 'blink_web_tests (with patch)',
] ]
] ]
]) ])

2
third_party/blink/tools/blinkpy/web_tests/layout_package/bot_test_expectations.py vendored

@ -109,7 +109,7 @@ class ResultsJSON(object):
class BotTestExpectationsFactory(object): class BotTestExpectationsFactory(object):
RESULTS_URL_FORMAT = ( RESULTS_URL_FORMAT = (
'https://test-results.appspot.com/testfile?testtype=webkit_layout_tests' 'https://test-results.appspot.com/testfile?testtype=blink_web_tests'
'&name=results-small.json&master=%s&builder=%s') '&name=results-small.json&master=%s&builder=%s')
def __init__(self, builders): def __init__(self, builders):

2
third_party/blink/tools/blinkpy/web_tests/layout_package/bot_test_expectations_unittest.py vendored

@ -53,7 +53,7 @@ class BotTestExpectationsFactoryTest(unittest.TestCase):
factory = bot_test_expectations.BotTestExpectationsFactory(self.fake_builder_list()) factory = bot_test_expectations.BotTestExpectationsFactory(self.fake_builder_list())
self.assertEqual(factory._results_url_for_builder('Dummy builder name'), self.assertEqual(factory._results_url_for_builder('Dummy builder name'),
'https://test-results.appspot.com/testfile?testtype=webkit_layout_tests' 'https://test-results.appspot.com/testfile?testtype=blink_web_tests'
'&name=results-small.json&master=dummy.master&builder=Dummy%20builder%20name') '&name=results-small.json&master=dummy.master&builder=Dummy%20builder%20name')
def test_expectations_for_builder(self): def test_expectations_for_builder(self):

2
third_party/blink/tools/blinkpy/web_tests/port/base.py vendored

@ -1122,7 +1122,7 @@ class Port(object):
return test_name return test_name
def bot_test_times_path(self): def bot_test_times_path(self):
# TODO(crbug.com/1030434): For the not_site_per_process_webkit_layout_tests step on linux, # TODO(crbug.com/1030434): For the not_site_per_process_blink_web_tests step on linux,
# an exception is raised when merging the bot times json files. This happens whenever they # an exception is raised when merging the bot times json files. This happens whenever they
# are outputted into the results directory. Temporarily we will return the bot times json # are outputted into the results directory. Temporarily we will return the bot times json
# file relative to the target directory. # file relative to the target directory.

2
third_party/blink/tools/blinkpy/web_tests/try_flag_unittest.py vendored

@ -138,7 +138,7 @@ class TryFlagTest(unittest.TestCase):
'https://test-results.appspot.com/data/layout_results', 'https://test-results.appspot.com/data/layout_results',
build.builder_name, build.builder_name,
build.build_number, build.build_number,
'webkit_layout_tests%20%28with%20patch%29' 'blink_web_tests%20%28with%20patch%29'
) )
self.assertEqual(host.stdout.getvalue(), '\n'.join([ self.assertEqual(host.stdout.getvalue(), '\n'.join([
'Fetching results...', 'Fetching results...',

6
third_party/blink/tools/blinkpy/web_tests/update_expectations_unittest.py vendored

@ -1361,7 +1361,7 @@ class UpdateTestExpectationsTest(LoggingTestCase):
'This change was made by the update_expectations.py script.\n\n' 'This change was made by the update_expectations.py script.\n\n'
'Recent test results history:\n' 'Recent test results history:\n'
'https://test-results.appspot.com/dashboards/flakiness_dashboard.html' 'https://test-results.appspot.com/dashboards/flakiness_dashboard.html'
'#testType=webkit_layout_tests&tests=test/a.html,test/b.html\n\n' '#testType=blink_web_tests&tests=test/a.html,test/b.html\n\n'
'Bug: 1111, 2222\n' 'Bug: 1111, 2222\n'
]) ])
@ -1402,7 +1402,7 @@ class UpdateTestExpectationsTest(LoggingTestCase):
'This change was made by the update_expectations.py script.\n\n' 'This change was made by the update_expectations.py script.\n\n'
'Recent test results history:\n' 'Recent test results history:\n'
'https://test-results.appspot.com/dashboards/flakiness_dashboard.html' 'https://test-results.appspot.com/dashboards/flakiness_dashboard.html'
'#testType=webkit_layout_tests&tests=test/b.html\n\n' '#testType=blink_web_tests&tests=test/b.html\n\n'
'Bug: 2222\n' 'Bug: 2222\n'
]) ])
@ -1452,6 +1452,6 @@ class UpdateTestExpectationsTest(LoggingTestCase):
'This change was made by the update_expectations.py script.\n\n' 'This change was made by the update_expectations.py script.\n\n'
'Recent test results history:\n' 'Recent test results history:\n'
'https://test-results.appspot.com/dashboards/flakiness_dashboard.html' 'https://test-results.appspot.com/dashboards/flakiness_dashboard.html'
'#testType=webkit_layout_tests&tests=test/a.html,test/b.html,test/d.html\n\n' '#testType=blink_web_tests&tests=test/a.html,test/b.html,test/d.html\n\n'
'Bug: 1111, 3333\n' 'Bug: 1111, 3333\n'
]) ])

4
third_party/blink/web_tests/SlowTests vendored

@ -324,11 +324,11 @@ crbug.com/869800 http/tests/xmlhttprequest/timeout/xmlhttprequest-timeout-worker
### Slow only on Mac10.13 ### Slow only on Mac10.13
crbug.com/869662 [ Mac10.13 ] virtual/threaded-prefer-compositing/fast/scrolling/overflow-scrollability.html [ Slow ] crbug.com/869662 [ Mac10.13 ] virtual/threaded-prefer-compositing/fast/scrolling/overflow-scrollability.html [ Slow ]
# Tests with slowest_run >= 10s on flakiness dashboard for site_per_process_webkit_layout_tests # Tests with slowest_run >= 10s on flakiness dashboard for site_per_process_blink_web_tests
crbug.com/874695 http/tests/devtools/sources/debugger-breakpoints/breakpoints-ui-shifted-breakpoint.js [ Slow ] crbug.com/874695 http/tests/devtools/sources/debugger-breakpoints/breakpoints-ui-shifted-breakpoint.js [ Slow ]
crbug.com/874695 http/tests/devtools/sources/sourcemap-hot-reload.js [ Slow ] crbug.com/874695 http/tests/devtools/sources/sourcemap-hot-reload.js [ Slow ]
# Tests where |3s <= slowest_run < 10s| on flakiness dashboard for site_per_process_webkit_layout_tests # Tests where |3s <= slowest_run < 10s| on flakiness dashboard for site_per_process_blink_web_tests
crbug.com/874695 accessibility/is-ignored-change-sends-notification.html [ Slow ] crbug.com/874695 accessibility/is-ignored-change-sends-notification.html [ Slow ]
crbug.com/874695 animations/svg/animated-filter-svg-element.html [ Slow ] crbug.com/874695 animations/svg/animated-filter-svg-element.html [ Slow ]
crbug.com/874695 bluetooth/server/connect/garbage-collection-ran-during-error.html [ Slow ] crbug.com/874695 bluetooth/server/connect/garbage-collection-ran-during-error.html [ Slow ]

2
third_party/blink/web_tests/fast/peerconnection/RTCPeerConnection-many.html vendored

@ -40,7 +40,7 @@ promise_test(async function() {
assert_throws_dom("UnknownError", allocatePeerConnection, assert_throws_dom("UnknownError", allocatePeerConnection,
'Creating too many RTCPeerConnections gives UnknownError'); 'Creating too many RTCPeerConnections gives UnknownError');
} finally { } finally {
// On swarming bots running webkit_layout_tests (such as win7-rel), garbage // On swarming bots running blink_web_tests (such as win7-rel), garbage
// collection is not guaranteed to occur between test pages. As such we // collection is not guaranteed to occur between test pages. As such we
// cannot assume that the initial count is 0, and we must clean up after // cannot assume that the initial count is 0, and we must clean up after
// this test regardless of test result or unrelated tests may fail. // this test regardless of test result or unrelated tests may fail.