0

blinkpy: Update "layout test" in various places to "web test"

Bug: 843412
Change-Id: I1bd7de018b74d32c17ed36c3cf4b0e56a4b7b467
Reviewed-on: https://chromium-review.googlesource.com/c/1371404
Commit-Queue: Kent Tamura <tkent@chromium.org>
Reviewed-by: Robert Ma <robertma@chromium.org>
Cr-Commit-Position: refs/heads/master@{#615751}
This commit is contained in:
Kent Tamura
2018-12-12 01:36:15 +00:00
committed by Commit Bot
parent a0d2a9b26a
commit 81f891fd02
26 changed files with 56 additions and 56 deletions

@ -108,7 +108,7 @@ and MacPort (and maybe the android port class) should be extended as they
provide platform specific overrides/extensions that implement most of the
important functionality. While there are many functions in Port, overriding one
function will affect most of the other ones to get the desired behavior. For
example, if `layout_tests_dir()` is overridden, not only will the code look for
example, if `web_tests_dir()` is overridden, not only will the code look for
tests in that directory, but it will find the correct TestExpectations file, the
platform specific expected files, etc.
@ -125,7 +125,7 @@ Here are some of the functions that most likely need to be overridden.
* This should return the name of the program test p. By default it returns
content_shell, but you want to have it return the program you want to
run, such as `chrome` or `browser_tests`.
* `layout_tests_dir`
* `web_tests_dir`
* This tells the port where to look for all the and everything associated
with them such as resources files.
* By default it returns the absolute path to the web tests directory.
@ -172,7 +172,7 @@ and has stdout, stdin, stderr.
#### Goals
Your goal for this part of the project is to create a program (or extend a
program) to interface with the web test framework. The layout test framework
program) to interface with the web test framework. The web test framework
will communicate with this program to tell it what to do and it will accept data
from this program to perform the regression testing or create new base line
files.

@ -78,7 +78,7 @@ class DiffParserTest(unittest.TestCase):
self.assertEqual((87, 90), diff.lines[22][0:2])
# Check if a newly added file is correctly handled.
diff = parser.files['LayoutTests/platform/mac/fast/flexbox/box-orient-button-expected.checksum']
diff = parser.files['web_tests/platform/mac/fast/flexbox/box-orient-button-expected.checksum']
self.assertEqual(1, len(diff.lines))
self.assertEqual((0, 1), diff.lines[0][0:2])

@ -69,7 +69,7 @@ index ce21720..324929e 100644
}
bool StyleRareInheritedData::shadowDataEquivalent(const StyleRareInheritedData& o) const
diff --git a/LayoutTests/platform/mac/fast/flexbox/box-orient-button-expected.checksum b/LayoutTests/platform/mac/fast/flexbox/box-orient-button-expected.checksum
diff --git a/web_tests/platform/mac/fast/flexbox/box-orient-button-expected.checksum b/web_tests/platform/mac/fast/flexbox/box-orient-button-expected.checksum
new file mode 100644
index 0000000..6db26bd
--- /dev/null

@ -87,7 +87,7 @@ class WebTestResultsTest(unittest.TestCase):
},
"skipped": 450,
"num_regressions": 15,
"layout_tests_dir": "/b/build/slave/Webkit_Mac10_5/build/src/third_party/WebKit/LayoutTests",
"layout_tests_dir": "/b/build/slave/Webkit_Mac10_5/build/src/third_party/blink/web_tests",
"version": 3,
"num_passes": 77,
"fixable": 1220,

@ -2,7 +2,7 @@
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
"""A utility function to do text diffs of expected and actual layout test results."""
"""A utility function to do text diffs of expected and actual web test results."""
import difflib

@ -1,4 +1,4 @@
W3C Web Platform Tests in Blink Layout Tests
W3C Web Platform Tests in Blink Web Tests
Design Doc: https://goo.gl/iXUaZd
@ -28,7 +28,7 @@ changing the ports (HTTP/S, WS/S), make sure to also:
- update `WPT_HOST_AND_PORTS` in
`//third_party/blink/tools/blinkpy/web_tests/port/driver.py`
- update LayoutTestContentBrowserClient::GetOriginsRequiringDedicatedProcess()
- update WebTestContentBrowserClient::GetOriginsRequiringDedicatedProcess()
checkout.sh
===========

@ -13,7 +13,7 @@ class _FakeOptimizer(BaselineOptimizer):
def read_results_by_directory(self, baseline_name):
if baseline_name.endswith('txt'):
return {'LayoutTests/passes/text.html': '123456'}
return {'web_tests/passes/text.html': '123456'}
return {}

@ -101,7 +101,7 @@ class RebaselineCLTest(BaseTestCase, LoggingTestCase):
self._write(path, 'contents')
self.mac_port.host.filesystem.write_text_file(
'/test.checkout/LayoutTests/external/wpt/MANIFEST.json', '{}')
'/test.checkout/web_tests/external/wpt/MANIFEST.json', '{}')
def tearDown(self):
BaseTestCase.tearDown(self)

@ -29,8 +29,8 @@ class MockChromiumCommit(object):
def filtered_changed_files(self):
return [
'third_party/WebKit/LayoutTests/external/wpt/one.html',
'third_party/WebKit/LayoutTests/external/wpt/two.html',
'third_party/blink/web_tests/external/wpt/one.html',
'third_party/blink/web_tests/external/wpt/two.html',
]
def url(self):

@ -26,9 +26,9 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""Represents a set of builder bots running layout tests.
"""Represents a set of builder bots running web tests.
This class is used to hold a list of builder bots running layout tests and their
This class is used to hold a list of builder bots running web tests and their
corresponding port names and TestExpectations specifiers.
"""

@ -2,13 +2,13 @@
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
"""Classes for merging layout tests results directories together.
"""Classes for merging web tests results directories together.
This is split into three parts:
* Generic code to merge JSON data together.
* Generic code to merge directories together.
* Code to specifically merge the layout tests result data together.
* Code to specifically merge the web tests result data together.
The JSON data merger will recursively merge dictionaries by default.
@ -518,7 +518,7 @@ class DirMerger(Merger):
break
# Classes specific to merging LayoutTest results directory.
# Classes specific to merging web test results directory.
# ------------------------------------------------------------------------
@ -577,7 +577,7 @@ class JSONTestResultsMerger(JSONMerger):
NameRegexMatch(':interrupted$'),
lambda o, name=None: bool(sum(o)))
# Layout test directory value is randomly created on each shard, so
# Web test directory value is randomly created on each shard, so
# clear it.
self.add_helper(
NameRegexMatch(':layout_tests_dir$'),
@ -596,8 +596,8 @@ class JSONTestResultsMerger(JSONMerger):
return JSONMerger.fallback_matcher(self, objs, name)
class LayoutTestDirMerger(DirMerger):
"""Merge layout test result directory."""
class WebTestDirMerger(DirMerger):
"""Merge web test result directory."""
def __init__(self, filesystem=None,
results_json_value_overrides=None,
@ -674,7 +674,7 @@ def main(argv):
parser = argparse.ArgumentParser()
parser.description = """\
Merges sharded layout test results into a single output directory.
Merges sharded web test results into a single output directory.
"""
parser.epilog = """\
@ -812,7 +812,7 @@ directory. The script will be given the arguments plus
results_json_value_overrides[result_key] = build_properties[build_prop_key]
logging.debug('results_json_value_overrides: %r', results_json_value_overrides)
merger = LayoutTestDirMerger(
merger = WebTestDirMerger(
results_json_value_overrides=results_json_value_overrides,
results_json_allow_unknown_if_matching=args.results_json_allow_unknown_if_matching)

@ -541,7 +541,7 @@ class JSONTestResultsMerger(unittest.TestCase):
merger.merge([{'seconds_since_epoch': 12}, {}]))
class LayoutTestDirMergerTests(unittest.TestCase):
class WebTestDirMergerTests(unittest.TestCase):
# JSON files for shard 1
# Shard1 has the following tests;
@ -555,7 +555,7 @@ class LayoutTestDirMergerTests(unittest.TestCase):
"chromium_revision": "123",
"fixable": 1,
"interrupted": false,
"layout_tests_dir": "/b/s/w/irJ1McdS/third_party/WebKit/LayoutTests",
"layout_tests_dir": "/b/s/w/irJ1McdS/third_party/blink/web_tests",
"num_failures_by_type": {
"AUDIO": 2,
"CRASH": 3
@ -677,7 +677,7 @@ ADD_RESULTS({
"chromium_revision": "123",
"fixable": 9,
"interrupted": false,
"layout_tests_dir": "/b/s/w/sadfa124/third_party/WebKit/LayoutTests",
"layout_tests_dir": "/b/s/w/sadfa124/third_party/blink/web_tests",
"num_failures_by_type": {
"AUDIO": 10,
"CRASH": 11
@ -772,7 +772,7 @@ ADD_RESULTS({
[Wed Mar 01 22:20:07.400802 2017] [ssl:warn] [pid 15010] AH01909: RSA certificate configured for 127.0.0.1:443 does NOT include an ID which matches the server name
"""
layout_test_filesystem = {
web_test_filesystem = {
# Files for shard0
'/shards/0/layout-test-results/access_log.txt': shard0_access_log,
'/shards/0/layout-test-results/archived_results.json': shard0_archived_results_json,
@ -1008,7 +1008,7 @@ ADD_RESULTS({
[Wed Mar 01 22:20:07.400802 2017] [ssl:warn] [pid 15010] AH01909: RSA certificate configured for 127.0.0.1:443 does NOT include an ID which matches the server name
"""
layout_test_output_filesystem = {
web_test_output_filesystem = {
'/out/layout-test-results/access_log.txt': output_access_log,
'/out/layout-test-results/archived_results.json': output_archived_results_json,
'/out/layout-test-results/error_log.txt': output_error_log,
@ -1040,11 +1040,11 @@ ADD_RESULTS({
}
def test(self):
fs = MockFileSystem(self.layout_test_filesystem)
fs = MockFileSystem(self.web_test_filesystem)
merger = merge_results.LayoutTestDirMerger(fs, results_json_value_overrides={'layout_tests_dir': 'src'})
merger = merge_results.WebTestDirMerger(fs, results_json_value_overrides={'layout_tests_dir': 'src'})
merger.merge('/out', ['/shards/0', '/shards/1'])
for fname, contents in self.layout_test_output_filesystem.items():
for fname, contents in self.web_test_output_filesystem.items():
self.assertIn(fname, fs.files)
self.assertMultiLineEqual(contents, fs.files[fname])

@ -26,7 +26,7 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""A helper class for reading in and dealing with tests expectations for layout tests."""
"""A helper class for reading in and dealing with tests expectations for web tests."""
from collections import defaultdict
@ -880,7 +880,7 @@ class TestExpectationsModel(object):
class TestExpectations(object):
"""Test expectations consist of lines with specifications of what
to expect from layout test cases. The test cases can be directories
to expect from web test cases. The test cases can be directories
in which case the expectations apply to all test cases in that
directory and any subdirectory. The format is along the lines of:

@ -61,7 +61,7 @@ class TestRunResults(object):
self.unexpected_failures = 0
self.unexpected_timeouts = 0
# The wall clock time spent running the tests (layout_test_runner.run()).
# The wall clock time spent running the tests (web_test_runner.run()).
self.run_time = 0
# Map of test name to the *last* result for the test.

@ -43,7 +43,7 @@ class BrowserTestDriver(Driver):
assign it to the stdin of the process that is owned by this driver's
server process.
"""
# FIXME(ivandavid): Need to handle case where the layout test doesn't
# FIXME(ivandavid): Need to handle case where the web test doesn't
# get a file name.
new_cmd_line = self.cmd_line(per_test_args)
if not self._server_process or new_cmd_line != self._current_cmd_line:
@ -52,7 +52,7 @@ class BrowserTestDriver(Driver):
self._open_stdin_path(deadline)
# Gets the path of the directory that the file for stdin communication is
# in. Since the browser test cannot clean it up, the layout test framework
# in. Since the browser test cannot clean it up, the web test framework
# will. Everything the browser test uses is stored in the same directory as
# the stdin file, so deleting that directory recursively will remove all the
# other temp data, like the printed pdf. This function assumes the correct

@ -29,10 +29,10 @@
import unittest
from blinkpy.common.host_mock import MockHost
from blinkpy.web_tests.print_layout_test_times import main
from blinkpy.web_tests.print_web_test_times import main
class PrintLayoutTestTimesTest(unittest.TestCase):
class PrintWebTestTimesTest(unittest.TestCase):
def check(self, args, expected_output, files=None):
host = MockHost()

@ -114,7 +114,7 @@ def parse_args(args):
action='append',
default=[],
dest='adb_devices',
help='Run Android layout tests on these devices'),
help='Run Android web tests on these devices'),
# FIXME: Flip this to be off by default once we can log the
# device setup more cleanly.
optparse.make_option(
@ -300,7 +300,7 @@ def parse_args(args):
help='Number of times to run the set of tests (e.g. ABCABCABC)'),
optparse.make_option(
'--layout-tests-directory',
help=('Path to a custom layout tests directory')),
help=('Path to a custom web tests directory')),
optparse.make_option(
'--max-locked-shards',
type='int',
@ -464,7 +464,7 @@ def parse_args(args):
'--zero-tests-executed-ok',
action='store_true',
help='If set, exit with a success code when no tests are run.'
' Used on trybots when layout tests are retried without patch.')
' Used on trybots when web tests are retried without patch.')
]))
option_group_definitions.append(
@ -509,7 +509,7 @@ def parse_args(args):
option_parser = optparse.OptionParser(
prog='run_web_tests.py',
usage='%prog [options] [tests]',
description='Runs Blink layout tests as described in docs/testing/web_tests.md')
description='Runs Blink web tests as described in docs/testing/web_tests.md')
for group_name, group_options in option_group_definitions:
option_group = optparse.OptionGroup(option_parser, group_name)

@ -29,7 +29,7 @@
"""A utility module for making standalone scripts to start servers.
Scripts in tools/ can use this module to start servers that are normally used
for layout tests, outside of the layout test runner.
for web tests, outside of the web test runner.
"""
import logging

@ -26,7 +26,7 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""Base class used to start servers used by the layout tests."""
"""Base class used to start servers used by the web tests."""
import errno
import logging
@ -42,7 +42,7 @@ class ServerError(Exception):
class ServerBase(object):
"""A skeleton class for starting and stopping servers used by the layout tests."""
"""A skeleton class for starting and stopping servers used by the web tests."""
def __init__(self, port_obj, output_dir):
self._port_obj = port_obj

@ -165,7 +165,7 @@ class Printer(object):
if self._options.timing:
parallel_time = sum(result.total_run_time for result in run_results.results_by_name.values())
# There is serial overhead in layout_test_runner.run() that we can't easily account for when
# There is serial overhead in web_test_runner.run() that we can't easily account for when
# really running in parallel, but taking the min() ensures that in the worst case
# (if parallel time is less than run_time) we do account for it.
serial_time = total_time - min(run_results.run_time, parallel_time)

@ -4,9 +4,9 @@
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
"""Check if a LayoutTest expected file is an all-PASS testharness result.
"""Check if a web test expected file is an all-PASS testharness result.
LayoutTests/PRESUBMIT.py uses this script to identify generic all-PASS
web_tests/PRESUBMIT.py uses this script to identify generic all-PASS
testharness baselines, which are redundant because run_web_tests.py assumes
all-PASS results for testharness tests when baselines are not found.
"""

@ -31,6 +31,6 @@
import sys
from blinkpy.common import host
from blinkpy.web_tests import print_layout_test_times
from blinkpy.web_tests import print_web_test_times
print_layout_test_times.main(host.Host(), sys.argv[1:])
print_web_test_times.main(host.Host(), sys.argv[1:])

@ -27,18 +27,18 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""Runs an Apache HTTP server to manually run layout tests locally.
"""Runs an Apache HTTP server to manually run web tests locally.
After running this script, you can locally navigate to URLs where
the path is relative to LayoutTests/http/tests/. For example, to run
LayoutTests/http/tests/cachestorage/window-cache-add.html, navigate to:
the path is relative to web_tests/http/tests/. For example, to run
web_tests/http/tests/cachestorage/window-cache-add.html, navigate to:
http://127.0.0.1:8000/cachestorage/window/cache-add.html
When using HTTPS, for example:
https://127.0.0.1:8443/https/verify-ssl-enabled.php
you will may a certificate warning, which you need to bypass.
After starting the server, you can also run individual layout tests
After starting the server, you can also run individual web tests
via content_shell, e.g.
$ out/Release/content_shell --run-web-tests \
http://127.0.0.1:8000/security/cross-frame-access-get.html

@ -33,7 +33,7 @@ Some tests require both an HTTP server and WebSocket server. You can start
both servers by running both run_blink_httpd.py and
run_blink_websocketserver.py.
Tests served by the HTTP server have paths relative to LayoutTests/http/tests/.
Tests served by the HTTP server have paths relative to web_tests/http/tests/.
For example, to run a test http/tests/websocket/binary-type.html which depends
on WebSocket, you can navigate to:
http://127.0.0.1:8000/websocket/close-unref-websocket.html

@ -8,7 +8,7 @@
The main HTTP server is run on 8001, while the main HTTPS server is run on 8444.
URL paths are relative to the web-platform-tests root, e.g. the test:
LayoutTests/external/wpt/referrer-policy/origin/http-rp/same-origin/http-http/img-tag/generic.no-redirect.http.html
web_tests/external/wpt/referrer-policy/origin/http-rp/same-origin/http-http/img-tag/generic.no-redirect.http.html
Could be tried by running this scrip then navigating to:
http://localhost:8001/referrer-policy/origin/http-rp/same-origin/http-http/img-tag/generic.no-redirect.http.html
"""