Added docs on accessibility tests
BUG=none NOTRY=true TBR=nektar@chromium.org Review-Url: https://codereview.chromium.org/2728663004 Cr-Commit-Position: refs/heads/master@{#454197}
This commit is contained in:
@@ -1,4 +1,4 @@
|
|||||||
DumpAccessibilityTreeTest and DumpAccessibilityEventsTest Notes:
|
# DumpAccessibilityTreeTest and DumpAccessibilityEventsTest Notes:
|
||||||
|
|
||||||
Both sets of tests use a similar format for files.
|
Both sets of tests use a similar format for files.
|
||||||
|
|
||||||
@@ -28,6 +28,7 @@ ninja -C out/Debug content_browsertests
|
|||||||
out/Debug/content_browsertests --gtest_filter="DumpAccessibility*"
|
out/Debug/content_browsertests --gtest_filter="DumpAccessibility*"
|
||||||
|
|
||||||
Files used:
|
Files used:
|
||||||
|
|
||||||
* foo.html -- a file to be tested
|
* foo.html -- a file to be tested
|
||||||
* foo-expected-android.txt -- expected Android AccessibilityNodeInfo output
|
* foo-expected-android.txt -- expected Android AccessibilityNodeInfo output
|
||||||
* foo-expected-auralinux.txt -- expected Linux ATK output
|
* foo-expected-auralinux.txt -- expected Linux ATK output
|
||||||
@@ -36,6 +37,7 @@ Files used:
|
|||||||
* foo-expected-win.txt -- expected Win IAccessible/IAccessible2 output
|
* foo-expected-win.txt -- expected Win IAccessible/IAccessible2 output
|
||||||
|
|
||||||
Format for expected files:
|
Format for expected files:
|
||||||
|
|
||||||
* Blank lines and lines beginning with # are ignored
|
* Blank lines and lines beginning with # are ignored
|
||||||
* Skipped files: if first line of file begins with #<skip then the
|
* Skipped files: if first line of file begins with #<skip then the
|
||||||
test passes. This can be used to indicate desired output with a link
|
test passes. This can be used to indicate desired output with a link
|
||||||
@@ -43,6 +45,7 @@ Format for expected files:
|
|||||||
* Use 2 plus signs for indent to show hierarchy
|
* Use 2 plus signs for indent to show hierarchy
|
||||||
|
|
||||||
Filters:
|
Filters:
|
||||||
|
|
||||||
* By default only some attributes of nodes in the accessibility tree, or
|
* By default only some attributes of nodes in the accessibility tree, or
|
||||||
events fired (when running DumpAccessibilityEvents) are output.
|
events fired (when running DumpAccessibilityEvents) are output.
|
||||||
This is to keep the tests robust and not prone to failure when unrelated
|
This is to keep the tests robust and not prone to failure when unrelated
|
||||||
@@ -51,11 +54,13 @@ Filters:
|
|||||||
They can appear anywhere but typically they're in an HTML comment block,
|
They can appear anywhere but typically they're in an HTML comment block,
|
||||||
and must be one per line.
|
and must be one per line.
|
||||||
* Filters are platform-specific:
|
* Filters are platform-specific:
|
||||||
|
```
|
||||||
@WIN-
|
@WIN-
|
||||||
@MAC-
|
@MAC-
|
||||||
@BLINK-
|
@BLINK-
|
||||||
@ANDROID-
|
@ANDROID-
|
||||||
@AURALINUX-
|
@AURALINUX-
|
||||||
|
```
|
||||||
* To dump all attributes while writing or debugging a test, add this filter:
|
* To dump all attributes while writing or debugging a test, add this filter:
|
||||||
@WIN-ALLOW:*
|
@WIN-ALLOW:*
|
||||||
(and similarly for other platforms).
|
(and similarly for other platforms).
|
||||||
@@ -64,16 +69,18 @@ Filters:
|
|||||||
ALLOW filter means to include the attribute, and a DENY filter means to
|
ALLOW filter means to include the attribute, and a DENY filter means to
|
||||||
exclude it. Filters can contain simple wildcards ('*') only, they're not
|
exclude it. Filters can contain simple wildcards ('*') only, they're not
|
||||||
regular expressions. Examples:
|
regular expressions. Examples:
|
||||||
|
```
|
||||||
- @WIN-ALLOW:name* - this will output the name attribute on Windows
|
- @WIN-ALLOW:name* - this will output the name attribute on Windows
|
||||||
- @WIN-ALLOW:name='Foo' - this will only output the name attribute if it
|
- @WIN-ALLOW:name='Foo' - this will only output the name attribute if it
|
||||||
exactly matches 'Foo'.
|
exactly matches 'Foo'.
|
||||||
- @WIN-DENY:name='X* - this will skip outputting any name that begins with
|
- @WIN-DENY:name='X* - this will skip outputting any name that begins with
|
||||||
the letter X.
|
the letter X.
|
||||||
|
```
|
||||||
* By default empty attributes are skipped. To output the value an attribute
|
* By default empty attributes are skipped. To output the value an attribute
|
||||||
even if it's empty, use @WIN-ALLOW-EMPTY:name, for example, and similarly
|
even if it's empty, use @WIN-ALLOW-EMPTY:name, for example, and similarly
|
||||||
for other platforms.
|
for other platforms.
|
||||||
|
|
||||||
Advanced:
|
## Advanced:
|
||||||
|
|
||||||
Normally the system waits for the document to finish loading before dumping
|
Normally the system waits for the document to finish loading before dumping
|
||||||
the accessibility tree.
|
the accessibility tree.
|
||||||
@@ -98,7 +105,7 @@ To load an iframe from a different site, forcing it into a different process,
|
|||||||
use /cross-site/HOSTNAME/ in the url, for example:
|
use /cross-site/HOSTNAME/ in the url, for example:
|
||||||
<iframe src="cross-site/1.com/accessibility/html/frame.html"></iframe>
|
<iframe src="cross-site/1.com/accessibility/html/frame.html"></iframe>
|
||||||
|
|
||||||
Generating expectations and rebaselining:
|
## Generating expectations and rebaselining:
|
||||||
|
|
||||||
If you want to populate the expectation file directly rather than typing it
|
If you want to populate the expectation file directly rather than typing it
|
||||||
or copying-and-pasting it, first make sure the file exists (it can be empty),
|
or copying-and-pasting it, first make sure the file exists (it can be empty),
|
||||||
@@ -113,7 +120,7 @@ This will replace the -expected-*.txt file with the current output. It's
|
|||||||
a great way to rebaseline a bunch of tests after making a change. Please
|
a great way to rebaseline a bunch of tests after making a change. Please
|
||||||
manually check the diff, of course!
|
manually check the diff, of course!
|
||||||
|
|
||||||
Adding a new test:
|
## Adding a new test:
|
||||||
|
|
||||||
If you are adding a new test file remember to add a corresponding test case in
|
If you are adding a new test file remember to add a corresponding test case in
|
||||||
content/browser/accessibility/dump_accessibility_events_browsertest.cc
|
content/browser/accessibility/dump_accessibility_events_browsertest.cc
|
@@ -1,6 +1,7 @@
|
|||||||
# Accessibility
|
# Accessibility
|
||||||
|
|
||||||
* [Accessibility Overview](accessibility/overview.md)
|
* [Accessibility Overview](accessibility/overview.md)
|
||||||
|
* [Accessibility Tests](accessibility/tests.md)
|
||||||
|
|
||||||
## Chrome OS
|
## Chrome OS
|
||||||
|
|
||||||
|
123
docs/accessibility/tests.md
Normal file
123
docs/accessibility/tests.md
Normal file
@@ -0,0 +1,123 @@
|
|||||||
|
# Accessibility
|
||||||
|
|
||||||
|
Here's a quick overview of all of the locations in the codebase where
|
||||||
|
you'll find accessibility tests, and a brief overview of the purpose of
|
||||||
|
all of them.
|
||||||
|
|
||||||
|
## Layout Tests
|
||||||
|
|
||||||
|
This is the primary place where we test accessibility code in Blink. This
|
||||||
|
code should have no platform-specific code. Use this to test anything
|
||||||
|
where there's any interesting web platform logic, or where you need to be
|
||||||
|
able to query things synchronously from the renderer thread to test them.
|
||||||
|
|
||||||
|
Don't add tests for trivial features like ARIA attributes that we just
|
||||||
|
expose directly to the next layer up. In those cases the Blink tests are
|
||||||
|
trivial and it's more valuable to test these features at a higher level
|
||||||
|
where we can ensure they actually work.
|
||||||
|
|
||||||
|
Note that many of these tests are inherited from WebKit and the coding style
|
||||||
|
has evolved a lot since then. Look for more recent tests as a guide if writing
|
||||||
|
a new one.
|
||||||
|
|
||||||
|
Test files:
|
||||||
|
```
|
||||||
|
third_party/WebKit/LayoutTests/accessibility
|
||||||
|
```
|
||||||
|
|
||||||
|
Source code to AccessibilityController and WebAXObjectProxy:
|
||||||
|
```
|
||||||
|
content/shell/test_runner
|
||||||
|
```
|
||||||
|
|
||||||
|
To run all accessibility LayoutTests:
|
||||||
|
```
|
||||||
|
ninja -C out/release blink_tests
|
||||||
|
third_party/WebKit/Tools/Scripts/run-webkit-tests --build-directory=out --target=release accessibility/
|
||||||
|
```
|
||||||
|
|
||||||
|
To run just one test by itself without the script:
|
||||||
|
```
|
||||||
|
ninja -C out/release blink_tests
|
||||||
|
out/release/content_shell --run-layout-test third_party/WebKit/LayoutTests/accessibility/name-calc-inputs.html
|
||||||
|
```
|
||||||
|
|
||||||
|
## DumpAccessibilityTree tests
|
||||||
|
|
||||||
|
This is the best way to write both cross-platform and platform-specific tests
|
||||||
|
using only an input HTML file, some magic strings to describe what attributes
|
||||||
|
you're interested in, and one or more expectation files to enable checking
|
||||||
|
whether the resulting accessibility tree is correct or not. In particular,
|
||||||
|
almost no test code is required.
|
||||||
|
|
||||||
|
[More documentation on DumpAccessibilityTree](../../content/test/data/accessibility/readme.md)
|
||||||
|
|
||||||
|
Test files:
|
||||||
|
```
|
||||||
|
content/test/data/accessibility
|
||||||
|
```
|
||||||
|
|
||||||
|
Test runner:
|
||||||
|
```
|
||||||
|
content/browser/accessibility/dump_accessibility_tree_browsertest.cc
|
||||||
|
```
|
||||||
|
|
||||||
|
To run all tests:
|
||||||
|
```
|
||||||
|
ninja -C out/release content_browsertests
|
||||||
|
out/release/content_browsertests --gtest_filter="DumpAccessibilityTree*"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Other content_browsertests
|
||||||
|
|
||||||
|
There are many other tests in content/ that relate to accessibility.
|
||||||
|
All of these tests work by launching a full multi-process browser shell,
|
||||||
|
loading a web page in a renderer, then accessing the resulting accessibility
|
||||||
|
tree from the browser process, and running some test from there.
|
||||||
|
|
||||||
|
To run all tests:
|
||||||
|
```
|
||||||
|
ninja -C out/release content_browsertests
|
||||||
|
out/release/content_browsertests --gtest_filter="*ccessib*"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Accessibility unittests
|
||||||
|
|
||||||
|
This tests the core accessibility code that's shared by both web and non-web
|
||||||
|
accessibility infrastructure.
|
||||||
|
|
||||||
|
Code location:
|
||||||
|
```
|
||||||
|
ui/accessibility
|
||||||
|
```
|
||||||
|
|
||||||
|
To run all tests:
|
||||||
|
```
|
||||||
|
ninja -C out/release accessibility_unittests
|
||||||
|
out/release/accessibility_unittests
|
||||||
|
```
|
||||||
|
|
||||||
|
## ChromeVox tests
|
||||||
|
|
||||||
|
You must build with ```target_os = "chromeos"``` in your GN args.
|
||||||
|
|
||||||
|
To run all tests:
|
||||||
|
```
|
||||||
|
ninja -C out/release chromevox_tests
|
||||||
|
out/release/chromevox_tests --test-launcher-jobs=10
|
||||||
|
```
|
||||||
|
|
||||||
|
## Other locations of accessibility tests:
|
||||||
|
|
||||||
|
Even this isn't a complete list. The tests described above cover more
|
||||||
|
than 90% of the accessibility tests, and the remainder are scattered
|
||||||
|
throughout the codebase. Here are a few other locations to check:
|
||||||
|
|
||||||
|
```
|
||||||
|
chrome/android/javatests/src/org/chromium/chrome/browser/accessibility
|
||||||
|
chrome/browser/accessibility
|
||||||
|
chrome/browser/chromeos/accessibility/
|
||||||
|
ui/chromeos
|
||||||
|
ui/views/accessibility
|
||||||
|
```
|
||||||
|
|
Reference in New Issue
Block a user