0

Added docs on accessibility tests

BUG=none
NOTRY=true
TBR=nektar@chromium.org

Review-Url: https://codereview.chromium.org/2728663004
Cr-Commit-Position: refs/heads/master@{#454197}
This commit is contained in:
dmazzoni
2017-03-01 22:58:46 -08:00
committed by Commit bot
parent 59ed1a91fd
commit 9cd1712465
3 changed files with 135 additions and 4 deletions
content/test/data/accessibility
docs

@ -1,4 +1,4 @@
DumpAccessibilityTreeTest and DumpAccessibilityEventsTest Notes:
# DumpAccessibilityTreeTest and DumpAccessibilityEventsTest Notes:
Both sets of tests use a similar format for files.
@ -28,6 +28,7 @@ ninja -C out/Debug content_browsertests
out/Debug/content_browsertests --gtest_filter="DumpAccessibility*"
Files used:
* foo.html -- a file to be tested
* foo-expected-android.txt -- expected Android AccessibilityNodeInfo output
* foo-expected-auralinux.txt -- expected Linux ATK output
@ -36,6 +37,7 @@ Files used:
* foo-expected-win.txt -- expected Win IAccessible/IAccessible2 output
Format for expected files:
* Blank lines and lines beginning with # are ignored
* Skipped files: if first line of file begins with #<skip then the
test passes. This can be used to indicate desired output with a link
@ -43,6 +45,7 @@ Format for expected files:
* Use 2 plus signs for indent to show hierarchy
Filters:
* By default only some attributes of nodes in the accessibility tree, or
events fired (when running DumpAccessibilityEvents) are output.
This is to keep the tests robust and not prone to failure when unrelated
@ -51,11 +54,13 @@ Filters:
They can appear anywhere but typically they're in an HTML comment block,
and must be one per line.
* Filters are platform-specific:
```
@WIN-
@MAC-
@BLINK-
@ANDROID-
@AURALINUX-
```
* To dump all attributes while writing or debugging a test, add this filter:
@WIN-ALLOW:*
(and similarly for other platforms).
@ -64,16 +69,18 @@ Filters:
ALLOW filter means to include the attribute, and a DENY filter means to
exclude it. Filters can contain simple wildcards ('*') only, they're not
regular expressions. Examples:
```
- @WIN-ALLOW:name* - this will output the name attribute on Windows
- @WIN-ALLOW:name='Foo' - this will only output the name attribute if it
exactly matches 'Foo'.
- @WIN-DENY:name='X* - this will skip outputting any name that begins with
the letter X.
```
* By default empty attributes are skipped. To output the value an attribute
even if it's empty, use @WIN-ALLOW-EMPTY:name, for example, and similarly
for other platforms.
Advanced:
## Advanced:
Normally the system waits for the document to finish loading before dumping
the accessibility tree.
@ -98,7 +105,7 @@ To load an iframe from a different site, forcing it into a different process,
use /cross-site/HOSTNAME/ in the url, for example:
<iframe src="cross-site/1.com/accessibility/html/frame.html"></iframe>
Generating expectations and rebaselining:
## Generating expectations and rebaselining:
If you want to populate the expectation file directly rather than typing it
or copying-and-pasting it, first make sure the file exists (it can be empty),
@ -113,7 +120,7 @@ This will replace the -expected-*.txt file with the current output. It's
a great way to rebaseline a bunch of tests after making a change. Please
manually check the diff, of course!
Adding a new test:
## Adding a new test:
If you are adding a new test file remember to add a corresponding test case in
content/browser/accessibility/dump_accessibility_events_browsertest.cc

@ -1,6 +1,7 @@
# Accessibility
* [Accessibility Overview](accessibility/overview.md)
* [Accessibility Tests](accessibility/tests.md)
## Chrome OS

123
docs/accessibility/tests.md Normal file

@ -0,0 +1,123 @@
# Accessibility
Here's a quick overview of all of the locations in the codebase where
you'll find accessibility tests, and a brief overview of the purpose of
all of them.
## Layout Tests
This is the primary place where we test accessibility code in Blink. This
code should have no platform-specific code. Use this to test anything
where there's any interesting web platform logic, or where you need to be
able to query things synchronously from the renderer thread to test them.
Don't add tests for trivial features like ARIA attributes that we just
expose directly to the next layer up. In those cases the Blink tests are
trivial and it's more valuable to test these features at a higher level
where we can ensure they actually work.
Note that many of these tests are inherited from WebKit and the coding style
has evolved a lot since then. Look for more recent tests as a guide if writing
a new one.
Test files:
```
third_party/WebKit/LayoutTests/accessibility
```
Source code to AccessibilityController and WebAXObjectProxy:
```
content/shell/test_runner
```
To run all accessibility LayoutTests:
```
ninja -C out/release blink_tests
third_party/WebKit/Tools/Scripts/run-webkit-tests --build-directory=out --target=release accessibility/
```
To run just one test by itself without the script:
```
ninja -C out/release blink_tests
out/release/content_shell --run-layout-test third_party/WebKit/LayoutTests/accessibility/name-calc-inputs.html
```
## DumpAccessibilityTree tests
This is the best way to write both cross-platform and platform-specific tests
using only an input HTML file, some magic strings to describe what attributes
you're interested in, and one or more expectation files to enable checking
whether the resulting accessibility tree is correct or not. In particular,
almost no test code is required.
[More documentation on DumpAccessibilityTree](../../content/test/data/accessibility/readme.md)
Test files:
```
content/test/data/accessibility
```
Test runner:
```
content/browser/accessibility/dump_accessibility_tree_browsertest.cc
```
To run all tests:
```
ninja -C out/release content_browsertests
out/release/content_browsertests --gtest_filter="DumpAccessibilityTree*"
```
## Other content_browsertests
There are many other tests in content/ that relate to accessibility.
All of these tests work by launching a full multi-process browser shell,
loading a web page in a renderer, then accessing the resulting accessibility
tree from the browser process, and running some test from there.
To run all tests:
```
ninja -C out/release content_browsertests
out/release/content_browsertests --gtest_filter="*ccessib*"
```
## Accessibility unittests
This tests the core accessibility code that's shared by both web and non-web
accessibility infrastructure.
Code location:
```
ui/accessibility
```
To run all tests:
```
ninja -C out/release accessibility_unittests
out/release/accessibility_unittests
```
## ChromeVox tests
You must build with ```target_os = "chromeos"``` in your GN args.
To run all tests:
```
ninja -C out/release chromevox_tests
out/release/chromevox_tests --test-launcher-jobs=10
```
## Other locations of accessibility tests:
Even this isn't a complete list. The tests described above cover more
than 90% of the accessibility tests, and the remainder are scattered
throughout the codebase. Here are a few other locations to check:
```
chrome/android/javatests/src/org/chromium/chrome/browser/accessibility
chrome/browser/accessibility
chrome/browser/chromeos/accessibility/
ui/chromeos
ui/views/accessibility
```