Add brief platform documentation for UI Automation
This is a short document targeted at engineers working on accessibility who aren't familiar with UIA. The intent is to identify the most important points to consider when looking at UIA-specific test results and to provide links to deeper documentation on MSDN. This CL also fixes up incorrectly marked-up links I noticed on the Android accessibility documentation page. Bug: None Change-Id: Iccfaf19eeae1659384244ac3f0a27e4596422c1d Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2957347 Reviewed-by: Daniel Libby <dlibby@microsoft.com> Reviewed-by: Dominic Mazzoni <dmazzoni@chromium.org> Commit-Queue: Kevin Babbitt <kbabbitt@microsoft.com> Cr-Commit-Position: refs/heads/master@{#892172}
This commit is contained in:

committed by
Chromium LUCI CQ

parent
331a87ce82
commit
805b7817dc
docs/accessibility
@ -8,16 +8,16 @@ This document covers some of the technical details of how Chrome
|
||||
implements its accessibility support on Android.
|
||||
|
||||
As background reading, you should be familiar with
|
||||
[https://developer.android.com/guide/topics/ui/accessibility](Android Accessibility)
|
||||
[Android Accessibility](https://developer.android.com/guide/topics/ui/accessibility)
|
||||
and in particular
|
||||
[https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo](AccessibilityNodeInfo)
|
||||
[AccessibilityNodeInfo](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo)
|
||||
and
|
||||
[https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeProvider](AccessibilityNodeProvider).
|
||||
[AccessibilityNodeProvider](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeProvider).
|
||||
|
||||
## WebContentsAccessibility
|
||||
|
||||
The main Java class that implements the accessibility protocol in Chrome is
|
||||
[https://cs.chromium.org/chromium/src/content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java](WebContentsAccessibilityImpl.java). It implements the AccessibilityNodeProvider
|
||||
[WebContentsAccessibilityImpl.java](https://cs.chromium.org/chromium/src/content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java). It implements the AccessibilityNodeProvider
|
||||
interface, so a single Android View can be represented by an entire tree
|
||||
of virtual views. Note that WebContentsAccessibilityImpl represents an
|
||||
entire web page, including all frames. The ids in the java code are unique IDs,
|
||||
@ -44,7 +44,7 @@ point. That's called by the Android framework when we need to provide the
|
||||
info about one virtual view (a web node).
|
||||
|
||||
We call into C++ code -
|
||||
[https://cs.chromium.org/chromium/src/content/browser/accessibility/web_contents_accessibility_android.cc](web_contents_accessibility_android.cc) from
|
||||
[web_contents_accessibility_android.cc](https://cs.chromium.org/chromium/src/content/browser/accessibility/web_contents_accessibility_android.cc) from
|
||||
there, because all of the information about the accessibility tree is
|
||||
using the shared C++ BrowserAccessibilityManager code.
|
||||
|
||||
|
@ -215,6 +215,14 @@ Automation Inspector Chrome extension).
|
||||
- macOS: Accessibility Inspector
|
||||
- Windows: Inspect, AViewer, accProbe (and many others)
|
||||
|
||||
### Supported Platforms and APIs
|
||||
|
||||
* Windows: IAccessible (also known as Microsoft Active Accessibility or MSAA),
|
||||
IAccessible2, [UI Automation](uiautomation.md). Chromium also supports
|
||||
[mapping between IAccessible2 and UI Automation nodes](ia2_to_uia.md).
|
||||
* Mac: NSAccessibility
|
||||
* Linux: ATK
|
||||
* Android: [AccessibilityNodeInfo and AccessibilityNodeProvider](android.md)
|
||||
|
||||
## Chromium's multi-process architecture
|
||||
|
||||
|
53
docs/accessibility/uiautomation.md
Normal file
53
docs/accessibility/uiautomation.md
Normal file
@ -0,0 +1,53 @@
|
||||
# UI Automation
|
||||
|
||||
[UI Automation (UIA)](https://docs.microsoft.com/en-us/windows/win32/winauto/entry-uiauto-win32)
|
||||
is the modern accessibility API on Windows.
|
||||
|
||||
## Key Features
|
||||
|
||||
### Clients and Providers
|
||||
|
||||
UI Automation exposes two different sets of interfaces. One is intended for
|
||||
clients such as assistive technologies and automation frameworks. The other is
|
||||
intended for providers such as UI widget frameworks and applications that render
|
||||
their own content. Chromium implements the UI Automation provider APIs.
|
||||
|
||||
Clients and providers do not talk directly to one another. Instead, the
|
||||
operating system gathers data from providers to present a unified tree view
|
||||
across all open applications to the client.
|
||||
|
||||
Further reading:
|
||||
|
||||
* [UI Automation Provider Programmer's Guide](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-providerportal)
|
||||
* [UI Automation Client Programmer's Guide](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-clientportal)
|
||||
|
||||
### Views of the UI Automation tree
|
||||
|
||||
Clients have the ability to filter the UI Automation tree to various subsets of
|
||||
nodes. Tools such as Windows Narrator take advantage of this capability to skip
|
||||
over structural details that might be interesting to an automation framework but
|
||||
aren't relevant to a screen reader.
|
||||
|
||||
Providers can set two properties on a node to determine which views it appears
|
||||
in: IsControlElement and IsContentElement. Getting the value of these properties
|
||||
right is critical to ensuring assistive technologies can get the information
|
||||
they need. Despite the names, there are many cases where nodes that might not be
|
||||
considered a "control" should appear in the Control view of the tree. A good
|
||||
litmus test is: if there's a reason a screen reader might be interested in a
|
||||
node, it should appear in the control view.
|
||||
|
||||
Further reading:
|
||||
|
||||
* [UI Automation Tree Overview](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-treeoverview)
|
||||
|
||||
### TextPattern
|
||||
|
||||
In addition to the tree view, UI Automation exposes a linear reading view
|
||||
through an API known as TextPattern. This API allows a screen reader to navigate
|
||||
text in more natural ways - characters, words, sentences, paragraphs, pages -
|
||||
without worrying about the underlying tree structure. Windows Narrator relies
|
||||
heavily on TextPattern for reading.
|
||||
|
||||
Further reading:
|
||||
|
||||
* [Text and TextRange Control Patterns](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-implementingtextandtextrange)
|
Reference in New Issue
Block a user