[a11y] Move some of serialization pipeline from RAI to cache.
This CL aims to start moving some of the code path starting from HandleAXEvent to ScheduleImmediateAXUpdate over to cache side. The old code is still left in RAI for legacy support. The aim is to eventually delete the code from RAI side, and then RAI would only be called from ProcessDeferredAccessibilityEvents to AXReadyCallback. ---------------------------- Details (only explaining post-lifecyle mode): Before this CL, a11y events (kind of) go like this: AXObjectCacheImpl::PostPlatformNotification -> RenderAccessibilityImpl::HandleAXEvent which also calls AXObjectCacheImpl::AddPendingEvent --> RenderAccessibilityImpl::ScheduleImmediateAXUpdate -> AXObjectCacheImpl::ScheduleAXUpdate --> AXObjectCacheImpl::ProcessDeferredAccessibilityEvents -> RenderAccessibilityImpl::AXReadyCallback Note that -> doesn't mean direct calls. Some calls in the middle may have been skipped. The CL prevents these excessively unnecessary calls in between RAI (RenderAccessibilityImpl) and AXObjectCacheImpl. The calls now go like this (again kinda): AXObjectCacheImpl::PostPlatformNotification -> AXObjectCacheImpl::AddEventToSerializationQueue -> AXObjectCacheImpl::ScheduleImmediateSerialization -> AXObjectCacheImpl::ScheduleAXUpdate -> AXObjectCacheImpl::ProcessDeferredAccessibilityEvents -> RenderAccessibilityImpl::AXReadyCallback Notice how RAI is only called at the very end. Of course, to make this possible many of the side logic had to be moved too like the delaying code that used to be AXReadyCallback is now at AXObjectCacheImpl::ProcessDeferredAccessibilityEvents. This image describes the above: https://i.imgur.com/U3tODTX.png What the CL achieves is moving the code in the blue square over to the left-hand side, thus making the code stay in AXObjectCache longer and only move to RAI at RenderAccessibilityImpl::AXReadyCallback. The serialization delay is moved to ProcessDeferredAccessibilityEvents. ---------------------------- Legacy scheduling mode (as opposed to post-lifecycle serialization) still uses the old code which goes back and forth between RAI and AXObjectCache. The reason they were not moved as well is that legacy mode will get removed eventually anyway. When legacy mode is removed, we can safely remove the old code from RAI. ---------------------------- Also, with the cleaner code, it'll be feasible to change the scheduling logic further to limit some events like scrolling. Scrolling improvements discussed here https://docs.google.com/document/d/1fBK1BBrG8souqZpyv7q_3kxCYF1IYoXNPXmoVL7X8Gk/edit#heading=h.uo8ebdp7cxl ---------------------------- Change-Id: I4662c29ee0185b8554b80b71db5ffb0aa6fc2319 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4994320 Reviewed-by: Peter Beverloo <peter@chromium.org> Auto-Submit: Ahmed Elwasefi (Ahmad45123) <a.m.elwasefi@gmail.com> Reviewed-by: Aaron Leventhal <aleventhal@chromium.org> Reviewed-by: Jeremy Roman <jbroman@chromium.org> Commit-Queue: Ahmed Elwasefi (Ahmad45123) <a.m.elwasefi@gmail.com> Cr-Commit-Position: refs/heads/main@{#1227944}
This commit is contained in:

committed by
Chromium LUCI CQ

parent
b534b3ef0e
commit
b0fb6c7b87
content
renderer
accessibility
web_test
third_party/blink
public
renderer
core
accessibility
modules
web_tests
external
wpt
long-animation-frame
tentative
@ -186,9 +186,11 @@ void RenderAccessibilityImpl::DidCommitProvisionalLoad(
|
||||
ax_image_annotator_->Destroy();
|
||||
ax_image_annotator_.reset();
|
||||
page_language_.clear();
|
||||
serialization_in_flight_ = false;
|
||||
|
||||
// New document has started. Do not expect to receive the ACK for a
|
||||
// serialization sent by the old document.
|
||||
ax_context_->OnSerializationCancelled();
|
||||
weak_factory_for_pending_events_.InvalidateWeakPtrs();
|
||||
weak_factory_for_serialization_pipeline_.InvalidateWeakPtrs();
|
||||
}
|
||||
|
||||
void RenderAccessibilityImpl::AccessibilityModeChanged(const ui::AXMode& mode) {
|
||||
@ -473,6 +475,8 @@ void RenderAccessibilityImpl::Reset(uint32_t reset_token) {
|
||||
FireLoadCompleteIfLoaded();
|
||||
}
|
||||
|
||||
// TODO(accessibility): When legacy mode is deleted, calls to this function may
|
||||
// be replaced with obj.AddDirtyObjectToSerializationQueue
|
||||
void RenderAccessibilityImpl::MarkWebAXObjectDirty(
|
||||
const WebAXObject& obj,
|
||||
bool subtree,
|
||||
@ -483,38 +487,59 @@ void RenderAccessibilityImpl::MarkWebAXObjectDirty(
|
||||
DCHECK(obj.AccessibilityIsIncludedInTree())
|
||||
<< "Cannot serialize unincluded object: " << obj.ToString(true).Utf8();
|
||||
|
||||
obj.MarkAXObjectDirtyWithDetails(subtree, event_from, event_from_action,
|
||||
event_intents);
|
||||
obj.AddDirtyObjectToSerializationQueue(subtree, event_from, event_from_action,
|
||||
event_intents);
|
||||
|
||||
NotifyWebAXObjectMarkedDirty(obj, event_type);
|
||||
// The logic below here is handled now in AXObjectCache and thus only runs in
|
||||
// legacy mode.
|
||||
if (!serialize_post_lifecycle_) {
|
||||
NotifyWebAXObjectMarkedDirty(obj, event_type);
|
||||
}
|
||||
}
|
||||
|
||||
// TODO(accessibility) Move this logic to AXObjectCacheImpl.
|
||||
// TODO(accessibility): Delete once legacy mode is removed.
|
||||
void RenderAccessibilityImpl::NotifyWebAXObjectMarkedDirty(
|
||||
const blink::WebAXObject& obj,
|
||||
ax::mojom::Event event_type) {
|
||||
DCHECK(!serialize_post_lifecycle_);
|
||||
|
||||
// The root is an exception because it often has focus while the page is
|
||||
// loading. In that case the event type is used as the signal (see
|
||||
// HandleAXEvent() and IsImmediateProcessingRequiredForEvent()).
|
||||
if (serialize_post_lifecycle_) {
|
||||
if (obj != ComputeRoot() && obj.IsFocused()) {
|
||||
ScheduleImmediateAXUpdate();
|
||||
}
|
||||
} else {
|
||||
if (legacy_event_schedule_mode_ ==
|
||||
LegacyEventScheduleMode::kProcessEventsImmediately) {
|
||||
return;
|
||||
}
|
||||
if (legacy_event_schedule_mode_ ==
|
||||
LegacyEventScheduleMode::kProcessEventsImmediately) {
|
||||
return;
|
||||
}
|
||||
if (obj != ComputeRoot() && obj.IsFocused()) {
|
||||
legacy_event_schedule_mode_ =
|
||||
LegacyEventScheduleMode::kProcessEventsImmediately;
|
||||
}
|
||||
LegacyScheduleSendPendingAccessibilityEvents();
|
||||
}
|
||||
}
|
||||
|
||||
// TODO(accessibility): Replace all instances of HandleAXEvent with
|
||||
// ax_context_->AddEventToSerializationQueue(event, true);. But we'll need to
|
||||
// make sure to handle the loading_stage_ variable below.
|
||||
void RenderAccessibilityImpl::HandleAXEvent(const ui::AXEvent& event) {
|
||||
DCHECK(ax_context_);
|
||||
|
||||
if (event.event_type == ax::mojom::Event::kLoadStart) {
|
||||
loading_stage_ = LoadingStage::kPreload;
|
||||
} else if (event.event_type == ax::mojom::Event::kLoadComplete) {
|
||||
loading_stage_ = LoadingStage::kLoadCompleted;
|
||||
}
|
||||
|
||||
if (serialize_post_lifecycle_) {
|
||||
ax_context_->AddEventToSerializationQueue(
|
||||
event, true); // All events sent to AXObjectCache from RAI need
|
||||
// immediate serialization!
|
||||
return;
|
||||
}
|
||||
|
||||
// TODO(accessibility): Delete this code when legacy mode is removed.
|
||||
// In lifecycle mode, the below logic is handled in ax_object_cache via
|
||||
// AddEventToSerializationQueue.
|
||||
DCHECK(!serialize_post_lifecycle_);
|
||||
const WebDocument& document = GetMainDocument();
|
||||
DCHECK(!document.IsNull());
|
||||
|
||||
@ -531,40 +556,21 @@ void RenderAccessibilityImpl::HandleAXEvent(const ui::AXEvent& event) {
|
||||
event.event_type);
|
||||
|
||||
if (IsImmediateProcessingRequiredForEvent(event)) {
|
||||
if (serialize_post_lifecycle_) {
|
||||
ScheduleImmediateAXUpdate();
|
||||
} else {
|
||||
legacy_event_schedule_mode_ =
|
||||
LegacyEventScheduleMode::kProcessEventsImmediately;
|
||||
}
|
||||
if (event.event_type == ax::mojom::Event::kLoadStart) {
|
||||
loading_stage_ = LoadingStage::kPreload;
|
||||
} else if (event.event_type == ax::mojom::Event::kLoadComplete) {
|
||||
loading_stage_ = LoadingStage::kLoadCompleted;
|
||||
}
|
||||
legacy_event_schedule_mode_ =
|
||||
LegacyEventScheduleMode::kProcessEventsImmediately;
|
||||
}
|
||||
|
||||
if (!serialize_post_lifecycle_) {
|
||||
LegacyScheduleSendPendingAccessibilityEvents();
|
||||
}
|
||||
LegacyScheduleSendPendingAccessibilityEvents();
|
||||
}
|
||||
|
||||
// TODO(accessibility) Move this and other serialization timing code to
|
||||
// AXObjectCacheImpl, scheduling immediate serialization there when necessary
|
||||
// (and enabling removal of WebDocument::IsLoaded()). When it's moved there, we
|
||||
// can restore the rule that if the focus is marked dirty but there is no event,
|
||||
// we should schedule an immediate serialization.
|
||||
// TODO(accessibility): When legacy mode is deleted, this function can go as
|
||||
// it's only used in RenderAccessibilityImpl::HandleAXEvent legacy mode.
|
||||
bool RenderAccessibilityImpl::IsImmediateProcessingRequiredForEvent(
|
||||
const ui::AXEvent& event) const {
|
||||
if (serialize_post_lifecycle_) {
|
||||
if (last_serialization_timestamp_ == kSerializeAtNextOpportunity) {
|
||||
return true; // Already scheduled for immediate mode.
|
||||
}
|
||||
} else {
|
||||
if (legacy_event_schedule_mode_ ==
|
||||
LegacyEventScheduleMode::kProcessEventsImmediately) {
|
||||
return true; // Already scheduled for immediate mode.
|
||||
}
|
||||
DCHECK(!serialize_post_lifecycle_);
|
||||
if (legacy_event_schedule_mode_ ==
|
||||
LegacyEventScheduleMode::kProcessEventsImmediately) {
|
||||
return true; // Already scheduled for immediate mode.
|
||||
}
|
||||
|
||||
if (event.event_from == ax::mojom::EventFrom::kAction) {
|
||||
@ -645,6 +651,8 @@ bool RenderAccessibilityImpl::IsImmediateProcessingRequiredForEvent(
|
||||
}
|
||||
}
|
||||
|
||||
// TODO(accessibility): When legacy mode is deleted, this function can go. All
|
||||
// deferring is now done in ax_object_cache.
|
||||
int RenderAccessibilityImpl::GetDeferredEventsDelay() {
|
||||
// The amount of time, in milliseconds, to wait before sending non-interactive
|
||||
// events that are deferred before the initial page load.
|
||||
@ -663,21 +671,14 @@ int RenderAccessibilityImpl::GetDeferredEventsDelay() {
|
||||
}
|
||||
|
||||
void RenderAccessibilityImpl::AXReadyCallback() {
|
||||
if (!serialize_post_lifecycle_) {
|
||||
return;
|
||||
}
|
||||
DCHECK(serialize_post_lifecycle_);
|
||||
DCHECK(ax_context_);
|
||||
DCHECK(ax_context_->HasDirtyObjects())
|
||||
<< "Should not call AXReadyCallback() unless there is something to "
|
||||
"serialize.";
|
||||
DCHECK(render_frame_);
|
||||
DCHECK(render_frame_->in_frame_tree());
|
||||
|
||||
if (serialization_in_flight_) {
|
||||
// Another serialization is in flight. When it's finished, a new
|
||||
// serialization will be triggered if necessary.
|
||||
return;
|
||||
}
|
||||
DCHECK(!ax_context_->IsSerializationInFlight());
|
||||
|
||||
// Don't send accessibility events for frames that don't yet have an tree id
|
||||
// as doing so will cause the browser to discard that message and all
|
||||
@ -693,52 +694,15 @@ void RenderAccessibilityImpl::AXReadyCallback() {
|
||||
return;
|
||||
}
|
||||
|
||||
const auto& now = base::Time::Now();
|
||||
const auto& delay_between_serializations =
|
||||
base::Milliseconds(GetDeferredEventsDelay());
|
||||
const auto& elapsed_since_last_serialization =
|
||||
now - last_serialization_timestamp_;
|
||||
const auto& delay_until_next_serialization =
|
||||
delay_between_serializations - elapsed_since_last_serialization;
|
||||
if (delay_until_next_serialization.is_positive()) {
|
||||
// If not loaded yet, ensure that AXReadyCallback() will occur again,
|
||||
// avoiding the possibility that the pipeline will stall with dirty
|
||||
// objects still in it.
|
||||
if (!weak_factory_for_serialization_pipeline_.HasWeakPtrs()) {
|
||||
render_frame_->GetTaskRunner(blink::TaskType::kInternalDefault)
|
||||
->PostDelayedTask(
|
||||
FROM_HERE,
|
||||
base::BindOnce(
|
||||
&RenderAccessibilityImpl::ScheduleImmediateAXUpdate,
|
||||
weak_factory_for_serialization_pipeline_.GetWeakPtr()),
|
||||
delay_until_next_serialization);
|
||||
}
|
||||
return; // No serialization needed yet.
|
||||
}
|
||||
|
||||
// There may be a delayed task queued up that was called to enable batching,
|
||||
// and ensure that the pipeline doesn't stall. However, at this point we will
|
||||
// serialize all current dirty objects, and the pipeline will be activated
|
||||
// again via ProcessDeferredAccessibilityEvents() if there are any changes in
|
||||
// the document.
|
||||
weak_factory_for_serialization_pipeline_.InvalidateWeakPtrs();
|
||||
|
||||
SendPendingAccessibilityEvents();
|
||||
}
|
||||
|
||||
// TODO(accessibility): When legacy mode is deleted, calls to this function may
|
||||
// be replaced with ax_context_->ScheduleImmediateSerialization()
|
||||
void RenderAccessibilityImpl::ScheduleImmediateAXUpdate() {
|
||||
if (serialize_post_lifecycle_) {
|
||||
// This makes sure that we'll serialize at the next available opportunity.
|
||||
last_serialization_timestamp_ = kSerializeAtNextOpportunity;
|
||||
if (serialization_in_flight_) {
|
||||
immediate_update_required_after_ack_ = true;
|
||||
return; // Wait until current serialization message has been received.
|
||||
}
|
||||
|
||||
// Call ScheduleAXUpdate() to ensure lifecycle does not get stalled.
|
||||
// Will call AXReadyCallback() at the next available opportunity.
|
||||
DCHECK(ax_context_);
|
||||
ax_context_->ScheduleAXUpdate();
|
||||
ax_context_->ScheduleImmediateSerialization();
|
||||
} else {
|
||||
// This method is currently only used for RenderAccessibilityImplLegacyTest
|
||||
// tests, which is expected to change in synchronous a11y implementation.
|
||||
@ -822,8 +786,8 @@ void RenderAccessibilityImpl::LegacyScheduleSendPendingAccessibilityEvents(
|
||||
// When no accessibility events are in-flight post a task to send
|
||||
// the events to the browser. We use PostTask so that we can queue
|
||||
// up additional events.
|
||||
DCHECK(!serialization_in_flight_);
|
||||
serialization_in_flight_ = true;
|
||||
DCHECK(!ax_context_->IsSerializationInFlight());
|
||||
ax_context_->OnSerializationStartSend();
|
||||
render_frame_->GetTaskRunner(blink::TaskType::kInternalDefault)
|
||||
->PostDelayedTask(
|
||||
FROM_HERE,
|
||||
@ -1202,8 +1166,7 @@ void RenderAccessibilityImpl::SendPendingAccessibilityEvents() {
|
||||
// scheduling. If serialize_post_lifecycle_ is true, then this method is
|
||||
// called synchronously, but should never be called if there's a previous
|
||||
// serialization still in flight.
|
||||
DCHECK(!serialize_post_lifecycle_ || !serialization_in_flight_);
|
||||
serialization_in_flight_ = true;
|
||||
DCHECK(!serialize_post_lifecycle_ || !ax_context_->IsSerializationInFlight());
|
||||
|
||||
if (!serialize_post_lifecycle_) {
|
||||
// Clear status here in case we return early.
|
||||
@ -1212,7 +1175,6 @@ void RenderAccessibilityImpl::SendPendingAccessibilityEvents() {
|
||||
WebDocument document = GetMainDocument();
|
||||
DCHECK(serialize_post_lifecycle_ || !document.IsNull());
|
||||
if (document.IsNull()) {
|
||||
serialization_in_flight_ = false;
|
||||
return;
|
||||
}
|
||||
|
||||
@ -1224,6 +1186,7 @@ void RenderAccessibilityImpl::SendPendingAccessibilityEvents() {
|
||||
CHECK(document.GetFrame()->GetEmbeddingToken());
|
||||
|
||||
DCHECK(ax_context_);
|
||||
ax_context_->OnSerializationStartSend();
|
||||
|
||||
if (!serialize_post_lifecycle_) {
|
||||
DCHECK(document.IsAccessibilityEnabled())
|
||||
@ -1269,7 +1232,7 @@ void RenderAccessibilityImpl::SendPendingAccessibilityEvents() {
|
||||
// defer so that the batch of events is larger. If any interactive events
|
||||
// come in, the batch will be processed immediately.
|
||||
legacy_event_schedule_mode_ = LegacyEventScheduleMode::kDeferEvents;
|
||||
serialization_in_flight_ = false;
|
||||
ax_context_->OnSerializationCancelled();
|
||||
return;
|
||||
}
|
||||
|
||||
@ -1325,7 +1288,7 @@ void RenderAccessibilityImpl::SendPendingAccessibilityEvents() {
|
||||
DCHECK(updates_and_events->events.empty())
|
||||
<< "If there are no updates, there also shouldn't be any events, "
|
||||
"because events always mark an object dirty.";
|
||||
serialization_in_flight_ = false;
|
||||
ax_context_->OnSerializationCancelled();
|
||||
return;
|
||||
}
|
||||
|
||||
@ -1408,7 +1371,7 @@ void RenderAccessibilityImpl::LegacyOnAccessibilityEventsHandled() {
|
||||
DCHECK(!serialize_post_lifecycle_);
|
||||
DCHECK_EQ(legacy_event_schedule_status_,
|
||||
LegacyEventScheduleStatus::kWaitingForAck);
|
||||
serialization_in_flight_ = false;
|
||||
ax_context_->OnSerializationCancelled();
|
||||
legacy_event_schedule_status_ = LegacyEventScheduleStatus::kNotWaiting;
|
||||
switch (legacy_event_schedule_mode_) {
|
||||
case LegacyEventScheduleMode::kDeferEvents:
|
||||
@ -1421,18 +1384,9 @@ void RenderAccessibilityImpl::LegacyOnAccessibilityEventsHandled() {
|
||||
}
|
||||
|
||||
void RenderAccessibilityImpl::OnSerializationReceived() {
|
||||
// Another serialization may be needed, in the case where the AXObjectCache is
|
||||
// dirty. In that case, make sure a visual update is scheduled so that
|
||||
// AXReadyCallback() will be called. ScheduleAXUpdate() will only schedule a
|
||||
// visual update if the AXObjectCache is dirty.
|
||||
serialization_in_flight_ = false;
|
||||
last_serialization_timestamp_ = base::Time::Now();
|
||||
if (immediate_update_required_after_ack_) {
|
||||
ScheduleImmediateAXUpdate();
|
||||
immediate_update_required_after_ack_ = false;
|
||||
} else {
|
||||
ax_context_->ScheduleAXUpdate();
|
||||
}
|
||||
DCHECK(serialize_post_lifecycle_);
|
||||
DCHECK(ax_context_);
|
||||
ax_context_->OnSerializationReceived();
|
||||
}
|
||||
|
||||
void RenderAccessibilityImpl::OnLoadInlineTextBoxes(
|
||||
@ -1480,7 +1434,7 @@ void RenderAccessibilityImpl::OnGetImageData(const ui::AXActionTarget* target,
|
||||
|
||||
void RenderAccessibilityImpl::LegacyCancelScheduledEvents() {
|
||||
DCHECK(!serialize_post_lifecycle_);
|
||||
serialization_in_flight_ = false;
|
||||
ax_context_->OnSerializationCancelled();
|
||||
switch (legacy_event_schedule_status_) {
|
||||
case LegacyEventScheduleStatus::kScheduledDeferred:
|
||||
case LegacyEventScheduleStatus::kScheduledImmediate: // Fallthrough
|
||||
@ -1670,7 +1624,7 @@ void RenderAccessibilityImpl::ConnectionClosed() {
|
||||
if (!serialize_post_lifecycle_) {
|
||||
legacy_event_schedule_status_ = LegacyEventScheduleStatus::kNotWaiting;
|
||||
}
|
||||
serialization_in_flight_ = false;
|
||||
ax_context_->OnSerializationCancelled();
|
||||
}
|
||||
|
||||
void RenderAccessibilityImpl::RecordInaccessiblePdfUkm() {
|
||||
|
@ -328,19 +328,6 @@ class CONTENT_EXPORT RenderAccessibilityImpl : public RenderAccessibility,
|
||||
};
|
||||
LoadingStage loading_stage_ = LoadingStage::kPreload;
|
||||
|
||||
// This stores the last time a serialization was ACK'ed after being sent to
|
||||
// the browser, so that serializations can be skipped if the time since the
|
||||
// last serialization is less than GetDeferredEventsDelay(). Setting to
|
||||
// "beginning of time" causes the upcoming serialization to occur at the next
|
||||
// available opportunity. Batching is used to reduce the number of
|
||||
// serializations, in order to provide overall faster content updates while
|
||||
// using less CPU, because nodes that change multiple times in a short time
|
||||
// period only need to be serialized once, e.g. during page loads or
|
||||
// animations.
|
||||
static constexpr base::Time kSerializeAtNextOpportunity =
|
||||
base::Time::UnixEpoch();
|
||||
base::Time last_serialization_timestamp_ = kSerializeAtNextOpportunity;
|
||||
|
||||
// The amount of time since the last UKM upload.
|
||||
std::unique_ptr<base::ElapsedTimer> ukm_timer_;
|
||||
|
||||
@ -363,16 +350,6 @@ class CONTENT_EXPORT RenderAccessibilityImpl : public RenderAccessibility,
|
||||
// A set of IDs for which we should always load inline text boxes.
|
||||
std::set<int32_t> load_inline_text_boxes_ids_;
|
||||
|
||||
// This will flip to true when we initiate the process of sending AX data to
|
||||
// the browser, and will flip back to false once we receive back an ACK.
|
||||
bool serialization_in_flight_ = false;
|
||||
|
||||
// This flips to true if a request for an immediate update was not honored
|
||||
// because serialization_in_flight_ was true. It flips back to false once
|
||||
// serialization_in_flight_ has flipped to false and an immediate update has
|
||||
// been requested.
|
||||
bool immediate_update_required_after_ack_ = false;
|
||||
|
||||
// Controls whether serialization should be run synchronously at the end of a
|
||||
// main frame update, or scheduled as an asynchronous task.
|
||||
bool serialize_post_lifecycle_;
|
||||
@ -389,11 +366,6 @@ class CONTENT_EXPORT RenderAccessibilityImpl : public RenderAccessibility,
|
||||
LegacyEventScheduleMode legacy_event_schedule_mode_ =
|
||||
LegacyEventScheduleMode::kDeferEvents;
|
||||
|
||||
// So we can ensure the serialization pipeline never stalls with dirty objects
|
||||
// remaining to be serialized.
|
||||
base::WeakPtrFactory<RenderAccessibilityImpl>
|
||||
weak_factory_for_serialization_pipeline_{this};
|
||||
|
||||
// So we can queue up tasks to be executed later.
|
||||
base::WeakPtrFactory<RenderAccessibilityImpl>
|
||||
weak_factory_for_pending_events_{this};
|
||||
|
@ -1301,7 +1301,7 @@ TEST_F(AXImageAnnotatorTest, MAYBE_OnImageAdded) {
|
||||
// Show node "B".
|
||||
ExecuteJavaScriptForTests(
|
||||
"document.getElementById('B').style.visibility = 'visible';");
|
||||
GetRenderAccessibilityImpl()->GetAXContext()->UpdateAXForAllDocuments();
|
||||
SendPendingAccessibilityEvents();
|
||||
ClearHandledUpdates();
|
||||
|
||||
// This should update the annotations of all images on the page, including the
|
||||
@ -1309,7 +1309,6 @@ TEST_F(AXImageAnnotatorTest, MAYBE_OnImageAdded) {
|
||||
GetRenderAccessibilityImpl()->MarkWebAXObjectDirty(root_obj,
|
||||
true /* subtree */);
|
||||
SendPendingAccessibilityEvents();
|
||||
task_environment_.RunUntilIdle();
|
||||
|
||||
EXPECT_THAT(mock_annotator().image_ids_,
|
||||
ElementsAre("test1.jpg", "test2.jpg", "test1.jpg", "test2.jpg"));
|
||||
@ -1348,7 +1347,6 @@ TEST_F(AXImageAnnotatorTest, OnImageUpdated) {
|
||||
GetRenderAccessibilityImpl()->MarkWebAXObjectDirty(root_obj,
|
||||
true /* subtree */);
|
||||
SendPendingAccessibilityEvents();
|
||||
task_environment_.RunUntilIdle();
|
||||
|
||||
EXPECT_THAT(mock_annotator().image_ids_,
|
||||
ElementsAre("test1.jpg", "test1.jpg"));
|
||||
@ -1359,7 +1357,7 @@ TEST_F(AXImageAnnotatorTest, OnImageUpdated) {
|
||||
|
||||
// Update node "A".
|
||||
ExecuteJavaScriptForTests("document.querySelector('img').src = 'test2.jpg';");
|
||||
GetRenderAccessibilityImpl()->GetAXContext()->UpdateAXForAllDocuments();
|
||||
SendPendingAccessibilityEvents();
|
||||
|
||||
ClearHandledUpdates();
|
||||
// This should update the annotations of all images on the page, including the
|
||||
@ -1367,7 +1365,6 @@ TEST_F(AXImageAnnotatorTest, OnImageUpdated) {
|
||||
GetRenderAccessibilityImpl()->MarkWebAXObjectDirty(root_obj,
|
||||
true /* subtree */);
|
||||
SendPendingAccessibilityEvents();
|
||||
task_environment_.RunUntilIdle();
|
||||
|
||||
EXPECT_THAT(mock_annotator().image_ids_,
|
||||
ElementsAre("test1.jpg", "test1.jpg", "test2.jpg"));
|
||||
|
@ -547,6 +547,61 @@ void WebFrameTestProxy::BeginNavigation(
|
||||
}
|
||||
|
||||
void WebFrameTestProxy::PostAccessibilityEvent(const ui::AXEvent& event) {
|
||||
HandleWebAccessibilityEventForTest(event);
|
||||
RenderFrameImpl::PostAccessibilityEvent(event);
|
||||
}
|
||||
|
||||
void WebFrameTestProxy::NotifyWebAXObjectMarkedDirty(
|
||||
const blink::WebAXObject& object) {
|
||||
HandleWebAccessibilityEventForTest(object, "MarkDirty",
|
||||
std::vector<ui::AXEventIntent>());
|
||||
|
||||
// Guard against the case where |this| was deleted as a result of an
|
||||
// accessibility listener detaching a frame. If that occurs, the
|
||||
// WebAXObject will be detached.
|
||||
if (object.IsDetached()) {
|
||||
return; // |this| is invalid.
|
||||
}
|
||||
|
||||
RenderFrameImpl::NotifyWebAXObjectMarkedDirty(object);
|
||||
}
|
||||
|
||||
void WebFrameTestProxy::HandleWebAccessibilityEventForTest(
|
||||
const blink::WebAXObject& object,
|
||||
const char* event_name,
|
||||
const std::vector<ui::AXEventIntent>& event_intents) {
|
||||
// Only hook the accessibility events that occurred during the test run.
|
||||
// This check prevents false positives in BlinkLeakDetector.
|
||||
// The pending tasks in browser/renderer message queue may trigger
|
||||
// accessibility events,
|
||||
// and AccessibilityController will hold on to their target nodes if we don't
|
||||
// ignore them here.
|
||||
if (!test_runner()->TestIsRunning()) {
|
||||
return;
|
||||
}
|
||||
|
||||
accessibility_controller_.NotificationReceived(GetWebFrame(), object,
|
||||
event_name, event_intents);
|
||||
|
||||
if (accessibility_controller_.ShouldLogAccessibilityEvents()) {
|
||||
std::string message("AccessibilityNotification - ");
|
||||
message += event_name;
|
||||
|
||||
blink::WebNode node = object.GetNode();
|
||||
if (!node.IsNull() && node.IsElementNode()) {
|
||||
blink::WebElement element = node.To<blink::WebElement>();
|
||||
if (element.HasAttribute("id")) {
|
||||
message += " - id:";
|
||||
message += element.GetAttribute("id").Utf8().data();
|
||||
}
|
||||
}
|
||||
|
||||
test_runner()->PrintMessage(message + "\n");
|
||||
}
|
||||
}
|
||||
|
||||
void WebFrameTestProxy::HandleWebAccessibilityEventForTest(
|
||||
const ui::AXEvent& event) {
|
||||
const char* event_name = nullptr;
|
||||
switch (event.event_type) {
|
||||
case ax::mojom::Event::kActiveDescendantChanged:
|
||||
@ -667,57 +722,8 @@ void WebFrameTestProxy::PostAccessibilityEvent(const ui::AXEvent& event) {
|
||||
|
||||
blink::WebDocument document = GetWebFrame()->GetDocument();
|
||||
auto object = blink::WebAXObject::FromWebDocumentByID(document, event.id);
|
||||
HandleWebAccessibilityEvent(std::move(object), event_name,
|
||||
event.event_intents);
|
||||
|
||||
RenderFrameImpl::PostAccessibilityEvent(event);
|
||||
}
|
||||
|
||||
void WebFrameTestProxy::NotifyWebAXObjectMarkedDirty(
|
||||
const blink::WebAXObject& object) {
|
||||
HandleWebAccessibilityEvent(object, "MarkDirty",
|
||||
std::vector<ui::AXEventIntent>());
|
||||
|
||||
// Guard against the case where |this| was deleted as a result of an
|
||||
// accessibility listener detaching a frame. If that occurs, the
|
||||
// WebAXObject will be detached.
|
||||
if (object.IsDetached())
|
||||
return; // |this| is invalid.
|
||||
|
||||
RenderFrameImpl::NotifyWebAXObjectMarkedDirty(object);
|
||||
}
|
||||
|
||||
void WebFrameTestProxy::HandleWebAccessibilityEvent(
|
||||
const blink::WebAXObject& object,
|
||||
const char* event_name,
|
||||
const std::vector<ui::AXEventIntent>& event_intents) {
|
||||
// Only hook the accessibility events that occurred during the test run.
|
||||
// This check prevents false positives in BlinkLeakDetector.
|
||||
// The pending tasks in browser/renderer message queue may trigger
|
||||
// accessibility events,
|
||||
// and AccessibilityController will hold on to their target nodes if we don't
|
||||
// ignore them here.
|
||||
if (!test_runner()->TestIsRunning())
|
||||
return;
|
||||
|
||||
accessibility_controller_.NotificationReceived(GetWebFrame(), object,
|
||||
event_name, event_intents);
|
||||
|
||||
if (accessibility_controller_.ShouldLogAccessibilityEvents()) {
|
||||
std::string message("AccessibilityNotification - ");
|
||||
message += event_name;
|
||||
|
||||
blink::WebNode node = object.GetNode();
|
||||
if (!node.IsNull() && node.IsElementNode()) {
|
||||
blink::WebElement element = node.To<blink::WebElement>();
|
||||
if (element.HasAttribute("id")) {
|
||||
message += " - id:";
|
||||
message += element.GetAttribute("id").Utf8().data();
|
||||
}
|
||||
}
|
||||
|
||||
test_runner()->PrintMessage(message + "\n");
|
||||
}
|
||||
HandleWebAccessibilityEventForTest(std::move(object), event_name,
|
||||
event.event_intents);
|
||||
}
|
||||
|
||||
void WebFrameTestProxy::CheckIfAudioSinkExistsAndIsAuthorized(
|
||||
|
@ -90,6 +90,11 @@ class WebFrameTestProxy : public RenderFrameImpl,
|
||||
bool should_reset_browser_interface_broker,
|
||||
const blink::ParsedPermissionsPolicy& permissions_policy_header,
|
||||
const blink::DocumentPolicyFeatureState& document_policy_header) override;
|
||||
void HandleWebAccessibilityEventForTest(
|
||||
const blink::WebAXObject& object,
|
||||
const char* event_name,
|
||||
const std::vector<ui::AXEventIntent>& event_intents) override;
|
||||
void HandleWebAccessibilityEventForTest(const ui::AXEvent& event) override;
|
||||
|
||||
// mojom::WebTestRenderFrame implementation.
|
||||
void SynchronouslyCompositeAfterTest(
|
||||
@ -106,11 +111,6 @@ class WebFrameTestProxy : public RenderFrameImpl,
|
||||
void BindReceiver(
|
||||
mojo::PendingAssociatedReceiver<mojom::WebTestRenderFrame> receiver);
|
||||
|
||||
void HandleWebAccessibilityEvent(
|
||||
const blink::WebAXObject& object,
|
||||
const char* event_name,
|
||||
const std::vector<ui::AXEventIntent>& event_intents);
|
||||
|
||||
TestRunner* test_runner();
|
||||
|
||||
const raw_ptr<TestRunner, ExperimentalRenderer> test_runner_;
|
||||
|
42
third_party/blink/public/web/web_ax_context.h
vendored
42
third_party/blink/public/web/web_ax_context.h
vendored
@ -98,12 +98,48 @@ class BLINK_EXPORT WebAXContext {
|
||||
// popup document. Ensures layout is clean as well.
|
||||
void UpdateAXForAllDocuments();
|
||||
|
||||
// Ensure that a layout and accessibility update will occur soon.
|
||||
void ScheduleAXUpdate();
|
||||
|
||||
// If the document is loaded, fire a load complete event.
|
||||
void FireLoadCompleteIfLoaded();
|
||||
|
||||
// Ensures that a serialization of all pending events and dirty objects is
|
||||
// sent to the client as soon as possible at the next lifecycle update.
|
||||
// Technically, ensures that a call to
|
||||
// RenderAccessibilityImpl::AXReadyCallback() will occur as soon as possible.
|
||||
void ScheduleImmediateSerialization();
|
||||
|
||||
// Add an event to the queue of events to be processed as well as mark the
|
||||
// AXObject dirty. If immediate_serialization is set, it schedules a
|
||||
// serialization to be done at the next lifecycle update without delays.
|
||||
void AddEventToSerializationQueue(const ui::AXEvent& event,
|
||||
bool immediate_serialization);
|
||||
|
||||
// Inform AXObjectCacheImpl that the last serialization was received by the
|
||||
// browser successfully.
|
||||
void OnSerializationReceived();
|
||||
|
||||
// Inform AXObjectCacheImpl that a serialization was cancelled. It's only
|
||||
// required for legacy, non-lifecycle mode. Check IsSerializationInFlight
|
||||
// details.
|
||||
// TODO(accessibility): This method can eventually be moved to AXObjectCache
|
||||
// when legacy mode is removed.
|
||||
void OnSerializationCancelled();
|
||||
|
||||
// Inform AXObjectCacheImpl that a serialization just started to be sent to
|
||||
// the browser. Check IsSerializationInFlight details.
|
||||
// TODO(accessibility): This method can eventually be moved to AXObjectCache
|
||||
// when legacy mode is removed.
|
||||
void OnSerializationStartSend();
|
||||
|
||||
// Determine if a serialization is in progress or not. Sometimes when
|
||||
// serializing the events, more events are generated and a new lifecycle
|
||||
// update occurs. Without this variable, we could end up in an infinite loop
|
||||
// of sending updates so we keep track when an update is in progress and avoid
|
||||
// starting any new updates while it's true. It becomes false again when the
|
||||
// update reaches the browser via a call to OnSerializationReceived().
|
||||
// TODO(accessibility): Again, this method is here only for legacy mode and
|
||||
// would be moved to AXObjectCache later on.
|
||||
bool IsSerializationInFlight() const;
|
||||
|
||||
private:
|
||||
std::unique_ptr<AXContext> private_;
|
||||
};
|
||||
|
2
third_party/blink/public/web/web_ax_object.h
vendored
2
third_party/blink/public/web/web_ax_object.h
vendored
@ -272,7 +272,7 @@ class BLINK_EXPORT WebAXObject {
|
||||
|
||||
// Marks ths object as dirty (needing serialization). If subtree is true,
|
||||
// the entire AX subtree should be invalidated as well.
|
||||
void MarkAXObjectDirtyWithDetails(
|
||||
void AddDirtyObjectToSerializationQueue(
|
||||
bool subtree,
|
||||
ax::mojom::EventFrom event_from,
|
||||
ax::mojom::Action event_from_action,
|
||||
|
@ -808,6 +808,21 @@ class BLINK_EXPORT WebLocalFrameClient {
|
||||
// a window.print() call.
|
||||
virtual void ScriptedPrint() {}
|
||||
|
||||
// This method is ONLY for web tests and is not supposed to be overridden in
|
||||
// classes other than web_frame_test_proxy. It's called from accessibility and
|
||||
// is used as a way to tunnel events to the accessibility_controller in web
|
||||
// tests.
|
||||
virtual void HandleWebAccessibilityEventForTest(
|
||||
const blink::WebAXObject& object,
|
||||
const char* event_name,
|
||||
const std::vector<ui::AXEventIntent>& event_intents) {}
|
||||
|
||||
// This method is ONLY for web tests and is not supposed to be overridden in
|
||||
// classes other than web_frame_test_proxy. It's called from accessibility and
|
||||
// is used as a way to tunnel events to the accessibility_controller in web
|
||||
// tests.
|
||||
virtual void HandleWebAccessibilityEventForTest(const ui::AXEvent& event) {}
|
||||
|
||||
// Create a new related WebView. This method must clone its session storage
|
||||
// so any subsequent calls to createSessionStorageNamespace conform to the
|
||||
// WebStorage specification.
|
||||
|
@ -236,9 +236,9 @@ class CORE_EXPORT AXObjectCache : public GarbageCollected<AXObjectCache> {
|
||||
// page occurs, such as an inertness change or a fullscreen toggle.
|
||||
// This keeps the existing nodes, but recomputes all of their properties and
|
||||
// reserializes everything.
|
||||
// Compared with ResetSerializer() and MarkAXObjectDirtyWithDetails() with
|
||||
// subtree = true, this does more work, because it recomputes the entire tree
|
||||
// structure and properties of each node.
|
||||
// Compared with ResetSerializer() and AddDirtyObjectToSerializationQueue()
|
||||
// with subtree = true, this does more work, because it recomputes the entire
|
||||
// tree structure and properties of each node.
|
||||
virtual void MarkDocumentDirty() = 0;
|
||||
|
||||
// Compared with MarkDocumentDirty(), this does less work, because it assumes
|
||||
@ -254,7 +254,7 @@ class CORE_EXPORT AXObjectCache : public GarbageCollected<AXObjectCache> {
|
||||
// |event_from| and |event_from_action| annotate this node change with info
|
||||
// about the event which caused the change. For example, an event from a user
|
||||
// or an event from a focus action.
|
||||
virtual void MarkAXObjectDirtyWithDetails(
|
||||
virtual void AddDirtyObjectToSerializationQueue(
|
||||
AXObject* obj,
|
||||
bool subtree,
|
||||
ax::mojom::blink::EventFrom event_from,
|
||||
@ -286,6 +286,28 @@ class CORE_EXPORT AXObjectCache : public GarbageCollected<AXObjectCache> {
|
||||
// Ensure that a call to ProcessDeferredAccessibilityEvents() will occur soon.
|
||||
virtual void ScheduleAXUpdate() const = 0;
|
||||
|
||||
// Ensure that a call to RenderAccessibilityImpl::AXReadyCallback() will occur
|
||||
// as soon as possible.
|
||||
virtual void ScheduleImmediateSerialization() = 0;
|
||||
|
||||
// Add an event to the queue of events to be processed as well as mark as
|
||||
// dirty if needed.
|
||||
virtual void AddEventToSerializationQueue(const ui::AXEvent& event,
|
||||
bool immediate_serialization) = 0;
|
||||
|
||||
// Called from browser to RAI and then to AXCache to notify that a
|
||||
// serialization has arrived to Browser.
|
||||
virtual void OnSerializationReceived() = 0;
|
||||
|
||||
// Inform AXObjectCacheImpl that a serialization was cancelled.
|
||||
virtual void OnSerializationCancelled() = 0;
|
||||
|
||||
// Inform AXObjectCacheImpl that a serialization was sent.
|
||||
virtual void OnSerializationStartSend() = 0;
|
||||
|
||||
// Determine if a serialization is in the process or not.
|
||||
virtual bool IsSerializationInFlight() const = 0;
|
||||
|
||||
protected:
|
||||
friend class ScopedBlinkAXEventIntent;
|
||||
FRIEND_TEST_ALL_PREFIXES(ScopedBlinkAXEventIntentTest, SingleIntent);
|
||||
|
@ -4352,7 +4352,7 @@ void AXNodeObject::LoadInlineTextBoxesHelper() {
|
||||
// If inline text box children were added, mark the node dirty so that the
|
||||
// results are serialized.
|
||||
if (!CachedChildrenIncludingIgnored().empty()) {
|
||||
AXObjectCache().MarkAXObjectDirtyWithDetails(
|
||||
AXObjectCache().AddDirtyObjectToSerializationQueue(
|
||||
this, /*subtree*/ false, ax::mojom::blink::EventFrom::kNone,
|
||||
ax::mojom::blink::Action::kNone, {});
|
||||
}
|
||||
|
@ -110,6 +110,7 @@
|
||||
#include "third_party/blink/renderer/modules/accessibility/ax_virtual_object.h"
|
||||
#include "third_party/blink/renderer/modules/permissions/permission_utils.h"
|
||||
#include "third_party/blink/renderer/platform/instrumentation/tracing/trace_event.h"
|
||||
#include "third_party/blink/renderer/platform/wtf/functional.h"
|
||||
#include "ui/accessibility/ax_common.h"
|
||||
#include "ui/accessibility/ax_enums.mojom-blink.h"
|
||||
#include "ui/accessibility/ax_event.h"
|
||||
@ -2935,6 +2936,21 @@ void AXObjectCacheImpl::CheckTreeIsUpdated() {
|
||||
#endif
|
||||
}
|
||||
|
||||
int AXObjectCacheImpl::GetDeferredEventsDelay() const {
|
||||
// The amount of time, in milliseconds, to wait before sending non-interactive
|
||||
// events that are deferred before the initial page load.
|
||||
constexpr int kDelayForDeferredUpdatesBeforePageLoad = 350;
|
||||
|
||||
// The amount of time, in milliseconds, to wait before sending non-interactive
|
||||
// events that are deferred after the initial page load.
|
||||
// Shync with same constant in CrossPlatformAccessibilityBrowserTest.
|
||||
constexpr int kDelayForDeferredUpdatesAfterPageLoad = 150;
|
||||
|
||||
return GetDocument().IsLoadCompleted()
|
||||
? kDelayForDeferredUpdatesAfterPageLoad
|
||||
: kDelayForDeferredUpdatesBeforePageLoad;
|
||||
}
|
||||
|
||||
void AXObjectCacheImpl::ProcessDeferredAccessibilityEvents(Document& document) {
|
||||
if (IsPopup(document)) {
|
||||
// Only process popup document together with main document.
|
||||
@ -3022,6 +3038,36 @@ void AXObjectCacheImpl::ProcessDeferredAccessibilityEvents(Document& document) {
|
||||
UpdateTreeIfNeeded();
|
||||
}
|
||||
|
||||
if (serialize_post_lifecycle_) {
|
||||
if (IsSerializationInFlight()) {
|
||||
// Another serialization is in flight. When it's finished, a new
|
||||
// serialization will be triggered if necessary.
|
||||
return;
|
||||
}
|
||||
|
||||
const auto& now = base::Time::Now();
|
||||
const auto& delay_between_serializations =
|
||||
base::Milliseconds(GetDeferredEventsDelay());
|
||||
const auto& elapsed_since_last_serialization =
|
||||
now - last_serialization_timestamp_;
|
||||
const auto& delay_until_next_serialization =
|
||||
delay_between_serializations - elapsed_since_last_serialization;
|
||||
if (delay_until_next_serialization.is_positive()) {
|
||||
if (!weak_factory_for_serialization_pipeline_.HasWeakPtrs()) {
|
||||
document.GetTaskRunner(blink::TaskType::kInternalDefault)
|
||||
->PostDelayedTask(
|
||||
FROM_HERE,
|
||||
WTF::BindOnce(
|
||||
&AXObjectCacheImpl::ScheduleAXUpdate,
|
||||
weak_factory_for_serialization_pipeline_.GetWeakPtr()),
|
||||
delay_until_next_serialization);
|
||||
}
|
||||
return; // No serialization needed yet.
|
||||
}
|
||||
|
||||
weak_factory_for_serialization_pipeline_.InvalidateWeakPtrs();
|
||||
}
|
||||
|
||||
// ------------------------ Freeze and serialize ---------------------------
|
||||
{
|
||||
// The frozen state begins immediately after processing deferred events.
|
||||
@ -3032,7 +3078,7 @@ void AXObjectCacheImpl::ProcessDeferredAccessibilityEvents(Document& document) {
|
||||
// TODO(accessibility) It's a bit confusing that this can be true when the
|
||||
// IsDirty() is false, but this is the case for objects marked dirty from
|
||||
// RenderAccessibilityImpl, e.g. for the kEndOfTest event.
|
||||
if (HasDirtyObjects()) {
|
||||
if (serialize_post_lifecycle_ && HasDirtyObjects()) {
|
||||
if (auto* client = GetWebLocalFrameClient()) {
|
||||
client->AXReadyCallback();
|
||||
}
|
||||
@ -4245,6 +4291,165 @@ WebLocalFrameClient* AXObjectCacheImpl::GetWebLocalFrameClient() const {
|
||||
return client;
|
||||
}
|
||||
|
||||
bool AXObjectCacheImpl::IsImmediateProcessingRequiredForEvent(
|
||||
const ui::AXEvent& event) const {
|
||||
DCHECK(serialize_post_lifecycle_);
|
||||
if (last_serialization_timestamp_ == kSerializeAtNextOpportunity) {
|
||||
return true; // Already scheduled for immediate mode.
|
||||
}
|
||||
|
||||
if (event.event_from == ax::mojom::blink::EventFrom::kAction) {
|
||||
return true; // Actions should result in an immediate response.
|
||||
}
|
||||
|
||||
switch (event.event_type) {
|
||||
case ax::mojom::blink::Event::kActiveDescendantChanged:
|
||||
case ax::mojom::blink::Event::kBlur:
|
||||
case ax::mojom::blink::Event::kCheckedStateChanged:
|
||||
case ax::mojom::blink::Event::kClicked:
|
||||
case ax::mojom::blink::Event::kDocumentSelectionChanged:
|
||||
case ax::mojom::blink::Event::kFocus:
|
||||
case ax::mojom::blink::Event::kHover:
|
||||
case ax::mojom::blink::Event::kLoadComplete:
|
||||
case ax::mojom::blink::Event::kLoadStart:
|
||||
case ax::mojom::blink::Event::kValueChanged:
|
||||
return true;
|
||||
|
||||
case ax::mojom::blink::Event::kDocumentTitleChanged:
|
||||
case ax::mojom::blink::Event::kExpandedChanged:
|
||||
case ax::mojom::blink::Event::kHide:
|
||||
case ax::mojom::blink::Event::kLayoutComplete:
|
||||
case ax::mojom::blink::Event::kLocationChanged:
|
||||
case ax::mojom::blink::Event::kMenuListValueChanged:
|
||||
case ax::mojom::blink::Event::kRowCollapsed:
|
||||
case ax::mojom::blink::Event::kRowCountChanged:
|
||||
case ax::mojom::blink::Event::kRowExpanded:
|
||||
case ax::mojom::blink::Event::kScrollPositionChanged:
|
||||
case ax::mojom::blink::Event::kScrolledToAnchor:
|
||||
case ax::mojom::blink::Event::kSelectedChildrenChanged:
|
||||
case ax::mojom::blink::Event::kShow:
|
||||
case ax::mojom::blink::Event::kTextChanged:
|
||||
return false;
|
||||
|
||||
// These events are not fired from Blink.
|
||||
// This list is duplicated in WebFrameTestProxy::PostAccessibilityEvent().
|
||||
case ax::mojom::blink::Event::kAlert:
|
||||
case ax::mojom::blink::Event::kAriaAttributeChangedDeprecated:
|
||||
case ax::mojom::blink::Event::kAutocorrectionOccured:
|
||||
case ax::mojom::blink::Event::kChildrenChanged:
|
||||
case ax::mojom::blink::Event::kControlsChanged:
|
||||
case ax::mojom::blink::Event::kEndOfTest:
|
||||
case ax::mojom::blink::Event::kFocusAfterMenuClose:
|
||||
case ax::mojom::blink::Event::kFocusContext:
|
||||
case ax::mojom::blink::Event::kHitTestResult:
|
||||
case ax::mojom::blink::Event::kImageFrameUpdated:
|
||||
case ax::mojom::blink::Event::kLiveRegionCreated:
|
||||
case ax::mojom::blink::Event::kLiveRegionChanged:
|
||||
case ax::mojom::blink::Event::kMediaStartedPlaying:
|
||||
case ax::mojom::blink::Event::kMediaStoppedPlaying:
|
||||
case ax::mojom::blink::Event::kMenuEnd:
|
||||
case ax::mojom::blink::Event::kMenuPopupEnd:
|
||||
case ax::mojom::blink::Event::kMenuPopupStart:
|
||||
case ax::mojom::blink::Event::kMenuStart:
|
||||
case ax::mojom::blink::Event::kMouseCanceled:
|
||||
case ax::mojom::blink::Event::kMouseDragged:
|
||||
case ax::mojom::blink::Event::kMouseMoved:
|
||||
case ax::mojom::blink::Event::kMousePressed:
|
||||
case ax::mojom::blink::Event::kMouseReleased:
|
||||
case ax::mojom::blink::Event::kNone:
|
||||
case ax::mojom::blink::Event::kSelection:
|
||||
case ax::mojom::blink::Event::kSelectionAdd:
|
||||
case ax::mojom::blink::Event::kSelectionRemove:
|
||||
case ax::mojom::blink::Event::kStateChanged:
|
||||
case ax::mojom::blink::Event::kTextSelectionChanged:
|
||||
case ax::mojom::blink::Event::kTooltipClosed:
|
||||
case ax::mojom::blink::Event::kTooltipOpened:
|
||||
case ax::mojom::blink::Event::kTreeChanged:
|
||||
case ax::mojom::blink::Event::kWindowActivated:
|
||||
case ax::mojom::blink::Event::kWindowDeactivated:
|
||||
case ax::mojom::blink::Event::kWindowVisibilityChanged:
|
||||
// Never fired from Blink.
|
||||
NOTREACHED() << "Event not expected from Blink: " << event.event_type;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// The lifecycle serialization works as follows:
|
||||
// 1) Dirty objects and events are fired through
|
||||
// AXObjectCacheImpl::PostPlatformNotification which in turn makes a call to
|
||||
// AXObjectCacheImpl::AddEventToSerializationQueue to queue it.
|
||||
//
|
||||
// 2) When the lifecycle is ready to be serialized,
|
||||
// AXObjectCacheImpl::ProcessDeferredAccessibilityEvents is called which first
|
||||
// checks if it's time to make a new serialization, and if not, it will early
|
||||
// return in order to add a delay between serializations.
|
||||
//
|
||||
// 3) AXObjectCacheImpl::ProcessDeferredAccessibilityEvents then calls
|
||||
// RenderAccessibilityImpl:AXReadyCallback to start serialization process.
|
||||
//
|
||||
// Check the below CL for more information:
|
||||
// https://chromium-review.googlesource.com/c/chromium/src/+/4994320
|
||||
void AXObjectCacheImpl::AddEventToSerializationQueue(
|
||||
const ui::AXEvent& event,
|
||||
bool immediate_serialization) {
|
||||
AXObject* obj = ObjectFromAXID(event.id);
|
||||
DCHECK(!obj->IsDetached());
|
||||
|
||||
pending_events_.push_back(event);
|
||||
|
||||
AddDirtyObjectToSerializationQueue(obj, false, event.event_from,
|
||||
event.event_from_action,
|
||||
event.event_intents);
|
||||
|
||||
if (immediate_serialization) {
|
||||
ScheduleImmediateSerialization();
|
||||
}
|
||||
}
|
||||
|
||||
void AXObjectCacheImpl::OnSerializationCancelled() {
|
||||
serialization_in_flight_ = false;
|
||||
}
|
||||
|
||||
void AXObjectCacheImpl::OnSerializationStartSend() {
|
||||
serialization_in_flight_ = true;
|
||||
}
|
||||
|
||||
bool AXObjectCacheImpl::IsSerializationInFlight() const {
|
||||
return serialization_in_flight_;
|
||||
}
|
||||
|
||||
void AXObjectCacheImpl::OnSerializationReceived() {
|
||||
serialization_in_flight_ = false;
|
||||
last_serialization_timestamp_ = base::Time::Now();
|
||||
|
||||
// Another serialization may be needed, in the case where the AXObjectCache is
|
||||
// dirty. In that case, make sure a visual update is scheduled so that
|
||||
// AXReadyCallback() will be called. ScheduleAXUpdate() will only schedule a
|
||||
// visual update if the AXObjectCache is dirty.
|
||||
if (serialize_immediately_after_current_serialization_) {
|
||||
serialize_immediately_after_current_serialization_ = false;
|
||||
ScheduleImmediateSerialization();
|
||||
} else {
|
||||
ScheduleAXUpdate();
|
||||
}
|
||||
}
|
||||
|
||||
void AXObjectCacheImpl::ScheduleImmediateSerialization() {
|
||||
DCHECK(serialize_post_lifecycle_);
|
||||
|
||||
// This makes sure that we'll serialize at the next available opportunity.
|
||||
last_serialization_timestamp_ = kSerializeAtNextOpportunity;
|
||||
|
||||
if (IsSerializationInFlight()) {
|
||||
serialize_immediately_after_current_serialization_ = true;
|
||||
return; // Wait until current serialization message has been received.
|
||||
}
|
||||
|
||||
// Call ScheduleAXUpdate() to ensure lifecycle does not get stalled.
|
||||
// Will call AXReadyCallback() at the next available opportunity.
|
||||
ScheduleAXUpdate();
|
||||
}
|
||||
|
||||
void AXObjectCacheImpl::PostPlatformNotification(
|
||||
AXObject* obj,
|
||||
ax::mojom::blink::Event event_type,
|
||||
@ -4268,15 +4473,21 @@ void AXObjectCacheImpl::PostPlatformNotification(
|
||||
for (auto agent : agents_)
|
||||
agent->AXEventFired(obj, event_type);
|
||||
|
||||
if (auto* client = GetWebLocalFrameClient()) {
|
||||
// TODO(accessibility) This doesn't need to call into RAI -- it
|
||||
// can add to pending events and dirty objects here. The only reason to call
|
||||
// into RAI would be during a page load, to inform in the case of an
|
||||
// event that requires immediate serialization, such as focus.
|
||||
// MarkAXObjectDirtyWithDetails(obj, false, event_from, event_from_action,
|
||||
// event.event_intents);
|
||||
// AddPendingEvent(event);
|
||||
client->PostAccessibilityEvent(event);
|
||||
if (serialize_post_lifecycle_) {
|
||||
AddEventToSerializationQueue(event,
|
||||
IsImmediateProcessingRequiredForEvent(event));
|
||||
|
||||
// TODO(aleventhal) This is for web tests only, in order to record MarkDirty
|
||||
// events. Is there a way to avoid these calls for normal browsing?
|
||||
// Maybe we should use dependency injection from AccessibilityController.
|
||||
if (auto* client = GetWebLocalFrameClient()) {
|
||||
client->HandleWebAccessibilityEventForTest(event);
|
||||
}
|
||||
} else {
|
||||
// legacy mode, go through RAI again!
|
||||
if (auto* client = GetWebLocalFrameClient()) {
|
||||
client->PostAccessibilityEvent(event);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -4309,12 +4520,29 @@ void AXObjectCacheImpl::MarkAXObjectDirtyWithCleanLayoutHelper(
|
||||
// TODO(aleventhal) This is for web tests only, in order to record MarkDirty
|
||||
// events. Is there a way to avoid these calls for normal browsing?
|
||||
// Maybe we should use dependency injection from AccessibilityController.
|
||||
if (auto* client = GetWebLocalFrameClient())
|
||||
client->NotifyWebAXObjectMarkedDirty(WebAXObject(obj));
|
||||
if (auto* client = GetWebLocalFrameClient()) {
|
||||
if (serialize_post_lifecycle_) {
|
||||
client->HandleWebAccessibilityEventForTest(
|
||||
WebAXObject(obj), "MarkDirty", std::vector<ui::AXEventIntent>());
|
||||
} else {
|
||||
client->NotifyWebAXObjectMarkedDirty(WebAXObject(obj));
|
||||
}
|
||||
}
|
||||
|
||||
if (serialize_post_lifecycle_) {
|
||||
// It's important for the user to have access to any changes to the
|
||||
// currently focused object, so schedule serializations immediately if that
|
||||
// object changes. The root is an exception because it often has focus while
|
||||
// the page is loading. In that case the event type is used as the signal
|
||||
// (see IsImmediateProcessingRequiredForEvent()).
|
||||
if (obj != Root() && obj->IsFocused()) {
|
||||
ScheduleImmediateSerialization();
|
||||
}
|
||||
}
|
||||
|
||||
std::vector<ui::AXEventIntent> event_intents;
|
||||
MarkAXObjectDirtyWithDetails(obj, subtree, event_from, event_from_action,
|
||||
event_intents);
|
||||
AddDirtyObjectToSerializationQueue(obj, subtree, event_from,
|
||||
event_from_action, event_intents);
|
||||
|
||||
obj->UpdateCachedAttributeValuesIfNeeded(true);
|
||||
for (auto agent : agents_)
|
||||
@ -4655,7 +4883,7 @@ bool AXObjectCacheImpl::SerializeEntireTree(
|
||||
return true;
|
||||
}
|
||||
|
||||
void AXObjectCacheImpl::MarkAXObjectDirtyWithDetails(
|
||||
void AXObjectCacheImpl::AddDirtyObjectToSerializationQueue(
|
||||
AXObject* obj,
|
||||
bool subtree,
|
||||
ax::mojom::blink::EventFrom event_from,
|
||||
@ -4870,6 +5098,7 @@ void AXObjectCacheImpl::GetImagesToAnnotate(
|
||||
}
|
||||
}
|
||||
|
||||
// TODO(accessibility): This function can go when legacy mode is deleted.
|
||||
bool AXObjectCacheImpl::AddPendingEvent(const ui::AXEvent& event,
|
||||
bool insert_at_beginning) {
|
||||
if (insert_at_beginning)
|
||||
|
@ -130,6 +130,19 @@ class MODULES_EXPORT AXObjectCacheImpl
|
||||
|
||||
AXObject* FocusedObject();
|
||||
|
||||
// This stores the last time a serialization was ACK'ed after being sent to
|
||||
// the browser, so that serializations can be skipped if the time since the
|
||||
// last serialization is less than GetDeferredEventsDelay(). Setting to
|
||||
// "beginning of time" causes the upcoming serialization to occur at the next
|
||||
// available opportunity. Batching is used to reduce the number of
|
||||
// serializations, in order to provide overall faster content updates while
|
||||
// using less CPU, because nodes that change multiple times in a short time
|
||||
// period only need to be serialized once, e.g. during page loads or
|
||||
// animations.
|
||||
static constexpr base::Time kSerializeAtNextOpportunity =
|
||||
base::Time::UnixEpoch();
|
||||
base::Time last_serialization_timestamp_ = kSerializeAtNextOpportunity;
|
||||
|
||||
const ui::AXMode& GetAXMode() override;
|
||||
void SetAXMode(const ui::AXMode&) override;
|
||||
|
||||
@ -289,6 +302,11 @@ class MODULES_EXPORT AXObjectCacheImpl
|
||||
const PhysicalRect&) override;
|
||||
|
||||
void InlineTextBoxesUpdated(LayoutObject*) override;
|
||||
|
||||
// Get the amount of time, in ms, that event processing should be deferred
|
||||
// in order to more efficiently batch changes.
|
||||
int GetDeferredEventsDelay() const;
|
||||
|
||||
// Called during the accessibility lifecycle to refresh the AX tree.
|
||||
void ProcessDeferredAccessibilityEvents(Document&) override;
|
||||
// Remove AXObject subtrees (once flat tree traversal is safe).
|
||||
@ -518,7 +536,8 @@ class MODULES_EXPORT AXObjectCacheImpl
|
||||
ui::AXTreeUpdate*,
|
||||
std::set<ui::AXSerializationErrorFlag>* out_error = nullptr) override;
|
||||
|
||||
void MarkAXObjectDirtyWithDetails(
|
||||
// Marks an object as dirty to be serialized in the next serialization.
|
||||
void AddDirtyObjectToSerializationQueue(
|
||||
AXObject* obj,
|
||||
bool subtree,
|
||||
ax::mojom::blink::EventFrom event_from,
|
||||
@ -588,7 +607,33 @@ class MODULES_EXPORT AXObjectCacheImpl
|
||||
return whitespace_ignored_map_;
|
||||
}
|
||||
|
||||
// Adds an event to the list of pending_events_ and mark the object as dirty
|
||||
// via AXObjectCache::AddDirtyObjectToSerializationQueue. If
|
||||
// immediate_serialization is set, it schedules a serialization to be done at
|
||||
// the next available time without delays.
|
||||
void AddEventToSerializationQueue(const ui::AXEvent& event,
|
||||
bool immediate_serialization) override;
|
||||
|
||||
// Called from browser to RAI and then to AXCache to notify that a
|
||||
// serialization has arrived to Browser.
|
||||
void OnSerializationReceived() override;
|
||||
|
||||
// Used by outside classes to determine if a serialization is in the process
|
||||
// or not.
|
||||
bool IsSerializationInFlight() const override;
|
||||
|
||||
// Used by outside classes, mainly RenderAccessibilityImpl, to inform
|
||||
// AXObjectCacheImpl that a serialization was cancelled.
|
||||
void OnSerializationCancelled() override;
|
||||
|
||||
// Used by outside classes, mainly RenderAccessibilityImpl, to inform
|
||||
// AXObjectCacheImpl that a serialization was sent.
|
||||
void OnSerializationStartSend() override;
|
||||
|
||||
protected:
|
||||
bool IsImmediateProcessingRequiredForEvent(const ui::AXEvent& event) const;
|
||||
void ScheduleImmediateSerialization() override;
|
||||
|
||||
void PostPlatformNotification(
|
||||
AXObject* obj,
|
||||
ax::mojom::blink::Event event_type,
|
||||
@ -1032,6 +1077,16 @@ class MODULES_EXPORT AXObjectCacheImpl
|
||||
wtf_size_t max_pending_updates_ = 1UL << 16;
|
||||
bool tree_updates_paused_ = false;
|
||||
|
||||
// This will flip to true when we initiate the process of sending AX data to
|
||||
// the browser, and will flip back to false once we receive back an ACK.
|
||||
bool serialization_in_flight_ = false;
|
||||
|
||||
// This flips to true if a request for an immediate update was not honored
|
||||
// because serialization_in_flight_ was true. It flips back to false once
|
||||
// serialization_in_flight_ has flipped to false and an immediate update has
|
||||
// been requested.
|
||||
bool serialize_immediately_after_current_serialization_ = false;
|
||||
|
||||
// Maps ids to their object's autofill suggestion availability.
|
||||
HashMap<AXID, WebAXAutofillSuggestionAvailability>
|
||||
autofill_suggestion_availability_map_;
|
||||
@ -1090,6 +1145,11 @@ class MODULES_EXPORT AXObjectCacheImpl
|
||||
|
||||
FRIEND_TEST_ALL_PREFIXES(AccessibilityTest, PauseUpdatesAfterMaxNumberQueued);
|
||||
FRIEND_TEST_ALL_PREFIXES(AccessibilityTest, RemoveReferencesToAXID);
|
||||
|
||||
// So we can ensure the serialization pipeline never stalls with dirty objects
|
||||
// remaining to be serialized.
|
||||
base::WeakPtrFactory<AXObjectCacheImpl>
|
||||
weak_factory_for_serialization_pipeline_{this};
|
||||
};
|
||||
|
||||
// This is the only subclass of AXObjectCache.
|
||||
|
@ -131,19 +131,57 @@ void WebAXContext::UpdateAXForAllDocuments() {
|
||||
return private_->GetAXObjectCache().UpdateAXForAllDocuments();
|
||||
}
|
||||
|
||||
void WebAXContext::ScheduleAXUpdate() {
|
||||
void WebAXContext::ScheduleImmediateSerialization() {
|
||||
if (!HasActiveDocument()) {
|
||||
return;
|
||||
}
|
||||
|
||||
const auto& cache = private_->GetAXObjectCache();
|
||||
auto& cache = private_->GetAXObjectCache();
|
||||
cache.ScheduleImmediateSerialization();
|
||||
}
|
||||
|
||||
// If no dirty objects are queued, it's not necessary to schedule an extra
|
||||
// visual update.
|
||||
if (!cache.HasDirtyObjects())
|
||||
void WebAXContext::AddEventToSerializationQueue(const ui::AXEvent& event,
|
||||
bool immediate_serialization) {
|
||||
if (!HasActiveDocument()) {
|
||||
return;
|
||||
}
|
||||
|
||||
return cache.ScheduleAXUpdate();
|
||||
auto& cache = private_->GetAXObjectCache();
|
||||
cache.AddEventToSerializationQueue(event, immediate_serialization);
|
||||
}
|
||||
|
||||
void WebAXContext::OnSerializationCancelled() {
|
||||
if (!HasActiveDocument()) {
|
||||
return;
|
||||
}
|
||||
|
||||
auto& cache = private_->GetAXObjectCache();
|
||||
cache.OnSerializationCancelled();
|
||||
}
|
||||
|
||||
void WebAXContext::OnSerializationStartSend() {
|
||||
if (!HasActiveDocument()) {
|
||||
return;
|
||||
}
|
||||
|
||||
auto& cache = private_->GetAXObjectCache();
|
||||
cache.OnSerializationStartSend();
|
||||
}
|
||||
|
||||
bool WebAXContext::IsSerializationInFlight() const {
|
||||
if (!HasActiveDocument()) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const auto& cache = private_->GetAXObjectCache();
|
||||
return cache.IsSerializationInFlight();
|
||||
}
|
||||
|
||||
void WebAXContext::OnSerializationReceived() {
|
||||
if (!HasActiveDocument()) {
|
||||
return;
|
||||
}
|
||||
return private_->GetAXObjectCache().OnSerializationReceived();
|
||||
}
|
||||
|
||||
void WebAXContext::FireLoadCompleteIfLoaded() {
|
||||
|
@ -214,14 +214,14 @@ void WebAXObject::MarkSerializerSubtreeDirty() const {
|
||||
private_->AXObjectCache().MarkSerializerSubtreeDirty(*private_);
|
||||
}
|
||||
|
||||
void WebAXObject::MarkAXObjectDirtyWithDetails(
|
||||
void WebAXObject::AddDirtyObjectToSerializationQueue(
|
||||
bool subtree,
|
||||
ax::mojom::blink::EventFrom event_from,
|
||||
ax::mojom::blink::Action event_from_action,
|
||||
std::vector<ui::AXEventIntent> event_intents) const {
|
||||
if (IsDetached())
|
||||
return;
|
||||
private_->AXObjectCache().MarkAXObjectDirtyWithDetails(
|
||||
private_->AXObjectCache().AddDirtyObjectToSerializationQueue(
|
||||
private_.Get(), subtree, event_from, event_from_action, event_intents);
|
||||
}
|
||||
|
||||
|
5
third_party/blink/web_tests/external/wpt/long-animation-frame/tentative/loaf-desired-exec-time.html
vendored
5
third_party/blink/web_tests/external/wpt/long-animation-frame/tentative/loaf-desired-exec-time.html
vendored
@ -16,6 +16,11 @@
|
||||
|
||||
const INTERNAL_OVERHEAD_DELAY_EPSILON = 5;
|
||||
|
||||
// Accessibility code may schedule tasks that confuse this test code.
|
||||
if (window.accessibilityController) {
|
||||
accessibilityController.reset();
|
||||
}
|
||||
|
||||
promise_test(async t => {
|
||||
const button = document.createElement("button");
|
||||
button.innerText = "Click";
|
||||
|
Reference in New Issue
Block a user