Add TYPE_VIEW_HOVER_ENTER to AccessibilityNodeInfo sendAccessibilityEvent #34969

fabOnReact · 2022-10-13T12:38:19Z

Summary

Adds AccessibilityEvent.TYPE_VIEW_HOVER_ENTER to AccessibilityNodeInfo sendAccessibilityEvent
Adds an example implementation.

Changelog

[Android] [Added] - Add TYPE_VIEW_HOVER_ENTER to AccessibilityNodeInfo sendAccessibilityEvent

Test Plan

Android: #34969 (comment) #34969 (comment)
iOS: #34969 (comment) #34969 (comment)

"mHostView.addView adds the react views to the modal. The modal is a wrapper around react views. They are added to the modal. mDialog.setTitle does not display the title. The react-native implementation must have changed the functionality. Commenting addTitle creates issues. Try to call setTitle on the Activity (the Dialog is included in a Custom Activity) Investigate PhoneWindow.setTitle in particular onWindowTitleChanged and dispatchWindowAttributesChanged UPDATE: The method is WindowManager.setTitle" "https://github.com/fabriziobertoglio1987/react-native/blob/62f83a9fad027ef0ed808f7e34973bb01cdf10e9/ReactAndroid/src/main/java/com/facebook/react/views/modal/ReactModalHostView.java#L114 https://github.com/aosp-mirror/platform_frameworks_base/blob/e648d3cb91f9a0156c729ca18ec27e71f59f9ce2/core/java/android/view/WindowManager.java#L3794" "Read TalkBack logic around dialog and title. Research for string associated with the Dialog role Research for logic related to the title attribute" Read logic mDialog https://github.com/fabriziobertoglio1987/react-native/blob/62f83a9fad027ef0ed808f7e34973bb01cdf10e9/ReactAndroid/src/main/java/com/facebook/react/views/modal/ReactModalHostView.java#L285 Read logic from AOSP Dialog, Window and WindowManager https://github.com/aosp-mirror/platform_frameworks_base/blob/e648d3cb91f9a0156c729ca18ec27e71f59f9ce2/core/java/android/app/Dialog.java#L641 https://github.com/aosp-mirror/platform_frameworks_base/blob/e648d3cb91f9a0156c729ca18ec27e71f59f9ce2/core/java/android/view/Window.java#L1444 https://github.com/aosp-mirror/platform_frameworks_base/blob/e648d3cb91f9a0156c729ca18ec27e71f59f9ce2/core/java/android/view/WindowManager.java#L3801 "Quickly test the following solution: Implement prop accessibilityTitle Implement method DialogManager accessibilityTitle Call mDialog.setTitle(“my title”) The setter method setAccessibilityTitle is not called" "The setter method setAccessibilityTitle is not called, while the other props are called: Read other files in the same folder Read cpp fabric files Add fabric configurations Read reactnative.dev codegen instructions" "Troubleshoot codegen generation of the accessibilityTitle prop. The prop is not added to jave codegen files generated in ReactAndroid/generated/codegen/Modal. Make sure the prop is added and test the outcome: Read instructions on reactnative.dev on codegen types to use for this prop (optional) Codegen does not include accessibilityTitle prop. Make sure codegen adds the title. Try testing on Paper. If the issue does not reproduce on Paper, it is caused by a missing CPP prop configuration" https://reactnative.dev/docs/the-new-architecture/pillars-codegen

analysis-bot · 2022-10-13T13:11:39Z

Platform	Engine	Arch	Size (bytes)	Diff
android	hermes	arm64-v8a	8,451,984	-927
android	hermes	armeabi-v7a	7,775,371	-996
android	hermes	x86	8,927,593	-549
android	hermes	x86_64	8,785,197	-363
android	jsc	arm64-v8a	6,668,621	-836
android	jsc	armeabi-v7a	7,462,581	-882
android	jsc	x86	9,192,744	-457
android	jsc	x86_64	6,893,751	-244

Base commit: 753bccc
Branch: main

fabOnReact · 2022-10-17T19:04:59Z

TODOs

Discuss if using the prop name title, accessibilityTitle or aria-title (link) https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Attributes [w3c] ☂️ Web Props (Part 1) umbrella issue #34424

- use Stringish instead of string (internal type used in react-native) - add Nullable to Java setAccessibilityTitle, we use setTitle(null) to clear and previous value

analysis-bot · 2022-10-17T21:19:03Z

Platform	Engine	Arch	Size (bytes)	Diff
ios	-	universal	n/a	--

Base commit: 8b00b4f
Branch: main

blavalla · 2022-10-17T22:02:19Z

I'm curious what impact (if any) this property should have on iOS. As far as I know, iOS doesn't have the concept of accessibility-specific titles in UIAlertController. They only have the option for a visible text title (which is announced on display). We could attempt to polyfill this behavior into iOS by making an announcement upon display of the modal, but this will be tricky as the modal will likely already be announcing something else (whatever gets focused), and I'm not sure it's worth the complexity.

Maybe we should simply have this prop be named "title" and set a visible title for both iOS and Android, rather than making this title for accessibility only to simplify things here.

As long as there is some way to give these dialogs a title for both platforms that is announced on display, I don't think it matters much if that title is only for accessibility services. It likely should be visible for all users anyways.

fabOnReact · 2022-10-18T20:32:39Z

I'm curious what impact (if any) this property should have on iOS. As far as I know, iOS doesn't have the concept of accessibility-specific titles in UIAlertController. They only have the option for a visible text title (which is announced on display). We could attempt to polyfill this behavior into iOS by making an announcement upon display of the modal, but this will be tricky as the modal will likely already be announcing something else (whatever gets focused), and I'm not sure it's worth the complexity.

This is the result of my research.

The modal is implemented in Fabric with the RCTModalHostViewComponentView (UIView) and RCTFabricModalHostViewController (UIViewController)

https://github.com/fabriziobertoglio1987/react-native/tree/accbbfc01af57eda34374ce3b13e36b6e7c12a93/React/Fabric/Mounting/ComponentViews/Modal

as explained in this conversation, it is possible to set a title in the navigation bar, but react-native does not handle navigation which is managed with other libraries (an ex. in react-native-navigation or react navigation), so I would not follow those solutions.

Another solution I tested, was triggering the announcement after delay of X seconds:

dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(3 * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{
    UIAccessibilityPostNotification(UIAccessibilityAnnouncementNotification, @"welcome to Italy!"); 
});

Video of the above delayed announcement https://drive.google.com/file/d/1MMXHFraj6q2LZTNqoORB69C8HZ2HtUzg/view?usp=sharing

announceForAccessibilityWithOptions queue would not work in this case as explained (StackOverflow explanation, detailed breakdown)

https://reactnative.dev/docs/accessibilityinfo#announceforaccessibilitywithoptions

The same issue affected PR #33468. The solution implemented in #33468 consisted of:

1) Trigger manually the announcement of the VoiceOver error message with method setAccessibilityErrorMessage

https://github.com/facebook/react-native/pull/33468/files#diff-533823c1cb870542c2480ff4fe6587e130dd05751339a8c6b7391d991a9c25feR134-R136

#30859 (comment)

2) Trigger manually the VoiceOver text announcement after text changes (setAttributedText)

https://github.com/facebook/react-native/pull/33468/files#diff-533823c1cb870542c2480ff4fe6587e130dd05751339a8c6b7391d991a9c25feR134-R136

#30859 (comment)

A queue mechanism could be built in the future to fix this issue (example includere here).
The queue mechanism would repeat the announcement if interrupted.

Maybe we should simply have this prop be named "title" and set a visible title for both iOS and Android, rather than making this title for accessibility only to simplify things here.
As long as there is some way to give these dialogs a title for both platforms that is announced on display, I don't think it matters much if that title is only for accessibility services. It likely should be visible for all users anyways.

Tomorrow I will try an alternative JavaScript approach in the Modal Component to implement the above requirement.

Thanks a lot for the feedback @blavalla.

changes discussed in facebook#34969 (comment) which consist in: - Modal accepts prop TitleComponent which is a React.Element or Component and represents the title of the modal. The solution is taken from ListHeaderComponent in VirtList - AccessibilityInfo.sendAccessibilityEvent is triggered to focus on the Title when the modal opens - sendAccessibilityEvent needs to set a timeout of 1s (see facebook#30097 and https://stackoverflow.com/questions/28472985/android-set-talkback-accessibility-focus-to-a-specific-view) - TitleComponent Styling and positioning is defined outside of reactnative This has to be reviewed as ListHeaderComponent uses ListHeaderComponentStyle Notes: "Improvements: Trigger focus on components using Fabric API (findNodeHandle is deprecated, setAccessibilityFocus). setAccessibilityFocus has several issues (stackoverflow), and I can not set a ref on a modal. componentDidMount seems to trigger before rendering the Modal children, the lifecycle does not work correctly with a modal on Fabric. You need to use forwardRef to call sendAccessibilityEvent Need to change the way you create the title component so that you create a forwardRef and then use this to send the accessibility event" "https://reactnative.dev/docs/new-architecture-library-intro#preparing-your-javascript-codebase-for-the-new-react-native-renderer-fabric https://reactnative.dev/docs/accessibilityinfo#setaccessibilityfocus https://reactnative.dev/docs/new-architecture-library-intro#preparing-your-javascript-codebase-for-the-new-react-native-renderer-fabric" "Test Text component solution without using getNativeRef, as the problem could be caused by timeout and not ref. Check the keys and try to use the _nativeRef Implement getNativeRef for the Text component and use it in Modal to call setAccessibilityFocus" https://github.com/fabriziobertoglio1987/react-native/blob/accbbfc01af57eda34374ce3b13e36b6e7c12a93/Libraries/Components/TextInput/TextInput.js#L1213 "Adding setTimeout with a wait of 1-second fixes issues with setAccessibilityFocus does not onLoad The reason is triggered focus on other elements, so we need to use an API to wait for the focus to display on that element Try to use runAfterInteraction" "facebook#30097 https://reactnative.dev/docs/next/interactionmanager#runafterinteractions" "Review meeting notes and Brett suggestion Consider removing the sendAccessibilityEvent as violates WCAG 2.0 regulations, including 3.2.1 and 3.2.3" Test Android "- The title component displays under the Modal on Android. The title receives focus with the ModalPresentation example. The title does not receive focus in the ModalOnShow example. => In both examples, the titles displays under the modal => In ModalOnShow, the title does not receive focus Apply the same prop as in the ModalPresentation and see if the title receives focus Remove style Put the title directly in the ModalOnShow and trigger the focus Trigger the focus manually with a button instead of using useEffect( _ref ) Trigger focus manually with a button in the Modal.js" "setAccessibilityFocus correctly moves the focus when triggered with Button. The issue could be caused: ref is undefined read conversation facebook#30097 alternative callback to _showModal (onLoad) TalkBack moves focus after calling sendAccessibilityEvent => increase timeout" facebook#30097 "Test functionality on iOS: test functionality on iOS with Xcode build" "Improve solution and finalize diff before final commit: Fix issue with sendAccessibilityEvent based on stackoverflow solution or runAfterIteraction (setAccessibilityFocus does not onLoad) TitleComponent should move to rn-tester ModalPresentation and ModalOnShow forwardRef may not work" "https://github.com/fabriziobertoglio1987/react-native/blob/accbbfc01af57eda34374ce3b13e36b6e7c12a93/ReactAndroid/src/main/java/com/facebook/react/fabric/mounting/SurfaceMountingManager.java#L968 https://stackoverflow.com/questions/28472985/android-set-talkback-accessibility-focus-to-a-specific-view facebook#30097"

pull-bot · 2022-10-24T12:32:24Z

PR build artifact for 5e1e72f is ready.
To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

pull-bot · 2022-10-24T12:32:46Z

PR build artifact for 5e1e72f is ready.
To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

fabOnReact · 2022-10-25T14:37:58Z

add documentation to rn-tester
implement sendAccessibilityEventWithDelay (issue #30097, logic in SurfaceMountingManager, relevant commit)

pull-bot · 2022-10-25T16:47:24Z

PR build artifact for 4213491 is ready.
To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

pull-bot · 2022-10-25T16:50:34Z

PR build artifact for 4213491 is ready.
To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

fabOnReact · 2022-11-28T21:23:40Z

Android - Video Test

example dc4c54e

2022-11-28.22-19-48.mp4

fabOnReact · 2022-11-28T21:24:24Z

iOS - View Test

example dc4c54e

ios_modal_title.mp4

pull-bot · 2022-11-28T21:36:39Z

PR build artifact for bf37a34 is ready.
To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

pull-bot · 2022-11-28T21:44:57Z

PR build artifact for bf37a34 is ready.
To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

pull-bot · 2022-11-29T11:15:54Z

PR build artifact for dc4c54e is ready.
To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

pull-bot · 2022-11-29T11:20:37Z

PR build artifact for dc4c54e is ready.
To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

fabOnReact · 2022-11-29T17:06:48Z

iOS - testing viewHoverEvent on iOS

view_hover_event_ios.mp4

facebook-github-bot · 2023-01-19T19:56:27Z

@lunaleaps has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2023-02-18T00:02:34Z

@lunaleaps has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2023-03-01T05:01:37Z

@lunaleaps merged this pull request in a0adf57.

…vent (facebook#34969) Summary: - Adds `AccessibilityEvent.TYPE_VIEW_HOVER_ENTER` to AccessibilityNodeInfo sendAccessibilityEvent - Adds an example implementation. fixes facebook#30860 fixes facebook#30097 Related Documentation facebook/react-native-website#3438 ## Changelog [Android] [Added] - Add TYPE_VIEW_HOVER_ENTER to AccessibilityNodeInfo sendAccessibilityEvent Pull Request resolved: facebook#34969 Test Plan: Android: facebook#34969 (comment) facebook#34969 (comment) iOS: facebook#34969 (comment) facebook#34969 (comment) Reviewed By: christophpurrer Differential Revision: D42613990 Pulled By: lunaleaps fbshipit-source-id: 8c8950610799dcc74067d2b47b44d4ff030f66e5

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 13, 2022

fabOnReact changed the title ~~adding accessibilityTitle prop to ReactAndroid~~ adding android prop accessibilityTitle Oct 17, 2022

react-native-bot added Platform: Android Android applications. Type: Enhancement A new feature or enhancement of an existing feature. labels Oct 17, 2022

fabOnReact mentioned this pull request Oct 17, 2022

Android: Modal announcements #30860

Closed

improve types in java and js

accbbfc

- use Stringish instead of string (internal type used in react-native) - add Nullable to Java setAccessibilityTitle, we use setTitle(null) to clear and previous value

fabOnReact added 2 commits October 24, 2022 12:47

Merge branch 'main' into modal-title

5e1e72f

fabOnReact closed this Oct 25, 2022

fabOnReact reopened this Oct 25, 2022

fabOnReact added 2 commits October 25, 2022 16:26

moving Title logic to modal rn-tester example

42ab164

removing changes from Modal

5c7ed5e

fix issues with sendAccessibilityEvent

98b4a94

fabOnReact changed the title ~~adding android prop accessibilityTitle~~ Modal title announcements example implementation Oct 25, 2022

refactor useTimer with useEffect

0fe1e3e

fabOnReact changed the title ~~Modal title announcements example implementation~~ Modal title announcements example rn-tester implementation Oct 25, 2022

fabOnReact changed the title ~~Modal title announcements example rn-tester implementation~~ Modal title accessibility announcement example rn-tester implementation Oct 25, 2022

minor fix to example

4213491

fix ref flow type in example

f89a01f

fabOnReact mentioned this pull request Nov 28, 2022

[docs] adding sendAccessibilityEvent to AccNodeInfo facebook/react-native-website#3438

Open

fabOnReact added 2 commits November 28, 2022 21:42

minor change

fd152d2

using correct flow type

bf37a34

remove example

dc4c54e

fabOnReact changed the title ~~Modal title accessibility announcement example rn-tester implementation~~ Add TYPE_VIEW_HOVER_ENTER to AccessibilityNodeInfo sendAccessibilityEvent Nov 29, 2022

Merge branch 'main' into modal-title

69f6a18

fabOnReact marked this pull request as ready for review November 29, 2022 17:07

fabOnReact mentioned this pull request Dec 1, 2022

🚧 repeat VoiceOver interrupted announcement 🚧 fabOnReact/react-native#5

Closed

fabOnReact added the Accessibility label Jan 9, 2023

lunaleaps added this to Blocked on others in Fabrizio Collaboration Jan 9, 2023

fabOnReact added 2 commits February 13, 2023 11:52

Merge branch 'main' into modal-title

a5c0cd8

remove AccInfo.flow.js

8debb60

fabOnReact moved this from Waiting For Others to In Progress in Fabrizio Collaboration Feb 21, 2023

Merge branch 'main' into modal-title

b5f3b21

fabOnReact moved this from In Progress to Waiting For Others in Fabrizio Collaboration Feb 21, 2023

facebook-github-bot closed this in a0adf57 Mar 1, 2023

facebook-github-bot added the Merged This PR has been merged. label Mar 1, 2023

lunaleaps moved this from Waiting For Others to Done in Fabrizio Collaboration Mar 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add TYPE_VIEW_HOVER_ENTER to AccessibilityNodeInfo sendAccessibilityEvent #34969

Add TYPE_VIEW_HOVER_ENTER to AccessibilityNodeInfo sendAccessibilityEvent #34969

fabOnReact commented Oct 13, 2022 •

edited

Loading

analysis-bot commented Oct 13, 2022 •

edited

Loading

fabOnReact commented Oct 17, 2022 •

edited

Loading

analysis-bot commented Oct 17, 2022 •

edited

Loading

blavalla commented Oct 17, 2022

fabOnReact commented Oct 18, 2022 •

edited

Loading

pull-bot commented Oct 24, 2022

pull-bot commented Oct 24, 2022

fabOnReact commented Oct 25, 2022 •

edited

Loading

pull-bot commented Oct 25, 2022

pull-bot commented Oct 25, 2022

fabOnReact commented Nov 28, 2022 •

edited

Loading

fabOnReact commented Nov 28, 2022 •

edited

Loading

pull-bot commented Nov 28, 2022

pull-bot commented Nov 28, 2022

pull-bot commented Nov 29, 2022

pull-bot commented Nov 29, 2022

fabOnReact commented Nov 29, 2022

facebook-github-bot commented Jan 19, 2023

facebook-github-bot commented Feb 18, 2023

facebook-github-bot commented Mar 1, 2023

Add TYPE_VIEW_HOVER_ENTER to AccessibilityNodeInfo sendAccessibilityEvent #34969

Add TYPE_VIEW_HOVER_ENTER to AccessibilityNodeInfo sendAccessibilityEvent #34969

Conversation

fabOnReact commented Oct 13, 2022 • edited Loading

Summary

Changelog

Test Plan

analysis-bot commented Oct 13, 2022 • edited Loading

fabOnReact commented Oct 17, 2022 • edited Loading

analysis-bot commented Oct 17, 2022 • edited Loading

blavalla commented Oct 17, 2022

fabOnReact commented Oct 18, 2022 • edited Loading

pull-bot commented Oct 24, 2022

pull-bot commented Oct 24, 2022

fabOnReact commented Oct 25, 2022 • edited Loading

pull-bot commented Oct 25, 2022

pull-bot commented Oct 25, 2022

fabOnReact commented Nov 28, 2022 • edited Loading

fabOnReact commented Nov 28, 2022 • edited Loading

pull-bot commented Nov 28, 2022

pull-bot commented Nov 28, 2022

pull-bot commented Nov 29, 2022

pull-bot commented Nov 29, 2022

fabOnReact commented Nov 29, 2022

facebook-github-bot commented Jan 19, 2023

facebook-github-bot commented Feb 18, 2023

facebook-github-bot commented Mar 1, 2023

fabOnReact commented Oct 13, 2022 •

edited

Loading

analysis-bot commented Oct 13, 2022 •

edited

Loading

fabOnReact commented Oct 17, 2022 •

edited

Loading

analysis-bot commented Oct 17, 2022 •

edited

Loading

fabOnReact commented Oct 18, 2022 •

edited

Loading

fabOnReact commented Oct 25, 2022 •

edited

Loading

fabOnReact commented Nov 28, 2022 •

edited

Loading

fabOnReact commented Nov 28, 2022 •

edited

Loading