Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TYPE_VIEW_HOVER_ENTER to AccessibilityNodeInfo sendAccessibilityEvent #34969

Closed
wants to merge 31 commits into from

Commits on Oct 13, 2022

  1. adding accessibilityTitle prop to ReactAndroid

    "mHostView.addView adds the react views to the modal. The modal is a wrapper around react views. They are added to the modal.
    mDialog.setTitle does not display the title. The react-native implementation must have changed the functionality. Commenting addTitle creates issues.
    Try to call setTitle on the Activity (the Dialog is included in a Custom Activity)
    Investigate PhoneWindow.setTitle in particular onWindowTitleChanged and dispatchWindowAttributesChanged
    UPDATE: The method is WindowManager.setTitle"
    
    "https://github.com/fabriziobertoglio1987/react-native/blob/62f83a9fad027ef0ed808f7e34973bb01cdf10e9/ReactAndroid/src/main/java/com/facebook/react/views/modal/ReactModalHostView.java#L114
    https://github.com/aosp-mirror/platform_frameworks_base/blob/e648d3cb91f9a0156c729ca18ec27e71f59f9ce2/core/java/android/view/WindowManager.java#L3794"
    
    "Read TalkBack logic around dialog and title.
    Research for string associated with the Dialog role
    Research for logic related to the title attribute"
    Read logic mDialog
    https://github.com/fabriziobertoglio1987/react-native/blob/62f83a9fad027ef0ed808f7e34973bb01cdf10e9/ReactAndroid/src/main/java/com/facebook/react/views/modal/ReactModalHostView.java#L285
    
    Read logic from AOSP Dialog, Window and WindowManager
    https://github.com/aosp-mirror/platform_frameworks_base/blob/e648d3cb91f9a0156c729ca18ec27e71f59f9ce2/core/java/android/app/Dialog.java#L641
    https://github.com/aosp-mirror/platform_frameworks_base/blob/e648d3cb91f9a0156c729ca18ec27e71f59f9ce2/core/java/android/view/Window.java#L1444
    https://github.com/aosp-mirror/platform_frameworks_base/blob/e648d3cb91f9a0156c729ca18ec27e71f59f9ce2/core/java/android/view/WindowManager.java#L3801
    
    "Quickly test the following solution:
    Implement prop accessibilityTitle
    Implement method DialogManager accessibilityTitle
    Call mDialog.setTitle(“my title”)
    The setter method setAccessibilityTitle is not called"
    
    "The setter method setAccessibilityTitle is not called, while the other props are called:
    Read other files in the same folder
    Read cpp fabric files
    Add fabric configurations
    Read reactnative.dev codegen instructions"
    
    "Troubleshoot codegen generation of the accessibilityTitle prop.
    The prop is not added to jave codegen files generated in ReactAndroid/generated/codegen/Modal. Make sure the prop is added and test the outcome:
    Read instructions on reactnative.dev on codegen types to use for this prop (optional)
    Codegen does not include accessibilityTitle prop. Make sure codegen adds the title.  
    Try testing on Paper. If the issue does not reproduce on Paper, it is caused by a missing CPP prop configuration"
    https://reactnative.dev/docs/the-new-architecture/pillars-codegen
    fabOnReact committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    fd0923e View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2022

  1. improve types in java and js

    - use Stringish instead of string (internal type used in react-native)
    - add Nullable to Java setAccessibilityTitle, we use setTitle(null) to
      clear and previous value
    fabOnReact committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    accbbfc View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2022

  1. implementing alternative solution for iOS/Android

    changes discussed in
    facebook#34969 (comment)
    which consist in:
    
    - Modal accepts prop TitleComponent which is a React.Element or
      Component and represents the title of the modal.
      The solution is taken from ListHeaderComponent in VirtList
    - AccessibilityInfo.sendAccessibilityEvent is triggered to focus on the
      Title when the modal opens
    - sendAccessibilityEvent needs to set a timeout of 1s (see facebook#30097 and https://stackoverflow.com/questions/28472985/android-set-talkback-accessibility-focus-to-a-specific-view)
    - TitleComponent Styling and positioning is defined outside of reactnative
      This has to be reviewed as ListHeaderComponent uses
      ListHeaderComponentStyle
    
    Notes:
    
    "Improvements:
    Trigger focus on components using Fabric API (findNodeHandle is deprecated, setAccessibilityFocus). setAccessibilityFocus has several issues (stackoverflow), and I can not set a ref on a modal.
    
    componentDidMount seems to trigger before rendering the Modal children, the lifecycle does not work correctly with a modal on Fabric.
    
    You need to use forwardRef to call sendAccessibilityEvent
    Need to change the way you create the title component so that you create a forwardRef and then use this to send the accessibility event"	"https://reactnative.dev/docs/new-architecture-library-intro#preparing-your-javascript-codebase-for-the-new-react-native-renderer-fabric
    https://reactnative.dev/docs/accessibilityinfo#setaccessibilityfocus
    https://reactnative.dev/docs/new-architecture-library-intro#preparing-your-javascript-codebase-for-the-new-react-native-renderer-fabric"
    "Test Text component solution without using getNativeRef, as the problem could be caused by timeout and not ref. Check the keys and try to use the _nativeRef
    Implement getNativeRef for the Text component and use it in Modal to call setAccessibilityFocus"	https://github.com/fabriziobertoglio1987/react-native/blob/accbbfc01af57eda34374ce3b13e36b6e7c12a93/Libraries/Components/TextInput/TextInput.js#L1213
    "Adding setTimeout with a wait of 1-second fixes issues with setAccessibilityFocus does not onLoad
    
    The reason is triggered focus on other elements, so we need to use an API to wait for the focus to display on that element
    
    Try to use runAfterInteraction"	"facebook#30097
    https://reactnative.dev/docs/next/interactionmanager#runafterinteractions"
    "Review meeting notes and Brett suggestion
    Consider removing the sendAccessibilityEvent as violates WCAG 2.0 regulations, including 3.2.1 and 3.2.3"
    Test Android
    "-  The title component displays under the Modal on Android.
    The title receives focus with the ModalPresentation example.
    The title does not receive focus in the ModalOnShow example.
    => In both examples, the titles displays under the modal
    => In ModalOnShow, the title does not receive focus
    Apply the same prop as in the ModalPresentation and see if the title receives focus
    Remove style
    Put the title directly in the ModalOnShow and trigger the focus
    Trigger the focus manually with a button instead of using useEffect( _ref )
    Trigger focus manually with a button in the Modal.js"
    "setAccessibilityFocus correctly moves the focus when triggered with Button. The issue could be caused:
    ref is undefined
    read conversation facebook#30097
    alternative callback to _showModal (onLoad)
    TalkBack moves focus after calling sendAccessibilityEvent => increase timeout"	facebook#30097
    "Test functionality on iOS:
    test functionality on iOS with Xcode build"
    "Improve solution and finalize diff before final commit:
    Fix issue with sendAccessibilityEvent based on stackoverflow solution or runAfterIteraction (setAccessibilityFocus does not onLoad)
    TitleComponent should move to rn-tester ModalPresentation and ModalOnShow
forwardRef may not work"	"https://github.com/fabriziobertoglio1987/react-native/blob/accbbfc01af57eda34374ce3b13e36b6e7c12a93/ReactAndroid/src/main/java/com/facebook/react/fabric/mounting/SurfaceMountingManager.java#L968
    https://stackoverflow.com/questions/28472985/android-set-talkback-accessibility-focus-to-a-specific-view
    facebook#30097"
    fabOnReact committed Oct 24, 2022
    Configuration menu
    Copy the full SHA
    17d7827 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5e1e72f View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2022

  1. Configuration menu
    Copy the full SHA
    42ab164 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5c7ed5e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    98b4a94 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    0fe1e3e View commit details
    Browse the repository at this point in the history
  5. minor fix to example

    fabOnReact committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    4213491 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2022

  1. Configuration menu
    Copy the full SHA
    f89a01f View commit details
    Browse the repository at this point in the history
  2. eslint fix check

    fabOnReact committed Oct 26, 2022
    Configuration menu
    Copy the full SHA
    9e7fe9f View commit details
    Browse the repository at this point in the history
  3. minor change

    fabOnReact committed Oct 26, 2022
    Configuration menu
    Copy the full SHA
    129d152 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    036e4b5 View commit details
    Browse the repository at this point in the history
  5. minor change

    fabOnReact committed Oct 26, 2022
    Configuration menu
    Copy the full SHA
    8a25a6f View commit details
    Browse the repository at this point in the history
  6. minor change

    fabOnReact committed Oct 26, 2022
    Configuration menu
    Copy the full SHA
    dbd01c6 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    3c0f531 View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2022

  1. Configuration menu
    Copy the full SHA
    23bf335 View commit details
    Browse the repository at this point in the history
  2. remove sendAccessibilityEvent setTimeout as not required on iOS

    iOS does not have issues with sendAccessibilityEvent (facebook#30097 (comment)), for this reason setTimeout is not required
    test on iOS
    https://www.icloud.com/iclouddrive/00cgfwQLkaoFQHvxh-Dj89zUA#ios_modal_title
    fabOnReact committed Nov 18, 2022
    Configuration menu
    Copy the full SHA
    43e2ffa View commit details
    Browse the repository at this point in the history
  3. adding flowfixme

    fabOnReact committed Nov 18, 2022
    Configuration menu
    Copy the full SHA
    8f2e0f0 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2022

  1. Configuration menu
    Copy the full SHA
    d351891 View commit details
    Browse the repository at this point in the history
  2. Add viewHoverEnter to sendAccessibilityEventFromJS

    Test sendAccessibilityEvent(TYPE_WINDOW_STATE_CHANGED)Replace setTimeout with sendAccessibilityEvent eventID 32App fails with runtime facebook::jni::JniException: java.lang.IllegalArgumentException: sendAccessibilityEventFromJS: invalid eventType 32'
    --
    Trigger sendAccessibilityEvent TYPE_WINDOW_STATE_CHANGED in Java.Calling sendAccessibilityEvent(ref.current, ’windowDidChange’) does not fix the issue. The method calls MountingManager.sendAccessibilityEvent which triggers sendAccessibilityEvent on the instance of ReactTextView.The method is triggered in SurfaceMountingManager sendAccessibilityEvent, which retrieves the ReactTextView using ref.current
    Read stackoverflow discussion
    Try to use ViewHoverEnter
    Add viewHoverEnter to sendAccessibilityEventFromJS (android docs)
    
    "Test [sendAccessibilityEvent(TYPE_WINDOW_STATE_CHANGED)](facebook#34969 (comment))
    Replace [setTimeout](https://github.com/fabriziobertoglio1987/react-native/blob/8f2e0f00ad9e7db302c246bbc0d509507043975a/packages/rn-tester/js/examples/Modal/ModalPresentation.js#L103) with [sendAccessibilityEvent](https://github.com/fabriziobertoglio1987/react-native/blob/8f2e0f00ad9e7db302c246bbc0d509507043975a/Libraries/Components/AccessibilityInfo/AccessibilityInfo.js#L359) eventID [32](https://developer.android.com/reference/android/view/accessibility/AccessibilityEvent#TYPE_WINDOW_STATE_CHANGED)
App fails with runtime facebook::jni::JniException: java.lang.IllegalArgumentException: sendAccessibilityEventFromJS: invalid eventType 32'"
    "Trigger sendAccessibilityEvent TYPE_WINDOW_STATE_CHANGED in Java.
Calling sendAccessibilityEvent(ref.current, ’windowDidChange’) does not fix the issue. The method calls [MountingManager.sendAccessibilityEvent](https://github.com/fabriziobertoglio1987/react-native/blob/8f2e0f00ad9e7db302c246bbc0d509507043975a/ReactAndroid/src/main/java/com/facebook/react/fabric/mounting/MountingManager.java#L293-L301) which triggers sendAccessibilityEvent on the instance of ReactTextView.
    The method is triggered in SurfaceMountingManager [sendAccessibilityEvent](https://github.com/fabriziobertoglio1987/react-native/blob/8f2e0f00ad9e7db302c246bbc0d509507043975a/ReactAndroid/src/main/java/com/facebook/react/fabric/mounting/SurfaceMountingManager.java#L934-L952), which retrieves the ReactTextView using ref.current "
    Read [stackoverflow](https://stackoverflow.com/questions/28472985/android-set-talkback-accessibility-focus-to-a-specific-view/28481095#28481095) discussion
    Try to use [ViewHoverEnter](dotnet/android#4322)
    Add viewHoverEnter to [sendAccessibilityEventFromJS](https://github.com/fabriziobertoglio1987/react-native/blob/8f2e0f00ad9e7db302c246bbc0d509507043975a/ReactAndroid/src/main/java/com/facebook/react/fabric/FabricUIManager.java#L1083) ([android docs](https://developer.android.com/reference/android/view/accessibility/AccessibilityEvent#TYPE_VIEW_HOVER_ENTER))
    fabOnReact committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    3126e49 View commit details
    Browse the repository at this point in the history
  3. remove state modalOpened

    fabOnReact committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    ef5e490 View commit details
    Browse the repository at this point in the history
  4. remove state modalOpened

    fabOnReact committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    4cf1605 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    e7f0023 View commit details
    Browse the repository at this point in the history
  6. minor change

    fabOnReact committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    fd152d2 View commit details
    Browse the repository at this point in the history
  7. using correct flow type

    fabOnReact committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    bf37a34 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2022

  1. remove example

    fabOnReact committed Nov 29, 2022
    Configuration menu
    Copy the full SHA
    dc4c54e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    69f6a18 View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2023

  1. Configuration menu
    Copy the full SHA
    a5c0cd8 View commit details
    Browse the repository at this point in the history
  2. remove AccInfo.flow.js

    fabOnReact committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    8debb60 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2023

  1. Configuration menu
    Copy the full SHA
    b5f3b21 View commit details
    Browse the repository at this point in the history