Proposal: Add Method to Hit Test and Manually Transfer Clicks (like in Android & iOS)

[jonmdev]

Proposal: Add Method to Hit Test and Manually Transfer Clicks (like in Android & iOS)

Summary

Currently there is no way in WinUI C# to hit test a click and then decide what to do based on the hit test (eg. let the platform activate the given clicked element, transfer the click to another element, or run some other function).

In Android .NET and iOS .NET there are working methods for this, but no such methods exist in WinUI3. Adding proper methods will allow more unified coding experiences with .NET MAUI and general modern designs. It will bring WinUI3 up to standard with the other OS's.

Rationale

.NET MAUI is supposed to represent a unified platform for development for Android, iOS, and WinUI3. Yet this major function is missing from WinUI3 disallowing us to build unified experiences where clicks can be custom managed across platforms.

WinUI3 should be brought up to spec with Android & iOS so it has a comparable feature set for custom and more advanced management of user click events. This will allow, for example, management of custom nested carousels or the creation of overlapping clickable objects with complex rules.

To illustrate:

1) In Android .NET:

• One can simply add an Android View overtop and subscribe to the Touch event: touchView.Touch += FunctionToRunOnTouch
• This event will have all active touches stored inside it with their positions and states as a MotionEvent.
• One can set e.Handled = true; if done with it.
• One can clone the click events by MotionEvent clonedEvent = MotionEvent.Obtain(e.Event)
• One can manually set click events into any View with pickedView.DispatchTouchEvent(clonedEvent)
• See for example here where Microsoft explains how you can do much this in .NET Android.

2) In iOS .NET:

• One can simply add a UIView overtop and override public override UIView HitTest(CGPoint point, UIEvent uievent) which returns the UIView that should receive the click.
• Calling base.HitTest() within this invokes the default behavior.
• We also can override TouchesBegan, TouchesCancelled, TouchesMoved & TouchesEnded which then lets us track the touch data for each touch as an array of UITouch objects on each update.
• See for example here for an explanation of how this can be done.

3) In WinUI3:

• A similar approach to Android or iOS could be taken.
• Either we can have a way to manually dispatch click events into a UI Element (like in Android) or we can have a method that lets us override the click logic to return whatever UI Element should get the click (like in iOS).
• Either way, we should also have a way to see all the click data on each update (as in Android and iOS) from the overlay that took the hit test so we can manage this data as needed.

Scope

Capability	Priority
This proposal will allow developers to Hit Test clicks from an overlay view	Must
This proposal will allow developers to read the status of all active clicks on each update from the overlay view	Must
This proposal will allow developers to manually control what UI Elements are activated or assigned to the clicks	Must

[github-actions]

Hi I'm an AI powered bot that finds similar issues based off the issue title.

Please view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one. Thank you!

Open similar issues:

• Proposal: Add pointer or keyboard event information to HyperlinkClickEventArgs (#4730), similarity score: 0.71
• Proposal: Update light-dismiss layer to pass the click through to underlying UI (#917), similarity score: 0.71

Note: You can give me feedback by thumbs upping or thumbs downing this comment.

[JesseCol]

Hi, have you seen the API Microsoft.UI.Xaml.Media.VisualTreeHelper.FindElementsInHostCoordinates? It might help you here. Thanks!

[jonmdev]

Microsoft.UI.Xaml.Media.VisualTreeHelper.FindElementsInHostCoordinates

Thank you, that is interesting. I wasn't aware of that. Maui actually has a built in cross-platform function like this as well:

VisualTreeElementExtensions.GetVisualTreeElements

https://learn.microsoft.com/en-us/dotnet/api/microsoft.maui.visualtreeelementextensions.getvisualtreeelements?view=net-maui-8.0

And one can deconstruct and alter the code easily enough that it uses in each platform.

The problem is not being able to find what is at a given coordinate, but rather deciding whether or not the component (or another over/under or simultaneous peripherally) should receive the click or be activated by it.

We have no way to control this in Windows.

We cannot DispatchTouchEvent like in Android from a click catching layer at will. Nor can we monitor all the active touch data on every update like in Android to decide what to do with it.

Nor can we return a custom element vs. decide to invoke base.HitTest() to invoke the default click behavior as in iOS. Or use the click data from the layer that caught it to do other tasks at will as in iOS.

That is where the WinUI3 deficit is.

If I am managing my own nested carousels made from basic Maui Layout/Image elements for example with 2D click and drag functions, I can determine in iOS and Android if the click is made in this region, and manipulate the elements myself based on this and the click data.

Or if I want a click to go straight through to a native text editor field, I can allow this and not interfere.

Such management is not possible in WinUI3, because we have no access to the ongoing touches/clicks and cannot pass through clicks selectively to the native element management.

All of those things are possible in Android and iOS for what I think are very good reasons.

Hopefully that makes sense. Please let me know if not. Thanks.

[JesseCol]

Thanks for the response, it makes sense. We'll leave this open and consider it in the future. Thanks!