Unable to build a MAUI iOS app using a self-hosted Azure DevOps build agent

[vecalion]

Package

Sentry

.NET Flavor

.NET

.NET Version

8.0.203

OS

iOS

SDK Version

4.2.1

Self-Hosted Sentry Version

No response

Steps to Reproduce

There is a self-hosted build agent on Mac M1 (agent version vsts-agent-osx-x64-3.236.1). After a minor code change, the pipeline that builds the app started to fail with an odd result (see below). Since the change didn't touch any dependencies and was more like a code clean up, I suspect that something has changed in the environment of the build machine. But I don't know what.

Expected Result

The app builds with no issues.

Actual Result

Apple clang version 15.0.0 (clang-1500.1.0.2.5)
  Target: arm64-apple-darwin23.4.0
  Thread model: posix
  InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
   "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -arch arm64 -dead_strip -exported_symbols_list obj/Release/net8.0-ios/ios-arm64/mtouch-symbols.list -platform_version ios 11.0.0 17.2 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.2.sdk -ObjC -o /Users/valeriykovalenko/sq-agents/agent-2/_work/1/s/STAmina.XF/obj/Release/net8.0-ios/ios-arm64/nativelibraries/STAmina.XF -framework libHarfBuzzSharp -framework <...> 
-framework Sentry -lc++ -lsqlite3 -lz -lc++ -lsqlite3 -lz /Users/valeriykovalenko/sq-agents/agent-2/_work/1/s/STAmina.XF/obj/Release/net8.0-ios/ios-arm64/nativelibraries/aot-output/arm64/System.Private.CoreLib.dll.o /Users/valeriykovalenko/sq-agents/agent-2/_work/1/s/STAmina.XF/obj/Release/net8.0-ios/ios-arm64/nativelibraries/aot-output/arm64/System.Private.CoreLib.dll.llvm.o <...>
2/_work/1/s/STAmina.XF/obj/Release/net8.0-ios/ios-arm64/linker-cache -F/Users/valeriykovalenko/sq-agents/agent-2/_work/1/s/STAmina.XF/obj/Release/net8.0-ios/ios-arm64/linker-cache
  Undefined symbols for architecture arm64:
    "_SentryVersionNumber", referenced from:
       -exported_symbol[s_list] command line option
    "_SentryVersionString", referenced from:
       -exported_symbol[s_list] command line option
  ld: symbol(s) not found for architecture arm64
  clang: error: linker command failed with exit code 1 (use -v to see invocation)

[jamescrosswell]

Hey @vecalion ,

I haven't bumped into this before... and our own CI isn't haven't any issues building the MAUI Sample apps we have in this repo.

Do you only get this on your CI machine (so does everything build on your local dev machine)? If so, I guess that would be a good place to start... how can you do a bit of a diff between your local dev environment and what's running on CI?

[vecalion]

Hey @jamescrosswell, yeah, this is what I've been trying to figure out for the past three days and I don't see any difference. When we remove the Sentry nuget and comment out all references, it works with no issues. I tried to reinstall the build agent - still getting that error. Any suggestions what else to try?

[jamescrosswell]

You're using github for CI right? I don't have much experience with self hosted agents on GitHub but you could check against the config we're using for our CI builds... all of which is here in this repo.

Most of our CI builds are triggered from .github/workflows/build.yml... you can see we're using macos-latest.

[vecalion]

@jamescrosswell, I took the sample project, replaced referenced Sentry.Maui with the latest nuget. Added to the .csproj:

  <PropertyGroup>
    <MtouchLink>SdkOnly</MtouchLink>
  </PropertyGroup>

Pushed into a Azure DevOpes repo, added a pipeline and got the same error:

        
  Undefined symbols for architecture arm64:
    "_SentryVersionNumber", referenced from:
       -exported_symbol[s_list] command line option
       (maybe you meant: ___Str_SentryVersionNumber, _lsda0_Sentry_Bindings_Cocoa_Sentry_CocoaSdk_Constants__get_SentryVersionNumber , _Sentry_Bindings_Cocoa_Sentry_CocoaSdk_Constants__get_SentryVersionNumber , _fram0_Sentry_Bindings_Cocoa_Sentry_CocoaSdk_Constants__get_SentryVersionNumber )
    "_SentryVersionString", referenced from:
       -exported_symbol[s_list] command line option
       (maybe you meant: _lsda0_Sentry_Bindings_Cocoa_Sentry_CocoaSdk_Constants__get_SentryVersionString, _fram0_Sentry_Bindings_Cocoa_Sentry_CocoaSdk_Constants__get_SentryVersionString , _Sentry_Bindings_Cocoa_Sentry_CocoaSdk_Constants__get_SentryVersionString , ___Str_SentryVersionString )
  ld: symbol(s) not found for architecture arm64
  clang: error: linker command failed with exit code 1 (use -v to see invocation)

The same is reproducible with the microsoft hosted agents (macos-13). I don't use GitHub Actions, so not sure if the issue will be reproducible there. But if you'd like to try, please use the nuget and the MtouchLink.

[vecalion]

@jamescrosswell, I have figured out what is causing the build issues: the version of microsoft.net.sdk.ios. In our project, we used the preview feed. The builds don't work with v17.2.8052. As soon as I switched to 8043, everything started working well. So, either it's an issue in the preview SDK or there are some upcoming changes that surface an issue in the Sentry SDK.

Here I created a repro project: https://github.com/vecalion/sentry-build-issue
This change breaks builds: https://github.com/vecalion/sentry-build-issue/commit/26cf02134045c16dafffd16744454e7fbad13f59

Please take a look. If this is something on the Sentry side, it would be great to have a hotfix until the next version of the microsoft.net.sdk.ios is publicly available.

[bruno-garcia]

Thanks for sharing the details.
I'll be honest it's unlikely we can put the effort to make things work with preview builds right now due to the volume of stuff we're dealing with, but if it's a RC for example, we need to make the time because those are often published as GA right after.

[bruno-garcia]

Perhaps worth checking with folks on https://github.com/xamarin/xamarin-macios might be something with their preview build not building things right on arm64?

[dpozimski]

i have the same issue on prod 8.0.203 with Azure DevOps pipeline for agent with os macOS-13

[dpozimski]

Is there any ETA? It's not possible to build the MAUI app without removing Sentry dependency from it.

[jamescrosswell]

Is there any ETA? It's not possible to build the MAUI app without removing Sentry dependency from it.

As it happens, we just started running into this on our own CI today... not sure what changed but be can't deploy any new versions of Sentry without fixing it so... probably pretty quickly 😉

[dpozimski]

great to hear thats its reproducible on your side ! 🔥

[vecalion]

Is there any ETA? It's not possible to build the MAUI app without removing Sentry dependency from it.

As it happens, we just started running into this on our own CI today... not sure what changed but be can't deploy any new versions of Sentry without fixing it so... probably pretty quickly 😉

As I mentioned in my investigation, the version of Microsoft.iOS.Sdk is causing it. New one was released 12 hours ago: https://www.nuget.org/packages/Microsoft.iOS.Sdk#versions-body-tab
So you can expect that all devs who use Maui (or probably Microsoft.iOS) + Sentry are going to experience this issue very soon. Imo it's critical.

[bitsandfoxes]

Imo it's critical.

We're working on it!

[bitsandfoxes]

I think I found what's going on. Something seems to be off with the bindings generation. A fix is coming.

[dpozimski]

When do you plan to publish newest package to nuget ?

[bitsandfoxes]

When do you plan to publish newest package to nuget ?

Any minute now. As soon as CI stops acting up.

[Redth]

Best guess here is that _SentryVersionNumber and _SentryVersionString are 'private' symbols and likely the change that started causing this was a newer Xcode version started now complaining about this and trying to bind to it when it previously let it go.

Glad you found the issue and it looks like there's a new package published :)

[Redth]

Adding a bit more context, it does look like there was a fix in .NET iOS: xamarin/xamarin-macios#20066 where they were not previously telling the compiler about some fields (which was broken/wrong/incorrect) and now that it was 'fixed' it's exposing this issue where the private fields are being bound when they should not be.

[bitsandfoxes]

I get where you're coming from, thanks for the explanation. But SentryVersionNumber and SentryVersionString do get exported and reside in the umbrella header and I assume they are public:
FOUNDATION_EXPORT double SentryVersionNumber;
FOUNDATION_EXPORT const unsigned char SentryVersionString[];