Can't initialize jni4net jni4net.n.l64.m26-0.8.9.0.dll invalid ELF header (Possible cause: endianness mismatch) #10

iansmirlis · 2023-10-31T22:03:37Z

When running native java on linux, it fails to load clr bridge dll with:

OpenJDK 64-Bit Server VM warning: You have loaded library jni4net.n.l64.m26-0.8.9.0.dll which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Can't initialize jni4net Bridgejni4net.n.l64.m26-0.8.9.0.dll: jni4net.n.l64.m26-0.8.9.0.dll: invalid ELF header (Possible cause: endianness mismatch)
Exception in thread "main" net.sf.jni4net.inj.INJException: Can't initialize jni4net Bridge
        at net.sf.jni4net.CLRLoader.init(CLRLoader.java:45)
        at net.sf.jni4net.Bridge.init(Bridge.java:35)
        at com.example.Main.main(Main.java:23)
Caused by: java.lang.UnsatisfiedLinkError: jni4net.n.l64.m26-0.8.9.0.dll: jni4net.n.l64.m26-0.8.9.0.dll: invalid ELF header (Possible cause: endianness mismatch)
        at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
        at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2445)
        at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2501)
        at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2700)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2630)
        at java.base/java.lang.Runtime.load0(Runtime.java:768)
        at java.base/java.lang.System.load(System.java:1837)
        at net.sf.jni4net.CLRLoader.init(CLRLoader.java:36)
        ... 2 more

It's pretty obvious why, since the JVM on linux expects an ELF .so library, however dotnet core produces a PE .dll.
(It strikes me thought why JavaToClrReflection works with the same dll through CallMeFromJava unit test)

Only viable solution seems to create a Java_net_sf_jni4net_Bridge_initDotNet for linux with hostfxr on C++, like jni4net.n.l32.m26 that was written for mono initialization.

@pavelsavara do you agree?

The text was updated successfully, but these errors were encountered:

pavelsavara · 2023-11-01T13:16:42Z

This looks promising https://medium.com/@sixpeteunder/how-to-build-a-shared-library-in-c-sharp-and-call-it-from-java-code-6931260d01e5

[UnmanagedCallersOnly(EntryPoint = "xxx")]

dotnet publish /p:NativeLib=Static --runtime linux-x64

pavelsavara · 2023-11-01T13:23:36Z

I would love to avoid adding C tool chain to the mix if possible.

iansmirlis · 2023-11-01T18:41:02Z

I would love to avoid adding C tool chain to the mix if possible.

Yeah I agree

This looks promising https://medium.com/@sixpeteunder/how-to-build-a-shared-library-in-c-sharp-and-call-it-from-java-code-6931260d01e5

[UnmanagedCallersOnly(EntryPoint = "xxx")]

dotnet publish /p:NativeLib=Static --runtime linux-x64

It is indeed interesting, thanks. I didn't know that .so are supported. However it needs NativeAOT and the reflection logic of BridgeExport will not play nice.

Maybe we should drop the idea of these BridgeExport drivers to bootstrap .net and use the native hostfxr library directly from java, like it's done with jni from .net?

pavelsavara · 2023-11-02T08:58:12Z

It is indeed interesting, thanks. I didn't know that .so are supported. However it needs NativeAOT

Oh, that's pity.

I also found
https://github.com/AaronRobinsonMSFT/DNNE
and
https://github.com/dotnet/samples/tree/main/core/hosting

directly from java, like it's done with jni from .net?

I'm not sure there is a way how to do the necessary native dance without C only from Java. If yes, that would be okay.

iansmirlis · 2023-11-02T22:58:30Z

I'm not sure there is a way how to do the necessary native dance without C only from Java. If yes, that would be okay.

I hope I can call directly dotnet native libraries libhostfxr.so and libnethost.so functions from java with System.loadLibrary(). I will try this on weekend

iansmirlis · 2023-11-04T10:21:37Z

I'm not sure there is a way how to do the necessary native dance without C only from Java. If yes, that would be okay.

Yeah you are right, of course. I saw that you called BridgeSetup.cs directly from Java, and I thought that C is not mandatory. Well, it is in this case. The problem is the JNI naming conventions of the dynamic library entry points, for which I do not control for the libhostfxr.so.

So I think we are left with the following options:

Use JNA instead of JNI
Never been there, never done that. Hosting CLR from native code is already a bit complicated, and I hope JNA can keep up effectively with such complexities. Moreover JNA creates a new dependency for jni4net.j, at least on linux.
Use JNI and write the bridge in C# instead of C++.
In order for the C# bridge to be ELF compatible, it will have to be compiled with Native AOT. This is a problem because it will not start the clr host by itself and as a consequence it will not be able to load net.sf.jni4net.Bridge type from jni4net.n. However no one forbids us to load the libhostfxr.so library from NativeAOT code, bootstrap the .net and dynamically load the jni4net.n assembly instead of using reflection.
DNNE library that you posted
It introduces c compiling dependency on build time

I think I prefer the second solution, but maybe I am biased due to language preferences.

pavelsavara · 2023-11-04T18:23:18Z

Now I remember more details. The Java -> C# bootstrap on windows was made possibly by the fact that MSIL has way how to make the native export. I got it from this article, I think. https://www.codeproject.com/Articles/16310/How-to-Automate-Exporting-NET-Function-to-Unmanage

The core of the trick is this MSIL syntax

.export [<ordinal>] as <export_name>

And here is a binary of the tool. I don't know if I still have the source code.
https://github.com/jni4net/jni4net.github.io/blob/master/mvnrepo/net/sf/jni4net/selvin.exportdll/0.2.5.0/selvin.exportdll-0.2.5.0.exe

And that's what all those DLLs like jni4net.n.w64.v40 are good for.

pavelsavara · 2023-11-04T18:37:10Z

I think I prefer the second solution

NativeAOT is not for everyone, let's stick with CoreCLR.

JNA stands on top of https://github.com/libffi/libffi

The problem is the JNI naming conventions of the dynamic library entry points, for which I do not control for the libhostfxr.so.

I wonder if we could have separate C project in which we wrap libhostfxr.so with the entrypoints named in a way that will be possible to consume directly from JNI. And we ship that as binaries for all necessary platforms.

Kind of bootstrap layer of this bridge without any marshaling/proxy/generators. Because that will be created once and the change only rarely, I would be happy to live with C toolchain for it.

jni4net would consume it.

iansmirlis · 2023-11-04T19:10:44Z

I wonder if we could have separate C project in which we wrap libhostfxr.so with the entrypoints named in a way that will be possible to consume directly from JNI. And we ship that as binaries for all necessary platforms.

Yeah it's easy, I just tried to avoid C.

By the way I wrote a few lines to test native aot, and indeed it's possible to call dotnet core native .so libs.

But let's stick with C, I will write this and create a pull request.

iansmirlis · 2023-11-04T20:37:03Z

I wonder if we could have separate C project in which we wrap libhostfxr.so with the entrypoints named in a way that will be possible to consume directly from JNI. And we ship that as binaries for all necessary platforms.

Oh wait, I reread your sentence, maybe I didnt understand. If you meant to bootstrap dotnet core from C/C++ and pass through the bridge pointers, using a C function with the expected naming convention, it's indeed easy.

If you meant to wrap all the nethost and hostfxr functions and write the boostrap logic in Java, begins to get things a bit messy, because then it's not about 2 integers anymore and naming convention. JNI C layer will also have to deal with expected character set, function pointers etc.

It gets more complicated than a thin later and in this case, bootstrapping from C would be an order of magnitude simpler solution. I think that if we are to open the door to C, it doesn't make any difference in maintenance and distribution means to deliver a .net bootstrap in C, as it is also a write once thing. Or at least until they change it again, which either way we will have to rewrite the C part.

Moreover, I don't think I would change the current way that the boostrap works on Windows.

Why you would avoid using C to boostrap?

pavelsavara · 2023-11-06T07:42:19Z

Why you would avoid using C to boostrap?

If we are not able to avoid it completely, I would like to keep C codebase as small as possible. Just because toolchain setup.

Moreover, I don't think I would change the current way that the boostrap works on Windows.

You mean Java to C# MSIL trick ? Does it still work ?

It gets more complicated than a thin later and in this case, bootstrapping from C would be an order of magnitude simpler solution.

I think that you need to do only one thing, execute one static C# method and finish the bootstrap logic from there.
C# is as strong low level language a C, and Java is not.

So C code should be trivial, just something like this https://github.com/dotnet/samples/blob/main/core/hosting/src/NativeHost/nativehost.cpp#L98

Maybe there are some embedding options which need to be set before you invoke libhostfxr.so

Are we on the same page now ? Or I'm confused now ?😆

iansmirlis · 2023-11-06T17:49:25Z

Are we on the same page now ? Or I'm confused now ?😆

Maybe we are saying the same thing and I didn't express it well.

If we are not able to avoid it completely, I would like to keep C codebase as small as possible. Just because toolchain setup.

So C code should be trivial, just something like this https://github.com/dotnet/samples/blob/main/core/hosting/src/NativeHost/nativehost.cpp#L98

This C++ code actually bootstraps .net framework and this is what I had in mind. Alternatively we can use exactly the same calls and write the equivalent code in C# nativeaot instead of C++. It would be exactly the same thing apart from using the C# compiler instead of a C one.

You mean Java to C# MSIL trick ? Does it still work ?

Maybe, but it seems that it's no longer needed since now there is

[System.Runtime.InteropServices.UnmanagedCallersOnly(EntryPoint = "Java_net_sf_jni4net_Bridge_initDotNet")]

I think that you need to do only one thing, execute one static C# method and finish the bootstrap logic from there.
C# is as strong low level language a C, and Java is not.

The problem is that if the static method is not native compiled, it will fail to load, since no one will start the framework.

(It's not the same as a windows .dll, because a static [UnmanagedCallersOnly] function is indeed exposed as an entry point, but there is also logic to bootstrap the .net framework there if it's not already running.

On the other hand on linux, if you create a .dll you can only load it from .net code, if you create an .so library, it can only have native code for the time being.)

pavelsavara · 2023-11-06T18:42:17Z

This C++ code actually bootstraps .net framework and this is what I had in mind.

Cool, let's do that.

Alternatively we can use exactly the same calls and write the equivalent code in C# nativeaot instead of C++.

Please don't.

Maybe, but it seems that it's no longer needed since now there is
[System.Runtime.InteropServices.UnmanagedCallersOnly(EntryPoint = "Java_net_sf_jni4net_Bridge_initDotNet")]

Are you sure this doesn't need AOT ?

iansmirlis · 2023-11-06T19:24:55Z

Are you sure this doesn't need AOT ?

Yes it's pure .net. I didn't test it yet though, I will have to setup a build system on a windows vm at some point.

One thing I don't get though is the following:

BridgeExport expects 2 pointer parameters here:
https://github.com/jni4net/jni4net/blob/ac2189c37253710e7b729797631419b0bf3b8559/jni4net.n.w64.v40/src/BridgeExport.cs#L27

The initialization from the Java side appears to be done here
https://github.com/jni4net/jni4net/blob/ac2189c37253710e7b729797631419b0bf3b8559/jni4net.j/src/main/java/net/sf/jni4net/CLRLoader.java#L36-L37C41

The Bridge.initdotnet() is declared here
https://github.com/jni4net/jni4net/blob/ac2189c37253710e7b729797631419b0bf3b8559/jni4net.j/src/main/java/net/sf/jni4net/Bridge.java#L119-L120

How do these 2 pointers manage to arrive from Java to BridgeExport?

pavelsavara · 2023-11-06T19:30:10Z

How do these 2 pointers manage to arrive from Java to BridgeExport?

not only name, but parameters are also by JNI convention. For instance method it would be env, this and for static env, class pointers.

iansmirlis · 2023-11-06T19:31:24Z

Oh ok I got it

iansmirlis · 2023-11-06T20:09:35Z

From what I read, it seems that the same problem will occur on windows as well. It seems that loading a dll from native code on windows, will not load .net framework as I thought on my previous post.

I'm wondering how and why this bridge from java->.net worked in the first place.

pavelsavara · 2023-11-07T10:20:52Z

I'm wondering how and why this bridge from java->.net worked in the first place.

selvin's MSIL magic as I described above.

iansmirlis added bug Something isn't working runtime labels Oct 31, 2023

iansmirlis self-assigned this Oct 31, 2023

iansmirlis mentioned this issue Oct 31, 2023

Memory leak on HeavyCallMeFromJava Unit Test #3

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't initialize jni4net jni4net.n.l64.m26-0.8.9.0.dll invalid ELF header (Possible cause: endianness mismatch) #10

Can't initialize jni4net jni4net.n.l64.m26-0.8.9.0.dll invalid ELF header (Possible cause: endianness mismatch) #10

iansmirlis commented Oct 31, 2023

pavelsavara commented Nov 1, 2023

pavelsavara commented Nov 1, 2023 •

edited

Loading

iansmirlis commented Nov 1, 2023

pavelsavara commented Nov 2, 2023 •

edited

Loading

iansmirlis commented Nov 2, 2023 •

edited

Loading

iansmirlis commented Nov 4, 2023 •

edited

Loading

pavelsavara commented Nov 4, 2023

pavelsavara commented Nov 4, 2023

iansmirlis commented Nov 4, 2023

iansmirlis commented Nov 4, 2023 •

edited

Loading

pavelsavara commented Nov 6, 2023

iansmirlis commented Nov 6, 2023 •

edited

Loading

pavelsavara commented Nov 6, 2023

iansmirlis commented Nov 6, 2023

pavelsavara commented Nov 6, 2023

iansmirlis commented Nov 6, 2023

iansmirlis commented Nov 6, 2023

pavelsavara commented Nov 7, 2023

Can't initialize jni4net jni4net.n.l64.m26-0.8.9.0.dll invalid ELF header (Possible cause: endianness mismatch) #10

Can't initialize jni4net jni4net.n.l64.m26-0.8.9.0.dll invalid ELF header (Possible cause: endianness mismatch) #10

Comments

iansmirlis commented Oct 31, 2023

pavelsavara commented Nov 1, 2023

pavelsavara commented Nov 1, 2023 • edited Loading

iansmirlis commented Nov 1, 2023

pavelsavara commented Nov 2, 2023 • edited Loading

iansmirlis commented Nov 2, 2023 • edited Loading

iansmirlis commented Nov 4, 2023 • edited Loading

pavelsavara commented Nov 4, 2023

pavelsavara commented Nov 4, 2023

iansmirlis commented Nov 4, 2023

iansmirlis commented Nov 4, 2023 • edited Loading

pavelsavara commented Nov 6, 2023

iansmirlis commented Nov 6, 2023 • edited Loading

pavelsavara commented Nov 6, 2023

iansmirlis commented Nov 6, 2023

pavelsavara commented Nov 6, 2023

iansmirlis commented Nov 6, 2023

iansmirlis commented Nov 6, 2023

pavelsavara commented Nov 7, 2023

pavelsavara commented Nov 1, 2023 •

edited

Loading

pavelsavara commented Nov 2, 2023 •

edited

Loading

iansmirlis commented Nov 2, 2023 •

edited

Loading

iansmirlis commented Nov 4, 2023 •

edited

Loading

iansmirlis commented Nov 4, 2023 •

edited

Loading

iansmirlis commented Nov 6, 2023 •

edited

Loading