Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New ARToolKitNFT class in the C++ Emscripten code #301

Merged
merged 22 commits into from
Apr 4, 2023

Conversation

kalwalt
Copy link
Member

@kalwalt kalwalt commented Mar 29, 2023

I will rewrite the C++ code, following this structure:

Create an ARToolKitNFT class and export it thanks to Emscripten bindings to WASM, instead of the function export in an extern C as was before.

Consequently change the Typescript, code i will add these functions/changes:

@kalwalt kalwalt added enhancement New feature or request C/C++ emscripten labels Mar 29, 2023
@kalwalt kalwalt self-assigned this Mar 29, 2023
src/index_improved.ts Outdated Show resolved Hide resolved
kalwalt and others added 8 commits March 29, 2023 23:19
- removing decorate() for instantitating methods
- static properties exposed explictly
- all methods declared inside ARToolkitNFT class
- removing some not necessary properties in ARControllerNFT_improved
- webpack and typescript
@kalwalt
Copy link
Member Author

kalwalt commented Apr 1, 2023

I think i have implemented all the planned features, i'll make a last check to be sure that it is ready to be merged.

@kalwalt
Copy link
Member Author

kalwalt commented Apr 2, 2023

Sometimes testing the new lib, i get this error:

Uncaught RuntimeError: null function or function signature mismatch
    at 001c84ce:0x5966
    at 001c84ce:0x3ca2a
    at 001c84ce:0x5fb60
    at 001c84ce:0xd44d
    at fA.ARToolKitNFT$detectNFTMarker [as detectNFTMarker] (eval at _A (ARToolkitNFT_simd.js:2:678066), <anonymous>:8:10)
    at A.value (ARToolkitNFT_simd.js:2:739639)
    at A.value (ARToolkitNFT_simd.js:2:746781)
    at A.value (ARToolkitNFT_simd.js:2:746000)
    at process (artoolkitNFT_ES6_gray.worker.js:151:8)
    at self.onmessage (artoolkitNFT_ES6_gray.worker.js:47:7)

When i recevive this error, after it's impossible to continue to detect and track. It looks strange, at first sight i was thinking that the reason was some bad bug in the typescript code ( a void function instead of function returning a number) but seems a silent issue instead in WASM (??) I think i nedd to debug adding some flags in makem.js (-g4 and other debug symbols).

@kalwalt
Copy link
Member Author

kalwalt commented Apr 2, 2023

Added some flags to have more info about this issue, running the ARToolkitNFT_ES6_example.html i got now:

artoolkitNFT_ES6.worker.js:125 RuntimeError: memory access out of bounds
    at icpDeleteHandle (003d3f82:0x11827)
    at ARToolKitNFT::deleteHandle() (003d3f82:0x29669)
    at ARToolKitNFT::setCamera(int, int) (003d3f82:0x29dc4)
    at ARToolKitNFT::setup(int, int, int) (003d3f82:0x32936)
    at emscripten::internal::MethodInvoker<int (ARToolKitNFT::*)(int, int, int), int, ARToolKitNFT*, int, int, int>::invoke(int (ARToolKitNFT::* const&)(int, int, int), ARToolKitNFT*, int, int, int) (003d3f82:0x39c44)
    at PA.ARToolKitNFT$setup [as setup] (eval at zI (ARToolkitNFT_simd.js:2:1410886), <anonymous>:11:10)
    at A.value (ARToolkitNFT_simd.js:2:1486634)
    at A.<anonymous> (ARToolkitNFT_simd.js:2:1498899)
    at s (ARToolkitNFT_simd.js:2:1015)
    at Generator.<anonymous> (ARToolkitNFT_simd.js:2:2370)
onError @ artoolkitNFT_ES6.worker.js:125
Promise.catch (asinc)
load @ artoolkitNFT_ES6.worker.js:133
self.onmessage @ artoolkitNFT_ES6.worker.js:39

Note deleteHandle is:

void ARToolKitNFT::deleteHandle() {
if (this->arhandle != NULL) {
arPattDetach(this->arhandle);
arDeleteHandle(this->arhandle);
this->arhandle = NULL;
}
if (this->ar3DHandle != NULL) {
ar3DDeleteHandle(&(this->ar3DHandle));
this->ar3DHandle = NULL;
}
if (this->paramLT != NULL) {
arParamLTFree(&(this->paramLT));
this->paramLT = NULL;
}
}

icpDeleteHandle is invoked by
ar3DDeleteHandle(&(this->ar3DHandle));

in our implementation
AR3DHandle *ar3DHandle;
we not assign a value during the initialization in the constructor. I tested and if init AR3DHandle *ar3DHandle to NULL this issue is avoided.
In general i think it's better to init the parameter in the constructor, i think i will opt to this code design.

@kalwalt
Copy link
Member Author

kalwalt commented Apr 2, 2023

I solved the issue above initializing the private fields in the .cpp file, now i got another error (while testing ARToolkitNFT_ES6_example.html) related to the passVideoData function. Exactly i get an heap-buffer-overflow on address 0x16204d00 in more specific it points to the memcpy invocations inside the passVideoData

int ARToolKitNFT::passVideoData(emscripten::val videoFrame,
emscripten::val videoLuma) {
std::vector<uint8_t> vf =
emscripten::convertJSArrayToNumberVector<uint8_t>(videoFrame);
std::vector<uint8_t> vl =
emscripten::convertJSArrayToNumberVector<uint8_t>(videoLuma);
memcpy(this->videoFrame, vf.data(), sizeof(uint8_t) * this->videoFrameSize);
memcpy(this->videoLuma, vl.data(),
sizeof(uint8_t) * this->videoFrameSize / 4);
return 0;
}

I receive an analog error with the basic.html, but without the additional infos because i added the -fsanitize=address and debug symbols only to the ES6 lib. But i'm quite sure that is caused by this.
Anyway it is not caused by this PR but this is a silent issue to be checked.

gist

@kalwalt
Copy link
Member Author

kalwalt commented Apr 2, 2023

this should be better:

// ARToolKitNFT_js.cpp
// inside passVideoData()
// instead of memcpy() stuff...
this->videoFrame = vf.data();
this->videoLuma = vl.data();

// ARToolKitJS.cpp
// inside passVideoData()
// instead of memcpy() stuff...
arc->videoFrame = vf.data();
arc->videoLuma = vl.data();

This works but i need to add -sSTACK_SIZE=20mb otherwise i got a stack overflow exception, maybe because the other flags?
I will test removing all the debug flags.

Why i used memcpy? i can not remember the reason...

- avoid memcpy in passVideoData
@kalwalt
Copy link
Member Author

kalwalt commented Apr 2, 2023

Without the debug flags i don't need to add the -sSTACK_SIZE=20mb, i don't encounter any other issue for now. but i need to report it in the issue tracker to take trace of it. I did most of the time test on a desktop machine i will test on Android to see if it is ok.

@kalwalt
Copy link
Member Author

kalwalt commented Apr 3, 2023

Debugging ES6 examples in Android i get this error:

artoolkitNFT_ES6.worker.js:125 RuntimeError: Aborted(RangeError: WebAssembly.instantiate(): Out of memory: Cannot allocate Wasm memory for new instance). Build with -sASSERTIONS for more info.

I got this in the past but i need to understand if it is caused by new changes.

@kalwalt
Copy link
Member Author

kalwalt commented Apr 3, 2023

adding an -sASSERTIONS=2 flag to only he ES6 version of the lib i get a stack overflow:

artoolkitNFT_ES6_wasm.js:8 Aborted(stack overflow (Attempt to set SP to 0x0000de00, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Aborted(stack overflow (Attempt to set SP to 0x0000dd10, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Aborted(stack overflow (Attempt to set SP to 0x0000dc20, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Aborted(stack overflow (Attempt to set SP to 0x0000db30, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Uncaught RuntimeError: Aborted(stack overflow (Attempt to set SP to 0x0000de00, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
    at abort (artoolkitNFT_ES6_wasm.js:8:14583)
    at ___handle_stack_overflow (artoolkitNFT_ES6_wasm.js:8:687719)
    at 001e4732:0x1c7f
    at 001e4732:0x2bdc9
    at 001e4732:0x2f5f9
    at ClassHandle.ARToolKitNFT$detectMarker [as detectMarker] (eval at new_ (artoolkitNFT_ES6_wasm.js:8:765162), <anonymous>:8:10)
    at ARToolkitNFT.detectMarker (ARToolkitNFT.ts:237:26)
    at ARControllerNFT.detectMarker (ARControllerNFT.ts:400:32)
    at ARControllerNFT.process (ARControllerNFT.ts:280:23)
    at process (artoolkitNFT_ES6.worker.js:140:8)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Uncaught RuntimeError: Aborted(stack overflow (Attempt to set SP to 0x0000dd10, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
    at abort (artoolkitNFT_ES6_wasm.js:8:14583)
    at ___handle_stack_overflow (artoolkitNFT_ES6_wasm.js:8:687719)
    at 001e4732:0x1c7f
    at 001e4732:0x2bdc9
    at 001e4732:0x2f5f9
    at ClassHandle.ARToolKitNFT$detectMarker [as detectMarker] (eval at new_ (artoolkitNFT_ES6_wasm.js:8:765162), <anonymous>:8:10)
    at ARToolkitNFT.detectMarker (ARToolkitNFT.ts:237:26)
    at ARControllerNFT.detectMarker (ARControllerNFT.ts:400:32)
    at ARControllerNFT.process (ARControllerNFT.ts:280:23)
    at process (artoolkitNFT_ES6.worker.js:140:8)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Uncaught RuntimeError: Aborted(stack overflow (Attempt to set SP to 0x0000dc20, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
    at abort (artoolkitNFT_ES6_wasm.js:8:14583)
    at ___handle_stack_overflow (artoolkitNFT_ES6_wasm.js:8:687719)
    at 001e4732:0x1c7f
    at 001e4732:0x2bdc9
    at 001e4732:0x2f5f9
    at ClassHandle.ARToolKitNFT$detectMarker [as detectMarker] (eval at new_ (artoolkitNFT_ES6_wasm.js:8:765162), <anonymous>:8:10)
    at ARToolkitNFT.detectMarker (ARToolkitNFT.ts:237:26)
    at ARControllerNFT.detectMarker (ARControllerNFT.ts:400:32)
    at ARControllerNFT.process (ARControllerNFT.ts:280:23)
    at process (artoolkitNFT_ES6.worker.js:140:8)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Uncaught RuntimeError: Aborted(stack overflow (Attempt to set SP to 0x0000db30, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
    at abort (artoolkitNFT_ES6_wasm.js:8:14583)
    at ___handle_stack_overflow (artoolkitNFT_ES6_wasm.js:8:687719)
    at 001e4732:0x1c7f
    at 001e4732:0x2bdc9
    at 001e4732:0x2f5f9
    at ClassHandle.ARToolKitNFT$detectMarker [as detectMarker] (eval at new_ (artoolkitNFT_ES6_wasm.js:8:765162), <anonymous>:8:10)
    at ARToolkitNFT.detectMarker (ARToolkitNFT.ts:237:26)
    at ARControllerNFT.detectMarker (ARControllerNFT.ts:400:32)
    at ARControllerNFT.process (ARControllerNFT.ts:280:23)
    at process (artoolkitNFT_ES6.worker.js:140:8)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44

@kalwalt
Copy link
Member Author

kalwalt commented Apr 3, 2023

if i enlarge the stack -sSTACK_SIZE=2mb it works for the first time but after the out of Memory error happens again and are affected all the examples. So maybe an issue only with my device? Simd or not Simd is the same. I may test with another device to see if there are differences. Note that on desktop i never had it.

@kalwalt
Copy link
Member Author

kalwalt commented Apr 4, 2023

last commit should fix issue RuntimeError: Aborted(RangeError: WebAssembly.instantiate(): Out of memory: Cannot allocate Wasm memory for new instance). and fix issue #97 and consequently will fix also webarkit/ARnft#212 but i will need to updated ARnft dependencies. This PR is ready to be merged.

@kalwalt kalwalt added dependencies Pull requests that update a dependency file Typescript labels Apr 4, 2023
@kalwalt kalwalt merged commit a7bfcf3 into dev Apr 4, 2023
@kalwalt kalwalt deleted the feat-ARToolKitNFT-em-class branch January 5, 2024 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C/C++ dependencies Pull requests that update a dependency file emscripten enhancement New feature or request Typescript
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant