[Windows Editor] Exported Package looks for .bytes files in local AppData Directory in new project, doesn't copy Streaming Assets folder

[ROBYER1]

Plugin Version or Commit ID

v0.9.1

Unity Version

2021.2.19f1

Your Host OS

Windows 10

Target Platform

UnityEditor, Windows Standalone

Target Device

Windows 10 Desktop/Editor

[Windows Only] Visual Studio C++ and Windows SDK Version

Visual Studio C++ : 14.29.30133
Windows SDK : 10.0.18362.0

[Linux Only] GCC/G++ and GLIBC Version

No response

[Android Only] Android Build Tools and NDK Version

No response

[iOS Only] XCode Version

No response

Build Command

python build.py build --desktop cpu --opencv cmake -vv

Bug Description

Using this project cloned from this commit, I built Mediapipe for Windows Desktop using Docker, and edited the Mediapipe_api>BUILD file to exclude the things we didn't need so it just had hand tracking for desktop.

We exported the package using the Tools > Export package command, and imported into a brand new project.

Exported package

Once imported, we had the errors due to hand_recrop.bytes not being in the package imported somehow.

HandTrackingGraph: System.NullReferenceException: Object reference not set to an instance of an object
  at Mediapipe.Unity.LocalResourceManager+<PrepareAssetAsync>d__11.MoveNext () [0x000b3] in C:\Unity_Projects\TestMedImport\Packages\com.github.homuler.mediapipe\Runtime\Scripts\Unity\ResourceManager\LocalResourceManager.cs:60 
  at Mediapipe.Unity.WaitForResult+<Run>d__20.MoveNext () [0x0007c] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\WaitForResult.cs:51 
UnityEngine.Logger:LogError (string,object)
Mediapipe.Unity.MemoizedLogger:LogError (string,object) (at Assets/MediaPipeUnity/Samples/Common/Scripts/MemoizedLogger.cs:199)
Mediapipe.Unity.Logger:LogError (string,object) (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/Logger.cs:90)
Mediapipe.Unity.GraphRunner/<Initialize>d__39:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/GraphRunner.cs:132)
Mediapipe.Unity.WaitForResult/<Run>d__20:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/WaitForResult.cs:51)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)

and

HandTrackingSolution: Mediapipe.InternalException: Failed to prepare dependent assets
  at Mediapipe.Unity.GraphRunner+<Initialize>d__39.MoveNext () [0x00180] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\GraphRunner.cs:134 
  at Mediapipe.Unity.WaitForResult+<Run>d__20.MoveNext () [0x0007c] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\WaitForResult.cs:51 
UnityEngine.Logger:LogError (string,object)
Mediapipe.Unity.MemoizedLogger:LogError (string,object) (at Assets/MediaPipeUnity/Samples/Common/Scripts/MemoizedLogger.cs:199)
Mediapipe.Unity.Logger:LogError (string,object) (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/Logger.cs:90)
Mediapipe.Unity.ImageSourceSolution`1/<Run>d__12<Mediapipe.Unity.HandTracking.HandTrackingGraph>:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/ImageSourceSolution.cs:79)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)

To fix this, I noticed that the Bootstrap Asset Loader Type was loading from 'Local' location by default which for us was this:
C:\Users####USERNAME####\AppData\LocalLow\DefaultCompany\MediaPipeUnityPlugin

Of course in a new project when importing the package, the files from AppData don't carry across to the new project directory so that rules out Local loading in the package by default unless users are told to move the files. I had t oexplicitly copy hand_recrop.bytes.

I then switched to 'Streaming Assets' Asset Loader Type then I noticed the Streaming Assets folder with the bytes file was not copied across in the exported package either, and for some reason the plugin built the Streaming Assets folder and placed it in the project directory > Packages folder (See in the example project.)

To finally get things working, I had to do this:

1. Create new Streaming Assets folder in Assets folder of new project
2. Copy files from the AppData folder from the Mediapipe project I used to build the package - in C:\Users####USERNAME####\AppData\LocalLow\DefaultCompany\MediaPipeUnityPlugin
3. Copy the files from Packages>Streaming Assets in the Mediapipe project I used to build the package to the new project's Streaming Assets folder
4. In the Hand tracking scene on Solution > Bootstrap, I had to set the Asset Loader Type to 'Streaming Assets'

If this behaviour is unique to Windows 10 editor/builds, can it be simplified or documented for users?

Steps to Reproduce the Bug

1. (Building not required as I already included Built mediapipe in this project) Using this project cloned from this commit, I built Mediapipe for Windows Desktop using Docker (BUILD file edited to just build hand tracking and a bit of Pose Tracking)
2. Make a new Unity project fresh
3. In the built mediapipe project, use Tools>Export Package
4. Open fresh project, import the package and try to run the sample scenes
5. See errors that hand_recrop.bytes is missing

Log

Errors from editor:

HandTrackingGraph: System.NullReferenceException: Object reference not set to an instance of an object
  at Mediapipe.Unity.LocalResourceManager+<PrepareAssetAsync>d__11.MoveNext () [0x000b3] in C:\Unity_Projects\TestMedImport\Packages\com.github.homuler.mediapipe\Runtime\Scripts\Unity\ResourceManager\LocalResourceManager.cs:60 
  at Mediapipe.Unity.WaitForResult+<Run>d__20.MoveNext () [0x0007c] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\WaitForResult.cs:51 
UnityEngine.Logger:LogError (string,object)
Mediapipe.Unity.MemoizedLogger:LogError (string,object) (at Assets/MediaPipeUnity/Samples/Common/Scripts/MemoizedLogger.cs:199)
Mediapipe.Unity.Logger:LogError (string,object) (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/Logger.cs:90)
Mediapipe.Unity.GraphRunner/<Initialize>d__39:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/GraphRunner.cs:132)
Mediapipe.Unity.WaitForResult/<Run>d__20:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/WaitForResult.cs:51)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)

and

HandTrackingSolution: Mediapipe.InternalException: Failed to prepare dependent assets
  at Mediapipe.Unity.GraphRunner+<Initialize>d__39.MoveNext () [0x00180] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\GraphRunner.cs:134 
  at Mediapipe.Unity.WaitForResult+<Run>d__20.MoveNext () [0x0007c] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\WaitForResult.cs:51 
UnityEngine.Logger:LogError (string,object)
Mediapipe.Unity.MemoizedLogger:LogError (string,object) (at Assets/MediaPipeUnity/Samples/Common/Scripts/MemoizedLogger.cs:199)
Mediapipe.Unity.Logger:LogError (string,object) (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/Logger.cs:90)
Mediapipe.Unity.ImageSourceSolution`1/<Run>d__12<Mediapipe.Unity.HandTracking.HandTrackingGraph>:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/ImageSourceSolution.cs:79)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)

Screenshot/Video

No response

Additional Context

No response

[LGuilhermeRodrigues]

1. Create new Streaming Assets folder in Assets folder of new project
2. Copy files from the AppData folder from the Mediapipe project I used to build the package - in C:\Users####USERNAME####\AppData\LocalLow\DefaultCompany\MediaPipeUnityPlugin
3. Copy the files from Packages>Streaming Assets in the Mediapipe project I used to build the package to the new project's Streaming Assets folder
4. In the Hand tracking scene on Solution > Bootstrap, I had to set the Asset Loader Type to 'Streaming Assets'

Doesn't it work without step 2?

[ROBYER1]

Doesn't it work without step 2?

Unfortunately not, that is why I linked the small repro project here to show it, and also the exported package here

[homuler]

It seems that you've forgotten to include hand_recrop.bytes in your custom package.

P.S. Whenever possible, please share complete logs, not excerpts.
If we can know what kind of logs were being output, it will not only be easier to determine the cause but also improve the output.

[ROBYER1]

It seems that you've forgotten to include hand_recrop.bytes in your custom package.

P.S. Whenever possible, please share complete logs, not excerpts. If we can know what kind of logs were being output, it will not only be easier to determine the cause but also improve the output.

Here is the hand_recrop.bytes from C:\Users###USERNAME###\AppData\LocalLow\DefaultCompany\MediaPipeUnityPlugin

I didn't include it as it should be included in the exported package file when I use Tools>Export Package
hand_recrop.zip

It seems when exporting the package, it misses out that hand_recrop.bytes that only gets built to the AppData\LocalLow folder on Windows when building mediapipe. Should it be built to streaming assets or at least the streaming assets folder in packages like the other .bytes files do?

So is the workflow once I export the package, I also have to go to the AppData\LocalLow folder and copy any bytes files from there to the Streaming Assets and AppData\LocalLow folder for the new project I am importing the package into?

Full logs:
Editor.log

I suspect that hand_recrop.bytes gets moved not copied to the AppData folder from StreamingAssets in the built mediapipe project, this might happen when you build mediapipe then hit play in one of the test scenes as I can see the logs detail copying of .bytes files to AppData folders. I cannot see hand_recrop.bytes anywhere in the local files of the Unity project I built Mediapipe in initially.

[homuler]

First, please forget about folders under C:\Users\USERNAME\AppDatal\LocalLow.
This error occurs because hand_recrop.bytes is not included in the package (AssetDatabase fails to load the file and asset is null, so at least the Null Check is missing here).

MediaPipeUnityPlugin/Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/ResourceManager/LocalResourceManager.cs

Lines 52 to 60 in 77ba1a4

	var assetPath = GetAssetPathFor(name);
	var asset = AssetDatabase.LoadAssetAtPath<TextAsset>(assetPath);
	Logger.LogVerbose(_TAG, $"Writing {name} data to {destFilePath}...");
	if (!Directory.Exists(_CachePathRoot))
	{
	var _ = Directory.CreateDirectory(_CachePathRoot);
	}
	File.WriteAllBytes(destFilePath, asset.bytes);

I didn't include it as it should be included in the exported package file when I use Tools>Export Package

I mean MediaPipeUnity.0.9.1.unitypackage you uploaded actually does not include hand_recrop.bytes.
I guess hand_recrop.bytes does not exist in the repository used to build the package (not the project folder used to test the package) in the first place.

I built Mediapipe for Windows Desktop using Docker, and edited the Mediapipe_api>BUILD file to exclude the things we didn't need so it just had hand tracking for desktop.

Have you included hand_recrop.bytes?
I recommend that you first check to see if the same error occurs without changing mediapipe_api/BUILD.

[ROBYER1]

I found the issue, in the BUILD file I commented out the hand_recrop in one of the dependencies but I still had it kicking around in the AppData/Local folder. Totally my mistake and apologies for wasting anyone's time, this was my first time using the package export feature and I was wondering what was going on with the AppData/Local files but I can see in the scripts they just get safely copied there if they are actually in the build.

Hopefully this issue history just helps other people if they make the same mistake, now I know better what is going on I will do more investigation of the build file edited vs unedited in future. Sorry again!

[homuler]

Sorry, hand_recrop.bytes is actually not needed for HandTrackingGraph.
#606 fixed it and also enables us to select required solutions at build time.

[fhn-fhn]

Thank you. Your thread helped me a lot to figure it out. I just copied the files generated through the editor from AppData to StreamingAssets, and everything worked

[Leka1B]

Thanks! I was very confused by this exact problem and this thread helped a lot. I imported the unitypackage from the latest release, and only copying the files manually fixed it for me