feat: add FloatVectorPacket and MatrixPacket #767
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* cc: matrix_frame as input to graph - "matrix_frame" as name in order to avoid confusion with matrix.cc - matrix_frame is a 2D input data modality that gets converted to Eigen::MatrixXf internally - suited for non-image input to tflite models * cc: float_vector_frame as input to graph - "float_vector_frame" as name in order to avoid confusion with FloatArrayPacket - float_vector_frame is a 1D output data modality that accepts std::vector<float> outputs from a mediapipe / tflite graph - suited for tflite classification results packaged as vector of floats * MatrixFramePacket - c# helper functions - send MatrixData to C++ as byte array * FloatVectorFramePacket - c# helper functions - return std::vector<float> from C++ to Unity as List<float> * Unity: Matrix Classification - Example scene * Matrix Classification.cs - driver code for the newly added MatrixFramePacket and FloatVectorFramePacket - feeds an example matrix of size [ 2 x 3 ] into a mediapipe graph - the graph runs a simple tflite model (adds +1 to every input) - then the graph returns the result back to Unity as List<float> - only tested on Unity-Editor-Mode on Windows 10 Pro * refactor: rename FloatVectorFrame -> FloatVector - rename variables - rename cs files - rename cc files * refactor: rename MatrixFrame -> Matrix - rename variables - rename cs files - rename cc files matrix_frame -> matrix_data -> avoid matrix.cc as it is already used as a name in mediapipe * move MatrixClassification example scene to Tutorials - does not represent an official solution - not sure where else to place this - MatrixClassification.cs is an important example for showcasing the usage of a tflite model with a matrix data input * GetArrayPtr() - change access to private * MatrixPacket: accept MatrixData as input - before it was byte[] * add license * move native functions to Packet_Unsafe - delete FloatVector_Unsafe.cs * float_vector.cc -> faster vector allocation * float_vector.cc remove unused function - delete(...) * float_vector.h - remove unused headers * refactor: float_vector.cc TODO: - implement GetFloatVector with vector size as argument * removed unused headers * refactor: float_vector.h * refactor: apply autoformatter on cc files - using format file ".clang-format" in project root * refactor: mp__MakeMatrixFramePacket_At__PA_i_Rt -> mp__MakeMatrixPacket_At__PKc_i_Rt * FloatVectorPacketTest added - build similar to FloatArrayPacketTest - not yet tested * fix: float_vector.cc * fix: MatrixPacket.cs * fix: Test: FloatVectorPacketTest - Consume_ShouldThrowNotSupportedException * MatrixPacketTest - add - all tests involving packet.Get() do not work - function is not yet implemented * fix: Make MatrixClassification.cs run on Android - adding StreamingAssets to ResourceManager [skip actions] * Update mediapipe_api/framework/formats/matrix_data.h Co-authored-by: Junrou Nishida <[email protected]> * Apply suggestions from code review Co-authored-by: Junrou Nishida <[email protected]> * float_vector - return vector size (+2 squashed commit) Squashed commit: [e409b05] refactor: vector_float.cc - naming aligns with files like packet.cc [bad3cd6] float_vector - return vector size * fix: matrix_data.cc - wrong func name (+3 squashed commit) Squashed commit: [9245a37] fix: Revert "Apply suggestions from code review" - the below mentioned commit is not working - return value of inline function is invalid -> probably due to inline function This reverts commit c374d61. [f597e83] fix: remove duplicate cpp func [6def10c] fix: semicolon omitted * FloatVectorPacket - replace list by array fix: FloatVectorPacket [skip actions] (+1 squashed commits) Squashed commits: [69302b1] FloatVectorPacket - replace list by array - list is slow * Add license headers [skip actions] * Remove Tutorial Scene: MatrixClassification as per request: - deleted demo / tutorial scene that showcasts a simple tflite graph [skip actions] * fix: MatrixPacket tests - new GetMatrix function Caveat: - MatrixPacket: Consume throws NotSupportedException() -> not sure if this is a useful test, but such tests exists in similar classes as well (cherry picked from commit 707af5f454e87312a86b60deaebec18463e47ded) Co-authored-by: Junrou Nishida <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.