numerical instabilities & occlusion

[mockup-partner]

Hi

Thanks for your amazing work. Arteq is able to really provide quickly better performance compared to other baselines. I had two questions related to your repo.

I noticed that in your training script you manually set to zero any gradient that would be nan and indeed some nans occur time to time during training. Do you have any ideas where they could from ?

Also on a side note, after some experiment and training, I observed that arteq seems to work even on partial point clouds (for example front view like depth map from stereo images) but struggles more when it comes to a complete temporary occlusion for example an obstacle hiding the whole elbows. Do you have any suggestions at training time as how to improve its performance in this kind of situation ? For example would conditioning pose estimation to its parent but also grand-parent or even child make sense ?

Thanks in advance and again congratulation for your great work.

[mathXin112]

@mockup-partner Hello, I would like to ask, when you are training the network, what kind of training resources do you use

[mockup-partner]

Hi @mathXin112, if your question is about the dataset, I am using AMASS and if it's about the compute power, I simply use a RTX 3070 on cuda 12.3.
Hope it answers your question.

[mockup-partner]

Hi @mathXin112 , if your question is about the dataset, I am using AMASS. And if it's about the compute power, I am simply using my RTX3070 on cuda 12.3.

Hope it answers your question.

[mathXin112]

----------------The message has been received，best wishes!您好，邮件已经收到，祝好。

[DavidTu21]

Hi @mockup-partner. I hope you are doing well! I am also interested in the model's performance on partial input human point cloud, but after some testing or inferencing, the model does not produce good results on partial point cloud (please find the attached inference results for partial input and output, and this model is trained on the default setting). May I ask did you train the model on any other data? Thank you in advance for your time!

Input:

Output: