Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Discussion options

Hello,

I've been working on a project in Unity that involves utilizing TensorFlow Lite's BlazePose model to detect and map the 3D coordinates of human body joints to a 3D Mecanim humanoid. While I've managed to get the model to detect the joints and return the coordinates, I've been experiencing issues with the mapping itself.

The problem seems to be that the coordinates I'm obtaining from the BlazePose model aren't working correctly when I try to map them to the 3D Mecanim humanoid. I'm not sure if this is a coordinate issue or if there's another problem, such as an orientation issue.

I've tried to troubleshoot the issue by comparing the coordinates I'm obtaining from the BlazePose model to the actual coordinates of the Mecanim humanoid, but so far I haven't been able to identify any discrepancies.

Additionally, I've just attempted a simple test where I create a direction vector from the shoulder 3D coordinate to the elbow 3D coordinate and try to apply it to the 3D Mecanim avatar. However, the avatar behaves strangely, which suggests that there may be a fundamental problem with the coordinate system or orientation.

I would like to clarify that I am not referring to any stuttering issues, but rather a possible orientation issue that is affecting the behavior of the avatar.

If anyone has any insights or suggestions on how to resolve this issue, I would greatly appreciate it.
Thank you in advance for your help!

You must be logged in to vote

Replies: 1 comment

Comment options

@tolotratlt do you have a capture video?

MediaPipe provides 2 types of keypoints: pose_landmarks and pose_world_landmarks. In the Unity example

public Vector4[] viewportLandmarks;
public Vector4[] worldLandmarks;

If you are using viewportLandmarks, please try to use worldLandmarks, which might have a more accurate z coordinate kye-point.

Please refer to the Here's the original document in the MediaPipe repo.

I am unfamiliar with avatar systems using body tracking, but it might be better to use a system like IK rather than mapping 3D coordinates directly.

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
🙏
Q&A
Labels
None yet
2 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.