Repository navigation
Inquiry About Supporting Additional Approaches and WholeBody Models #124
Description
Hello,
First, thank you for the incredible work you've put into this library! Your dedication to maintaining and improving it has clearly created a valuable resource for many of us in the community.
I wanted to ask if you have any plans to support additional approaches for pose estimation and body tracking, beyond the existing options like MediaPipe Holistic and OpenPose. Specifically, I’m curious about the possibility of integrating solutions such as OpenPifPaf WholeBody, which offers whole-body tracking capabilities.
Additionally, I’m interested in whether there are any plans to enable support for models trained on datasets like COCO-WholeBody, such as the lightweight models in the Sapiens project. These could potentially enhance functionality for specific use cases where detailed whole-body keypoints are essential.
I realize that expanding support can be a significant undertaking, so I completely understand if this is not currently a priority or in scope. Regardless, I appreciate any insights you can share about future directions or potential considerations for these approaches.
Thank you again for all your hard work on this project—it’s greatly appreciated!
Best regards.
Activity
AmitMY commentedon Nov 14, 2024
Hi! This repository is mainly designed to work with the binary file format, and is mostly agnostic to the specific pose estimation model used.
For example, let's say you run
OpenPosewhich gives you a directory of JSON files. What this library provides is a loader (https://github.com/sign-language-processing/pose/blob/master/src/python/pose_format/utils/openpose.py) that defines what pose components OpenPose has (hands, body, face), what is the names of the points and how they are connected, and then, how to load it from the JSON files. Once defined and loaded, you can now save it as a.posefile which would work with the rest of the infrastructure and utilities.AmitMY commentedon Jul 23, 2026
Closing as supported: #231 added an MMPose WholeBody estimator (
pose_format.utils.mmposewholebody), joining the existing MediaPipe Holistic, OpenPose, OpenPose-135, and AlphaPose loaders. As discussed above, the format itself is estimator-agnostic — a loader for OpenPifPaf WholeBody or any other model would follow the same pattern, and PRs for additional loaders are welcome.