mimicrig

Mapping 2D photo poses to Mixamo Blender rigs.

Single command line: takes a reference image and a .blend containing a Mixamo-rigged character, and produces a posed .blend plus a front-view preview PNG.

./pose.sh <reference.png> <input.blend> <output.blend>

Outputs:

• <output.blend> — the posed scene
• <output>.png — front-view preview render (same basename)

Example

./pose.sh inputs/pose-test.png inputs/rosella-hunyuan-online.blend out/pose-test_posed.blend

How it works

1. RTMPose3D inference on the reference image, in the rtmpose3d conda env, produces 133 3D keypoint coordinates (COCO-WholeBody).
2. apply_pose.py runs inside Blender:
   • Picks the first armature whose bone names start with mixamorig: and which has a mesh child (skips OpenPose debug rigs etc.).
   • Clears any animation and resets to rest pose.
   • Pure FK direction matching: for each major bone, rotates so its Y axis points from one keypoint to the next, processed parent-first.
   • Head yaw via orient_bone(): the neck uses a full two-axis orientation (vertical Y + horizontal ear axis projection) to capture head turn around the vertical axis. point_bone alone can't represent rotation around the bone's own axis.
   • Forward-monotone arm chain: mirrors arm joints whose Y is behind their parent — a safety net for monocular depth flips that put wrists behind the body.
   • Floor settle: after rotations, translates the whole armature in Z so the lowest foot point sits at world Z=0. Fixes the "floating squat" problem for low-stance poses.
3. Preview render: front-view, framed to the posed character, lit and saved alongside the output .blend.

Bones the script does NOT rotate

• Hips, Spine, Spine1, Spine2: keeping the spine straight avoids unnatural torso lean from monocular depth noise.
• Head: rest pose has the face pointing forward; rotating it from a single nose keypoint causes chin-down tilt because RTMPose3D places the nose well in front of the shoulders. Yaw is captured via the neck instead.
• Feet/toes: RTMPose3D's toe Z is consistently too low, which would make the feet point downward like ballet pointe. Rest pose (feet flat) is more accurate for standing/sitting poses.
• Clavicle (Shoulder): doubly-rotating clavicle + upper arm causes the shoulder joint to over-extend.

Best results

• Use a head-on full-body reference image. RTMPose3D's monocular depth estimation is much cleaner from a frontal view; angled views get the wrong front/back for individual joints.
• The character mesh and rig should be in Mixamo bone-name convention (mixamorig:LeftArm etc.).
• Rest pose should have the character facing -Y in world (Mixamo default).

Layout

mimicrig/
├── pose.sh             # CLI wrapper
├── apply_pose.py       # Blender script (retargeting + preview render)
├── inputs/             # Test reference images and the rig blend
│   ├── *.png
│   └── rosella-hunyuan-online.blend
└── out/                # Outputs (created on first run)

Dependencies

• Blender 4.x at /opt/blender-4.3.2-linux-x64/blender (override via $BLENDER)
• Conda env rtmpose3d with mmpose + mmdet + mmcv installed
• mmpose checkout with the projects/rtmpose3d directory, defaulting to ~/dev/playground/mmpose (override via $MMPOSE_DIR)
• RTMPose3D model weights cached at ~/.cache/torch/hub/checkpoints/ — auto-downloaded by the demo script on first run

The mmpose checkout is treated as an external library here; this repo only holds the retargeting glue.