# mimicrig

Mapping 2D photo poses to Mixamo Blender rigs.

Single command line: takes a reference image and a `.blend` containing a Mixamo-rigged character, and produces a posed `.blend` plus a front-view preview PNG.

```bash
./pose.sh <reference.png> <input.blend> <output.blend>
```

Outputs:
- `<output.blend>` — the posed scene
- `<output>.png` — front-view preview render (same basename)

## Example

```bash
./pose.sh inputs/pose-test.png inputs/rosella-hunyuan-online.blend out/pose-test_posed.blend
```

## How it works

1. **RTMPose3D inference** on the reference image, in the `rtmpose3d` conda env, produces 133 3D keypoint coordinates (COCO-WholeBody).
2. **`apply_pose.py`** runs inside Blender:
   - Picks the first armature whose bone names start with `mixamorig:` and which has a mesh child (skips OpenPose debug rigs etc.).
   - Clears any animation and resets to rest pose.
   - **Pure FK direction matching**: for each major bone, rotates so its Y axis points from one keypoint to the next, processed parent-first.
   - **Head yaw via `orient_bone()`**: the neck uses a full two-axis orientation (vertical Y + horizontal ear axis projection) to capture head turn around the vertical axis. `point_bone` alone can't represent rotation around the bone's own axis.
   - **Forward-monotone arm chain**: mirrors arm joints whose Y is behind their parent — a safety net for monocular depth flips that put wrists behind the body.
   - **Floor settle**: after rotations, translates the whole armature in Z so the lowest foot point sits at world Z=0. Fixes the "floating squat" problem for low-stance poses.
3. **Preview render**: front-view, framed to the posed character, lit and saved alongside the output `.blend`.

### Bones the script does NOT rotate

- **Hips, Spine, Spine1, Spine2**: keeping the spine straight avoids unnatural torso lean from monocular depth noise.
- **Head**: rest pose has the face pointing forward; rotating it from a single nose keypoint causes chin-down tilt because RTMPose3D places the nose well in front of the shoulders. Yaw is captured via the neck instead.
- **Feet/toes**: RTMPose3D's toe Z is consistently too low, which would make the feet point downward like ballet pointe. Rest pose (feet flat) is more accurate for standing/sitting poses.
- **Clavicle (Shoulder)**: doubly-rotating clavicle + upper arm causes the shoulder joint to over-extend.

## Best results

- Use a **head-on full-body reference image**. RTMPose3D's monocular depth estimation is much cleaner from a frontal view; angled views get the wrong front/back for individual joints.
- The character mesh and rig should be in **Mixamo bone-name convention** (`mixamorig:LeftArm` etc.).
- Rest pose should have the character **facing -Y** in world (Mixamo default).

## Layout

```
mimicrig/
├── pose.sh             # CLI wrapper
├── apply_pose.py       # Blender script (retargeting + preview render)
├── inputs/             # Test reference images and the rig blend
│   ├── *.png
│   └── rosella-hunyuan-online.blend
└── out/                # Outputs (created on first run)
```

## Dependencies

- **Blender 4.x** at `/opt/blender-4.3.2-linux-x64/blender` (override via `$BLENDER`)
- **Conda env `rtmpose3d`** with mmpose + mmdet + mmcv installed
- **mmpose checkout** with the `projects/rtmpose3d` directory, defaulting to `~/dev/playground/mmpose` (override via `$MMPOSE_DIR`)
- **RTMPose3D model weights** cached at `~/.cache/torch/hub/checkpoints/` — auto-downloaded by the demo script on first run

The mmpose checkout is treated as an external **library** here; this repo only holds the retargeting glue.