228 lines
9.7 KiB
Markdown
228 lines
9.7 KiB
Markdown
# Specification: Compass Image Loader
|
|
|
|
## Overview
|
|
|
|
A ComfyUI custom node that loads images from a structured directory hierarchy under ComfyUI's `input/` directory. The node navigates via a user-selected subdirectory, an optional compass direction, and a modality folder, then outputs matching image(s) with optional resizing.
|
|
|
|
## Directory Structure Convention
|
|
|
|
```
|
|
<comfyui_root>/input/
|
|
<project>/
|
|
<action>/
|
|
[direction]/
|
|
<modality>/
|
|
frame_0001.png
|
|
frame_0002.png
|
|
...
|
|
```
|
|
|
|
When `direction` is left blank, the direction folder is omitted:
|
|
|
|
```
|
|
<comfyui_root>/input/
|
|
<project>/
|
|
<action>/
|
|
<modality>/
|
|
frame_0001.png
|
|
...
|
|
```
|
|
|
|
**Example resolved path (with direction):** `input/rosella/walk/s/depth/frame_0001.png`
|
|
**Example resolved path (without direction):** `input/rosella/walk/image/frame_0001.png`
|
|
|
|
---
|
|
|
|
## Node Definition
|
|
|
|
### Node Name
|
|
`CompassImageLoader`
|
|
|
|
### Display Name
|
|
`Compass Image Loader`
|
|
|
|
### Category
|
|
`image/loaders`
|
|
|
|
---
|
|
|
|
## Widgets / Inputs
|
|
|
|
| Name | Type | Widget | Default | Description |
|
|
|------|------|--------|---------|-------------|
|
|
| `directory` | `STRING` | Combo with autocomplete | `""` | Recursive subdirectory path under `input/`. Populated dynamically from all directories containing at least one valid direction subfolder (n, ne, e, se, s, sw, w, nw) **or** at least one valid modality subfolder (image, depth, openpose). Supports typing to filter. |
|
|
| `direction` | `STRING` | Combo | `""` | One of: `""`, `n`, `ne`, `e`, `se`, `s`, `sw`, `w`, `nw`. Empty string omits the direction folder entirely. |
|
|
| `modality` | `STRING` | Combo | `"image"` | One of: `image`, `depth`, `openpose` |
|
|
| `frame` | `STRING` | Text | `""` | Frame index as string. Empty string or whitespace-only = load all images as a batch. `"0"` = first image (0-based). |
|
|
| `width` | `INT` | Number | `0` | Target resize width in pixels. `0` = leave original width. |
|
|
| `height` | `INT` | Number | `0` | Target resize height in pixels. `0` = leave original height. |
|
|
|
|
### Resize Behavior
|
|
- If `width == 0` and `height == 0`: no resize.
|
|
- If **only one** of `width` or `height` is non-zero: scale proportionally so that the non-zero dimension matches the target. The other dimension is computed to maintain aspect ratio. No cropping occurs.
|
|
- If **both** `width` and `height` are non-zero: use a **cover + center crop** strategy:
|
|
1. Compute `scale = max(target_width / source_width, target_height / source_height)`.
|
|
2. Resize by `scale` using Lanczos resampling.
|
|
3. Center crop to exactly `width x height`.
|
|
|
|
---
|
|
|
|
## Outputs
|
|
|
|
| Name | Type | Description |
|
|
|------|------|-------------|
|
|
| `IMAGE` | `IMAGE` | Tensor of shape `(B, H, W, C)` where `B` = batch size. `B = 1` for a single frame; `B = N` when `frame` is empty (all images loaded as a batch). |
|
|
| `path` | `STRING` | The resolved file system path. For single frame: absolute path to the image file. For batch mode: absolute path to the directory containing the images. |
|
|
| `width` | `INT` | Final width in pixels after any resizing. |
|
|
| `height` | `INT` | Final height in pixels after any resizing. |
|
|
| `frame_count` | `INT` | Number of frames loaded. `1` for single frame, `N` for batch mode. |
|
|
|
|
---
|
|
|
|
## Core Behavior
|
|
|
|
### 1. Base Directory Resolution
|
|
- Base is always `folder_paths.get_input_directory()`.
|
|
- The user does **not** configure this.
|
|
|
|
### 2. Directory Discovery (Autocomplete)
|
|
- The backend recursively scans the `input/` directory.
|
|
- A directory is eligible for the `directory` combo if it satisfies **either** of the following:
|
|
- It contains at least one valid direction subfolder (`n`, `ne`, `e`, `se`, `s`, `sw`, `w`, `nw`), **or**
|
|
- It contains at least one valid modality subfolder (`image`, `depth`, `openpose`).
|
|
- The list of eligible paths is provided as the combo values for the `directory` widget.
|
|
- The frontend extension enhances this combo with typing-to-filter autocomplete behavior.
|
|
|
|
### 3. Path Construction
|
|
```python
|
|
if direction and direction.strip():
|
|
full_dir = os.path.join(input_dir, directory, direction, modality)
|
|
else:
|
|
full_dir = os.path.join(input_dir, directory, modality)
|
|
```
|
|
|
|
### 4. Image Discovery
|
|
- List all files in `full_dir`.
|
|
- Filter to supported image content types. Use `folder_paths.filter_files_content_types` if available, otherwise filter by standard image extensions (`png`, `jpg`, `jpeg`, `webp`, `bmp`, `gif`, `tiff`).
|
|
- Sort alphabetically by filename. This sorted order defines frame indices (`0 = first file alphabetically`).
|
|
|
|
### 5. Frame Selection
|
|
- If `frame` is empty or whitespace-only: select all images (batch mode).
|
|
- Else: parse `frame` as a base-10 integer. Select the single image at that 0-based index.
|
|
- If the parsed index is negative or `>= len(images)`, raise `RuntimeError` with a clear message.
|
|
|
|
### 6. Image Loading & Resizing
|
|
- Open each selected file with PIL.
|
|
- Convert to RGB (`image.convert("RGB")`).
|
|
- If resize is required:
|
|
- **Proportional resize (one dimension zero):**
|
|
- If `width > 0` and `height == 0`: `new_height = int(source_height * (width / source_width))`.
|
|
- If `height > 0` and `width == 0`: `new_width = int(source_width * (height / source_height))`.
|
|
- Use `Image.Resampling.LANCZOS`.
|
|
- **Cover + Crop (both dimensions non-zero):**
|
|
- `scale = max(width / source_width, height / source_height)`.
|
|
- `new_size = (int(source_width * scale), int(source_height * scale))`.
|
|
- `image = image.resize(new_size, Image.Resampling.LANCZOS)`.
|
|
- Compute crop box to center the crop:
|
|
- `left = (new_size[0] - width) // 2`
|
|
- `top = (new_size[1] - height) // 2`
|
|
- `right = left + width`
|
|
- `bottom = top + height`
|
|
- `image = image.crop((left, top, right, bottom))`.
|
|
|
|
### 7. Tensor Conversion
|
|
- `np.array(image).astype(np.float32) / 255.0`
|
|
- `torch.from_numpy(image)[None,]` (adds batch dimension of 1)
|
|
- For batch mode, `torch.cat(tensors, dim=0)` to merge into `(N, H, W, C)`.
|
|
|
|
### 8. Return Values
|
|
- `path`:
|
|
- Single frame: `os.path.join(full_dir, filename)` (absolute path).
|
|
- Batch mode: `full_dir` (absolute path).
|
|
- `width`, `height`: Dimensions of the final tensor.
|
|
- `frame_count`: `len(images)`.
|
|
|
|
---
|
|
|
|
## Error Handling
|
|
|
|
| Scenario | Behavior |
|
|
|----------|----------|
|
|
| `directory` does not exist | `RuntimeError("Compass directory not found: {full_dir}")` |
|
|
| `full_dir` contains no supported images | `RuntimeError("No images found in: {full_dir}")` |
|
|
| `frame` index out of bounds | `RuntimeError("Frame index {n} out of bounds. Found {m} images in {full_dir}.")` |
|
|
| `frame` string is not a valid integer | `RuntimeError("Invalid frame number: '{frame}'. Must be an integer.")` |
|
|
|
|
---
|
|
|
|
## Frontend Extension Requirements
|
|
|
|
### File
|
|
`js/compass_image_loader.js`
|
|
|
|
### Responsibilities
|
|
1. **Register Extension:** Use `app.registerExtension({ name: "CompassImageLoader", ... })`.
|
|
2. **Dynamic Combo Refresh:** Listen for the `fresh-node-defs` event fired by ComfyUI's `api.js` (which happens on startup and when the user clicks Refresh). When received, update the `directory` combo widget on existing node instances with the latest values from the node's definition.
|
|
3. **Autocomplete / Filtering:** Override the `directory` combo widget behavior to allow the user to type and filter the dropdown list in real-time (rgthree-style). The filtered list should be case-insensitive.
|
|
4. **State Preservation:** Ensure the user's selected `directory` value is preserved across page reloads and workflow loads. Do not reset the value when refreshing the combo list unless the previously selected value is no longer in the new list.
|
|
|
|
---
|
|
|
|
## Backend API for Frontend Refresh
|
|
|
|
To support the frontend refreshing the directory list without reloading the page, the backend exposes a lightweight HTTP endpoint. However, the standard ComfyUI pattern is to rely on the node definition refresh (clicking the Refresh button in the UI), which re-fetches `api.getNodeDefs()`. Therefore, the node must ensure its `INPUT_TYPES` classmethod dynamically rebuilds the directory list every time it is called.
|
|
|
|
No custom API endpoint is required. The frontend simply refreshes the combo values from the re-fetched node definition.
|
|
|
|
---
|
|
|
|
## Dependencies
|
|
|
|
- `folder_paths` (ComfyUI builtin)
|
|
- `torch`
|
|
- `numpy`
|
|
- `PIL` (Pillow)
|
|
- `os`
|
|
|
|
No external pip dependencies are required beyond what ComfyUI already provides.
|
|
|
|
---
|
|
|
|
## Example Workflows
|
|
|
|
### Example 1: Single Frame, with Direction, Resized
|
|
- `directory`: `rosella/walk`
|
|
- `direction`: `s`
|
|
- `modality`: `depth`
|
|
- `frame`: `0`
|
|
- `width`: `512`
|
|
- `height`: `768`
|
|
|
|
**Result:** Loads `input/rosella/walk/s/depth/<first_image>`, resizes to 512x768 with cover-crop + Lanczos. Returns `IMAGE` [1, 768, 512, 3], path string, width=512, height=768, frame_count=1.
|
|
|
|
### Example 2: Batch Mode, No Direction, No Resize
|
|
- `directory`: `hero/run`
|
|
- `direction`: *(blank)*
|
|
- `modality`: `image`
|
|
- `frame`: *(blank)*
|
|
- `width`: `0`
|
|
- `height`: `0`
|
|
|
|
**Result:** Loads all images from `input/hero/run/image/`, sorted alphabetically, as a batch tensor [N, H, W, 3]. Returns directory path, original dimensions, frame_count=N.
|
|
|
|
---
|
|
|
|
## Testing Criteria
|
|
|
|
| # | Test | Expected Result |
|
|
|---|------|-----------------|
|
|
| 1 | `frame="0"`, no resize | Loads first image alphabetically, outputs [1, H, W, 3] |
|
|
| 2 | `frame=""`, no resize | Loads all images, outputs [N, H, W, 3] |
|
|
| 3 | `width=512, height=768` | Image is resized to cover 512x768 and center-cropped |
|
|
| 4 | `width=512, height=0` | Image is proportionally scaled so width=512, height maintained |
|
|
| 5 | Direction blank | Path omits direction folder; loads from `input/dir/modality/` |
|
|
| 6 | Invalid directory | Clear `RuntimeError` |
|
|
| 7 | `frame` out of bounds | Clear `RuntimeError` |
|
|
| 8 | Refresh button clicked | Directory combo updates with newly added/removed folders |
|
|
| 9 | Type in directory combo | Dropdown filters to matching entries |
|