229 lines
10 KiB
Markdown
229 lines
10 KiB
Markdown
# Specification: Compass Image Loader
|
|
|
|
## Overview
|
|
|
|
A ComfyUI custom node that loads images from a structured directory hierarchy under ComfyUI's `input/` directory. The node navigates via a user-selected subdirectory, an optional compass direction, and a modality folder, then outputs matching image(s) with optional resizing.
|
|
|
|
## Directory Structure Convention
|
|
|
|
```
|
|
<comfyui_root>/input/
|
|
<project>/
|
|
<action>/
|
|
[direction]/
|
|
<modality>/
|
|
frame_0001.png
|
|
frame_0002.png
|
|
...
|
|
```
|
|
|
|
When `direction` is left blank, the direction folder is omitted:
|
|
|
|
```
|
|
<comfyui_root>/input/
|
|
<project>/
|
|
<action>/
|
|
<modality>/
|
|
frame_0001.png
|
|
...
|
|
```
|
|
|
|
**Example resolved path (with direction):** `input/rosella/walk/s/depth/frame_0001.png`
|
|
**Example resolved path (without direction):** `input/rosella/walk/image/frame_0001.png`
|
|
|
|
---
|
|
|
|
## Node Definition
|
|
|
|
### Node Name
|
|
`CompassImageLoader`
|
|
|
|
### Display Name
|
|
`Compass Image Loader`
|
|
|
|
### Category
|
|
`image/loaders`
|
|
|
|
---
|
|
|
|
## Widgets / Inputs
|
|
|
|
| Name | Type | Widget | Default | Description |
|
|
|------|------|--------|---------|-------------|
|
|
| `directory` | `STRING` | Combo with autocomplete | `""` | Recursive subdirectory path under `input/`. Populated dynamically from all directories containing at least one valid direction subfolder (n, ne, e, se, s, sw, w, nw) **or** at least one valid modality subfolder (image, depth, openpose). Supports typing to filter. |
|
|
| `direction` | `STRING` | Combo | `""` | One of: `""`, `n`, `ne`, `e`, `se`, `s`, `sw`, `w`, `nw`. Empty string omits the direction folder entirely. |
|
|
| `modality` | `STRING` | Combo | `"image"` | One of: `image`, `depth`, `openpose` |
|
|
| `frame` | `STRING` | Text | `""` | Frame index as string. Empty string or whitespace-only = load all images as a batch. `"0"` = first image (0-based). |
|
|
| `width` | `INT` | Number | `0` | Target resize width in pixels. `0` = leave original width. |
|
|
| `height` | `INT` | Number | `0` | Target resize height in pixels. `0` = leave original height. |
|
|
| `direction_in` | `STRING` | Optional input | `""` | Optional STRING input that, if connected, takes precedence over `direction`. Enables chaining a direction output from another node into this input. |
|
|
|
|
### Resize Behavior
|
|
- If `width == 0` and `height == 0`: no resize.
|
|
- If **only one** of `width` or `height` is non-zero: scale proportionally so that the non-zero dimension matches the target. The other dimension is computed to maintain aspect ratio. No cropping occurs.
|
|
- If **both** `width` and `height` are non-zero: use a **cover + center crop** strategy:
|
|
1. Compute `scale = max(target_width / source_width, target_height / source_height)`.
|
|
2. Resize by `scale` using Lanczos resampling.
|
|
3. Center crop to exactly `width x height`.
|
|
|
|
---
|
|
|
|
## Outputs
|
|
|
|
| Name | Type | Description |
|
|
|------|------|-------------|
|
|
| `IMAGE` | `IMAGE` | Tensor of shape `(B, H, W, C)` where `B` = batch size. `B = 1` for a single frame; `B = N` when `frame` is empty (all images loaded as a batch). |
|
|
| `path` | `STRING` | The resolved file system path. For single frame: absolute path to the image file. For batch mode: absolute path to the directory containing the images. |
|
|
| `direction` | `STRING` | The resolved direction string (`"n"`, `"s"`, `"nw"`, `""`, etc.). Can be fed into the `direction_in` input of another `CompassImageLoader` node to chain direction values. |
|
|
| `width` | `INT` | Final width in pixels after any resizing. |
|
|
| `height` | `INT` | Final height in pixels after any resizing. |
|
|
| `frame_count` | `INT` | Number of frames loaded. `1` for single frame, `N` for batch mode. |
|
|
|
|
---
|
|
|
|
## Core Behavior
|
|
|
|
### 1. Base Directory Resolution
|
|
- Base is always `folder_paths.get_input_directory()`.
|
|
- The user does **not** configure this.
|
|
|
|
### 2. Directory Discovery (Autocomplete)
|
|
- The backend recursively scans the `input/` directory.
|
|
- A directory is eligible for the `directory` combo if it satisfies **either** of the following:
|
|
- It contains at least one valid direction subfolder (`n`, `ne`, `e`, `se`, `s`, `sw`, `w`, `nw`), **or**
|
|
- It contains at least one valid modality subfolder (`image`, `depth`, `openpose`).
|
|
- The list of eligible paths is provided as the combo values for the `directory` widget.
|
|
- The frontend extension enhances this combo with typing-to-filter autocomplete behavior.
|
|
|
|
### 3. Path Construction
|
|
```python
|
|
if direction and direction.strip():
|
|
full_dir = os.path.join(input_dir, directory, direction, modality)
|
|
else:
|
|
full_dir = os.path.join(input_dir, directory, modality)
|
|
```
|
|
|
|
### 4. Image Discovery
|
|
- List all files in `full_dir`.
|
|
- Filter to supported image content types. Use `folder_paths.filter_files_content_types` if available, otherwise filter by standard image extensions (`png`, `jpg`, `jpeg`, `webp`, `bmp`, `gif`, `tiff`).
|
|
- Sort alphabetically by filename. This sorted order defines frame indices (`0 = first file alphabetically`).
|
|
|
|
### 5. Frame Selection
|
|
- If `frame` is empty or whitespace-only: select all images (batch mode).
|
|
- Else: parse `frame` as a base-10 integer. Select the single image at that 0-based index.
|
|
- If the parsed index is negative or `>= len(images)`, raise `RuntimeError` with a clear message.
|
|
|
|
### 6. Image Loading & Resizing
|
|
- Open each selected file with PIL.
|
|
- Convert to RGB (`image.convert("RGB")`).
|
|
- If resize is required:
|
|
- **Proportional resize (one dimension zero):**
|
|
- If `width > 0` and `height == 0`: `new_height = int(source_height * (width / source_width))`.
|
|
- If `height > 0` and `width == 0`: `new_width = int(source_width * (height / source_height))`.
|
|
- Use `Image.Resampling.LANCZOS`.
|
|
- **Cover + Crop (both dimensions non-zero):**
|
|
- `scale = max(width / source_width, height / source_height)`.
|
|
- `new_size = (int(source_width * scale), int(source_height * scale))`.
|
|
- `image = image.resize(new_size, Image.Resampling.LANCZOS)`.
|
|
- Compute crop box to center the crop:
|
|
- `left = (new_size[0] - width) // 2`
|
|
- `top = (new_size[1] - height) // 2`
|
|
- `right = left + width`
|
|
- `bottom = top + height`
|
|
- `image = image.crop((left, top, right, bottom))`.
|
|
|
|
### 7. Tensor Conversion
|
|
- `np.array(image).astype(np.float32) / 255.0`
|
|
- `torch.from_numpy(image)[None,]` (adds batch dimension of 1)
|
|
- For batch mode, `torch.cat(tensors, dim=0)` to merge into `(N, H, W, C)`.
|
|
|
|
### 8. Return Values
|
|
- `path`:
|
|
- Single frame: `os.path.join(full_dir, filename)` (absolute path).
|
|
- Batch mode: `full_dir` (absolute path).
|
|
- `width`, `height`: Dimensions of the final tensor.
|
|
- `frame_count`: `len(images)`.
|
|
|
|
---
|
|
|
|
## Error Handling
|
|
|
|
| Scenario | Behavior |
|
|
|----------|----------|
|
|
| `directory` does not exist | `RuntimeError("Compass directory not found: {full_dir}")` |
|
|
| `full_dir` contains no supported images | `RuntimeError("No images found in: {full_dir}")` |
|
|
| `frame` index out of bounds | `RuntimeError("Frame index {n} out of bounds. Found {m} images in {full_dir}.")` |
|
|
| `frame` string is not a valid integer | `RuntimeError("Invalid frame number: '{frame}'. Must be an integer.")` |
|
|
|
|
---
|
|
|
|
## Frontend Extension Requirements
|
|
|
|
### File
|
|
`js/compass_image_loader.js`
|
|
|
|
### Responsibilities
|
|
1. **Register Extension:** Use `app.registerExtension({ name: "CompassImageLoader", ... })`.
|
|
2. **Dynamic Combo Refresh:** Listen for the `fresh-node-defs` event fired by ComfyUI's `api.js` (which happens on startup and when the user clicks Refresh). When received, update the `directory` combo widget on existing node instances with the latest values from the node's definition.
|
|
3. **Hierarchical Directory Autocomplete:** The `directory` field is driven by a text input that builds a path segment-by-segment. As the user types, the dropdown shows only **direct child directories** of the currently-typed prefix. When the user selects a child, it is appended with a trailing `/`, and the next focus/input shows only the children of that new prefix. This continues recursively, allowing the user to drill down into arbitrarily-deep directory trees.
|
|
4. **Stop on Direction Detection:** The autocomplete filters are applied against the **full list of eligible parent paths** discovered by the backend. For each level, only the immediate next segment is shown — not any deeper paths. This means the user naturally stops at the level where they intend to set the `direction` widget.
|
|
5. **State Preservation:** Ensure the user's selected `directory` value is preserved across page reloads and workflow loads.
|
|
|
|
---
|
|
|
|
## Backend API for Frontend Refresh
|
|
|
|
The frontend extension works by maintaining the **full list** of eligible directory paths (discovered by `INPUT_TYPES`) in `_all_values`. It does not need to call any custom endpoint — it simply filters that full list to show only the next level down on each interaction. The `_discover_children` helper function is defined in the backend for completeness but is not currently called by the frontend.
|
|
|
|
---
|
|
|
|
## Dependencies
|
|
|
|
- `folder_paths` (ComfyUI builtin)
|
|
- `torch`
|
|
- `numpy`
|
|
- `PIL` (Pillow)
|
|
- `os`
|
|
|
|
No external pip dependencies are required beyond what ComfyUI already provides.
|
|
|
|
---
|
|
|
|
## Example Workflows
|
|
|
|
### Example 1: Single Frame, with Direction, Resized
|
|
- `directory`: `rosella/walk`
|
|
- `direction`: `s`
|
|
- `modality`: `depth`
|
|
- `frame`: `0`
|
|
- `width`: `512`
|
|
- `height`: `768`
|
|
|
|
**Result:** Loads `input/rosella/walk/s/depth/<first_image>`, resizes to 512x768 with cover-crop + Lanczos. Returns `IMAGE` [1, 768, 512, 3], path string, width=512, height=768, frame_count=1.
|
|
|
|
### Example 2: Batch Mode, No Direction, No Resize
|
|
- `directory`: `hero/run`
|
|
- `direction`: *(blank)*
|
|
- `modality`: `image`
|
|
- `frame`: *(blank)*
|
|
- `width`: `0`
|
|
- `height`: `0`
|
|
|
|
**Result:** Loads all images from `input/hero/run/image/`, sorted alphabetically, as a batch tensor [N, H, W, 3]. Returns directory path, original dimensions, frame_count=N.
|
|
|
|
---
|
|
|
|
## Testing Criteria
|
|
|
|
| # | Test | Expected Result |
|
|
|---|------|-----------------|
|
|
| 1 | `frame="0"`, no resize | Loads first image alphabetically, outputs [1, H, W, 3] |
|
|
| 2 | `frame=""`, no resize | Loads all images, outputs [N, H, W, 3] |
|
|
| 3 | `width=512, height=768` | Image is resized to cover 512x768 and center-cropped |
|
|
| 4 | `width=512, height=0` | Image is proportionally scaled so width=512, height maintained |
|
|
| 5 | Direction blank | Path omits direction folder; loads from `input/dir/modality/` |
|
|
| 6 | Invalid directory | Clear `RuntimeError` |
|
|
| 7 | `frame` out of bounds | Clear `RuntimeError` |
|
|
| 8 | Refresh button clicked | Directory combo updates with newly added/removed folders |
|
|
| 9 | Type in directory combo | Dropdown filters to matching entries |
|