ai-game-2/tools/ora_editor/ORA_EDITOR.md

ORA Editor - Browser-Based Image Layer Editor

Overview

A Flask + Alpine.js web application for editing OpenRaster (ORA) files and PNG images with layer management, polygon-based mask extraction hints, and Krita integration.

Location: tools/ora_editor/

Tech Stack:

• Backend: Flask (Python 3)
• Frontend: Alpine.js + Tailwind CSS (CDN)
• Image Processing: Pillow
• Custom JS: Minimal (~50 lines for polygon canvas)

---

Project Structure

tools/ora_editor/
├── app.py                 # Flask application
├── ora_ops.py             # ORA operations wrapper (functions from ora_edit.py)
├── templates/
│   └── editor.html       # Single-page Alpine.js UI
├── requirements.txt       # Python dependencies
├── temp/                  # Krita temp files (gitignored)
└── .gitignore

---

Dependencies

Python Packages (requirements.txt)

flask>=3.0.0
pillow>=10.0.0

External Requirements

• ComfyUI running at configurable address (default: 127.0.0.1:8188)
• Krita desktop application (for "Open in Krita" feature)

---

Technology Choices

Why Flask + Alpine.js?

• Flask: Minimal Python web framework, mirrors the CLI tool workflow
• Alpine.js: Reactive UI with zero custom JS, perfect for this use case
• Tailwind CSS (CDN): Rapid styling without build step

Why Minimal Custom JS?

The polygon drawing requires capturing mouse clicks on a canvas. This is the only interaction that Alpine.js cannot handle natively. Everything else (layer toggles, modals, forms) is pure Alpine.js.

---

File Operations

Root Directory

All file paths are relative to the project root: /home/noti/dev/ai-game-2

Supported Formats

• ORA (OpenRaster): Native format with layer support
• PNG: Auto-converted to ORA on open

Open Flow

1. User enters path relative to project root (e.g., scenes/kq4_010/pic.png)
2. Backend checks if file exists
3. If PNG: auto-create ORA with single base layer
4. Parse ORA structure, extract layers
5. Return layer list + image data URLs

---

Layer System

Layer Data Structure

{
    "name": "door_0",           # Layer identifier
    "is_group": False,          # False for regular layers
    "visibility": True,        # Visible by default
    "src": "data/door_0.png",  # Path inside ORA zip
    "opacity": 1.0,            # Opacity 0.0-1.0
}

ORA Structure Created

📁 Group: <entity_name> (e.g., "door")
   └─ <entity_name>_0         (e.g., "door_0")
📁 Group: <another_entity>
   └─ <another_entity>_0
🖼️  Base: base

Layer Operations

Operation	Description
Toggle Visibility	Show/hide layer on canvas
Rename	Change layer/entity name
Delete	Remove layer from ORA
Reorder	Move layer up/down in stack (changes z-index)
Add Masked	Add new layer with alpha mask

---

Canvas Display

Image Rendering

• Layers rendered as stacked <img> elements with CSS positioning
• Base layer at bottom, entity layers stacked above
• Visibility controlled by opacity: 0 or opacity: 1
• No server-side compositing for preview

Layer DOM Structure

<div class="relative w-full h-full">
    <img src="/api/image/base" class="absolute inset-0">
    <img src="/api/image/door_0" class="absolute inset-0" style="opacity: 1">
    <img src="/api/image/chest_0" class="absolute inset-0" style="opacity: 0">
    <!-- Polygon overlay when active -->
    <canvas id="polygon-canvas" class="absolute inset-0 pointer-events-auto"></canvas>
</div>

---

Polygon Drawing

Purpose

The polygon is temporary - it's a visual guide to help the AI mask extraction understand what area to focus on. It does NOT affect the final ORA file directly.

Coordinate System

• Points stored as percentages (0.0-1.0) of image dimensions
• Sent to backend for mask extraction hint
• Drawn on a canvas overlay

User Interaction

1. Click "Start Drawing" button
2. Click on image to add points (shown as small circles)
3. Right-click or double-click to close polygon
4. Polygon renders with selected color and line width
5. "Clear" removes polygon

Backend Storage

Polygon points stored in memory on server (per-request, not session). Used only when submitting mask extraction with use_polygon: true.

Implementation (Minimal JS)

// Canvas click handler (only custom JS needed)
canvas.addEventListener('click', (e) => {
    const rect = canvas.getBoundingClientRect();
    const x = (e.clientX - rect.left) / rect.width;
    const y = (e.clientY - rect.top) / rect.height;
    alpineStore.addPolygonPoint(x, y);
});

---

Mask Extraction (ComfyUI)

Workflow

1. User describes subject (e.g., "the wooden door")
2. Optionally enable "Use polygon hint"
3. Click "Extract Mask"

API: POST /api/mask/extract

{
    "subject": "the wooden door",
    "use_polygon": true
}

Backend Process

1. If use_polygon: true, fetch stored polygon points
2. Draw polygon overlay on input image (using draw_polygon.py logic)
3. Build prompt: "Create a black and white alpha mask of {subject}"
4. Queue workflow to ComfyUI at configured server URL
5. Poll for completion (timeout: 4 minutes)
6. Download resulting mask to temp/ directory
7. Return mask path

ComfyUI Integration

• Server address stored in localStorage (client-side setting)
• Uses existing tools/image_mask_extraction.json workflow
• Same extraction logic as extract_mask.py

Mask Preview Modal

When mask is ready:

1. Full-screen modal appears
2. Shows base image with mask applied as colored tint overlay
3. Three buttons:
   • Re-roll: Re-run extraction with same params
   • Use This Mask: Add masked layer to ORA, close modal
   • Cancel: Discard mask, close modal

Modal Display

┌─────────────────────────────────────────────────────────┐
│                                                         │
│     [Base image with mask applied as tint overlay]     │
│                                                         │
│        [Re-roll]  [Use This Mask]  [Cancel]            │
│                                                         │
└─────────────────────────────────────────────────────────┘

---

Krita Integration

Flow

1. User selects a layer and clicks "Open in Krita"
2. Backend extracts layer PNG to temp/{layer_name}_{timestamp}.png
3. Returns file:// URL
4. Browser opens URL (Krita launches with file)
5. User edits in Krita, saves

File Watching

Since browser cannot detect when Krita saves:

• On tab visibility change: Check file mtime, auto-reload if changed
• Manual "Refresh" button: Force reload from disk
• No automatic save back: User explicitly saves in Krita, then refreshes

Temp File Management

• Temp files stored in tools/ora_editor/temp/
• Named: {layer_name}_{unix_timestamp}.png
• Not automatically deleted (low priority)

---

API Reference

File Operations

POST /api/open

Open a PNG or ORA file.

Request:

{"path": "scenes/kq4_010/pic.png"}

Response:

{
    "success": true,
    "file_path": "/home/noti/dev/ai-game-2/scenes/kq4_010/pic.ora",
    "ora_path": "/home/noti/dev/ai-game-2/scenes/kq4_010/pic.ora",
    "layers": [
        {"name": "base", "visibility": true, "src": "data/base.png"},
        {"name": "door_0", "visibility": true, "src": "data/door/door_0.png"}
    ]
}

POST /api/save

Persist current layer state to ORA file.

Request:

{"ora_path": "scenes/kq4_010/pic.ora"}

Response:

{"success": true}

---

Layer Operations

GET /api/layers

Get current layer structure.

Response:

{
    "layers": [
        {"name": "base", "visibility": true, "src": "data/base.png"},
        {"name": "door_0", "visibility": true, "src": "data/door/door_0.png"}
    ]
}

POST /api/layer/add

Add a new masked layer.

Request:

{
    "entity_name": "door",
    "mask_path": "/tmp/mask.png",
    "source_layer": "base"
}

Response:

{"success": true, "layer_name": "door_1"}

POST /api/layer/rename

Rename a layer.

Request:

{"old_name": "door_0", "new_name": "wooden_door_0"}

Response:

{"success": true}

POST /api/layer/delete

Delete a layer.

Request:

{"layer_name": "door_0"}

Response:

{"success": true}

POST /api/layer/reorder

Move layer up or down.

Request:

{"layer_name": "door_0", "direction": "up"}

Response:

{"success": true}

---

Image Serving

GET /api/image/<layer_name>

Serve a layer PNG. Returns image data.

GET /api/image/base

Serve base/merged image.

GET /api/image/polygon

Serve polygon overlay image (for drawing mode).

---

Polygon

POST /api/polygon

Draw polygon overlay on current image.

Request:

{
    "points": [{"x": 0.1, "y": 0.2}, {"x": 0.5, "y": 0.8}],
    "color": "#FF0000",
    "width": 2
}

Response:

{"success": true, "overlay_url": "/api/image/polygon"}

POST /api/polygon/points

Store polygon points for mask extraction.

Request:

{"points": [{"x": 0.1, "y": 0.2}, {"x": 0.5, "y": 0.8}]}

Response:

{"success": true}

POST /api/polygon/clear

Clear polygon overlay.

Response:

{"success": true}

---

Mask Extraction

POST /api/mask/extract

Extract mask using ComfyUI.

Request:

{
    "subject": "the wooden door",
    "use_polygon": true
}

Response:

{
    "success": true,
    "mask_path": "/home/noti/dev/ai-game-2/tools/ora_editor/temp/mask.png"
}

---

Krita

POST /api/krita/open

Copy layer to temp and return file URL.

Request:

{"layer_name": "base"}

Response:

{
    "success": true,
    "file_url": "file:///home/noti/dev/ai-game-2/tools/ora_editor/temp/base_1234567890.png",
    "temp_path": "/home/noti/dev/ai-game-2/tools/ora_editor/temp/base_1234567890.png"
}

GET /api/krita/status/<layer_name>

Check if temp file was modified.

Response:

{
    "modified": true,
    "mtime": 1234567890
}

---

Frontend State (Alpine.js)

Main Store (x-data)

{
    // File state
    filePath: '',
    oraPath: '',
    layers: [],
    baseImage: '',

    // UI state
    selectedLayer: null,
    isDrawingPolygon: false,
    polygonPoints: [],
    polygonColor: '#FF0000',
    polygonWidth: 2,

    // Mask extraction
    maskSubject: '',
    usePolygonHint: true,
    isExtracting: false,
    showMaskModal: false,
    tempMaskPath: null,

    // Krita
    kritaLayer: null,
    kritaMtime: null,

    // Settings
    comfyUrl: localStorage.getItem('comfy_url') || '127.0.0.1:8188',
}

Layer Object

{
    name: 'door_0',
    visibility: true,
    src: 'data/door/door_0.png'
}

---

UI Layout

┌──────────────────────────────────────────────────────────────────────┐
│ [Open: ________________________] [Open File]   [Settings ⚙]        │
├───────────────────┬──────────────────────────────────────────────────┤
│ LAYERS            │                                                   │
│ ☑ base            │                                                   │
│ ☑ door_0         │           [Image Canvas with                      │
│ ☐ chest_0        │            stacked layers]                        │
│                   │                                                   │
│ [Rename] [Delete] │                                                   │
│ [▲ Up] [▼ Down]  │                                                   │
│                   │                                                   │
│ ─────────────────│                                                   │
│ POLYGON TOOL     │                                                   │
│ Color: [#FF0000] │                                                   │
│ Width: [2____]   │                                                   │
│ [Start Drawing]  │                                                   │
│ Points: 5        │                                                   │
│ [Clear]          │                                                   │
│                   │                                                   │
│ ─────────────────│                                                   │
│ MASK EXTRACTION  │                                                   │
│ Subject: [______]│                                                   │
│ ☑ Use polygon    │                                                   │
│ [Extract Mask]   │                                                   │
│                   │                                                   │
│ ─────────────────│                                                   │
│ [Open in Krita]  │                                                   │
│                   │                                                   │
├───────────────────┴──────────────────────────────────────────────────┤
│ [Save]                                         [Refresh from Krita]  │
└──────────────────────────────────────────────────────────────────────┘

---

Mask Preview Modal

┌─────────────────────────────────────────────────────────┐
│                      EXTRACTED MASK                     │
│  ─────────────────────────────────────────────────────  │
│                                                         │
│     [Base image with mask applied as tinted overlay]   │
│                                                         │
│        [Re-roll]  [Use This Mask]  [Cancel]            │
│                                                         │
└─────────────────────────────────────────────────────────┘

• Re-roll: Re-runs extraction with same params
• Use This Mask: Calls POST /api/layer/add, closes modal
• Cancel: Closes modal, mask discarded

---

Settings Modal

┌─────────────────────────────────────────────────────────┐
│                      SETTINGS                          │
│  ─────────────────────────────────────────────────────  │
│                                                         │
│  ComfyUI Server: [127.0.0.1:8188____________]          │
│                                                         │
│                         [Save]  [Cancel]                │
│                                                         │
└─────────────────────────────────────────────────────────┘

• Settings stored in localStorage
• ComfyURL stored as comfy_url key

---

Implementation Tasks

Phase 1: Project Setup

• Create directory structure (templates/, temp/)
• Create requirements.txt
• Create .gitignore
• Test virtualenv/dependencies

Phase 2: Backend Core

• Create ora_ops.py with wrapper functions:
  • load_ora(path) - Parse ORA, return layers + images
  • save_ora(ora_path) - Rebuild ORA from current state
  • add_masked_layer(ora_path, entity, mask_path, source) - Apply mask
  • rename_layer(ora_path, old, new) - Update layer name
  • delete_layer(ora_path, name) - Remove layer
  • reorder_layer(ora_path, name, direction) - Move up/down
  • get_layer_image(ora_path, layer_name) - Extract PNG
  • get_base_image(ora_path) - Get merged/composited base
• Create app.py with Flask routes:
  • File operations (/api/open, /api/save)
  • Layer operations (all /api/layer/*)
  • Image serving (/api/image/*)
  • Polygon (/api/polygon*)
  • Mask extraction (/api/mask/extract)
  • Krita (/api/krita/*)

Phase 3: Frontend UI

• Create editor.html with Tailwind + Alpine.js
• File open input and handler
• Layer list with visibility toggles
• Canvas area with stacked layer images
• Layer operations (rename, delete, reorder buttons)

Phase 4: Polygon Drawing

• Polygon tool UI (color, width, start/clear)
• Canvas overlay for polygon rendering
• Minimal JS for click-to-add-points
• Backend polygon drawing endpoint

Phase 5: Mask Extraction

• Mask extraction form UI
• Backend ComfyUI integration
• Mask preview modal
• Re-roll / Use This Mask / Cancel logic

Phase 6: Krita Integration

• "Open in Krita" button and endpoint
• File URL generation
• Refresh button and logic
• Visibility-change auto-reload (JS)

Phase 7: Polish

• Settings modal for ComfyUI URL
• Error handling and user feedback
• Loading states
• Empty states

---

Key Implementation Details

ORA File Structure

ORA is a ZIP file containing:

mimetype                    # "image/openraster"
stack.xml                   # Layer structure XML
mergedimage.png             # Composited full image
Thumbnails/thumbnail.png    # Preview thumbnail
data/
  base.png                  # Base layer
  <entity>/
    <entity>_0.png          # Masked layer

stack.xml Format

<image version="0.0.3" w="800" h="600">
  <stack>
    <layer name="base" src="data/base.png" opacity="1.0"/>
    <stack name="door">
      <layer name="door_0" src="data/door/door_0.png" opacity="1.0"/>
    </stack>
  </stack>
</image>

Image Compositing for Preview

• Frontend receives individual layer PNG URLs
• Layers stacked with CSS position: absolute
• Visibility toggled via CSS opacity
• No server-side compositing needed

Polygon Overlay Rendering

1. Backend receives points as [{x: 0.1, y: 0.2}, ...]
2. Convert to absolute pixel coordinates
3. Use Pillow ImageDraw.polygon() to draw
4. Return overlay as PNG

ComfyUI Workflow

Uses existing tools/image_mask_extraction.json:

1. Load input image
2. If polygon hint: composite polygon overlay
3. Set prompt text
4. Queue prompt to ComfyUI
5. Poll for completion
6. Download output mask

---

Color Reference

Default Polygon Colors

Name	Hex
Red	#FF0000
Green	#00FF00
Blue	#0000FF
Yellow	#FFFF00

Mask Tint Colors (Preview)

Default: #00FF0080 (semi-transparent green)

---

Error Handling

Error	User Feedback
File not found	"File not found: {path}"
Invalid ORA	"Invalid ORA file format"
ComfyUI offline	"Cannot connect to ComfyUI at {url}"
ComfyUI timeout	"Mask extraction timed out (4 min)"
Layer not found	"Layer '{name}' not found"

---

Browser Compatibility

Tested/supported:

• Modern browsers with ES6+ support
• Chrome/Edge (primary)
• Firefox
• Safari

Required features:

• ES6 modules
• CSS Grid/Flexbox
• Canvas API (for polygon)
• localStorage (for settings)

---

Running the Application

cd tools/ora_editor
source ../venv/bin/activate  # or your Python env
pip install -r requirements.txt
python app.py
# Opens at http://localhost:5000

---

Future Considerations (Out of Scope)

• Multi-select layers
• Drag-and-drop reordering
• Undo/redo history
• Collaborative editing
• Additional export formats
• Layer groups in UI (show hierarchy)