ai-game-2/tools/ora_editor/ORA_EDITOR.md

ORA Editor - Browser-Based Image Layer Editor

Overview

A Flask + Alpine.js web application for editing OpenRaster (ORA) files and PNG images with layer management, polygon-based mask extraction hints, and Krita integration.

Location: tools/ora_editor/

Tech Stack:

• Backend: Flask (Python 3)
• Frontend: Alpine.js + Tailwind CSS (CDN)
• Image Processing: Pillow
• Custom JS: Minimal (~50 lines for polygon canvas)

---

Project Structure

tools/ora_editor/
├── app.py                 # Flask application
├── ora_ops.py             # ORA operations wrapper (functions from ora_edit.py)
├── templates/
│   └── editor.html       # Single-page Alpine.js UI
├── requirements.txt       # Python dependencies
├── temp/                  # Krita temp files (gitignored)
└── .gitignore

---

Dependencies

Python Packages (requirements.txt)

flask>=3.0.0
pillow>=10.0.0

External Requirements

• ComfyUI running at configurable address (default: 127.0.0.1:8188)
• Krita desktop application (for "Open in Krita" feature)

---

Technology Choices

Why Flask + Alpine.js?

• Flask: Minimal Python web framework, mirrors the CLI tool workflow
• Alpine.js: Reactive UI with zero custom JS, perfect for this use case
• Tailwind CSS (CDN): Rapid styling without build step

Why Minimal Custom JS?

The polygon drawing requires capturing mouse clicks on a canvas. This is the only interaction that Alpine.js cannot handle natively. Everything else (layer toggles, modals, forms) is pure Alpine.js.

---

File Operations

Root Directory

All file paths are relative to the project root: /home/noti/dev/ai-game-2

Supported Formats

• ORA (OpenRaster): Native format with layer support
• PNG: Auto-converted to ORA on open

Open Flow

1. User clicks "Browse" button to open file browser dialog
2. Backend lists directories and files (PNG, ORA) from project root
3. User navigates directories, selects file
4. Backend checks if file exists
5. If PNG: auto-create ORA with single base layer
6. Parse ORA structure, extract layers
7. Return layer list + image data URLs

---

Browse Dialog

Purpose

Allows users to visually navigate the project directory and select PNG/ORA files without manually typing paths.

UI Layout

┌─────────────────────────────────────────────────────────┐
│                      OPEN FILE                          │
│  ─────────────────────────────────────────────────────  │
│  📁 ..                                                  │
│  📁 scenes/                                             │
│  📁 tools/                                              │
│  🖼️ pic_010.ora                                        │
│  🖼️ pic_011.png                                        │
│                                                         │
│  Current: scenes/kq4_010/                               │
│                                                         │
│              [Cancel]  [Open Selected]                  │
└─────────────────────────────────────────────────────────┘

Behavior

• Double-click folder: Navigate into folder
• Double-click file: Select and open immediately
• Single-click + Enter: Select and open
• ** ".." entry**: Navigate to parent directory
• File types: Only show .png and .ora files
• Hidden files: Excluded (dot-prefixed)

Backend Implementation

GET /api/browse

List directory contents for browsing.

Query params:

• path: Directory path relative to project root (default: "")

Response:

{
    "current_path": "scenes/kq4_010",
    "parent_path": "scenes",
    "directories": [
        {"name": "subdir1", "path": "scenes/kq4_010/subdir1"}
    ],
    "files": [
        {"name": "pic.png", "path": "scenes/kq4_010/pic.png", "type": "png"},
        {"name": "scene.ora", "path": "scenes/kq4_010/scene.ora", "type": "ora"}
    ]
}

Frontend State

// Added to main store
browseModal: false,
browsePath: '',           // Current directory being browsed
browseDirectories: [],    // List of {name, path}
browseFiles: [],          // List of {name, path, type}
browseSelected: null,     // Currently selected file path

Entry Point

• Button next to path input: [Browse...]
• Or clicking into the path input field (if empty)

---

Layer System

Layer Data Structure

{
    "name": "door_0",           # Layer identifier
    "is_group": False,          # False for regular layers
    "visibility": True,        # Visible by default
    "src": "data/door_0.png",  # Path inside ORA zip
    "opacity": 1.0,            # Opacity 0.0-1.0
}

ORA Structure Created

📁 Group: <entity_name> (e.g., "door")
   └─ <entity_name>_0         (e.g., "door_0")
📁 Group: <another_entity>
   └─ <another_entity>_0
🖼️  Base: base

Layer Operations

Operation	Description
Toggle Visibility	Show/hide specific layer image on canvas
Tint Red Preview	Apply red tint to layer for mask verification (client-only)
Rename	Change layer/entity name
Delete	Remove layer from ORA
Reorder	Move layer up/down in stack (changes z-index)
Add Masked	Add new layer with alpha mask

---

Canvas Display

Image Rendering

• Individual layers rendered as stacked <img> elements with CSS positioning
• Layers stacked in order from list (visual z-index matches layer order)
• Visibility togglecheckbox hides/shows each layer's image on canvas
• Tint checkbox applies semi-transparent red overlay for mask verification (client-only, visual aid)
• No server-side compositing for preview

Layer DOM Structure

<div class="relative w-full h-full">
    <!-- Individual layers rendered in order (x-show controls visibility) -->
    <template x-for="layer in layers">
        <img 
            x-show="layer.visible"
            :src="'/api/image/layer/' + layer.name" 
            class="absolute inset-0"
            :style="layer.tintRed ? 'mix-blend-multiply; opacity: 0.6;' : ''"
        >
    </template>
    <!-- Polygon overlay when active -->
    <canvas id="polygon-canvas" class="absolute inset-0 pointer-events-auto"></canvas>
</div>

---

Polygon Drawing

Purpose

The polygon is temporary - it's a visual guide to help the AI mask extraction understand what area to focus on. It does NOT affect the final ORA file directly.

Coordinate System

• Points stored as percentages (0.0-1.0) of image dimensions
• Sent to backend for mask extraction hint
• Drawn on a canvas overlay

User Interaction

1. Click "Start Drawing" button
2. Click on image to add points (shown as small circles)
3. Right-click or double-click to close polygon
4. Polygon renders with selected color and line width
5. "Clear" removes polygon

Backend Storage

Polygon points stored in memory on server (per-request, not session). Used only when submitting mask extraction with use_polygon: true.

Implementation (Minimal JS)

// Canvas click handler (only custom JS needed)
canvas.addEventListener('click', (e) => {
    const rect = canvas.getBoundingClientRect();
    const x = (e.clientX - rect.left) / rect.width;
    const y = (e.clientY - rect.top) / rect.height;
    alpineStore.addPolygonPoint(x, y);
});

---

Mask Extraction (ComfyUI)

Workflow

1. User describes subject (e.g., "the wooden door")
2. Optionally enable "Use polygon hint"
3. Click "Extract Mask"

API: POST /api/mask/extract

{
    "subject": "the wooden door",
    "use_polygon": true
}

Backend Process

1. If use_polygon: true, fetch stored polygon points
2. Draw polygon overlay on input image (using draw_polygon.py logic)
3. Build prompt: "Create a black and white alpha mask of {subject}"
4. Queue workflow to ComfyUI at configured server URL
5. Poll for completion (timeout: 4 minutes)
6. Download resulting mask to temp/ directory
7. Return mask path

ComfyUI Integration

• Server address stored in localStorage (client-side setting)
• Uses existing tools/image_mask_extraction.json workflow
• Same extraction logic as extract_mask.py

Mask Preview Modal

When mask is ready:

1. Full-screen modal appears
2. Shows base image with semi-transparent red overlay where the mask is white (selected area)
   • Uses CSS mask-image property to use grayscale values as alpha channel
   • White pixels = fully opaque red tint, black/dark pixels = transparent (no tint)
   • Dark/gray areas of the mask remain invisible so you can clearly see the mask boundary
3. Three buttons:
   • Re-roll: Re-run extraction with same params
   • Use This Mask: Add masked layer to ORA, close modal
   • Cancel: Discard mask, close modal

Modal Display

┌─────────────────────────────────────────────────────────┐
│                                                         │
│     [Base image with mask applied as tint overlay]     │
│                                                         │
│        [Re-roll]  [Use This Mask]  [Cancel]            │
│                                                         │
└─────────────────────────────────────────────────────────┘

---

Krita Integration

Flow

1. User selects a layer and clicks "Open in Krita"
2. Backend extracts layer PNG to temp/{layer_name}_{timestamp}.png
3. Returns file:// URL
4. Browser opens URL (Krita launches with file)
5. User edits in Krita, saves

File Watching

Since browser cannot detect when Krita saves:

• On tab visibility change: Check file mtime, auto-reload if changed
• Manual "Refresh" button: Force reload from disk
• No automatic save back: User explicitly saves in Krita, then refreshes

Temp File Management

• Temp files stored in tools/ora_editor/temp/
• Named: {layer_name}_{unix_timestamp}.png
• Not automatically deleted (low priority)

---

API Reference

File Browser

GET /api/browse

List directory contents for file browsing.

Query params:

• path: Directory path relative to project root (default: "")

Response:

{
    "current_path": "scenes/kq4_010",
    "parent_path": "scenes",
    "directories": [
        {"name": "subdir1", "path": "scenes/kq4_010/subdir1"}
    ],
    "files": [
        {"name": "pic.png", "path": "scenes/kq4_010/pic.png", "type": "png"},
        {"name": "scene.ora", "path": "scenes/kq4_010/scene.ora", "type": "ora"}
    ]
}

File Operations

POST /api/open

Open a PNG or ORA file.

Request:

{"path": "scenes/kq4_010/pic.png"}

Response:

{
    "success": true,
    "file_path": "/home/noti/dev/ai-game-2/scenes/kq4_010/pic.ora",
    "ora_path": "/home/noti/dev/ai-game-2/scenes/kq4_010/pic.ora",
    "layers": [
        {"name": "base", "visibility": true, "src": "data/base.png"},
        {"name": "door_0", "visibility": true, "src": "data/door/door_0.png"}
    ]
}

POST /api/save

Persist current layer state to ORA file.

Request:

{"ora_path": "scenes/kq4_010/pic.ora"}

Response:

{"success": true}

---

Layer Operations

GET /api/layers

Get current layer structure.

Response:

{
    "layers": [
        {"name": "base", "visibility": true, "src": "data/base.png"},
        {"name": "door_0", "visibility": true, "src": "data/door/door_0.png"}
    ]
}

POST /api/layer/add

Add a new masked layer.

Request:

{
    "entity_name": "door",
    "mask_path": "/tmp/mask.png",
    "source_layer": "base"
}

Response:

{"success": true, "layer_name": "door_1"}

POST /api/layer/rename

Rename a layer.

Request:

{"old_name": "door_0", "new_name": "wooden_door_0"}

Response:

{"success": true}

POST /api/layer/delete

Delete a layer.

Request:

{"layer_name": "door_0"}

Response:

{"success": true}

POST /api/layer/reorder

Move layer up or down.

Request:

{"layer_name": "door_0", "direction": "up"}

Response:

{"success": true}

---

Image Serving

GET /api/image/<layer_name>

Serve a layer PNG. Returns image data.

GET /api/image/base

Serve base/merged image.

GET /api/image/layer/<layer_name>

Serve a specific layer as image.

Query params:

• ora_path: Path to ORA file

Response: PNG image data or 404 if layer not found.

GET /api/image/polygon

Serve polygon overlay image (for drawing mode).

---

Polygon

POST /api/polygon

Draw polygon overlay on current image.

Request:

{
    "points": [{"x": 0.1, "y": 0.2}, {"x": 0.5, "y": 0.8}],
    "color": "#FF0000",
    "width": 2
}

Response:

{"success": true, "overlay_url": "/api/image/polygon"}

POST /api/polygon/points

Store polygon points for mask extraction.

Request:

{"points": [{"x": 0.1, "y": 0.2}, {"x": 0.5, "y": 0.8}]}

Response:

{"success": true}

POST /api/polygon/clear

Clear polygon overlay.

Response:

{"success": true}

---

Mask Extraction

POST /api/mask/extract

Extract mask using ComfyUI.

Request:

{
    "subject": "the wooden door",
    "use_polygon": true
}

Response:

{
    "success": true,
    "mask_path": "/home/noti/dev/ai-game-2/tools/ora_editor/temp/mask.png"
}

---

Krita

POST /api/krita/open

Copy layer to temp and return file URL.

Request:

{"layer_name": "base"}

Response:

{
    "success": true,
    "file_url": "file:///home/noti/dev/ai-game-2/tools/ora_editor/temp/base_1234567890.png",
    "temp_path": "/home/noti/dev/ai-game-2/tools/ora_editor/temp/base_1234567890.png"
}

GET /api/krita/status/<layer_name>

Check if temp file was modified.

Response:

{
    "modified": true,
    "mtime": 1234567890
}

---

Frontend State (Alpine.js)

Main Store (x-data)

{
    // File state
    filePath: '',
    oraPath: '',
    layers: [],
    baseImage: '',

    // UI state
    selectedLayer: null,
    isDrawingPolygon: false,
    polygonPoints: [],
    polygonColor: '#FF0000',
    polygonWidth: 2,

    // Mask extraction
    maskSubject: '',
    usePolygonHint: true,
    isExtracting: false,
    showMaskModal: false,
    tempMaskPath: null,

    // Krita
    kritaLayer: null,
    kritaMtime: null,

    // Settings
    comfyUrl: localStorage.getItem('comfy_url') || '127.0.0.1:8188',
}

Layer Object

{
    name: 'door_0',
    visibility: true,
    src: 'data/door/door_0.png'
}

---

UI Layout

┌──────────────────────────────────────────────────────────────────────┐
│ [Open: ________________________] [📁 Browse] [🌀 Open]   [Settings ⚙] │
├───────────────────┬──────────────────────────────────────────────────┤
│ LAYERS            │                                                   │
│ ☑  □ base         │                                                   │
│ ☑  □ door_0      │           [Image Canvas with                      │
│ ☐  ☒ chest_0     │            stacked layers - visibility toggles    │
│                  │             show/hide individual layer images]    │
│                   │                                                   │
│ [Rename] [Delete] │                                                   │
│ [▲ Up] [▼ Down]  │                                                   │
│                   │                                                   │
│ ─────────────────│                                                   │
│ POLYGON TOOL     │                                                   │
│ Color: [#FF0000] │                                                   │
│ Width: [2____]   │                                                   │
│ [Start Drawing]  │                                                   │
│ Points: 5        │                                                   │
│ [Clear]          │                                                   │
│                   │                                                   │
│ ─────────────────│                                                   │
│ MASK EXTRACTION  │                                                   │
│ Subject: [______]│                                                   │
│ ☑ Use polygon    │                                                   │
│ [🌀 Extract Mask]│ (spinner shown when extracting)                    │
│                   │                                                   │
│ ─────────────────│                                                   │
│ [Open in Krita]  │                                                   │
│                   │                                                   │
├───────────────────┴──────────────────────────────────────────────────┤
│ [Save]                                         [Refresh from Krita]  │
└──────────────────────────────────────────────────────────────────────┘

Legend: ☑ = visible checkbox, ☒ = tint red checkbox (red when checked)

---

Mask Preview Modal

┌─────────────────────────────────────────────────────────┐
│                      EXTRACTED MASK                     │
│  ─────────────────────────────────────────────────────  │
│                                                         │
│     [Base image with RED mask overlay]                 │
│         (mask shown in semi-transparent red)           │
│                                                         │
│        [Re-roll]  [Use This Mask]  [Cancel]            │
│                                                         │
└─────────────────────────────────────────────────────────┘

• Re-roll: Re-runs extraction with same params
• Use This Mask: Calls POST /api/layer/add, closes modal
• Cancel: Closes modal, mask discarded
• Red tint: Uses CSS filter to render grayscale mask as semi-transparent red

---

Browse Modal

┌─────────────────────────────────────────────────────────┐
│                      OPEN FILE                          │
│  ─────────────────────────────────────────────────────  │
│  📁 ..                    (parent directory)           │
│  📁 scenes/                                             │
│  📁 tools/                                              │
│  🖼️ pic_010.ora                                        │
│  🖼️ pic_011.png                                        │
│                                                         │
│  Path: scenes/kq4_010/                                  │
│                                                         │
│                      [Cancel]  [Open Selected]          │
└─────────────────────────────────────────────────────────┘

• Double-click folder to navigate
• Double-click file to open immediately
• Single-click to select, then click "Open Selected"
• Only shows .png and .ora files

---

Settings Modal

┌─────────────────────────────────────────────────────────┐
│                      SETTINGS                          │
│  ─────────────────────────────────────────────────────  │
│                                                         │
│  ComfyUI Server: [127.0.0.1:8188____________]          │
│                                                         │
│                         [Save]  [Cancel]                │
│                                                         │
└─────────────────────────────────────────────────────────┘

• Settings stored in localStorage
• ComfyURL stored as comfy_url key

---

Implementation Tasks

Phase 1: Project Setup

• Create directory structure (templates/, temp/)
• Create requirements.txt
• Create .gitignore
• Test virtualenv/dependencies

Phase 2: Backend Core

• Create ora_ops.py with wrapper functions:
  • load_ora(path) - Parse ORA, return layers + images
  • save_ora(ora_path) - Rebuild ORA from current state
  • add_masked_layer(ora_path, entity, mask_path, source) - Apply mask
  • rename_layer(ora_path, old, new) - Update layer name
  • delete_layer(ora_path, name) - Remove layer
  • reorder_layer(ora_path, name, direction) - Move up/down
  • get_layer_image(ora_path, layer_name) - Extract PNG
  • get_base_image(ora_path) - Get merged/composited base
• Create app.py with Flask routes:
  • File browser (/api/browse)
  • File operations (/api/open, /api/save)
  • Layer operations (all /api/layer/*)
  • Image serving (/api/image/*)
  • Polygon (/api/polygon*)
  • Mask extraction (/api/mask/extract)
  • Krita (/api/krita/*)

Phase 3: Frontend UI

• Create editor.html with Tailwind + Alpine.js
• File open input and handler
• Browse modal for file selection
• Layer list with visibility toggles
• Canvas area with stacked layer images
• Layer operations (rename, delete, reorder buttons)

Phase 4: Polygon Drawing

• Polygon tool UI (color, width, start/clear)
• Canvas overlay for polygon rendering
• Minimal JS for click-to-add-points
• Backend polygon drawing endpoint

Phase 5: Mask Extraction

• Mask extraction form UI
• Backend ComfyUI integration
• Mask preview modal
• Re-roll / Use This Mask / Cancel logic

Phase 6: Krita Integration

• "Open in Krita" button and endpoint
• File URL generation
• Refresh button and logic
• Visibility-change auto-reload (JS)

Phase 7: Polish

• Settings modal for ComfyUI URL
• Error handling and user feedback
• Loading states
• Empty states

---

Key Implementation Details

ORA File Structure

ORA is a ZIP file containing:

mimetype                    # "image/openraster"
stack.xml                   # Layer structure XML
mergedimage.png             # Composited full image
Thumbnails/thumbnail.png    # Preview thumbnail
data/
  base.png                  # Base layer
  <entity>/
    <entity>_0.png          # Masked layer

stack.xml Format

<image version="0.0.3" w="800" h="600">
  <stack>
    <layer name="base" src="data/base.png" opacity="1.0"/>
    <stack name="door">
      <layer name="door_0" src="data/door/door_0.png" opacity="1.0"/>
    </stack>
  </stack>
</image>

Image Compositing for Preview

• Frontend receives individual layer PNG URLs
• Layers stacked with CSS position: absolute
• Visibility toggled via CSS opacity
• No server-side compositing needed

Polygon Overlay Rendering

1. Backend receives points as [{x: 0.1, y: 0.2}, ...]
2. Convert to absolute pixel coordinates
3. Use Pillow ImageDraw.polygon() to draw
4. Return overlay as PNG

ComfyUI Workflow

Uses existing tools/image_mask_extraction.json:

1. Load input image
2. If polygon hint: composite polygon overlay
3. Set prompt text
4. Queue prompt to ComfyUI
5. Poll for completion
6. Download output mask

---

Color Reference

Default Polygon Colors

Name	Hex
Red	#FF0000
Green	#00FF00
Blue	#0000FF
Yellow	#FFFF00

Mask Tint Colors (Preview)

Default: #00FF0080 (semi-transparent green)

---

Error Handling

Error	User Feedback
File not found	"File not found: {path}"
Invalid ORA	"Invalid ORA file format"
ComfyUI offline	"Cannot connect to ComfyUI at {url}"
ComfyUI timeout	"Mask extraction timed out (4 min)"
Layer not found	"Layer '{name}' not found"

---

Browser Compatibility

Tested/supported:

• Modern browsers with ES6+ support
• Chrome/Edge (primary)
• Firefox
• Safari

Required features:

• ES6 modules
• CSS Grid/Flexbox
• Canvas API (for polygon)
• localStorage (for settings)

---

Running the Application

cd tools/ora_editor
source ../venv/bin/activate  # or your Python env
pip install -r requirements.txt
python app.py
# Opens at http://localhost:5000

---

Future Considerations (Out of Scope)

• Multi-select layers
• Drag-and-drop reordering
• Undo/redo history
• Collaborative editing
• Additional export formats
• Layer groups in UI (show hierarchy)