# ORA Editor - Browser-Based Image Layer Editor ## Overview A Flask + Alpine.js web application for editing OpenRaster (ORA) files and PNG images with layer management, polygon-based mask extraction hints, and Krita integration. **Location:** `tools/ora_editor/` **Tech Stack:** - Backend: Flask (Python 3) - Frontend: Alpine.js + Tailwind CSS (CDN) - Image Processing: Pillow - Custom JS: Minimal (~50 lines for polygon canvas) --- ## Project Structure ``` tools/ora_editor/ ├── app.py # Flask application ├── ora_ops.py # ORA operations wrapper (functions from ora_edit.py) ├── templates/ │ └── editor.html # Single-page Alpine.js UI ├── requirements.txt # Python dependencies ├── temp/ # Krita temp files (gitignored) └── .gitignore ``` --- ## Dependencies ### Python Packages (`requirements.txt`) ``` flask>=3.0.0 pillow>=10.0.0 ``` ### External Requirements - **ComfyUI** running at configurable address (default: `127.0.0.1:8188`) - **Krita** desktop application (for "Open in Krita" feature) --- ## Technology Choices ### Why Flask + Alpine.js? - Flask: Minimal Python web framework, mirrors the CLI tool workflow - Alpine.js: Reactive UI with zero custom JS, perfect for this use case - Tailwind CSS (CDN): Rapid styling without build step ### Why Minimal Custom JS? The polygon drawing requires capturing mouse clicks on a canvas. This is the only interaction that Alpine.js cannot handle natively. Everything else (layer toggles, modals, forms) is pure Alpine.js. --- ## File Operations ### Root Directory All file paths are relative to the project root: `/home/noti/dev/ai-game-2` ### Supported Formats - **ORA** (OpenRaster): Native format with layer support - **PNG**: Auto-converted to ORA on open ### Open Flow 1. User enters path relative to project root (e.g., `scenes/kq4_010/pic.png`) 2. Backend checks if file exists 3. If PNG: auto-create ORA with single `base` layer 4. Parse ORA structure, extract layers 5. Return layer list + image data URLs --- ## Layer System ### Layer Data Structure ```python { "name": "door_0", # Layer identifier "is_group": False, # False for regular layers "visibility": True, # Visible by default "src": "data/door_0.png", # Path inside ORA zip "opacity": 1.0, # Opacity 0.0-1.0 } ``` ### ORA Structure Created ``` 📁 Group: (e.g., "door") └─ _0 (e.g., "door_0") 📁 Group: └─ _0 🖼️ Base: base ``` ### Layer Operations | Operation | Description | |-----------|-------------| | **Toggle Visibility** | Show/hide layer on canvas | | **Rename** | Change layer/entity name | | **Delete** | Remove layer from ORA | | **Reorder** | Move layer up/down in stack (changes z-index) | | **Add Masked** | Add new layer with alpha mask | --- ## Canvas Display ### Image Rendering - Layers rendered as stacked `` elements with CSS positioning - Base layer at bottom, entity layers stacked above - Visibility controlled by `opacity: 0` or `opacity: 1` - No server-side compositing for preview ### Layer DOM Structure ```html
``` --- ## Polygon Drawing ### Purpose The polygon is **temporary** - it's a visual guide to help the AI mask extraction understand what area to focus on. It does NOT affect the final ORA file directly. ### Coordinate System - Points stored as percentages (0.0-1.0) of image dimensions - Sent to backend for mask extraction hint - Drawn on a canvas overlay ### User Interaction 1. Click "Start Drawing" button 2. Click on image to add points (shown as small circles) 3. Right-click or double-click to close polygon 4. Polygon renders with selected color and line width 5. "Clear" removes polygon ### Backend Storage Polygon points stored in memory on server (per-request, not session). Used only when submitting mask extraction with `use_polygon: true`. ### Implementation (Minimal JS) ```javascript // Canvas click handler (only custom JS needed) canvas.addEventListener('click', (e) => { const rect = canvas.getBoundingClientRect(); const x = (e.clientX - rect.left) / rect.width; const y = (e.clientY - rect.top) / rect.height; alpineStore.addPolygonPoint(x, y); }); ``` --- ## Mask Extraction (ComfyUI) ### Workflow 1. User describes subject (e.g., "the wooden door") 2. Optionally enable "Use polygon hint" 3. Click "Extract Mask" ### API: `POST /api/mask/extract` ```json { "subject": "the wooden door", "use_polygon": true } ``` ### Backend Process 1. If `use_polygon: true`, fetch stored polygon points 2. Draw polygon overlay on input image (using `draw_polygon.py` logic) 3. Build prompt: "Create a black and white alpha mask of {subject}" 4. Queue workflow to ComfyUI at configured server URL 5. Poll for completion (timeout: 4 minutes) 6. Download resulting mask to `temp/` directory 7. Return mask path ### ComfyUI Integration - Server address stored in localStorage (client-side setting) - Uses existing `tools/image_mask_extraction.json` workflow - Same extraction logic as `extract_mask.py` ### Mask Preview Modal When mask is ready: 1. Full-screen modal appears 2. Shows base image with mask applied as colored tint overlay 3. Three buttons: - **Re-roll**: Re-run extraction with same params - **Use This Mask**: Add masked layer to ORA, close modal - **Cancel**: Discard mask, close modal ### Modal Display ``` ┌─────────────────────────────────────────────────────────┐ │ │ │ [Base image with mask applied as tint overlay] │ │ │ │ [Re-roll] [Use This Mask] [Cancel] │ │ │ └─────────────────────────────────────────────────────────┘ ``` --- ## Krita Integration ### Flow 1. User selects a layer and clicks "Open in Krita" 2. Backend extracts layer PNG to `temp/{layer_name}_{timestamp}.png` 3. Returns `file://` URL 4. Browser opens URL (Krita launches with file) 5. User edits in Krita, saves ### File Watching Since browser cannot detect when Krita saves: - **On tab visibility change**: Check file mtime, auto-reload if changed - **Manual "Refresh" button**: Force reload from disk - **No automatic save back**: User explicitly saves in Krita, then refreshes ### Temp File Management - Temp files stored in `tools/ora_editor/temp/` - Named: `{layer_name}_{unix_timestamp}.png` - Not automatically deleted (low priority) --- ## API Reference ### File Operations #### `POST /api/open` Open a PNG or ORA file. **Request:** ```json {"path": "scenes/kq4_010/pic.png"} ``` **Response:** ```json { "success": true, "file_path": "/home/noti/dev/ai-game-2/scenes/kq4_010/pic.ora", "ora_path": "/home/noti/dev/ai-game-2/scenes/kq4_010/pic.ora", "layers": [ {"name": "base", "visibility": true, "src": "data/base.png"}, {"name": "door_0", "visibility": true, "src": "data/door/door_0.png"} ] } ``` #### `POST /api/save` Persist current layer state to ORA file. **Request:** ```json {"ora_path": "scenes/kq4_010/pic.ora"} ``` **Response:** ```json {"success": true} ``` --- ### Layer Operations #### `GET /api/layers` Get current layer structure. **Response:** ```json { "layers": [ {"name": "base", "visibility": true, "src": "data/base.png"}, {"name": "door_0", "visibility": true, "src": "data/door/door_0.png"} ] } ``` #### `POST /api/layer/add` Add a new masked layer. **Request:** ```json { "entity_name": "door", "mask_path": "/tmp/mask.png", "source_layer": "base" } ``` **Response:** ```json {"success": true, "layer_name": "door_1"} ``` #### `POST /api/layer/rename` Rename a layer. **Request:** ```json {"old_name": "door_0", "new_name": "wooden_door_0"} ``` **Response:** ```json {"success": true} ``` #### `POST /api/layer/delete` Delete a layer. **Request:** ```json {"layer_name": "door_0"} ``` **Response:** ```json {"success": true} ``` #### `POST /api/layer/reorder` Move layer up or down. **Request:** ```json {"layer_name": "door_0", "direction": "up"} ``` **Response:** ```json {"success": true} ``` --- ### Image Serving #### `GET /api/image/` Serve a layer PNG. Returns image data. #### `GET /api/image/base` Serve base/merged image. #### `GET /api/image/polygon` Serve polygon overlay image (for drawing mode). --- ### Polygon #### `POST /api/polygon` Draw polygon overlay on current image. **Request:** ```json { "points": [{"x": 0.1, "y": 0.2}, {"x": 0.5, "y": 0.8}], "color": "#FF0000", "width": 2 } ``` **Response:** ```json {"success": true, "overlay_url": "/api/image/polygon"} ``` #### `POST /api/polygon/points` Store polygon points for mask extraction. **Request:** ```json {"points": [{"x": 0.1, "y": 0.2}, {"x": 0.5, "y": 0.8}]} ``` **Response:** ```json {"success": true} ``` #### `POST /api/polygon/clear` Clear polygon overlay. **Response:** ```json {"success": true} ``` --- ### Mask Extraction #### `POST /api/mask/extract` Extract mask using ComfyUI. **Request:** ```json { "subject": "the wooden door", "use_polygon": true } ``` **Response:** ```json { "success": true, "mask_path": "/home/noti/dev/ai-game-2/tools/ora_editor/temp/mask.png" } ``` --- ### Krita #### `POST /api/krita/open` Copy layer to temp and return file URL. **Request:** ```json {"layer_name": "base"} ``` **Response:** ```json { "success": true, "file_url": "file:///home/noti/dev/ai-game-2/tools/ora_editor/temp/base_1234567890.png", "temp_path": "/home/noti/dev/ai-game-2/tools/ora_editor/temp/base_1234567890.png" } ``` #### `GET /api/krita/status/` Check if temp file was modified. **Response:** ```json { "modified": true, "mtime": 1234567890 } ``` --- ## Frontend State (Alpine.js) ### Main Store (`x-data`) ```javascript { // File state filePath: '', oraPath: '', layers: [], baseImage: '', // UI state selectedLayer: null, isDrawingPolygon: false, polygonPoints: [], polygonColor: '#FF0000', polygonWidth: 2, // Mask extraction maskSubject: '', usePolygonHint: true, isExtracting: false, showMaskModal: false, tempMaskPath: null, // Krita kritaLayer: null, kritaMtime: null, // Settings comfyUrl: localStorage.getItem('comfy_url') || '127.0.0.1:8188', } ``` ### Layer Object ```javascript { name: 'door_0', visibility: true, src: 'data/door/door_0.png' } ``` --- ## UI Layout ``` ┌──────────────────────────────────────────────────────────────────────┐ │ [Open: ________________________] [Open File] [Settings ⚙] │ ├───────────────────┬──────────────────────────────────────────────────┤ │ LAYERS │ │ │ ☑ base │ │ │ ☑ door_0 │ [Image Canvas with │ │ ☐ chest_0 │ stacked layers] │ │ │ │ │ [Rename] [Delete] │ │ │ [▲ Up] [▼ Down] │ │ │ │ │ │ ─────────────────│ │ │ POLYGON TOOL │ │ │ Color: [#FF0000] │ │ │ Width: [2____] │ │ │ [Start Drawing] │ │ │ Points: 5 │ │ │ [Clear] │ │ │ │ │ │ ─────────────────│ │ │ MASK EXTRACTION │ │ │ Subject: [______]│ │ │ ☑ Use polygon │ │ │ [Extract Mask] │ │ │ │ │ │ ─────────────────│ │ │ [Open in Krita] │ │ │ │ │ ├───────────────────┴──────────────────────────────────────────────────┤ │ [Save] [Refresh from Krita] │ └──────────────────────────────────────────────────────────────────────┘ ``` --- ## Mask Preview Modal ``` ┌─────────────────────────────────────────────────────────┐ │ EXTRACTED MASK │ │ ───────────────────────────────────────────────────── │ │ │ │ [Base image with mask applied as tinted overlay] │ │ │ │ [Re-roll] [Use This Mask] [Cancel] │ │ │ └─────────────────────────────────────────────────────────┘ ``` - **Re-roll**: Re-runs extraction with same params - **Use This Mask**: Calls `POST /api/layer/add`, closes modal - **Cancel**: Closes modal, mask discarded --- ## Settings Modal ``` ┌─────────────────────────────────────────────────────────┐ │ SETTINGS │ │ ───────────────────────────────────────────────────── │ │ │ │ ComfyUI Server: [127.0.0.1:8188____________] │ │ │ │ [Save] [Cancel] │ │ │ └─────────────────────────────────────────────────────────┘ ``` - Settings stored in `localStorage` - ComfyURL stored as `comfy_url` key --- ## Implementation Tasks ### Phase 1: Project Setup - [ ] Create directory structure (`templates/`, `temp/`) - [ ] Create `requirements.txt` - [ ] Create `.gitignore` - [ ] Test virtualenv/dependencies ### Phase 2: Backend Core - [ ] Create `ora_ops.py` with wrapper functions: - [ ] `load_ora(path)` - Parse ORA, return layers + images - [ ] `save_ora(ora_path)` - Rebuild ORA from current state - [ ] `add_masked_layer(ora_path, entity, mask_path, source)` - Apply mask - [ ] `rename_layer(ora_path, old, new)` - Update layer name - [ ] `delete_layer(ora_path, name)` - Remove layer - [ ] `reorder_layer(ora_path, name, direction)` - Move up/down - [ ] `get_layer_image(ora_path, layer_name)` - Extract PNG - [ ] `get_base_image(ora_path)` - Get merged/composited base - [ ] Create `app.py` with Flask routes: - [ ] File operations (`/api/open`, `/api/save`) - [ ] Layer operations (all `/api/layer/*`) - [ ] Image serving (`/api/image/*`) - [ ] Polygon (`/api/polygon*`) - [ ] Mask extraction (`/api/mask/extract`) - [ ] Krita (`/api/krita/*`) ### Phase 3: Frontend UI - [ ] Create `editor.html` with Tailwind + Alpine.js - [ ] File open input and handler - [ ] Layer list with visibility toggles - [ ] Canvas area with stacked layer images - [ ] Layer operations (rename, delete, reorder buttons) ### Phase 4: Polygon Drawing - [ ] Polygon tool UI (color, width, start/clear) - [ ] Canvas overlay for polygon rendering - [ ] Minimal JS for click-to-add-points - [ ] Backend polygon drawing endpoint ### Phase 5: Mask Extraction - [ ] Mask extraction form UI - [ ] Backend ComfyUI integration - [ ] Mask preview modal - [ ] Re-roll / Use This Mask / Cancel logic ### Phase 6: Krita Integration - [ ] "Open in Krita" button and endpoint - [ ] File URL generation - [ ] Refresh button and logic - [ ] Visibility-change auto-reload (JS) ### Phase 7: Polish - [ ] Settings modal for ComfyUI URL - [ ] Error handling and user feedback - [ ] Loading states - [ ] Empty states --- ## Key Implementation Details ### ORA File Structure ORA is a ZIP file containing: ``` mimetype # "image/openraster" stack.xml # Layer structure XML mergedimage.png # Composited full image Thumbnails/thumbnail.png # Preview thumbnail data/ base.png # Base layer / _0.png # Masked layer ``` ### stack.xml Format ```xml ``` ### Image Compositing for Preview - Frontend receives individual layer PNG URLs - Layers stacked with CSS `position: absolute` - Visibility toggled via CSS `opacity` - No server-side compositing needed ### Polygon Overlay Rendering 1. Backend receives points as `[{x: 0.1, y: 0.2}, ...]` 2. Convert to absolute pixel coordinates 3. Use Pillow `ImageDraw.polygon()` to draw 4. Return overlay as PNG ### ComfyUI Workflow Uses existing `tools/image_mask_extraction.json`: 1. Load input image 2. If polygon hint: composite polygon overlay 3. Set prompt text 4. Queue prompt to ComfyUI 5. Poll for completion 6. Download output mask --- ## Color Reference ### Default Polygon Colors | Name | Hex | |------|-----| | Red | #FF0000 | | Green | #00FF00 | | Blue | #0000FF | | Yellow | #FFFF00 | ### Mask Tint Colors (Preview) Default: #00FF0080 (semi-transparent green) --- ## Error Handling | Error | User Feedback | |-------|---------------| | File not found | "File not found: {path}" | | Invalid ORA | "Invalid ORA file format" | | ComfyUI offline | "Cannot connect to ComfyUI at {url}" | | ComfyUI timeout | "Mask extraction timed out (4 min)" | | Layer not found | "Layer '{name}' not found" | --- ## Browser Compatibility Tested/supported: - Modern browsers with ES6+ support - Chrome/Edge (primary) - Firefox - Safari Required features: - ES6 modules - CSS Grid/Flexbox - Canvas API (for polygon) - localStorage (for settings) --- ## Running the Application ```bash cd tools/ora_editor source ../venv/bin/activate # or your Python env pip install -r requirements.txt python app.py # Opens at http://localhost:5000 ``` --- ## Future Considerations (Out of Scope) - Multi-select layers - Drag-and-drop reordering - Undo/redo history - Collaborative editing - Additional export formats - Layer groups in UI (show hierarchy)