AI-Editor

An AI-first image editor that runs entirely on the local GPU: a real layer/canvas editor plus a chat agent that operates on the same document, where every AI edit lands as a normal, undoable operation. Tauri 2 shell + React frontend + Python inference sidecar driving a headless ComfyUI.

See PLAN.md for the architecture, the model stack and the milestone plan. This file is only about getting a clean clone running.

Status: M0 (skeleton) and M1 (manual essentials) are done. M2 (first AI ops) is the next milestone — that is where the model weights below start being used.

---

Setting up from a clean clone

Everything except the model weights is reproducible from this repo. The four untracked things — frontend/node_modules, src-tauri/target, server/.venv and the whole comfyui/ checkout — are rebuilt by the commands below from lockfiles and pinned refs. Only the weights have to come off the network again.

0. Prerequisites

Tool	Version used	Notes
uv	0.11+	Drives all Python. Must be on PATH — the Tauri shell spawns the sidecar with uv run.
Node.js	24.x	npm ci uses the committed lockfile.
Rust	1.94+	Plus the Tauri 2 prerequisites (on Windows: MSVC build tools + WebView2).
Git	any	Used by the provisioning script to clone pinned repos.
NVIDIA GPU	RTX 5070 Ti (16 GB, Blackwell sm_120)	Driver new enough for CUDA 12.8. Everything is pinned for sm_120.

Disk: ~60 GB for the core weights, ~90 GB for the full model stack, plus ~15 GB for the ComfyUI venv (torch alone is ~2.5 GB downloaded, more unpacked).

1. Clone

git clone https://git.ossalali.com/oss/AI-Editor.git
cd AI-Editor

2. Python sidecar

uv sync --directory server --extra dev

Reproduces server/.venv exactly from server/uv.lock. (You can skip this — uv run syncs on demand — but doing it up front makes the first app launch fast instead of mysteriously slow.)

3. Frontend

npm ci --prefix frontend

npm ci, not npm install: it installs exactly what frontend/package-lock.json pins and fails rather than silently drifting.

4. Rust shell

cargo build --manifest-path src-tauri/Cargo.toml

First build is slow (a few hundred crates); after that it is incremental.

5. Headless ComfyUI

uv run scripts/provision_comfyui.py --check       # prerequisites only, changes nothing
uv run scripts/provision_comfyui.py --boot-test   # full provision, then boot and probe

Clones ComfyUI at its pinned tag into comfyui/ (gitignored), creates a dedicated venv, installs Blackwell-targeted torch (cu128) and the pinned custom nodes, and writes comfyui/extra_model_paths.yaml so every model category resolves to the E: drive. Idempotent — re-running fast-forwards each repo to its pin. Details and the pin list: scripts/README.md.

6. Model weights

uv run scripts/download_models.py --list          # the manifest and what it costs
uv run scripts/download_models.py --check --all   # what is already on disk
uv run scripts/download_models.py --group core    # the M2 set (~51 GB)

This is the one step that always hits the network on a fresh machine. Weights go to AI_EDITOR_COMFY_MODELS (default E:\Models\ComfyUI) in the layout extra_model_paths.yaml expects; the agent's llama.cpp GGUF goes to the shared HuggingFace cache at HF_HOME. Groups (core, agent, people, quality, speed) map to milestones — see scripts/README.md.

Interrupted downloads resume, and re-running skips whatever is already there, so it is safe to do this in chunks.

7. Run it

npm run tauri dev --prefix frontend

The Tauri CLI is a pinned devDependency, so step 3 already installed it — no global cargo install tauri-cli needed. (If you have one anyway, cargo tauri dev --manifest-path src-tauri/Cargo.toml is equivalent.)

The Rust shell reserves a free port, spawns the sidecar with uv run, waits for /health, and kills the whole process tree on exit. That is also why uv has to be on PATH rather than just installed somewhere.

---

Verifying a setup

Run the full suite — it is headless, binds no ports and steals no focus, so it is safe to run alongside anything else:

uv run --directory server pytest        # sidecar: ops, store, compositor, API
npm test --prefix frontend              # frontend: Vitest + RTL, jsdom
cargo build --manifest-path src-tauri/Cargo.toml

For the inference stack specifically:

uv run scripts/provision_comfyui.py --boot-test   # ComfyUI answers /system_stats
uv run scripts/download_models.py --check --all   # every weight accounted for

---

Where things live

AI-Editor/
  PLAN.md             architecture, model stack, milestones
  design/             DESIGN_PROMPT.md + baseline/ (the binding style reference)
  src-tauri/          Rust shell: window, sidecar lifecycle (spawn/health/kill-tree)
  frontend/           React + Vite + TypeScript
  server/             Python sidecar: FastAPI, project store, ops, agent, ComfyUI client
  scripts/            provisioning + model download
  comfyui/            pinned headless ComfyUI (gitignored, provisioned)

Nothing generated is committed. comfyui/, node_modules/, dist/, target/ and .venv/ are all gitignored and rebuilt by the steps above.

Paths and environment

Model weights never land on C: — that is a standing rule for this machine, and the provisioning script encodes it.

Variable	Default	Purpose
AI_EDITOR_COMFY_MODELS	E:\Models\ComfyUI	Root for all ComfyUI weights. Both scripts read it.
HF_HOME	E:\Models\huggingface	Shared HuggingFace cache (set machine-wide). The agent GGUF lives here.
AI_EDITOR_COMFY_PORT	8188	Port for the ComfyUI boot probe.
AI_EDITOR_COMFY_CPU	(unset)	If set, boot ComfyUI in --cpu mode.
AI_EDITOR_SERVER_DIR	<repo>/server	Override where the Rust shell looks for the sidecar.

On a machine without an E: drive, set AI_EDITOR_COMFY_MODELS and HF_HOME before running either script and everything follows.

Known follow-ups

Carried from M0, both due in M2 (also documented in scripts/README.md):

• Nunchaku — the ComfyUI wrapper node installs, but the NVFP4 draft tier needs the separate hardware-specific nunchaku backend wheel. The speed weight group is useless until that is installed.
• SeedVR2 — imports triton, which has no stock Windows wheel. The node logs an import warning and is skipped at boot; the quality group waits on it.

Booting ComfyUI on Windows requires UTF-8 (PYTHONUTF8=1, PYTHONIOENCODING=utf-8). A failing node import makes ComfyUI log a traceback, and on a cp1252 console that raises an unhandled UnicodeEncodeError that kills the process outright.