Wayfinder: AI Studio end-to-end slice spec #1

New issue

Closed

opened 2026-10-06 03:55:21 +00:00 by crowmaki · 1 comment

crowmaki commented 2026-10-06 03:55:21 +00:00

Owner

Copy link

Destination

A PRD issue on Forgejo, split into build issues, for a thin end-to-end slice: OSF script → entities → locked references → atlases → setups/shots with boards and keyframes → takes (drafts, promotion, single-pass final) → animatic → Resolve FCPXML edit package. Source plan: scratch/scratch-plan.md (local, gitignored).

Notes

• Domain: read CONTEXT.md first and use its terms (Take, Rendition, Promotion, Reference set, Tile, Lock snapshot, Look, Wardrobe item, Preservation, Atlas, Board, Keyframe, Performance, Lighting…). ADRs in docs/adr/.
• Skills: grilling tickets call grilling + domain-modeling; prototypes call prototype; research calls research.
• Stack: Tauri 2 + SvelteKit + Svelte 5 (scaffold only). Everything local via ComfyUI on one RTX 5090 / 32 GB. Video backend MiniMax H3 (FL2VA + Ref2VA).
• Standing decisions from charting:
  • Built for the owner first, but shippable: the app runs a preflight check of the user-managed ComfyUI (nodes, models) and installs nothing. Each Take can pick its own checkpoint, quantization and LoRAs.
  • The app ships fixed workflows (ADR-0002). The owner's scratch/minimaxH3SEEDHUNTERLatent_v25.json is a technique reference, not a runtime dependency.
  • A Take is one roll; it has draft and/or final Renditions; Promotion turns a draft into a final; single-pass final is allowed.
  • .studio = SQLite project file, with Renditions in a sibling media folder (ADR-0001).
  • The app owns the render queue and feeds ComfyUI one job at a time.
  • No LLM in the slice: props come from text selection or manual creation; all Entities can be created manually.
  • References are imported (with cropping a sheet into Tiles) or generated by one t2i workflow; editing happens outside the app. Tiles are versioned and Reference sets are locked as Lock snapshots.
  • Each Setup gets a hand-picked location angle and a framing note, with a blocking-plan UI placeholder.
  • Performance is optional (from a chosen Take's audio or imported); by default H3 generates audio with the Take.
  • Looks and Wardrobe items are in, with full and partial Preservation.
  • Boards: paint tool, Annotation layer, Notation, and a no-sketch t2i path. Raw sketches go to H3.
  • A minimal Animatic player. Segmented Shots ≤ 15 s. Keyframes are in.
  • One OSF import (custom urn:drafter:1 dialect; scene headings carry d:id; <lists> are ignored). Scenes and lines get stable IDs.
  • Locations are flat, with merge. Lighting is a Scene field inferred from the time slug.
  • Atlases are built per Setup and identified by content. Export targets Resolve FCPXML over a folder layout that doesn't depend on the NLE.
• H3 behaviour: the owner has hands-on H3 experience. Questions about how H3 behaves are grillings with the owner, not research.

Decisions so far

• ComfyUI API surface: all ComfyUI calls are made from Rust, because loopback Origin checks block the webview. Jobs follow WS + upload + /api/prompt with our own prompt_id; a 200 with node_errors counts as a failure. The app must save results right away because ComfyUI history is in-memory only. Preflight uses /api/object_info, and pickers use /api/models/*. Notes: branch research/comfyui-api.
• Local finishing pass for final Renditions: finals use the H3 latent upscaler 3D plus an H3 refine pass, which is also how Promotion works, at native 24 fps with no RIFE (RIFE stays an opt-in extra). SeedVR2 is the pixel-space fallback or the step to 2K/4K. Peak VRAM and runtime at 1080p on a 5090 have to be measured. Notes: branch research/finishing-pass.
• Reusing H3 AV latents across jobs: yes, across jobs and restarts. Split with LTXVSeparateAVLatent, use two SaveLatent nodes in the draft job, and two LoadLatent nodes in the Promotion job, which then upscales and re-concats. Saving the joint latent directly fails because it is a NestedTensor. Files are about 14–41 MB at 768p. The Promotion job must rebuild conditioning and re-apply the audio noise mask. Not yet checked by running it in ComfyUI. Notes: branch research/h3-latent-reuse.
• Resolve FCPXML import fidelity: emit hand-written FCPXML 1.10, used only for the Animatic timeline (cuts, linked audio, Board stills). Bins come from per-scene folders, and clip names from 12A-3.mov file names. Circled status and notes go through a metadata CSV into Resolve's Good Take, Comments and Scene/Shot/Take fields. The OTIO FCPXML adapter is not viable, but native .otio is a candidate. Not yet tested in a real Resolve. Notes: branch research/resolve-fcpxml.
• OSF import mapping: each action paragraph and each whole dialogue block is a Line, with ULID ids; Scenes keep d:id. Shots cover whole Lines only. Characters come from cues only and can be merged. Cue extensions are stored on the Line, and SUBTITLE Lines get hand-entered spoken language and text. Headings split into Location (INT and EXT kept separate, auto-grouped into Location folders) and a slug that maps to Lighting through an editable table. Dual dialogue gets a shared simultaneous marker. Scene numbers are frozen at import, and omitted Scenes stay as placeholders. Import is lenient and ends with a summary of warnings.
• Rust core vs TypeScript frontend split: a thick Rust studio-core crate (no Tauri) owns the domain, SQLite, worker, ComfyUI, ffmpeg (an LGPL sidecar), all derived pixels and export, with a thin src-tauri adapter. Svelte is a view: taurpc commands with adapter DTOs, coarse invalidation events, and Channels for job progress. One studio: scheme serves blobs and Renditions by id. Undo is an in-memory Rust command log for authored edits only. One Project per process. ADR-0003.
• Default atlas layout: one row per Look (face → front → side → back) in uniform 4:9 cells, scaled to fit and padded on mid-grey, with wider gaps between rows and no burned-in text. Scaled to a megapixel budget (1.0 MP default, with a native toggle; per Project, overridable per Setup). Rows shrink to the Look count. Default grouping follows subject order, and the user can regroup per Setup. A missing Tile collapses its cell, and Atlases only combine Looks with the same Tile set (parity). Wardrobe items take their own slot.
• Import and crop a sheet into Tiles: import opens one stage with all four free-form crop boxes, auto-fitted to detected figures in reading order and snapping to their edges. A side panel compares each new crop with the Tile's current version. Re-crop and provenance live on the Reference set's Tile cards. A Source sheet is an immutable blob owned by the Project, so one sheet can feed several sets. Each imported version stores its own pixels plus the sheet and its crop box. A new version is made only when the crop differs from the current one, and committing never locks. Notes: branch prototype/crop-sheet.
• Board paint tool and Notation: a Panel-strip editor with a Sketch/Annotate switch over a Scene-wide Board timeline, which is for storyboarding only (the Animatic is a separate surface). Boards can have several panels, tiled into one image slot by default with a per-Shot override. The canvas is a fixed low resolution at the Project's aspect ratio (short side 180 px), sent without upscaling. Annotations are structured: Notation symbols with a subject and direction, read back as prompt lines. They are composited onto the image by default, with a per-Shot clean-sketch toggle. The Notation set is a fixed glyph library with editable entries. t2i fills the Sketch layer as generated, with opt-in tone reduction. Notes: branch prototype/board-paint.
• Dailies review and circling: one Dailies screen per Scene with a toggle between a coverage wall (Take tiles per Shot, grouped by Setup, with a synced compare tray) and a sortable table. A keyboard screening mode for unseen Takes opens from either view, and a details panel holds an A/B wipe and the inputs comparison. A Shot can have several circled Takes; the Animatic plays the Pinned take or else the most recently circled one. Stale Takes can be promoted with a warning. A new seed uses the current inputs by default. Notes are a list with optional timecodes. The seen flag is stored. Failed Takes offer Retry or Delete, and NG Takes are hidden but kept. Notes: branch prototype/dailies.
• Animatic duration rules: the Animatic is a per-Scene Cut over Lines. Each Line plays from one covering Shot, and only user overrides are stored; the default is the first Setup in Setup order. Lines carry time, from the Performance, then a manual override, then a text estimate. Circled Takes play between per-Take Line marks. Take events carry their own audio; Board events carry the Performance slice or captioned silence. A Shot's length is its Lines plus handles, with a 2–15 s override; a split is offered past 15 s, and a single Line over 15 s blocks queuing. Uncovered Lines get slates, and dual dialogue is one span. "Play from here" runs on across Scenes.
• Verify latent round trip and refine-pass cost: confirmed on the owner's ComfyUI 0.39.0. Split draft → restart → Promotion matches the single job to within GPU noise. Promotion at 1984×1120 takes ~90 s for 5 s and ~283 s for 15 s, and fits 32 GB. The 2nd pass re-noises generated audio, so a Promotion keeps the draft's audio only with a 0 audio noise mask. The 1.0 MP selector gives 1376×768 drafts. Notes: branch research/h3-latent-reuse.
• Test Resolve import of FCPXML and metadata CSV: tested on Resolve 21.1.1 free. FCPXML must hold only the Animatic project in one event, because browser-clip events cause "No valid sequences". Cuts, linked stereo audio and Board stills import, but FCPXML markers are lost. The metadata CSV needs Start TC/End TC (timecode matching is on by default); UTF-8 is fine. Circled status goes in the Tag column (1 = Good Take, 2 = Rejected), not the free-text Good Take column. Native .otio keeps marker names and colours. Test kit: local scratch/resolve-test/.
• Prompt composer phrasing: H3's six fields, always present. Characters are <Subject N> and Props <Prop N>, both built from structured fields with no names; the Location is <Environment> plus a lighting note; slots are <Picture N>/<Audio N>, numbered after the budget. Preservation goes in retention_analysis (fully/partially_preserved, attribute_transfer for Wardrobe items, weak_reference for the Board, Keyframes with timestamps, Performance fully_copy, which drops Voice references). The action text carries dialogue chips in script order, with parentheticals as inline tags. Notation becomes plain sentences, and the storyboard is never named. The soundscape is the Location's Ambience plus a Scene addition; music is always N/A. A Shot can hold a hand-edited prompt override, with a quiet stale marker and reconcile on demand.
• Reference slot budget: the limits that matter are 9 images and 3 audio; video is unused in the slice. Default image order: Keyframes → Atlases → Wardrobe mannequins → location angle → Board → Props, with a per-Panel Board falling back to tiled before anything drops. The order is per Project, drag-and-drop, and seeded from an app-wide default. Each Shot has a Slot list with a sent/not-sent line (excluded vs over budget); edited lists keep user placements when candidates change. No per-Take override; Takes record their slot assignment. Voices come from speaking Characters only, one slot each. Props are key props only, one packed slot each. An over-budget badge appears in Shot listings.
• .studio schema: a table-level model with ULID keys and a content-addressed blob table. Characters, Locations and Props get separate tables, and each owner points at its Reference set. Retired Tiles stay. The current Lock snapshot is the highest number. Atlases are a cache keyed by their inputs (ADR-0005). Shots cover a contiguous Line range, Setup letters are never reused, and a Scene cast picks Looks. Takes record take_input FK rows, the exact job as a blob, and take_link edges for the chain. A job table holds one row per ComfyUI submission, and the media folder is ID-named. Rows a Take consumed are soft-deleted (ADR-0004). Aspect ratio can change, with per-Shot overrides; Panels and Keyframes keep their pixels and are barred at send time. Keyframes can be taken from a Take's frame (in the slice). Performances are versioned, and WAL is used only while the file is open.
• Staleness rules: a Take is stale when a re-roll with current inputs (ignoring its seed and generation profile) would send a different job. Staleness is computed and never stored, and each difference is a named Stale reason. A shipped workflow change is informational only. Users can accept a reason per Take, until that input changes again. Promotion rebuilds from the recorded inputs. Staleness shows on every Take in Dailies, but only for circled Takes in Shot listings and the Animatic, and it never blocks export. A batch action re-rolls stale circled Takes as unseen drafts and never changes circles.
• Shipped workflow roster and slot contracts: five workflows. h3.draft (an MP4 plus a saved latent), h3.promotion (upscale + refine, with the draft's audio always frozen), h3.final (a Single-pass final generated directly at final size), and image.t2i and image.board (both Qwen-Image 2.1). Each is a JSON template with a manifest, and Rust splices in the slot loaders and the LoRA chain. The Generation profile picks the model and encoder files, LoRAs, a Distilled/Full speed mode, and the draft/final short sides (768/1120). Versions are hand-bumped integers, and Promotion always uses the current one. Preflight runs per workflow, and jobs for a Not-ready workflow wait. Finals are a ProRes master plus an H.264 proxy, and every H3 job writes a WAV. A Performance is both forced into the latent and sent as a slot. Lengths round up to 5 + 17k frames. Re-promotion uses a new refine seed and replaces the final in place.
• Render worker scheduling: jobs are built at queue time, so a queued Take has its number and inputs and can be stale. Order is image jobs, then drafts, then finals, FIFO within each class. Model affinity applies within a class only, with a skip limit. Jobs can be dragged within a class, and Run next puts them at the front; nothing is pre-empted. Controls are Pause, per-job Hold and a sticky Hold finals switch. Cancel deletes a Take that has no Rendition and burns its number. A Take has at most one pending final job. The queue always reopens paused. On reopen the in-flight job is reattached, collected or requeued. Errors fail with no retry, except OOM, which retries once after /free (a setting). ComfyUI offline is a waiting state. Post-steps run in a CPU lane. A Queue panel pins failed jobs to the top.
• Edit package layout: one Edit package folder per Project, synced in place. It holds Scene → Setup bins with one .mov per Take that isn't NG (optionally circled only): a hard-linked ProRes master, or else the draft remuxed with PCM. Clips carry TC 01:00:00:00, and segments are named 12B_1-3. Timelines are per Scene plus the whole film, as FCPXML (no markers) and .otio (markers), with burned-in captions and a .srt. Metadata goes in takes.csv (Tag=1 for circled) and provenance in manifest.json. Dropped clips move to _Removed/. The proxy uses NVENC, falling back to OpenH264.
• External media view and cleanup: a project-wide Storage view of everything that takes space, showing what uses each item and how much space removing it would really free (counted by hard-link count). Only media-folder Renditions and latents can be purged (the row stays, the files go, and it can't be undone). Circled Takes and latents with a pending job are blocked. In-file items can only be deleted when unused, and only Source sheets are deleted here. Storage can empty the Edit package's _Removed/ folders. Suggestions only select items. Compact is a manual VACUUM. Each Rendition keeps a poster frame, so purged Takes still show.
• Marking Lines in a Performance: a Performance is a Bounce of the Cut's audio (marks come free; Lines with no circled Take carry forward the old Performance or become silent holes) or imported audio (silencedetect auto-placement, then hand correction). Marks are a contiguous partition (pauses go to the earlier Line, action Lines share the gaps, a dual-dialogue group is one span). The audio is immutable, but marks are editable; Takes record their sent slice, so only Shots whose slice moved go stale, and a Take's Line marks are snapshotted at send. A Performance covers a set of Lines; Shots straddling covered and uncovered Lines block queuing. Marked in a Performance lane on the Animatic surface.

Not yet specified

Nothing at the moment.

Out of scope

Beyond the slice; returns only as a fresh effort:

• Blender previz (playblast as a video reference)
• MCP server and prompt templates
• Lined script export
• Blocking plan editor (the slice keeps only a UI placeholder), derived framing, board underlays
• Board cleanup pass (ControlNet / edit model)
• Continuity state
• Take extension (seamless continuation with overlap): firmly wanted next, so the slice's provenance must not rule out chained generations
• Script re-import with reconciliation
• A render daemon that runs while the app is closed
• In-app inpaint / edit-model reference editing
• Takes with internal cuts ([Shot 2] blocks, intercuts): Shots stay aligned with real-world shots; intercuts are a separate effort. From Prompt composer phrasing.
• SeedVR2 and NVIDIA upscaling as drop-in finishing plugins. From Shipped workflow roster and slot contracts.
• Alternate voice sources for Performances (local TTS etc.); the Performance source edge list must allow them. From Marking Lines in a Performance.
• Forced alignment / ASR auto-marking of imported Performances. From Marking Lines in a Performance.

Tickets

• OSF import mapping #2
• Rust core vs TypeScript frontend split #3
• Default atlas layout #4
• Import and crop a sheet into Tiles #5
• Board paint tool and Notation #6
• Dailies review and circling #7
• Animatic duration rules #8
• ComfyUI API surface #13
• Reusing H3 AV latents across jobs #14
• Local finishing pass for final Renditions #15
• Resolve FCPXML import fidelity #16
• Verify latent round trip and refine-pass cost #20
• Test Resolve import of FCPXML and metadata CSV #21
• Prompt composer phrasing #9
• Reference slot budget #10
• .studio schema #11
• Staleness rules #12
• Shipped workflow roster and slot contracts #17
• Render worker scheduling #18
• Edit package layout #19
• External media view and cleanup #22
• Marking Lines in a Performance #23

## Destination A PRD issue on Forgejo, split into build issues, for a thin end-to-end slice: OSF script → entities → locked references → atlases → setups/shots with boards and keyframes → takes (drafts, promotion, single-pass final) → animatic → Resolve FCPXML edit package. Source plan: `scratch/scratch-plan.md` (local, gitignored). ## Notes - **Domain:** read `CONTEXT.md` first and use its terms (Take, Rendition, Promotion, Reference set, Tile, Lock snapshot, Look, Wardrobe item, Preservation, Atlas, Board, Keyframe, Performance, Lighting…). ADRs in `docs/adr/`. - **Skills:** grilling tickets call `grilling` + `domain-modeling`; prototypes call `prototype`; research calls `research`. - **Stack:** Tauri 2 + SvelteKit + Svelte 5 (scaffold only). Everything local via ComfyUI on one RTX 5090 / 32 GB. Video backend MiniMax H3 (FL2VA + Ref2VA). - **Standing decisions from charting:** - Built for the owner first, but shippable: the app runs a preflight check of the user-managed ComfyUI (nodes, models) and installs nothing. Each Take can pick its own checkpoint, quantization and LoRAs. - The app ships fixed workflows (ADR-0002). The owner's `scratch/minimaxH3SEEDHUNTERLatent_v25.json` is a technique reference, not a runtime dependency. - A Take is one roll; it has draft and/or final Renditions; Promotion turns a draft into a final; single-pass final is allowed. - `.studio` = SQLite project file, with Renditions in a sibling media folder (ADR-0001). - The app owns the render queue and feeds ComfyUI one job at a time. - No LLM in the slice: props come from text selection or manual creation; all Entities can be created manually. - References are imported (with cropping a sheet into Tiles) or generated by one t2i workflow; editing happens outside the app. Tiles are versioned and Reference sets are locked as Lock snapshots. - Each Setup gets a hand-picked location angle and a framing note, with a blocking-plan UI placeholder. - Performance is optional (from a chosen Take's audio or imported); by default H3 generates audio with the Take. - Looks and Wardrobe items are in, with full and partial Preservation. - Boards: paint tool, Annotation layer, Notation, and a no-sketch t2i path. Raw sketches go to H3. - A minimal Animatic player. Segmented Shots ≤ 15 s. Keyframes are in. - One OSF import (custom `urn:drafter:1` dialect; scene headings carry `d:id`; `<lists>` are ignored). Scenes and lines get stable IDs. - Locations are flat, with merge. Lighting is a Scene field inferred from the time slug. - Atlases are built per Setup and identified by content. Export targets Resolve FCPXML over a folder layout that doesn't depend on the NLE. - **H3 behaviour:** the owner has hands-on H3 experience. Questions about how H3 behaves are grillings with the owner, not research. ## Decisions so far <!-- the index: one line per closed ticket --> - [ComfyUI API surface](https://git.crowmaki.me/crowmaki/ai-studio/issues/13): all ComfyUI calls are made from Rust, because loopback Origin checks block the webview. Jobs follow WS + upload + `/api/prompt` with our own prompt_id; a 200 with node_errors counts as a failure. The app must save results right away because ComfyUI history is in-memory only. Preflight uses `/api/object_info`, and pickers use `/api/models/*`. Notes: branch `research/comfyui-api`. - [Local finishing pass for final Renditions](https://git.crowmaki.me/crowmaki/ai-studio/issues/15): finals use the H3 latent upscaler 3D plus an H3 refine pass, which is also how Promotion works, at native 24 fps with no RIFE (RIFE stays an opt-in extra). SeedVR2 is the pixel-space fallback or the step to 2K/4K. Peak VRAM and runtime at 1080p on a 5090 have to be measured. Notes: branch `research/finishing-pass`. - [Reusing H3 AV latents across jobs](https://git.crowmaki.me/crowmaki/ai-studio/issues/14): yes, across jobs and restarts. Split with `LTXVSeparateAVLatent`, use two `SaveLatent` nodes in the draft job, and two `LoadLatent` nodes in the Promotion job, which then upscales and re-concats. Saving the joint latent directly fails because it is a NestedTensor. Files are about 14–41 MB at 768p. The Promotion job must rebuild conditioning and re-apply the audio noise mask. Not yet checked by running it in ComfyUI. Notes: branch `research/h3-latent-reuse`. - [Resolve FCPXML import fidelity](https://git.crowmaki.me/crowmaki/ai-studio/issues/16): emit hand-written FCPXML 1.10, used only for the Animatic timeline (cuts, linked audio, Board stills). Bins come from per-scene folders, and clip names from `12A-3.mov` file names. Circled status and notes go through a metadata CSV into Resolve's Good Take, Comments and Scene/Shot/Take fields. The OTIO FCPXML adapter is not viable, but native `.otio` is a candidate. Not yet tested in a real Resolve. Notes: branch `research/resolve-fcpxml`. - [OSF import mapping](https://git.crowmaki.me/crowmaki/ai-studio/issues/2): each action paragraph and each whole dialogue block is a Line, with ULID ids; Scenes keep `d:id`. Shots cover whole Lines only. Characters come from cues only and can be merged. Cue extensions are stored on the Line, and SUBTITLE Lines get hand-entered spoken language and text. Headings split into Location (INT and EXT kept separate, auto-grouped into Location folders) and a slug that maps to Lighting through an editable table. Dual dialogue gets a shared simultaneous marker. Scene numbers are frozen at import, and omitted Scenes stay as placeholders. Import is lenient and ends with a summary of warnings. - [Rust core vs TypeScript frontend split](https://git.crowmaki.me/crowmaki/ai-studio/issues/3): a thick Rust `studio-core` crate (no Tauri) owns the domain, SQLite, worker, ComfyUI, ffmpeg (an LGPL sidecar), all derived pixels and export, with a thin `src-tauri` adapter. Svelte is a view: `taurpc` commands with adapter DTOs, coarse invalidation events, and Channels for job progress. One `studio:` scheme serves blobs and Renditions by id. Undo is an in-memory Rust command log for authored edits only. One Project per process. ADR-0003. - [Default atlas layout](https://git.crowmaki.me/crowmaki/ai-studio/issues/4): one row per Look (face → front → side → back) in uniform 4:9 cells, scaled to fit and padded on mid-grey, with wider gaps between rows and no burned-in text. Scaled to a megapixel budget (1.0 MP default, with a native toggle; per Project, overridable per Setup). Rows shrink to the Look count. Default grouping follows subject order, and the user can regroup per Setup. A missing Tile collapses its cell, and Atlases only combine Looks with the same Tile set (parity). Wardrobe items take their own slot. - [Import and crop a sheet into Tiles](https://git.crowmaki.me/crowmaki/ai-studio/issues/5): import opens one stage with all four free-form crop boxes, auto-fitted to detected figures in reading order and snapping to their edges. A side panel compares each new crop with the Tile's current version. Re-crop and provenance live on the Reference set's Tile cards. A Source sheet is an immutable blob owned by the Project, so one sheet can feed several sets. Each imported version stores its own pixels plus the sheet and its crop box. A new version is made only when the crop differs from the current one, and committing never locks. Notes: branch `prototype/crop-sheet`. - [Board paint tool and Notation](https://git.crowmaki.me/crowmaki/ai-studio/issues/6): a Panel-strip editor with a Sketch/Annotate switch over a Scene-wide Board timeline, which is for storyboarding only (the Animatic is a separate surface). Boards can have several panels, tiled into one image slot by default with a per-Shot override. The canvas is a fixed low resolution at the Project's aspect ratio (short side 180 px), sent without upscaling. Annotations are structured: Notation symbols with a subject and direction, read back as prompt lines. They are composited onto the image by default, with a per-Shot clean-sketch toggle. The Notation set is a fixed glyph library with editable entries. t2i fills the Sketch layer as generated, with opt-in tone reduction. Notes: branch `prototype/board-paint`. - [Dailies review and circling](https://git.crowmaki.me/crowmaki/ai-studio/issues/7): one Dailies screen per Scene with a toggle between a coverage wall (Take tiles per Shot, grouped by Setup, with a synced compare tray) and a sortable table. A keyboard screening mode for unseen Takes opens from either view, and a details panel holds an A/B wipe and the inputs comparison. A Shot can have several circled Takes; the Animatic plays the Pinned take or else the most recently circled one. Stale Takes can be promoted with a warning. A new seed uses the current inputs by default. Notes are a list with optional timecodes. The seen flag is stored. Failed Takes offer Retry or Delete, and NG Takes are hidden but kept. Notes: branch `prototype/dailies`. - [Animatic duration rules](https://git.crowmaki.me/crowmaki/ai-studio/issues/8): the Animatic is a per-Scene Cut over Lines. Each Line plays from one covering Shot, and only user overrides are stored; the default is the first Setup in Setup order. Lines carry time, from the Performance, then a manual override, then a text estimate. Circled Takes play between per-Take Line marks. Take events carry their own audio; Board events carry the Performance slice or captioned silence. A Shot's length is its Lines plus handles, with a 2–15 s override; a split is offered past 15 s, and a single Line over 15 s blocks queuing. Uncovered Lines get slates, and dual dialogue is one span. "Play from here" runs on across Scenes. - [Verify latent round trip and refine-pass cost](https://git.crowmaki.me/crowmaki/ai-studio/issues/20): confirmed on the owner's ComfyUI 0.39.0. Split draft → restart → Promotion matches the single job to within GPU noise. Promotion at 1984×1120 takes ~90 s for 5 s and ~283 s for 15 s, and fits 32 GB. The 2nd pass re-noises generated audio, so a Promotion keeps the draft's audio only with a 0 audio noise mask. The 1.0 MP selector gives 1376×768 drafts. Notes: branch `research/h3-latent-reuse`. - [Test Resolve import of FCPXML and metadata CSV](https://git.crowmaki.me/crowmaki/ai-studio/issues/21): tested on Resolve 21.1.1 free. FCPXML must hold only the Animatic project in one event, because browser-clip events cause "No valid sequences". Cuts, linked stereo audio and Board stills import, but FCPXML markers are lost. The metadata CSV needs `Start TC`/`End TC` (timecode matching is on by default); UTF-8 is fine. Circled status goes in the `Tag` column (`1` = Good Take, `2` = Rejected), not the free-text `Good Take` column. Native `.otio` keeps marker names and colours. Test kit: local `scratch/resolve-test/`. - [Prompt composer phrasing](https://git.crowmaki.me/crowmaki/ai-studio/issues/9): H3's six fields, always present. Characters are `<Subject N>` and Props `<Prop N>`, both built from structured fields with no names; the Location is `<Environment>` plus a lighting note; slots are `<Picture N>`/`<Audio N>`, numbered after the budget. Preservation goes in retention_analysis (fully/partially_preserved, attribute_transfer for Wardrobe items, weak_reference for the Board, Keyframes with timestamps, Performance fully_copy, which drops Voice references). The action text carries dialogue chips in script order, with parentheticals as inline tags. Notation becomes plain sentences, and the storyboard is never named. The soundscape is the Location's Ambience plus a Scene addition; music is always N/A. A Shot can hold a hand-edited prompt override, with a quiet stale marker and reconcile on demand. - [Reference slot budget](https://git.crowmaki.me/crowmaki/ai-studio/issues/10): the limits that matter are 9 images and 3 audio; video is unused in the slice. Default image order: Keyframes → Atlases → Wardrobe mannequins → location angle → Board → Props, with a per-Panel Board falling back to tiled before anything drops. The order is per Project, drag-and-drop, and seeded from an app-wide default. Each Shot has a Slot list with a sent/not-sent line (excluded vs over budget); edited lists keep user placements when candidates change. No per-Take override; Takes record their slot assignment. Voices come from speaking Characters only, one slot each. Props are key props only, one packed slot each. An over-budget badge appears in Shot listings. - [.studio schema](https://git.crowmaki.me/crowmaki/ai-studio/issues/11): a table-level model with ULID keys and a content-addressed blob table. Characters, Locations and Props get separate tables, and each owner points at its Reference set. Retired Tiles stay. The current Lock snapshot is the highest number. Atlases are a cache keyed by their inputs (ADR-0005). Shots cover a contiguous Line range, Setup letters are never reused, and a Scene cast picks Looks. Takes record `take_input` FK rows, the exact job as a blob, and `take_link` edges for the chain. A `job` table holds one row per ComfyUI submission, and the media folder is ID-named. Rows a Take consumed are soft-deleted (ADR-0004). Aspect ratio can change, with per-Shot overrides; Panels and Keyframes keep their pixels and are barred at send time. Keyframes can be taken from a Take's frame (in the slice). Performances are versioned, and WAL is used only while the file is open. - [Staleness rules](https://git.crowmaki.me/crowmaki/ai-studio/issues/12): a Take is stale when a re-roll with current inputs (ignoring its seed and generation profile) would send a different job. Staleness is computed and never stored, and each difference is a named Stale reason. A shipped workflow change is informational only. Users can accept a reason per Take, until that input changes again. Promotion rebuilds from the recorded inputs. Staleness shows on every Take in Dailies, but only for circled Takes in Shot listings and the Animatic, and it never blocks export. A batch action re-rolls stale circled Takes as unseen drafts and never changes circles. - [Shipped workflow roster and slot contracts](https://git.crowmaki.me/crowmaki/ai-studio/issues/17): five workflows. `h3.draft` (an MP4 plus a saved latent), `h3.promotion` (upscale + refine, with the draft's audio always frozen), `h3.final` (a Single-pass final generated directly at final size), and `image.t2i` and `image.board` (both Qwen-Image 2.1). Each is a JSON template with a manifest, and Rust splices in the slot loaders and the LoRA chain. The Generation profile picks the model and encoder files, LoRAs, a Distilled/Full speed mode, and the draft/final short sides (768/1120). Versions are hand-bumped integers, and Promotion always uses the current one. Preflight runs per workflow, and jobs for a Not-ready workflow wait. Finals are a ProRes master plus an H.264 proxy, and every H3 job writes a WAV. A Performance is both forced into the latent and sent as a slot. Lengths round up to 5 + 17k frames. Re-promotion uses a new refine seed and replaces the final in place. - [Render worker scheduling](https://git.crowmaki.me/crowmaki/ai-studio/issues/18): jobs are built at queue time, so a queued Take has its number and inputs and can be stale. Order is image jobs, then drafts, then finals, FIFO within each class. Model affinity applies within a class only, with a skip limit. Jobs can be dragged within a class, and Run next puts them at the front; nothing is pre-empted. Controls are Pause, per-job Hold and a sticky Hold finals switch. Cancel deletes a Take that has no Rendition and burns its number. A Take has at most one pending final job. The queue always reopens paused. On reopen the in-flight job is reattached, collected or requeued. Errors fail with no retry, except OOM, which retries once after /free (a setting). ComfyUI offline is a waiting state. Post-steps run in a CPU lane. A Queue panel pins failed jobs to the top. - [Edit package layout](https://git.crowmaki.me/crowmaki/ai-studio/issues/19): one Edit package folder per Project, synced in place. It holds Scene → Setup bins with one `.mov` per Take that isn't NG (optionally circled only): a hard-linked ProRes master, or else the draft remuxed with PCM. Clips carry TC `01:00:00:00`, and segments are named `12B_1-3`. Timelines are per Scene plus the whole film, as FCPXML (no markers) and `.otio` (markers), with burned-in captions and a `.srt`. Metadata goes in `takes.csv` (`Tag=1` for circled) and provenance in `manifest.json`. Dropped clips move to `_Removed/`. The proxy uses NVENC, falling back to OpenH264. - [External media view and cleanup](https://git.crowmaki.me/crowmaki/ai-studio/issues/22): a project-wide **Storage** view of everything that takes space, showing what uses each item and how much space removing it would really free (counted by hard-link count). Only media-folder Renditions and latents can be **purged** (the row stays, the files go, and it can't be undone). Circled Takes and latents with a pending job are blocked. In-file items can only be deleted when unused, and only Source sheets are deleted here. Storage can empty the Edit package's `_Removed/` folders. Suggestions only select items. Compact is a manual VACUUM. Each Rendition keeps a poster frame, so purged Takes still show. - [Marking Lines in a Performance](https://git.crowmaki.me/crowmaki/ai-studio/issues/23): a Performance is a **Bounce** of the Cut's audio (marks come free; Lines with no circled Take carry forward the old Performance or become silent holes) or imported audio (silencedetect auto-placement, then hand correction). Marks are a contiguous partition (pauses go to the earlier Line, action Lines share the gaps, a dual-dialogue group is one span). The audio is immutable, but marks are editable; Takes record their sent slice, so only Shots whose slice moved go stale, and a Take's Line marks are snapshotted at send. A Performance covers a set of Lines; Shots straddling covered and uncovered Lines block queuing. Marked in a Performance lane on the Animatic surface. ## Not yet specified _Nothing at the moment._ ## Out of scope Beyond the slice; returns only as a fresh effort: - Blender previz (playblast as a video reference) - MCP server and prompt templates - Lined script export - Blocking plan editor (the slice keeps only a UI placeholder), derived framing, board underlays - Board cleanup pass (ControlNet / edit model) - Continuity state - **Take extension** (seamless continuation with overlap): firmly wanted next, so the slice's provenance must not rule out chained generations - Script re-import with reconciliation - A render daemon that runs while the app is closed - In-app inpaint / edit-model reference editing - **Takes with internal cuts** (`[Shot 2]` blocks, intercuts): Shots stay aligned with real-world shots; intercuts are a separate effort. From [Prompt composer phrasing](https://git.crowmaki.me/crowmaki/ai-studio/issues/9). - SeedVR2 and NVIDIA upscaling as drop-in finishing plugins. From [Shipped workflow roster and slot contracts](https://git.crowmaki.me/crowmaki/ai-studio/issues/17). - Alternate voice sources for Performances (local TTS etc.); the Performance source edge list must allow them. From [Marking Lines in a Performance](https://git.crowmaki.me/crowmaki/ai-studio/issues/23). - Forced alignment / ASR auto-marking of imported Performances. From [Marking Lines in a Performance](https://git.crowmaki.me/crowmaki/ai-studio/issues/23). ## Tickets - [x] OSF import mapping #2 - [x] Rust core vs TypeScript frontend split #3 - [x] Default atlas layout #4 - [x] Import and crop a sheet into Tiles #5 - [x] Board paint tool and Notation #6 - [x] Dailies review and circling #7 - [x] Animatic duration rules #8 - [x] ComfyUI API surface #13 - [x] Reusing H3 AV latents across jobs #14 - [x] Local finishing pass for final Renditions #15 - [x] Resolve FCPXML import fidelity #16 - [x] Verify latent round trip and refine-pass cost #20 - [x] Test Resolve import of FCPXML and metadata CSV #21 - [x] Prompt composer phrasing #9 - [x] Reference slot budget #10 - [x] .studio schema #11 - [x] Staleness rules #12 - [x] Shipped workflow roster and slot contracts #17 - [x] Render worker scheduling #18 - [x] Edit package layout #19 - [x] External media view and cleanup #22 - [x] Marking Lines in a Performance #23

crowmaki added the

wayfinder:map

label 2026-10-06 03:55:23 +00:00

crowmaki referenced this issue 2026-10-06 03:55:51 +00:00

OSF import mapping #2

crowmaki referenced this issue 2026-10-06 03:55:51 +00:00

Rust core vs TypeScript frontend split #3

crowmaki referenced this issue 2026-10-06 03:55:51 +00:00

Default atlas layout #4

crowmaki referenced this issue 2026-10-06 03:55:52 +00:00

Import and crop a sheet into Tiles #5

crowmaki referenced this issue 2026-10-06 03:55:52 +00:00

Board paint tool and Notation #6

crowmaki referenced this issue 2026-10-06 03:55:52 +00:00

Dailies review and circling #7

crowmaki referenced this issue 2026-10-06 03:55:53 +00:00

Animatic duration rules #8

crowmaki referenced this issue 2026-10-06 03:55:53 +00:00

Prompt composer phrasing #9

crowmaki referenced this issue 2026-10-06 03:55:53 +00:00

Reference slot budget #10

crowmaki referenced this issue 2026-10-06 03:55:54 +00:00

.studio schema #11

crowmaki referenced this issue 2026-10-06 03:55:54 +00:00

Staleness rules #12

crowmaki referenced this issue 2026-10-06 03:55:54 +00:00

ComfyUI API surface #13

crowmaki referenced this issue 2026-10-06 03:55:55 +00:00

Reusing H3 AV latents across jobs #14

crowmaki referenced this issue 2026-10-06 03:55:55 +00:00

Local finishing pass for final Renditions #15

crowmaki referenced this issue 2026-10-06 03:55:55 +00:00

Resolve FCPXML import fidelity #16

crowmaki referenced this issue 2026-10-06 03:55:56 +00:00

Shipped workflow roster and slot contracts #17

crowmaki referenced this issue 2026-10-06 03:55:56 +00:00

Render worker scheduling #18

crowmaki referenced this issue 2026-10-06 03:55:56 +00:00

Edit package layout #19

crowmaki referenced this issue 2026-10-06 04:01:22 +00:00

Verify latent round trip and refine-pass cost #20

crowmaki referenced this issue 2026-10-06 04:07:07 +00:00

Test Resolve import of FCPXML and metadata CSV #21

crowmaki referenced this issue 2026-10-06 05:41:54 +00:00

External media view and cleanup #22

crowmaki referenced this issue 2026-10-06 07:32:31 +00:00

Marking Lines in a Performance #23

crowmaki referenced this issue 2026-10-07 07:24:15 +00:00

Spec: AI Studio end-to-end slice #24

crowmaki commented 2026-10-07 07:24:59 +00:00

Author

Owner

Copy link

Destination reached: the spec for the end-to-end slice is #24 (#24), labelled ready-for-agent. It combines all 22 closed child tickets, CONTEXT.md and ADRs 0001–0005. The next step is splitting it into build issues.

Destination reached: the spec for the end-to-end slice is #24 (https://git.crowmaki.me/crowmaki/ai-studio/issues/24), labelled ready-for-agent. It combines all 22 closed child tickets, CONTEXT.md and ADRs 0001–0005. The next step is splitting it into build issues.

crowmaki closed this issue 2026-10-07 07:24:59 +00:00

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

No results found.

Labels

Clear labels   

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

help wanted

Need some help

  

invalid

Something is wrong

  

needs-info

Waiting on reporter for more information

  

needs-triage

Maintainer needs to evaluate this issue

  

question

More information is needed

  

ready-for-agent

Fully specified, ready for an AFK agent

  

ready-for-human

Requires human implementation

  

wayfinder:grilling

Needs a decision talked through with a human

  

wayfinder:map

Index of decisions and open questions for a large effort

  

wayfinder:prototype

Needs a rough artifact built to react to

  

wayfinder:research

Needs facts gathered from docs or external sources

  

wayfinder:task

Manual work that must happen before a decision

  

wontfix

Will not be actioned

No labels

bug

duplicate

enhancement

help wanted

invalid

needs-info

needs-triage

question

ready-for-agent

ready-for-human

wayfinder:grilling

wayfinder:map

wayfinder:prototype

wayfinder:research

wayfinder:task

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

crowmaki/ai-studio#1

No description provided.