Verify latent round trip and refine-pass cost #20

New issue

Closed

opened 2026-10-06 04:01:22 +00:00 by crowmaki · 1 comment

crowmaki commented

2026-10-06 04:01:22 +00:00

Owner

Part of #1

Question

Does the split-latent round trip from "Reusing H3 AV latents across jobs" (#14) actually work in the owner's ComfyUI? Run the reference workflow once as a single job. Then run it split in two: a draft job (LTXVSeparateAVLatent → two SaveLatent), restart ComfyUI, and a Promotion job (two LoadLatent → MinimaxH3LatentUpscaler3D → LTXVConcatAVLatent → 2nd pass, conditioning rebuilt, audio noise mask re-applied). Compare the outputs. While it runs, record the 2nd pass's peak VRAM and runtime at the intended final size on the 5090, which "Local finishing pass for final Renditions" (#15) left unmeasured. See branch research/h3-latent-reuse for the recipe.

Part of #1 ## Question Does the split-latent round trip from "Reusing H3 AV latents across jobs" (#14) actually work in the owner's ComfyUI? Run the reference workflow once as a single job. Then run it split in two: a draft job (`LTXVSeparateAVLatent` → two `SaveLatent`), restart ComfyUI, and a Promotion job (two `LoadLatent` → `MinimaxH3LatentUpscaler3D` → `LTXVConcatAVLatent` → 2nd pass, conditioning rebuilt, audio noise mask re-applied). Compare the outputs. While it runs, record the 2nd pass's peak VRAM and runtime at the intended final size on the 5090, which "Local finishing pass for final Renditions" (#15) left unmeasured. See branch `research/h3-latent-reuse` for the recipe.

crowmaki added the

wayfinder:task

label

2026-10-06 04:01:22 +00:00

crowmaki referenced this issue

2026-10-06 04:01:22 +00:00

Shipped workflow roster and slot contracts #17

crowmaki referenced this issue

2026-10-06 04:01:22 +00:00

Wayfinder: AI Studio end-to-end slice spec #1

crowmaki self-assigned this

2026-10-06 07:52:07 +00:00

crowmaki referenced this issue from a commit

2026-10-06 09:11:03 +00:00

research: measure H3 latent round trip and refine-pass cost (#20)

crowmaki commented

2026-10-06 09:11:14 +00:00

Author

Owner

Resolution

Done. The split-latent round trip works, including across a ComfyUI restart. Run on 2026-10-06, ComfyUI 0.39.0, RTX 5090.

The Promotion output (2× LoadLatent "… [output]" → MinimaxH3LatentUpscaler3D → LTXVConcatAVLatent → 2nd pass, conditioning rebuilt) matches the single-job baseline to within GPU nondeterminism. Video PSNR baseline vs Promotion is 41.7 dB mean, about the same as Promotion vs Promotion from identical latents (42.2 dB).
SaveLatent on the raw AV latent fails with 'NestedTensor' object has no attribute 'contiguous', as predicted.
New fact: with generated audio (no mask), the 0.4-denoise 2nd pass re-noises the audio on every run (~15 dB SNR run to run). A Promotion therefore does not keep the draft's audio unless the loaded audio latent gets a 0-valued noise mask.
New fact: ResolutionSelector at 1.0 MP / 16:9 / ×32 gives a 1376×768 draft (not 1344). Megapixels 2.1 / align 32 gives a 1984×1120 final.

Cost at ~1080p (1984×1120): Promotion only takes 92.5 s cold or 83.5 s warm for 5 s, and 283 s for 15 s (362 frames). The single job (draft + finish) takes 167 s for 5 s, and the 15 s draft takes 278 s. Peak VRAM saturates at ~31.8/32.6 GB in every run with no OOM, so 15 s finals at ~1080p fit. Latents: 14.7 MB (5 s) and 42.4 MB (15 s).

Details, job graphs and the job-splitting script: branch research/h3-latent-reuse, research/h3-latent-reuse-run/README.md.

## Resolution **Done. The split-latent round trip works, including across a ComfyUI restart.** Run on 2026-10-06, ComfyUI 0.39.0, RTX 5090. - The Promotion output (2× `LoadLatent "… [output]"` → `MinimaxH3LatentUpscaler3D` → `LTXVConcatAVLatent` → 2nd pass, conditioning rebuilt) matches the single-job baseline to within GPU nondeterminism. Video PSNR baseline vs Promotion is 41.7 dB mean, about the same as Promotion vs Promotion from identical latents (42.2 dB). - `SaveLatent` on the raw AV latent fails with `'NestedTensor' object has no attribute 'contiguous'`, as predicted. - **New fact:** with generated audio (no mask), the 0.4-denoise 2nd pass re-noises the audio on every run (~15 dB SNR run to run). A Promotion therefore does **not** keep the draft's audio unless the loaded audio latent gets a 0-valued noise mask. - **New fact:** ResolutionSelector at 1.0 MP / 16:9 / ×32 gives a 1376×768 draft (not 1344). Megapixels 2.1 / align 32 gives a 1984×1120 final. **Cost at ~1080p (1984×1120):** Promotion only takes 92.5 s cold or 83.5 s warm for 5 s, and 283 s for 15 s (362 frames). The single job (draft + finish) takes 167 s for 5 s, and the 15 s draft takes 278 s. Peak VRAM saturates at ~31.8/32.6 GB in every run with no OOM, so 15 s finals at ~1080p fit. Latents: 14.7 MB (5 s) and 42.4 MB (15 s). Details, job graphs and the job-splitting script: branch `research/h3-latent-reuse`, `research/h3-latent-reuse-run/README.md`.

crowmaki closed this issue

2026-10-06 09:11:14 +00:00

crowmaki referenced this issue

2026-10-06 09:11:15 +00:00

Shipped workflow roster and slot contracts #17

crowmaki referenced this issue

2026-10-07 07:24:15 +00:00

Spec: AI Studio end-to-end slice #24

crowmaki referenced this issue

2026-10-07 07:34:43 +00:00

Shipped workflow templates and manifests #33

crowmaki referenced this issue