ComfyUI Video Generation#

Automate AI video generation using ComfyUI + LTX-2.3 model. Ideal for music video (MV) production, multi-scene batch rendering, and AI video content creation.

Requirements#

ItemSpecGPU≥24GB VRAM (Turing/Ampere/Ada)ComfyUI0.17+PyTorch2.6+cu124AccessSSH tunnel forwarding port 18188

Model Setup#

ModelSizePathLTX-2.3 dev (bf16)43GBmodels/checkpoints/ltx-2.3-22b-dev.safetensorsGemma 3 12B23GBmodels/text_encoders/comfy_gemma_3_12B_it.safetensorsDistilled LoRA7.1GBmodels/loras/ltxv/ltx2/ltx-2.3-22b-distilled-lora-384.safetensorsVideo VAE (bf16)-models/vae/LTX23_video_vae_bf16.safetensors

Turing GPUs (e.g., Quadro RTX 8000) do NOT support fp8_e4m3fn. Use bf16/fp16 models only.

Performance Baseline#

text

Per-step time: ~221s (constant, regardless of frame count!)
15 steps: ~57 min
25 steps: ~1h45m
Frames: 72=3s, 121=5s, 480=20s (24fps)

Key insight: Frame count does NOT affect total time. Bottleneck is model forward pass.

Workflow Node Reference#

NodeIDPurposeLoadImage2004I2V reference inputCLIPTextEncode (positive)2483Positive promptCLIPTextEncode (negative)2612Negative promptEmptyLTXVLatentVideo3059Empty latentLTXVScheduler4966Steps/length paramsLoraLoaderModelOnly4922+LoRA loaderSaveVideo4823/4852Output mp4

Quick Start#

Generate a Single Video (I2V)#

1. Load workflow: /workspace/ComfyUI/custom_nodes/ComfyUI-LTXVideo/example_workflows/2.3/LTX-2.3_T2V_I2V_Single_Stage_Distilled_Full.json
2. Set params using scripts/batch_scenes.js
3. Click Run
4. Wait ~1 hour
5. Download from /workspace/ComfyUI/output/

Batch Scene Generation#

Use scripts/batch_scenes.js for automation:

javascript

// Load script first, then configure each scene:
await comfyui_batch.configureScene({
  name: "scene_01",
  prompt: "A lonely girl running through rain at night, neon reflections",
  image: "unified_ref.png",
  steps: 15,
  frames: 72
});
// Click Run, repeat for next scene

Step Count Guide#

StepsQualityTime/SceneUse Case8Rough30minQuick preview15Good57minRecommended sweet spot25Best~1h45mFinal quality output

I2V + LoRA at 15 steps achieves ~90% of 25-step quality with 40% less time.

Troubleshooting#

VAEDecode Validation Failed#

Error: Exception when validating node: 'VAEDecode' Cause: VAE load timing or insufficient VRAM Fix: Reload the entire workflow (fetch + loadGraphData), wait for models to fully load, then run. Never reload during execution.

Browser Tab Lost#

Cause: SSH tunnel disconnected Fix:

1. Rebuild tunnel: ssh -f -N -L 18188:localhost:18188 user@host -p port
2. Navigate to ComfyUI
3. Reload workflow

Inconsistent Characters Across Scenes#

Cause: Different reference images per scene Fix: Use the SAME reference image for all scenes. Extract a clear frame from an existing video if needed. The I2V input image dictates the visual baseline.

Output Video Not Saved#

Check: ssh -p PORT root@HOST "ls -lht /workspace/ComfyUI/output/*.mp4" Fix: Check for VAEDecode errors in log, then re-run.

Monitoring Progress#

bash

# Current sampling progress
ssh -p PORT root@HOST "grep 'it/s' /tmp/comfy.log | tail -1"

# Completion check
ssh -p PORT root@HOST "grep 'Prompt executed' /tmp/comfy.log | tail -1"

# Output files
ssh -p PORT root@HOST "ls -lht /workspace/ComfyUI/output/*.mp4"

Best Practices#

1. 15 steps is the sweet spot — I2V converges at 15-20 steps, 25 has diminishing returns
2. Unified reference image — Same input image for all scenes ensures character consistency
3. Reload workflow every time — Avoids VAEDecode validation failures
4. Never reload during execution — Current run will fail
5. Frame selection — 72 frames (3s) for testing, 480 frames (20s) for final output
6. VRAM management — Wait for each generation to complete before starting next

T2V vs I2V Comparison#

ModeStepsQualityNotesT2V (no LoRA)15❌ Very blurryNot recommendedI2V + LoRA25✅ ExcellentMajor quality improvementI2V + LoRA15✅ Very goodBest time/quality ratio

Conclusion: I2V + LoRA is the recommended combination.

Resources#

• scripts/batch_scenes.js — Batch scene automation
• references/workflow_nodes.md — Full node ID mapping
• references/tips.md — Prompt tips, VRAM management, optimization