This document provides a comprehensive overview of ComfyUI, a node-based visual AI engine for Stable Diffusion and related generative AI models. It covers the system's purpose, key features, and high-level architecture, serving as an entry point for understanding the codebase.
For installation instructions, see Installation and Setup. For a hands-on introduction, see Quick Start Guide. For detailed architectural information, see Core Architecture.
ComfyUI is the AI creation engine for visual professionals who demand control over every model, every parameter, and every output. It provides a node/flowchart interface for designing AI workflows—including image, video, 3D, and audio generation—without coding. The system runs locally on Windows, Linux, and macOS, supporting NVIDIA, AMD, Intel, Apple Silicon, and Ascend accelerators. README.md3-43 README.md68-104
Core Architecture: The system consists of a Python backend (server.py1-60 execution.py1-48) with a web-based frontend. Workflows are represented as directed acyclic graphs (DAGs) where nodes perform operations on data flowing through connections.
Sources: README.md3-43 README.md68-104 server.py1-60 execution.py1-48
ComfyUI implements foundational design principles focused on efficiency and modularity:
| Principle | Implementation | Key Code |
|---|---|---|
| Node-Based Composition | All operations are nodes in a DAG. Graph traversal uses topological sorting. | DynamicPrompt in execution.py34-39 |
| Incremental Execution | Only nodes with changed inputs re-execute. | IsChangedCache in execution.py58-64 |
| Caching System | Outputs are cached by input signature hash to prevent redundant compute. | CacheSet in execution.py114-130 |
| Hardware Flexibility | Smart offloading allows large models to run on low-VRAM hardware. | VRAMState in comfy/model_management.py43-50 |
Layered Design: High-level orchestration in execution.py calls into specialized subsystems for model loading, sampling, and device management in the comfy/ package.
Sources: execution.py34-130 comfy/model_management.py43-50 README.md105-117
ComfyUI natively supports the latest open-source state-of-the-art models. The system automatically detects architectures and configures the appropriate sampling logic.
Model Loading and Execution Pipeline
Analysis: The pipeline demonstrates sophisticated model handling—wrapping components in ModelPatcher for dynamic modification and managing VRAM states (comfy/model_management.py43-50) to support hardware from high-end GPUs to shared memory systems.
Supported Model Families
| Family | Examples | Features |
|---|---|---|
| Image Models | SD1.x, SDXL, SD3.5, Flux, AuraFlow, Hunyuan | EPS, V_PREDICTION, Flow Matching |
| Video Models | SVD, Mochi, LTX-Video, Hunyuan Video, Wan | Temporal Attention, Causal VAE |
| Audio Models | Stable Audio, ACE Step | Latent Audio Diffusion |
| 3D Models | Hunyuan3D 2.0 | Mesh and Geometry generation |
Sources: README.md71-104 comfy/model_management.py43-50 execution.py114-148
| Feature | Description | Implementation |
|---|---|---|
| Asynchronous Queue | Multiple prompts can be queued and executed sequentially | PromptQueue in execution.py |
| Memory Management | Automatic offloading and VRAM optimization | comfy/model_management.py193-212 |
| Model Patching | Dynamic injection of LoRAs and weight modifications | ModelPatcher (referenced in comfy/model_management.py40) |
| Workflow Serialization | Full workflows embedded in generated PNG/WebP/FLAC | README.md114-115 |
Sources: README.md105-117 comfy/model_management.py40-212 execution.py11-30
ComfyUI's architecture consists of five main layers: User Interfaces, Core Orchestration, Node System, Execution Engine, and Model Management.
Overall System Architecture and Component Relationships
Layer Responsibilities:
| Layer | Key Components | Primary Functions |
|---|---|---|
| User Interfaces | PromptServer, HTTP routes | Receives workflow JSON, serves web UI, manages WebSocket events |
| Core Orchestration | ModelFileManager, UserManager | Manages user profiles, assets, and model file organization |
| Execution Engine | DynamicPrompt, CacheSet | Builds dependency graphs, validates inputs, and manages memoization |
| Model Management | model_management.py, folder_paths.py | Handles VRAM allocation, device selection, and model path resolution |
Sources: server.py53-60 execution.py34-148 comfy/model_management.py43-50 folder_paths.py12-45
ComfyUI/
├── app/ # Application logic (Assets, Models, Users)
├── comfy/ # Core AI library (Sampling, Management, Ops)
│ ├── model_management.py # Device and VRAM orchestration
│ ├── cli_args.py # Global configuration and flags
│ └── options.py # Argument parsing entry
├── comfy_execution/ # Execution logic (Graph, Caching, Validation)
├── api_server/ # Internal API routes and protocol definitions
├── folder_paths.py # Path resolution for models and inputs
├── server.py # Primary HTTP/WebSocket server implementation
├── execution.py # Graph execution engine entry
└── main.py # System entry point and environment setup
Sources: server.py1-60 main.py1-35 folder_paths.py1-70 pyproject.toml1-10
To begin using ComfyUI:
Installation: See Installation and Setup for instructions on desktop application, portable packages, and manual installation for various GPU backends.
First Workflow: Follow the Quick Start Guide to understand node connections and load your first checkpoints.
Architecture Deep Dive: Explore the Core Architecture for detailed information on the server and execution engine.
Sources: README.md45-66