This page documents the implementation of the LLM Node in Dify's workflow engine. The LLM Node is responsible for orchestrating large language model invocations within workflows, including prompt construction with variable interpolation, integration with memory for conversational context, handling multimodal inputs such as images, and processing responses—including streamed outputs, structured data, and reasoning content extraction.
The discussion includes the architecture of the LLMNode class, the data flow during node execution, prompt formation, memory and knowledge context integration, model invocation, and response handling. For broader workflow and node factory context, see Workflow Definition and Execution Model (5.1) and Node Factory and Dependency Injection (5.2) For quota and model management, see Quota Management and Credit Pools (6.3)
The core implementation of the LLM Node is in the LLMNode class, defined in graphon.nodes.llm.node. It extends the base Node class and adds specialized logic to handle LLM prompts, memory, context, and invocation.
The LLMNodeType defines the node data payload including the model configuration, prompt templates, memory and context settings, and vision (e.g., image) configurations managed in the frontend and passed to the backend node.
LLMNode leverages:
ModelInstance to perform the actual model invocation.LLMFileSaver to manage file persistence for the node.VariablePool to access runtime variables bound to the execution context.PromptMessageSerializerProtocol.This class also handles features like streaming output events and structured output parsing.
Sources: web/app/components/workflow/nodes/llm/types.ts11-73 web/app/components/workflow/types.ts138-144 api/core/llm_generator/llm_generator.py155-165 api/tests/unit_tests/core/workflow/nodes/llm/test_node.py75-85
The LLM Node execution proceeds through a well-defined pipeline involving prompt preparation, context and memory fetching, model invocation, and response handling.
The _run() method of LLMNode orchestrates this. After formatting prompt templates by rendering Jinja2 variables and applying chat/completion mode logic, it loads vision files and retrieval context if enabled. The node then fetches any memory history via TokenBufferMemory. The LLM is invoked with these inputs and either streamed or awaited for the full result. Response chunks are parsed to separate text and reasoning content before building outputs for the workflow.
Sources: api/core/llm_generator/llm_generator.py162-170 api/tests/unit_tests/core/workflow/nodes/llm/test_node.py75-82 api/core/llm_generator/llm_generator.py29-34
Prompt construction is a key responsibility of the LLM Node, combining tiled templates, variable substitution, memory, context, and multimodal inputs.
For chat mode, the node builds a messages list including system messages, user and assistant messages with variable substitution. If memory is enabled, it includes conversation history. Vision-enabled nodes add image files as message content.
In completion (non-chat) mode, the prompt is constructed as a single user prompt by formatting the template.
| Variable Type | Source | Transformation |
|---|---|---|
{{input_var}} | User inputs or upstream outputs | Substituted via Jinja2 in prompt templates |
#context# | Knowledge retrieval output | Assembled from document segments for broader context |
#histories# | Conversation memory | Retrieved from TokenBufferMemory |
#files# | Vision files variable selector | Transformed to ImagePromptMessageContent instances |
These substitutions ensure the prompt is dynamically customized per invocation based on variable values and workflow state.
Sources: api/tests/unit_tests/core/workflow/nodes/llm/test_node.py78-82 api/core/llm_generator/llm_generator.py160-165 web/app/components/workflow/nodes/llm/panel.tsx156-171 web/app/components/workflow/nodes/_base/components/variable/utils.ts124-136
To handle conversational multi-turn contexts, the LLM Node integrates with a memory abstraction TokenBufferMemory which stores past prompt messages (user-assistant history).
The memory fetch typically occurs before LLM invocation to prepend the last N messages based on token or message count limits.
Sources: api/tests/unit_tests/core/workflow/nodes/llm/test_node.py93-102 api/tests/unit_tests/core/workflow/nodes/llm/test_node.py79-80
LLM nodes may enable vision support where image files are passed as prompt attachments.
The vision config references variables indicating files to embed alongside textual prompts as structured message content. This supports multimodal LLM models with image understanding.
Sources: api/tests/unit_tests/core/workflow/nodes/llm/test_node.py169-176 api/tests/unit_tests/core/workflow/nodes/llm/test_node.py33-38 web/app/components/workflow/nodes/llm/panel.tsx54-55
After prompt construction, the node invokes the model via a ModelInstance (core.model_manager.ModelInstance) which abstracts provider interactions.
The node supports both streaming and blocking invocation modes, optionally parsing structured output formats when enabled. Models that support reasoning content extraction are handled here as well.
Sources: api/tests/unit_tests/core/workflow/nodes/llm/test_node.py24-30 web/app/components/workflow/nodes/llm/panel.tsx59-60 api/core/llm_generator/llm_generator.py162-165
Response chunks from streaming calls are iterated and parsed for text and reasoning events, which the node yields to the workflow runtime.
The node separates reasoning tags or content from plain text, providing structured events to enable advanced downstream use cases.
Sources: api/tests/unit_tests/core/workflow/nodes/llm/test_node.py50-55 api/tests/unit_tests/core/workflow/nodes/llm/test_node.py84-85 api/core/llm_generator/llm_generator.py162-166
The node produces standardized output variables accessible to downstream nodes and workflow logic:
| Output Variable | Type | Description |
|---|---|---|
text | String | Clean text output content |
reasoning_content | String | Extracted reasoning or thinking logs |
usage | Object | LLM token usage and cost metadata |
These outputs are declared in constants used throughout the workflow system.
Sources: web/app/components/workflow/constants.ts142-155 api/tests/unit_tests/core/workflow/nodes/llm/test_node.py24-30
The frontend UI for configuring the LLM node is implemented in React. It uses the useConfig hook layered within the node panel component for managing model selection, prompt editing, memory, context variables, and vision options.
The panel supports editing prompt templates with variables (Jinja2), selecting multimodal file inputs, adjusting model parameters, and viewing output variables.
Sources: web/app/components/workflow/nodes/llm/panel.tsx27-63 web/app/components/workflow/nodes/llm/panel.tsx116-130 web/app/components/workflow/nodes/llm/panel.tsx157-171
The LLM Node implementation in Dify integrates tightly with workflow execution infrastructure to provide:
This architecture enables powerful LLM-augmented workflows with fine control and observability over model input/output.
Sources:
web/app/components/workflow/nodes/llm/types.ts11-73
web/app/components/workflow/types.ts138-144
api/core/llm_generator/llm_generator.py29-170
api/tests/unit_tests/core/workflow/nodes/llm/test_node.py24-176
web/app/components/workflow/nodes/llm/panel.tsx27-171
web/app/components/workflow/nodes/_base/components/variable/utils.ts124-136
api/core/workflow/node_factory.py75-143
Refresh this wiki
This wiki was recently refreshed. Please wait 6 days to refresh again.