The Workflow Testing and Mock System in Dify provides a comprehensive framework aimed at validating workflow execution in isolation from real external services such as LLM providers or HTTP endpoints. This is achieved by injecting mock implementations of workflow nodes, enabling deterministic and fast tests. The system further supports flexible table-driven tests for different node types by configuring input scenarios and expected behaviors.
This page discusses the design and implementation of the mock system, the factory patterns for node instantiation, table-driven testing strategies, and integration test workflows.
At the core of the system is the use of a customized Node Factory that enables substituting real node classes with mock variants. This allows unit tests and integration tests to simulate node behaviors without calling live LLMs, external APIs, or other side effects.
The main production factory for workflow nodes is the DifyNodeFactory class. It is responsible for instantiating workflow node classes according to node type and version declarations in the workflow graph api/core/workflow/node_factory.py11-18
The factory references a centralized node registry powered by the base Node class from the graphon library, which maps each workflow node type and version to a Python class.
For testing, special mock factories leverage Python's unittest.mock.MagicMock or custom lightweight mocks to replace components such as model instances inside LLM nodes, or HTTP clients for HTTP request nodes.
The workflow engine obtains node instances from the factory. When using a mock factory, these nodes return controlled fixed data or simulated results as configured by the test setup.
The test environment orchestrates these substitutions by configuring the GraphInitParams and GraphRuntimeState with mock data and injecting mock dependencies.
The DifyGraphInitContext dataclass represents explicit graph initialization parameters owned by the workflow system, encapsulating the workflow id, graph configuration, run context values, and call depth api/core/workflow/node_factory.py79-102
| Component | Role | Source |
|---|---|---|
DifyNodeFactory | Factory class for instantiating workflow node classes by type/version, supports mocking. | api/core/workflow/node_factory.py11-18 |
DifyGraphInitContext | Data container for explicit workflow graph initialization context used in node factory. | api/core/workflow/node_factory.py79-102 |
VariablePool | Maintains variable context during mock execution, supporting variable injection and retrieval. | api/graphon/runtime.py41-42 |
LLMNode (mocked) | LLMNode's internal ModelInstance is replaced with a MagicMock to simulate LLM responses. | api/tests/unit_tests/core/workflow/nodes/llm/test_node.py210-215 |
Executor | Executes HTTP requests, can be mocked to generate logs without real network calls. | api/tests/unit_tests/core/workflow/nodes/http_request/test_http_request_executor.py59-66 |
This diagram relates core production classes involved in workflow execution to their corresponding test/mock components for LLM node mocking.
Sources: api/core/workflow/workflow_entry.py131-140 api/tests/unit_tests/core/workflow/nodes/llm/test_node.py105-120 api/tests/integration_tests/workflow/nodes/test_llm.py137-172
Dify uses parameterized testing extensively to validate multiple scenarios of node execution using pytest.mark.parametrize. This allows running the same test logic on different node configuration data and expected outcomes, greatly increasing test coverage with less duplication.
Defined in integration tests under tests/integration_tests/workflow/nodes/test_http.py.
The helper function init_http_node constructs a minimal graph configuration including a start node and the HTTP request node configured from test parameters api/tests/integration_tests/workflow/nodes/test_http.py34-44
A VariablePool is initialized with system variables simulating user/environment variables needed for the node.
Different authorization modes are tested (no-auth, api-key, custom), ensuring the HTTP client executor serializes headers correctly and constructs request data per configuration api/tests/integration_tests/workflow/nodes/test_http.py92-157
Raw HTTP request strings are inspected with Executor.to_log() to verify headers, query params, and JSON body with variable interpolation api/tests/unit_tests/core/workflow/nodes/http_request/test_http_request_executor.py79-84
The tests mock the ModelInstance.invoke_llm call to return a controlled LLMResult object, allowing assertion of prompt correctness and output handling api/tests/integration_tests/workflow/nodes/test_llm.py166-172
Fixtures build a complete node instance with mock dependencies, including mock prompt message serializer and file savers api/tests/integration_tests/workflow/nodes/test_llm.py28-91
Tests verify that prompt templates interpolate variables from the VariablePool and that usage metadata (token counts, prices) is reported correctly after invocation api/tests/integration_tests/workflow/nodes/test_llm.py151-164
Code nodes are tested using mocks for the code_executor to emulate sandbox execution of Python or Node.js code snippets without real execution api/tests/integration_tests/workflow/nodes/test_code.py67-83
The mocks verify proper injection of input variables and validate outputs against configured response schema api/tests/integration_tests/workflow/nodes/test_code.py136-178
Sources: api/tests/integration_tests/workflow/nodes/test_http.py34-157 api/tests/integration_tests/workflow/nodes/test_llm.py28-172 api/tests/integration_tests/workflow/nodes/test_code.py24-178
Beyond unit- and node-level testing, Dify implements integration tests that validate end-to-end workflow execution with background tasks and database interactions.
Celery tasks handle async workflow runs with queue separation for different subscription levels (Professional, Team, Sandbox) api/tasks/async_workflow_tasks.py53-85
The tasks fetch workflow records and instantiate a WorkflowAppGenerator that runs the workflow graph via the GraphEngine api/tasks/async_workflow_tasks.py130-151
Execution results and node-level traces are persisted via SQLAlchemy ORM to the database.
Sources: api/tasks/async_workflow_tasks.py110-151 api/core/workflow/workflow_entry.py163-195 api/tasks/app_generate/workflow_execute_task.py153-179
init_llm_node(config: dict) -> LLMNodeConstructs a testable LLMNode instance by setting up a minimal graph with a start node and the LLM node under test.
Initializes a VariablePool pre-populated with system and user variables for use in prompt templates.
Injects mocked dependencies like CredentialsProvider, ModelFactory, and ModelInstance so no real LLM calls are made api/tests/integration_tests/workflow/nodes/test_llm.py28-91
init_http_node(config: dict) -> HttpRequestNodeCreates an HTTP request node inside a minimal test graph configuration.
Sets up GraphInitParams and a VariablePool with system variables for request template rendering.
Uses mocks or direct HTTP client instances for executing request logic in isolation, enabling inspection of generated requests without external calls api/tests/integration_tests/workflow/nodes/test_http.py34-89
init_code_node(code_config: dict) -> CodeNodeInitializes a CodeNode instance equipped with a mocked Python or Node.js code executor.
Provides limits and schema validation to ensure test control over code execution simulation.
Supports assertions on input variable injection and output processing without sandbox overhead api/tests/integration_tests/workflow/nodes/test_code.py24-85
| Helper Function | Usage | Source |
|---|---|---|
init_llm_node | Creates an LLM node instance with mocks for unit and integration tests | api/tests/integration_tests/workflow/nodes/test_llm.py28-91 |
init_http_node | Creates an HTTP request node setup for mock HTTP call testing | api/tests/integration_tests/workflow/nodes/test_http.py34-89 |
init_code_node | Creates a Code node with mocked executor for code execution tests | api/tests/integration_tests/workflow/nodes/test_code.py24-85 |
Sources: api/tests/integration_tests/workflow/nodes/test_llm.py28-91 api/tests/integration_tests/workflow/nodes/test_http.py34-89 api/tests/integration_tests/workflow/nodes/test_code.py24-85
The Dify workflow testing architecture centers on flexible mock factories and table-driven test cases. The DifyNodeFactory pattern facilitates replacing real node runtime components with mocks to simulate node execution reliably and repeatably in unit and integration tests. This infrastructure supports end-to-end workflow validation with real persistence layers and asynchronous task execution while isolating complex external interactions to mocks. Table-driven tests verify node behavior across diverse input cases to improve code quality and maintainability in this graph-driven AI platform.
This approach enables developers to confidently build, test, and iterate on complex workflows without costly dependencies or brittle external integrations.
Sources:
Refresh this wiki
This wiki was recently refreshed. Please wait 6 days to refresh again.