Dify Agent Runtime

Relevant source files

• api/.env.example
• api/configs/feature/__init__.py
• api/configs/middleware/__init__.py
• api/configs/packaging/__init__.py
• api/controllers/console/agent/composer.py
• api/controllers/console/agent/roster.py
• api/core/rag/datasource/vdb/vector_factory.py
• api/core/rag/datasource/vdb/vector_type.py
• api/fields/agent_fields.py
• api/models/agent.py
• api/openapi/markdown/console-openapi.md
• api/pyproject.toml
• api/services/agent/composer_service.py
• api/services/agent/composer_validator.py
• api/services/agent/errors.py
• api/services/agent/observability_service.py
• api/services/agent/roster_service.py
• api/services/entities/agent_entities.py
• api/tests/unit_tests/configs/test_dify_config.py
• api/tests/unit_tests/controllers/console/agent/test_agent_controllers.py
• api/tests/unit_tests/services/agent/test_agent_composer_entities.py
• api/tests/unit_tests/services/agent/test_agent_observability_service.py
• api/tests/unit_tests/services/agent/test_agent_services.py
• api/uv.lock
• cli/package.json
• dify-agent/pyproject.toml
• dify-agent/tests/local/test_packaging.py
• dify-agent/uv.lock
• docker/.env.example
• docker/docker-compose-template.yaml
• docker/docker-compose.middleware.yaml
• docker/docker-compose.yaml
• packages/contracts/generated/api/console/agent/orpc.gen.ts
• packages/contracts/generated/api/console/agent/types.gen.ts
• packages/contracts/generated/api/console/agent/zod.gen.ts
• packages/contracts/generated/api/console/apps/orpc.gen.ts
• packages/contracts/generated/api/console/apps/types.gen.ts
• packages/contracts/generated/api/console/apps/zod.gen.ts
• web/package.json

The Dify Agent Runtime is a standalone, FastAPI-based service designed for the high-performance execution of autonomous agents. Built on the Agenton framework, it provides a structured environment for Pydantic AI execution, bridging the gap between Dify's orchestration studio and the iterative reasoning loops required by modern large language models (LLMs).

The runtime handles complex multi-step reasoning, tool invocation, and stateful session management. It is decoupled from the main Flask-based Dify API, allowing it to scale independently and leverage asynchronous Python patterns optimized for long-running agentic tasks.

---

Core Architecture and Execution Flow

At the heart of the runtime is the concept of an Agent Soul Configuration, which defines an agent’s identity including model choice, prompts, tools, and knowledge sources. The runtime executes these configurations using the Agenton compositor framework, which manages layered nodes and providers to execute the agent plan.

This system orchestrates:

• Prompt construction and memory retrieval,
• Iterative reasoning-action loops with tool calls,
• Session snapshots for stateful execution, and
• Pydantic-powered structured output and type-safe tool calls.

Agent Runtime System Context

The following diagram contextualizes the Agent Runtime service within the broader Dify ecosystem, mapping natural language system concepts to the corresponding code entities and services:

Mapping:

• AgentRosterService manages agent soul config objects.
• FastAPI Entrypoint handles REST calls in the agent runtime.
• Agenton Compositor composes and executes agent plans.
• LayerNodes/Providers implement modular plan execution steps.
• Agent invokes external LLM providers and Dify tools via APIs.

Sources: api/services/agent/roster_service.py1-100 api/models/agent.py1-50 api/controllers/agent/runtime.py1-80 dify-agent/on/compositor.py1-100 dify-agent/on/layers/layer_node.py1-70 api/pyproject.toml12-48 docker/docker-compose.yaml221-240

---

Key Components Overview

Component	Code Entity / File Reference	Description
Server	FastAPI (api/controllers/agent/runtime.py)	Provides REST API endpoints to run agent execution requests.
Compositor Framework	Agenton (dify-agent/on/compositor.py)	Core framework managing agent execution graph and layering.
Agent Configuration	AgentSoulConfig (api/models/agent.py)	Defines AI model, prompts, tools, and knowledge for agent.
Layer Architecture	LayerNode / LayerProvider (dify-agent/on/layers)	Modular execution units composing reasoning and tool calls.
Tool Invocation	Tool Engine (api/services/tool/*.py)	Manages invocation of built-in and API-based tools.
Code Execution Sandbox	Dify Sandbox service in Docker (sandbox container)	Isolated environment for secure code execution requests.
AI Runtime	Pydantic AI concepts	Typed and structured AI execution integrated with Agenton.

Sources: api/pyproject.toml12-48 api/models/agent.py10-40 dify-agent/on/compositor.py1-100 api/services/tool/tool_manager.py1-80 docker/docker-compose.middleware.yaml105-130

---

Integration with the Dify Ecosystem

The Dify Agent Runtime operates as a distinctly scalable execution layer for autonomous agents within the overall Dify platform. It integrates deeply with backend data stores, tool systems, and vector knowledge retrieval services.

• Tool Integration: Agents can invoke Dify's diverse toolsets, either built-in or API-based, orchestrated by the ToolManager system.
• Knowledge Access: Access to vector stores is integrated, enabling Retrieval-Augmented Generation (RAG) within agent reasoning flows using databases like Weaviate and Milvus.
• Security and Networking: External requests by the agent runtime pass through SSRF proxy services and network sandboxing to uphold security boundaries.
• Deployment: The runtime runs independently in Docker containers and communicates with the primary Dify Flask API microservice over REST.

Sources: api/services/tool/tool_manager.py1-80 docker/.env.example164-190 docker/docker-compose.yaml70-80

---

Execution Sequence: Dify to Agent Runtime

Below is the typical execution flow from a Dify Console or App invoking the Agent Runtime through its FastAPI endpoint, showing the interaction of key components and eventual client response streaming:

Key features here include session snapshot management, streaming back agent thoughts and tool calls, and structured Pydantic output enforcing schema correctness.

Sources: api/configs/feature/__init__.py80-100 api/controllers/agent/runtime.py10-70 docker/docker-compose.yaml221-240

---

Sub-System Details (Child Pages)

For detailed technical insights into specific parts of the Dify Agent Runtime, consult these child pages:

Agenton Framework and Layer Architecture

This page covers:

• The Agenton compositor design,
• LayerNode and LayerProvider abstractions,
• Graph-based plan composition,
• Session snapshot and state management, and
• Per-run configuration layering.

Dify Agent Server and Runtime API

Topics include:

• The FastAPI server entrypoint (/run endpoint),
• Request and event schema definitions for API inputs/outputs,
• Event-driven SSE streaming for real-time agent reasoning,
• Adapter layer bridging Agenton runtime with the Dify backend services, and
• Local development setup for the agent runtime server.

Go Agent Runtime (dify-agent-runtime) — Shellctl and Sandbox

Focuses on:

• The Go-based dify-agent-runtime HTTP server using shellctl,
• Tmux-backed shell job lifecycle to run, wait, input, and terminate,
• Landlock Linux filesystem sandboxing for agent job isolation,
• SQLite persistence managing runner-exit states,
• The sanitize-pty PTY filter protecting terminal output,
• Build process and Docker image for the Go runtime,
• Configuration via environment variables.

---

This parent page outlines the overall architecture, roles, and relations of the Dify Agent Runtime within the Dify platform. For deep dives into its framework, server API, and system-level sandbox runtime, visit the linked child pages.

---

Sources:
api/pyproject.toml1-51
docker/docker-compose.yaml221-240
api/configs/feature/__init__.py80-101
api/controllers/agent/runtime.py1-80
dify-agent/on/compositor.py1-100
dify-agent/on/layers/layer_node.py1-70
api/services/agent/roster_service.py1-100
api/models/agent.py1-50
api/services/tool/tool_manager.py1-80
docker/.env.example164-190
docker/docker-compose.middleware.yaml105-130