This document provides a comprehensive technical overview of the Chat and Completion APIs within the Dify platform. These APIs constitute the two primary endpoints for sending user messages and receiving AI-generated responses, covering session management, request processing, and response structures. The document focuses on the implementation details, data flow, key classes/functions, and system integration points for the POST /chat-messages and /completion-messages endpoints.
The Chat and Completion APIs serve distinct conversational and text-generation needs within Dify:
POST /chat-messages): Designed for conversation-based applications requiring session persistence through conversation identifiers. Supports chat app modes, including agent and advanced chat modes with conversation history and tool integrations.POST /completion-messages): Supports stateless text generation scenarios such as translation, summarization, and content creation without session context.Both APIs support user authentication and permissions under multi-tenant tenancy using RBAC models. Execution can be synchronous (blocking) or asynchronous (streaming), with configurable response modes.
| Feature | Chat API | Completion API |
|---|---|---|
| Endpoint | /chat-messages | /completion-messages |
| Session Persistence | Yes (conversation_id) | No |
| Supported App Modes | CHAT, AGENT_CHAT, ADVANCED_CHAT, AGENT | COMPLETION |
| Auto-Conversation Title Generation | Supported | Not applicable |
Sources: api/controllers/service_api/app/message.py68-69 api/controllers/service_api/app/completion.py46-47 api/services/conversation_service.py130-142
Messages sent by clients traverse through layered namespaces, controllers, app generators, task pipelines, and back into database models. The flow below connects the user's natural language input to the underlying Dify code entities handling the request.
This flow emphasizes how user input reaches the Flask-RestX namespace service_api_ns, which routes to respective controllers for chat, completion, or workflow apps. These controllers delegate message generation to app generator classes, which utilize task pipeline classes to manage asynchronous execution. The task pipelines interact directly with the persistent database models.
Sources:
All APIs accept requests with app-level API tokens and optional user context tokens. Token validation occurs via the @validate_app_token decorator in the service API. This decorator fetches the App and EndUser models and enforces authentication before controller methods proceed.
Multi-tenant role-based access control (RBAC) is enforced, with decorators such as @rbac_permission_required controlling which users may invoke specific app endpoints. User context (console user or end user) also affects conversation ownership and filtering.
Sources:
The Chat API is implemented as a Flask-RestX resource in ChatMessageApi in the service API controller, exposed at POST /chat-messages. Its implementation includes:
ChatMessagePayload to validate query, inputs, conversation_id, parent_message_id, and response_mode fields. UUID fields are normalized and checked. api/controllers/console/app/completion.py79-115CHAT, AGENT_CHAT, ADVANCED_CHAT, and AGENT.blocking and streaming. Agent apps restrict to streaming only.AppGenerateService.generate to start message generation logic passing all validated parameters, handling exceptions related to LLM providers, quota, and model support.| Parameter | Type | Required | Description |
|---|---|---|---|
query | string | Yes | User input text of the question or prompt |
inputs | dict | No | Variables input defined by app prompt schema |
response_mode | enum | No | blocking or streaming, defaults to blocking |
conversation_id | UUID string | No | Identifies conversation session for context persistence |
parent_message_id | UUID string | No | For referencing a parent message in threaded chats |
files | list | No | Uploaded file inputs for multimodal data |
Sources: api/controllers/service_api/app/message.py101-115 api/controllers/console/app/completion.py79-115
The Completion API, exposed at POST /completion-messages through CompletionMessageApi controller, supports stateless text generation with:
CompletionMessagePayload.COMPLETION.blocking and streaming response modes.AppGenerateService.generate) as chat but without conversation context or session persistence.Sources: api/controllers/console/app/completion.py97-165 api/controllers/service_api/app/completion.py148-172
For chat apps, session management is centered on the Conversation entity, which captures metadata about a chat session. The ConversationService offers pagination, retrieval, and renaming APIs.
app_id, user ownership (Account or EndUser), and soft-deleted flag.last_id) supports infinite scroll UI patterns.updated_at timestamps to enable sorting and filtering.LLMGenerator.generate_conversation_name.| Operation | Endpoint | Service Call |
|---|---|---|
| List Conversations | GET /conversations | ConversationService.pagination_by_last_id |
| Rename Conversation | POST /conversations/{conversation_id}/name | ConversationService.rename or auto_generate_name |
Sources: api/controllers/service_api/app/conversation.py157-185 api/services/conversation_service.py33-147
This diagram outlines how API controllers interface with core services and ultimately operate on persistent database models for conversations and messages. The MessageCycleManager assists in generating conversation titles.
Sources: api/controllers/service_api/app/conversation.py156-157 api/services/conversation_service.py33-34 api/services/conversation_service.py166-168 api/core/app/task_pipeline/message_cycle_manager.py38-40
Dify abstracts core LLM calling and response handling using pipeline classes:
The pipeline choice depends on the app mode and API endpoint triggered.
Sources: api/core/app/task_pipeline/easy_ui_based_generate_task_pipeline.py131-140 api/core/app/apps/advanced_chat/generate_task_pipeline.py140-150 api/core/app/apps/workflow/generate_task_pipeline.py70-90
Dify allows users and admins to provide feedback and annotation (standard Q&A) data to improve app responses.
User Feedback: Via the endpoint POST /messages/{message_id}/feedbacks, users can submit like or dislike ratings and optional comments on individual messages. The MessageService.create_feedback method manages persistence. api/controllers/service_api/app/message.py158-178
Annotations: Admins configure "annotation reply" features to provide standard answers, backed by embedding-based similarity search. The AppAnnotationService is used to enable or disable annotation reply with async job management. Endpoints include POST /apps/annotation-reply/<action> and job status querying. api/controllers/service_api/app/annotation.py79-182 api/services/annotation_service.py90-200
Sources: api/controllers/service_api/app/message.py154-155 api/controllers/service_api/app/annotation.py79-182 api/services/annotation_service.py90-200
Dify supports audio-related services:
Audio services run alongside LLM generation pipelines and can return audio chunks in streaming responses.
Sources: api/services/audio_service.py44-93 api/core/app/apps/advanced_chat/generate_task_pipeline.py70-80
The Chat and Completion APIs use Pydantic models for request validation and response serialization. Errors map to standard HTTP statuses:
| HTTP Status | Description |
|---|---|
| 200 | Successful message generation and delivery |
| 400 | Bad request due to invalid parameters or mode mismatches |
| 401 | Unauthorized: invalid or missing API token |
| 403 | Forbidden: insufficient permissions |
| 404 | Not found: app, conversation, or message not found |
| 429 | Too many requests / rate limit exceeded |
Common exceptions such as ConversationNotExistsError, MessageNotExistsError, and InvokeRateLimitError are caught and converted to appropriate HTTP responses in controllers.
Sources: api/controllers/service_api/app/message.py122-130 api/controllers/service_api/app/workflow.py39 api/controllers/service_api/app/annotation.py167-168
The Chat and Completion APIs provide flexible, secure endpoints for external clients to interact with Dify-powered conversational AI and text generation applications. The APIs integrate with internal app generators and task pipelines that orchestrate conversation state, LLM calls, and async streaming responses. Multi-tenant RBAC ensures secure access, while async job mechanisms manage complex features like annotation replies. Session management enables continuous conversational contexts, while completion API usage supports stateless scenarios.
This page detailed the request and response flows, key classes such as ChatMessageApi, CompletionMessageApi, and ConversationService, and the associated database models involved. The layered architecture facilitates extensibility and robust handling of advanced chat and agent app modes.
Refresh this wiki
This wiki was recently refreshed. Please wait 6 days to refresh again.