Testing Infrastructure

Relevant source files

• .github/workflows/sep-tests.yml
• api/apps/restful_apis/_generation_params.py
• api/apps/restful_apis/bot_api.py
• api/apps/restful_apis/dify_retrieval_api.py
• api/apps/restful_apis/file2document_api.py
• api/apps/restful_apis/openai_api.py
• api/constants.py
• api/utils/configs.py
• lefthook.yml
• sdk/python/test/conftest.py
• sdk/python/test/test_frontend_api/common.py
• sdk/python/test/test_frontend_api/test_chunk.py
• sdk/python/test/test_frontend_api/test_dataset.py
• test/benchmark/README.md
• test/benchmark/auth.py
• test/benchmark/chat.py
• test/benchmark/cli.py
• test/testcases/conftest.py
• test/testcases/restful_api/helpers/client.py
• test/testcases/restful_api/test_agents.py
• test/testcases/restful_api/test_chunks.py
• test/testcases/restful_api/test_datasets.py
• test/testcases/restful_api/test_dify_retrieval_routes_unit.py
• test/testcases/restful_api/test_document_raw_routes.py
• test/testcases/restful_api/test_documents.py
• test/testcases/restful_api/test_file_routes_unit.py
• test/testcases/restful_api/test_message_routes_unit.py
• test/testcases/restful_api/test_openai_compatible.py
• test/testcases/restful_api/test_retrieval.py
• test/testcases/restful_api/test_searches.py
• test/testcases/restful_api/test_sessions.py
• test/testcases/test_http_api/common.py
• test/testcases/test_http_api/test_dataset_management/test_create_dataset.py
• test/testcases/test_http_api/test_dataset_management/test_delete_datasets.py
• test/testcases/test_http_api/test_dataset_management/test_dify_retrieval_routes_unit.py
• test/testcases/test_http_api/test_dataset_management/test_knowledge_graph.py
• test/testcases/test_http_api/test_dataset_management/test_list_datasets.py
• test/testcases/test_http_api/test_dataset_management/test_update_dataset.py
• test/testcases/test_http_api/test_file_management_within_dataset/test_update_document.py
• test/testcases/test_http_api/test_session_management/test_chat_completions_openai.py
• test/testcases/test_http_api/test_session_management/test_related_questions.py
• test/testcases/test_sdk_api/test_dataset_mangement/test_create_dataset.py
• test/testcases/test_sdk_api/test_dataset_mangement/test_delete_datasets.py
• test/testcases/test_sdk_api/test_dataset_mangement/test_list_datasets.py
• test/testcases/test_sdk_api/test_dataset_mangement/test_update_dataset.py
• test/testcases/test_sdk_api/test_file_management_within_dataset/test_update_document.py
• test/testcases/test_web_api/test_dataset_management/test_dataset_sdk_routes_unit.py
• test/testcases/test_web_api/test_file_app/test_file2document_routes_unit.py
• test/unit_test/api/apps/restful_apis/test_agentbots_access_control.py
• test/unit_test/api/apps/restful_apis/test_openai_stream_no_duplicate.py
• test/unit_test/api/apps/sdk/test_dify_retrieval.py
• test/unit_test/tools/test_check_files.py
• tools/hooks/check_files.py
• web/.prettierignore
• web/.prettierrc
• web/src/assets/svg/next-login-bg.svg
• web/src/constants/compilation.ts
• web/src/interfaces/request/file-manager.ts
• web/src/lib/editor/plugins/floating-selection-toolbar.tsx
• web/src/pages/chunk/representation/components/mindmap-g6-graph/index.tsx
• web/src/pages/chunk/representation/components/mindmap-g6-graph/types.ts
• web/src/pages/chunk/representation/components/representation-renderer.tsx
• web/src/pages/chunk/representation/components/representation-select.tsx
• web/src/pages/chunk/representation/utils/adapters.ts
• web/src/pages/dataset/dataset/generate-button/hook.ts
• web/src/pages/login-next/bg.tsx
• web/src/pages/login-next/card.tsx
• web/src/pages/login-next/index.less
• web/src/pages/login-next/index.tsx

The RAGFlow testing infrastructure provides a comprehensive validation suite covering the entire stack, from core document understanding logic to end-to-end browser interactions. The system utilizes pytest as the primary test runner and integrates with GitHub Actions for continuous integration against multiple backend configurations (Elasticsearch, Infinity, OpenSearch).

Test Suite Organization

The test suite is categorized into several distinct layers based on the interface they target and their position in the request lifecycle.

Category	Location	Purpose	Key Tools
SDK Tests	sdk/python/test/	Validates the Python SDK client logic and its interaction with the server sdk/python/test/conftest.py22-86	ragflow-sdk, pytest, hypothesis
HTTP API Tests	test/testcases/test_http_api/	Tests the high-level RESTful API endpoints directly via requests test/testcases/test_http_api/common.py18-33	requests, requests-toolbelt
Restful API Tests	test/testcases/restful_api/	Comprehensive suite for API contracts, including OpenAI compatibility and Go-proxy parity test/testcases/restful_api/test_datasets.py30-49	pytest, RestClient
Web API Tests	test/testcases/test_web_api/	Internal API validation for endpoints used by the React web frontend test/testcases/test_web_api/test_file_app/test_file2document_routes_unit.py1-5	pytest
Unit Tests	test/unit_test/	Tests isolated business logic using stubs for heavy dependencies like LLM providers test/unit_test/api/apps/restful_apis/test_openai_stream_no_duplicate.py1-10	pytest, monkeypatch
Playwright E2E	test/playwright/e2e/	End-to-end browser-based testing for user flows.	Playwright, pytest-playwright
Benchmark	test/benchmark/	Performance and retrieval accuracy testing test/benchmark/README.md1-5	requests, auth.py

Sources: test/testcases/test_http_api/common.py18-33 test/testcases/restful_api/test_datasets.py30-49 sdk/python/test/conftest.py22-86 test/benchmark/README.md1-5

System Data Flow in Testing

The following diagram illustrates how different test types interact with the RAGFlow ecosystem, bridging natural language test definitions to code entities.

Testing Interface Architecture

Sources: test/testcases/test_http_api/common.py23-33 api/apps/restful_apis/openai_api.py34-51 test/testcases/restful_api/test_datasets.py69-76 sdk/python/test/conftest.py52-60

Test Configuration and CI Integration

RAGFlow leverages pytest fixtures, markers, and environment variables to manage state and prioritize execution.

CI Workflow and Test Levels

The infrastructure uses a level-based marking system in GitHub Actions to control test execution depth based on the trigger .github/workflows/sep-tests.yml43-47:

• p1 (High): Critical path tests, such as basic CRUD and authorization test/testcases/restful_api/test_datasets.py68-76
• p2 (Medium): Validation tests, boundary conditions, and property-based tests using hypothesis test/testcases/test_http_api/test_dataset_management/test_update_dataset.py124-129
• p3 (Low): Full suite, concurrency tests, and deep validation test/testcases/test_http_api/test_dataset_management/test_update_dataset.py91-99

The CI pipeline detects changes across Python, Go, and Web components to optimize execution .github/workflows/sep-tests.yml44-46 It also supports testing against different backend document engines by setting the DOC_ENGINE environment variable test/testcases/restful_api/test_datasets.py51-65

Backend Parity Testing

A significant portion of the testing infrastructure is dedicated to ensuring parity between the Python backend and the Go proxy layer. Tests utilize the IS_GO_PROXY flag to handle differences in error codes and field names test/testcases/restful_api/test_datasets.py30-32

Sources: .github/workflows/sep-tests.yml40-48 test/testcases/restful_api/test_datasets.py30-65 test/testcases/test_http_api/test_dataset_management/test_update_dataset.py91-129

Specialized Testing Scenarios

API Contract Validation

Tests in test_http_api/common.py and restful_api/ provide helper functions that encapsulate the API contracts for major features:

• Dataset Management: create_dataset, list_datasets, update_dataset test/testcases/test_http_api/common.py37-49
• Document Lifecycle: upload_documents, parse_documents, stop_parse_documents test/testcases/test_http_api/common.py75-149
• OpenAI Compatibility: Validates SSE event shapes and chat.completion payloads api/apps/restful_apis/openai_api.py107-142
• Dify Integration: Specialized unit tests for Dify retrieval routes using module stubbing test/testcases/restful_api/test_dify_retrieval_routes_unit.py71-118

SDK to Backend Mapping

Sources: test/testcases/test_http_api/common.py24-33 sdk/python/test/test_frontend_api/test_dataset.py23-27 api/apps/restful_apis/openai_api.py97-106 sdk/python/test/conftest.py79-87

Property-Based Testing

RAGFlow uses hypothesis to generate valid and invalid inputs for API testing, ensuring robustness against edge cases in naming and metadata test/testcases/test_http_api/test_dataset_management/test_update_dataset.py125-138

• Name Validation: Tests name length limits and character constraints using valid_names() test/testcases/test_http_api/test_dataset_management/test_update_dataset.py125-134
• Metadata Enrichment: Validates the enrichment of chunks with document metadata using enrich_chunks_with_document_metadata api/apps/restful_apis/openai_api.py58-85

Pre-commit and Code Quality

The repository uses lefthook to manage pre-commit hooks, ensuring that code style and basic linting are enforced before commits are pushed .github/workflows/sep-tests.yml136-150

Sources: test/testcases/test_http_api/test_dataset_management/test_update_dataset.py125-138 api/apps/restful_apis/openai_api.py58-85 .github/workflows/sep-tests.yml136-150