This page provides an overview of deploying Docling in production environments. It covers the deployment architecture, dependency management, and key considerations for production use. For detailed information on specific deployment topics, see:
For configuration options and pipeline setup, see Configuration and Pipeline Options. For threading and concurrency patterns, see Threading and Concurrency. For hardware acceleration, see GPU and Hardware Acceleration.
The following diagram bridges the high-level deployment components to their respective code entities and configuration settings.
Deployment Architecture: DocumentConverter orchestrates models which are managed via download_models docling/utils/model_downloader.py38-60 Models are stored in the directory defined by settings.cache_dir docling/datamodel/settings.py10 and docling/utils/model_downloader.py61-62 Device selection is handled by decide_device docling/utils/accelerator_utils.py37 mapping requested AcceleratorOptions docling/datamodel/accelerator_options.py42-57 to physical AcceleratorDevice docling/datamodel/accelerator_options.py13-20 types.
Sources: docling/utils/model_downloader.py38-65 docling/datamodel/accelerator_options.py13-57 docling/datamodel/settings.py10 docling/document_converter.py48
Docling utilizes specialized ML models for document understanding. These models are typically hosted on HuggingFace and managed via the model_downloader utility.
The download_models function docling/utils/model_downloader.py38-60 serves as the central entry point for fetching artifacts. It supports granular selection of models including:
LayoutModel docling/utils/model_downloader.py67-73TableStructureModel docling/utils/model_downloader.py75-81 and TableStructureModelV2 docling/utils/model_downloader.py83-93RapidOcrModel docling/utils/model_downloader.py204-208 and EasyOcrModel docling/utils/model_downloader.py210-214ChartExtractionModelGraniteVision docling/utils/model_downloader.py181-191 and various Smol/Granite VLM specs like GRANITEDOCLING_TRANSFORMERS docling/utils/model_downloader.py126-133The docling-tools models CLI docling/cli/models.py39-44 allows for pre-provisioning models in production environments (e.g., during Docker image builds).
Sources: docling/utils/model_downloader.py38-214 docling/cli/models.py75-151
Docling uses a modular dependency structure. The docling-slim package docling/cli/models.py22-23 provides a minimal base for custom production builds, requiring specific extras for features like OCR or CLI support.
| Group | Key Dependencies | Purpose |
|---|---|---|
vlm | transformers | Vision-language parsing and chart extraction docling/models/stages/chart_extraction/granite_vision.py27 |
ocr | easyocr, rapidocr | Alternative OCR engines docling/utils/model_downloader.py204-214 |
cli | typer, rich | Command line interface support docling/cli/models.py9-12 |
Acceleration is resolved via decide_device docling/models/stages/chart_extraction/granite_vision.py62-65 which maps AcceleratorOptions to physical hardware. AcceleratorDevice.AUTO docling/datamodel/accelerator_options.py14 is often used to let the system determine the best available backend (CUDA, MPS, XPU, or CPU).
Sources: docling/cli/models.py9-26 docling/datamodel/accelerator_options.py13-20 docling/models/stages/chart_extraction/granite_vision.py62-65
Production setups should favor minimal images. The provided Dockerfile uses python:3.11-slim-bookworm Dockerfile1 and installs Docling with CPU-only torch by default to keep the image size manageable Dockerfile12
Key container considerations:
OMP_NUM_THREADS Dockerfile22 to avoid CPU congestion in container clusters.DOCLING_ARTIFACTS_PATH environment variable Dockerfile28 ensures the DocumentConverter uses models baked into the image.HF_HOME and TORCH_HOME are redirected to /tmp/ Dockerfile14-15 for compatibility with read-only filesystems.| Variable | Usage |
|---|---|
OMP_NUM_THREADS | Limits CPU thread budget to prevent congestion Dockerfile22 |
DOCLING_ARTIFACTS_PATH | Path to local model weights for offline usage Dockerfile28 |
HF_HOME | Directory for HuggingFace model caching Dockerfile14 |
Sources: Dockerfile1-30 docling/datamodel/accelerator_options.py42-57
Models like ChartExtractionModelGraniteVision docling/models/stages/chart_extraction/granite_vision.py42 require significant compute. When deploying, ensure AcceleratorOptions are tuned for the target environment docs/examples/run_with_accelerator.py43-60 For example, AcceleratorDevice.MPS is used for macOS, while AcceleratorDevice.CUDA is for NVIDIA GPUs docs/examples/run_with_accelerator.py49-57
Production deployments often require balancing OCR quality and speed. Toggling do_ocr or using force_full_page_ocr=True docs/examples/full_page_ocr.py54-68 significantly impacts throughput. Additionally, images_scale docs/examples/export_figures.py60 affects memory usage when generating page or element images.
For detailed tuning, see Production Considerations.
Sources: docling/models/stages/chart_extraction/granite_vision.py42-65 docs/examples/run_with_accelerator.py43-60 docs/examples/full_page_ocr.py54-68
Deployment of Docling involves:
docling-tools models download to fetch weights docling/cli/models.py75AcceleratorOptions docling/datamodel/accelerator_options.py42For detailed guides, refer to the child pages:
Refresh this wiki