Pipeline API

Relevant source files

• docs/source/ar/tasks/sequence_classification.md
• docs/source/en/kv_cache.md
• docs/source/en/main_classes/pipelines.md
• docs/source/en/model_doc/whisper.md
• docs/source/en/pipeline_tutorial.md
• docs/source/en/tasks/any_to_any.md
• docs/source/en/tasks/image_captioning.md
• docs/source/en/tasks/image_feature_extraction.md
• docs/source/en/tasks/image_text_to_text.md
• docs/source/en/tasks/knowledge_distillation_for_image_classification.md
• docs/source/en/tasks/mask_generation.md
• docs/source/en/tasks/monocular_depth_estimation.md
• docs/source/en/tasks/object_detection.md
• docs/source/en/tasks/semantic_segmentation.md
• docs/source/en/tasks/sequence_classification.md
• docs/source/en/tasks/text-to-speech.md
• docs/source/en/tasks/video_text_to_text.md
• docs/source/en/tasks/visual_document_retrieval.md
• docs/source/en/tasks/visual_question_answering.md
• docs/source/ja/main_classes/pipelines.md
• docs/source/ja/tasks/object_detection.md
• docs/source/ko/main_classes/pipelines.md
• docs/source/ko/model_doc/whisper.md
• docs/source/ko/tasks/object_detection.md
• docs/source/ko/tasks/sequence_classification.md
• docs/source/pt/tasks/sequence_classification.md
• docs/source/ro/pipeline_tutorial.md
• docs/source/zh/main_classes/pipelines.md
• docs/source/zh/tasks/sequence_classification.md
• src/transformers/models/gemma4_unified/convert_gemma4_unified_weights.py
• src/transformers/models/whisper/configuration_whisper.py
• src/transformers/models/whisper/convert_openai_to_hf.py
• src/transformers/models/whisper/generation_whisper.py
• src/transformers/models/whisper/modeling_whisper.py
• src/transformers/models/whisper/tokenization_whisper.py
• src/transformers/pipelines/__init__.py
• src/transformers/pipelines/any_to_any.py
• src/transformers/pipelines/automatic_speech_recognition.py
• src/transformers/pipelines/base.py
• src/transformers/pipelines/document_question_answering.py
• src/transformers/pipelines/image_feature_extraction.py
• src/transformers/pipelines/image_text_to_text.py
• src/transformers/pipelines/table_question_answering.py
• src/transformers/pipelines/text_generation.py
• tests/models/roberta/test_tokenization_roberta.py
• tests/models/roc_bert/test_tokenization_roc_bert.py
• tests/models/whisper/test_modeling_whisper.py
• tests/models/whisper/test_tokenization_whisper.py
• tests/pipelines/test_pipelines_any_to_any.py
• tests/pipelines/test_pipelines_automatic_speech_recognition.py
• tests/pipelines/test_pipelines_common.py
• tests/pipelines/test_pipelines_document_question_answering.py
• tests/pipelines/test_pipelines_feature_extraction.py
• tests/pipelines/test_pipelines_image_feature_extraction.py
• tests/pipelines/test_pipelines_image_text_to_text.py
• tests/pipelines/test_pipelines_text_generation.py
• tests/pipelines/test_pipelines_zero_shot_audio_classification.py
• tests/test_pipeline_mixin.py
• tests/utils/test_chat_parsing.py

Purpose and Scope

The Pipeline API provides a high-level, task-oriented interface for inference with transformer models. It abstracts the complexity of preprocessing inputs, running model inference, and postprocessing outputs into a simple, unified API. Pipelines handle tokenization, feature extraction, batching, device placement, and output formatting automatically.

This document covers the architecture, lifecycle, and usage of pipelines. For model loading and weight management, see Model Loading and Weight Management. For tokenization details, see Tokenization System. For multi-modal processing, see Multi-Modal Processing.

High-Level Architecture

The Pipeline system consists of three layers: a factory function for instantiation, a base class defining the inference lifecycle, and task-specific implementations.

Pipeline System Architecture

Title: Pipeline System Architecture

Sources: src/transformers/pipelines/__init__.py141-184 src/transformers/pipelines/base.py188-280 src/transformers/pipelines/base.py751-1000

Pipeline Lifecycle

Every pipeline follows a standardized three-stage lifecycle: preprocess → forward → postprocess. The base Pipeline class orchestrates this flow.

Inference Execution Flow

Title: Pipeline Execution Sequence

Key Lifecycle Methods

Method	Purpose	Responsibilities
_sanitize_parameters()	Separate kwargs into stage-specific params	Returns (preprocess_params, forward_params, postprocess_params) src/transformers/pipelines/text_generation.py128-146
preprocess()	Convert raw inputs to model-ready tensors	Tokenization, image processing, padding src/transformers/pipelines/base.py937-951
_forward()	Run model inference	Execute model.forward() or model.generate() src/transformers/pipelines/base.py953-967
postprocess()	Convert model outputs to user-friendly format	Decoding, score calculation src/transformers/pipelines/base.py969-983

Sources: src/transformers/pipelines/base.py932-983 src/transformers/pipelines/text_generation.py128-146

The pipeline() Factory Function

The pipeline() function is the primary entry point for creating pipelines. It handles task resolution, model loading, and component initialization.

Task Resolution

The SUPPORTED_TASKS dictionary src/transformers/pipelines/__init__.py141-184 maps task identifiers to pipeline configurations.

Key	Value Fields	Example
Task identifier	impl: Pipeline class	AutomaticSpeechRecognitionPipeline src/transformers/pipelines/__init__.py149
	pt: Model classes tuple	(AutoModelForCTC, AutoModelForSpeechSeq2Seq) src/transformers/pipelines/__init__.py150
	default: Default model dict	{"model": ("facebook/wav2vec2-base-960h", "22aad52")} src/transformers/pipelines/__init__.py151
	type: Input modality	"multimodal", "text", "audio" src/transformers/pipelines/__init__.py152

Sources: src/transformers/pipelines/__init__.py141-184

Component Loading

The pipeline loads components based on flags defined in the specific pipeline class.

Title: Natural Language Space to Pipeline Code Entity Mapping

Each task-specific pipeline sets these flags:

• TextGenerationPipeline: _load_tokenizer = True src/transformers/pipelines/text_generation.py90
• ImageTextToTextPipeline: _load_processor = True src/transformers/pipelines/image_text_to_text.py115
• AutomaticSpeechRecognitionPipeline: _load_feature_extractor = True and _load_tokenizer = True src/transformers/pipelines/automatic_speech_recognition.py157-158

Sources: src/transformers/pipelines/base.py773-777 src/transformers/pipelines/text_generation.py86-90 src/transformers/pipelines/automatic_speech_recognition.py155-158

Task-Specific Pipeline Implementations

Text Generation Pipeline

TextGenerationPipeline src/transformers/pipelines/text_generation.py23 handles causal language model generation with support for text prompts and chat conversations.

Key Features:

• Integrates with generation system via _pipeline_calls_generate = True src/transformers/pipelines/text_generation.py86
• Uses _default_generation_config with max_new_tokens=256, do_sample=True, and temperature=0.7 src/transformers/pipelines/text_generation.py93-97
• Automatically sets tokenizer.padding_side = "left" for decoder-only models to ensure correct batched generation src/transformers/pipelines/text_generation.py105-106
• Supports chat templates and continue_final_message logic src/transformers/pipelines/text_generation.py140

Sources: src/transformers/pipelines/text_generation.py23-146

Automatic Speech Recognition Pipeline

AutomaticSpeechRecognitionPipeline src/transformers/pipelines/automatic_speech_recognition.py112 extracts text from audio waveforms or files.

Key Features:

• Inherits from ChunkPipeline src/transformers/pipelines/automatic_speech_recognition.py112 to support processing long audio files via windowed chunking.
• Implements chunk_iter src/transformers/pipelines/automatic_speech_recognition.py61-85 to yield overlapping audio segments.
• Supports Whisper specific logic like WhisperTimeStampLogitsProcessor src/transformers/models/whisper/generation_whisper.py32 and dynamic time warping for timestamps src/transformers/models/whisper/generation_whisper.py64-115
• Provides timestamp extraction for both CTC (character/word) and Whisper (segment/token) src/transformers/pipelines/automatic_speech_recognition.py121-153

Sources: src/transformers/pipelines/automatic_speech_recognition.py112-174 src/transformers/models/whisper/generation_whisper.py64-115

Batching and Streaming

Pipelines support efficient batching via DataLoader and custom collate functions.

Batching Data Flow

Title: Batching Data Flow to Code Entities

Collation Functions:

• pad_collate_fn() src/transformers/pipelines/base.py125-185: Pads sequences to the same length within a batch. It handles input_ids, pixel_values, and input_features with appropriate padding values src/transformers/pipelines/base.py167-181
• _pad() src/transformers/pipelines/base.py87-122: Core padding logic handling 1D, 2D, and 4D tensors (like mel spectrograms) src/transformers/pipelines/base.py100-102

Sources: src/transformers/pipelines/base.py87-185

Advanced Features

Device Management

Pipelines handle device placement automatically in __init__ src/transformers/pipelines/base.py815-859

• Supports device_map for multi-GPU setups src/transformers/pipelines/base.py829
• Checks for framework-specific accelerators like is_torch_npu_available src/transformers/pipelines/base.py51 is_torch_mps_available src/transformers/pipelines/base.py49 or is_torch_xpu_available src/transformers/pipelines/base.py52

KV Cache Integration

For generation tasks, pipelines integrate with the library's Cache system.

• DynamicCache is used by default for growing sequences docs/source/en/kv_cache.md35
• StaticCache can be enabled via cache_implementation="static" to support torch.compile optimizations docs/source/en/kv_cache.md72-86

Sources: src/transformers/pipelines/base.py815-859 docs/source/en/kv_cache.md33-89