This document provides an overview of the Protocol Buffers development and testing infrastructure, including build systems, CI/CD pipelines, and quality assurance processes that ensure compatibility across platforms, compilers, and language implementations.
For detailed information about specific testing systems, see:
Protocol Buffers supports two primary build systems: Bazel (preferred for development and testing) and CMake (for C++ ecosystem integration). Each serves different use cases while maintaining consistent build outputs.
The following diagram bridges the high-level build concepts to the specific configuration files and entities in the codebase.
Sources: MODULE.bazel1-10 CMakeLists.txt1-10 protobuf_deps.bzl1-10 maven_install.json1-10
Bazel Configuration: The repository uses modern Bzlmod (MODULE.bazel) for dependency management. The protobuf_deps() function in protobuf_deps.bzl53 loads external dependencies like Abseil, zlib, and various Bazel rule sets.
CMake Configuration: CMakeLists.txt1-352 provides a complete build system supporting options like protobuf_BUILD_TESTS and protobuf_BUILD_CONFORMANCE. It handles platform-specific logic and links dependencies like Abseil and zlib.
Sources: protobuf_deps.bzl53-202 MODULE.bazel1-356 CMakeLists.txt1-352
The Protocol Buffers testing infrastructure is organized into three layers: unit tests per language, conformance tests for wire format compatibility, and integration tests for cross-version compatibility.
This diagram maps the testing abstractions to the concrete classes and runners used in the repository.
Sources: conformance/conformance_test_runner.cc1-50 conformance/binary_json_conformance_suite.cc8-62 .github/workflows/test_cpp.yml33-36
Unit Test Framework: C++ tests use GoogleTest, while language-specific tests use native frameworks. Bazel is the primary executor with targets like bazel test //src/... .github/workflows/test_cpp.yml41
Conformance Testing: The suite validates wire format compatibility across implementations. The conformance-test-runner executes test cases from BinaryJsonConformanceSuite conformance/binary_json_conformance_suite.cc8 against language-specific binaries. See Conformance Testing for details.
Sanitizer Coverage: CI includes comprehensive sanitizer builds to detect memory errors and threading issues .github/workflows/test_cpp.yml33-36:
--config=asan).--config=docker-msan).--config=tsan).--config=ubsan).Sources: .github/workflows/test_cpp.yml1-572 conformance/conformance_test_runner.cc1-100 conformance/binary_json_conformance_suite.cc1-100
The CI/CD pipeline uses GitHub Actions to execute comprehensive tests across multiple languages, platforms, and compilers.
The workflow is designed for parallel execution with selective "continuous-only" tests for expensive operations.
| Language | Workflow File | Primary Targets |
|---|---|---|
| C++ | .github/workflows/test_cpp.yml | //src/..., //pkg/... |
| μpb | .github/workflows/test_upb.yml | //upb/..., //python/... |
| Java | .github/workflows/test_java.yml | //java/..., //compatibility/... |
| Python | .github/workflows/test_python.yml | //python/... |
| Ruby | .github/workflows/test_ruby.yml | //ruby/... |
| PHP | .github/workflows/test_php.yml | composer test, //php:conformance_test |
| Rust | .github/workflows/test_rust.yml | //rust/... |
Test Selection: Workflows use a continuous-only flag to skip expensive tests (like MSAN or Debug builds) during presubmit pull requests, while running them on the main branch .github/workflows/test_cpp.yml32-36
Platform Coverage:
Cross-Compilation: For non-x86 architectures, the pipeline cross-compiles protoc using the cross-compile-protoc action .github/workflows/test_cpp.yml123 then runs tests via QEMU emulation in specialized Docker containers .github/workflows/test_cpp.yml138
Sources: .github/workflows/test_cpp.yml1-150 .github/workflows/test_upb.yml1-100 .github/workflows/test_java.yml26-60 .github/workflows/test_python.yml26-55
Each language implementation integrates with the core C++ testing infrastructure while maintaining its own ecosystem-specific tests.
Sources: .github/workflows/test_upb.yml1-100 .github/workflows/test_java.yml1-100 .github/workflows/test_python.yml1-100 .github/workflows/test_ruby.yml1-100 .github/workflows/test_php.yml1-100 .github/workflows/test_rust.yml1-100