Testing Infrastructure

Relevant source files

• CHANGELOG.md
• Cargo.lock
• Cargo.toml
• src/assets/build_assets.rs
• src/diff.rs
• src/vscreen.rs
• tests/examples/bat.conf
• tests/examples/system_config/bat/config
• tests/integration_tests.rs
• tests/system_wide_config.rs
• tests/tester/mod.rs
• tests/utils/command.rs
• tests/utils/mocked_pagers.rs
• tests/utils/mod.rs

This document provides an overview of the comprehensive testing infrastructure used in the bat project. The testing system ensures code quality, prevents regressions, and validates syntax highlighting functionality across various languages and platforms.

Overview

The bat testing infrastructure consists of several complementary testing approaches:

1. Integration Tests: End-to-end tests that validate bat's functionality as a command-line tool.
2. Syntax Highlighting Tests: Regression tests that verify syntax highlighting output remains consistent using Python-based comparison scripts.
3. Snapshot Tests: Tests that compare actual terminal output against stored reference files using the BatTester framework.
4. Benchmarking: Performance tests that measure highlighting speed and startup time.

For information about the build system and CI/CD pipeline that runs these tests, see Build System and CI/CD.

Sources: Cargo.toml97-108 tests/integration_tests.rs27-32

Integration and Snapshot Testing

Integration tests verify that bat works correctly as a complete application. These tests use the assert_cmd and predicates crates to run bat with various arguments and verify its output, exit codes, and behavior with external tools like pagers.

The BatTester Framework

The BatTester struct provides a controlled environment for snapshot testing. It initializes a temporary directory with a Git repository to test features like Git integration markers.

• Snapshot Testing: The test_snapshot function runs the bat binary against a sample file and compares the stdout against files in tests/snapshots/output/.
• Environment Isolation: The bat_raw_command_with_config utility ensures tests are reproducible by stripping environment variables like BAT_CONFIG_PATH, BAT_THEME, and PAGER.
• Mocked Pagers: Tests for pager integration use mocked versions of more and most to verify that bat correctly identifies and invokes terminal pagers.

For details, see Integration and Snapshot Testing.

Sources: tests/tester/mod.rs15-52 tests/utils/command.rs6-24 tests/utils/mocked_pagers.rs58-81 tests/integration_tests.rs44-52

Syntax Highlighting Tests

bat uses a specialized regression test system to ensure that syntax highlighting remains consistent across updates to syntax definitions or the underlying syntect engine.

Test Architecture

• Reference Files: Pre-generated highlighted versions containing ANSI color codes are stored in tests/syntax-tests/highlighted/.
• Regression Script: regression_test.sh orchestrates the creation of new output and comparison against references.
• Git Integration: tests/.gitattributes forces LF line endings for these fixtures to prevent cross-platform diff failures.

For details, see Syntax Testing and Regression Tests.

Sources: tests/integration_tests.rs49 tests/.gitattributes1-13

Benchmarking and Performance Testing

To maintain performance, bat includes a benchmarking suite focused on two critical areas: startup time and highlighting throughput.

Tool	Purpose
run-benchmarks.sh	Top-level script to execute the performance suite.
highlighting-speed	Measures lines-per-second processing for large files.
startup-time	Measures the overhead of loading assets and configuration.
find-slow-to-highlight-files.py	Identifies specific files or syntaxes causing performance bottlenecks.

For details, see Benchmarking and Performance Testing.

Sources: Cargo.toml130-134

Git Integration Testing

Testing Git integration requires a real Git repository environment. The BatTester framework uses the gix (Gitoxide) crate to programmatically create repositories, stage files, and commit changes to verify that bat correctly renders diff markers in the gutter.

Sources: tests/tester/mod.rs64-102 src/diff.rs43-85

System-Wide Configuration Tests

Special tests exist to verify the precedence of configuration files. These tests (often ignored by default) check if a system-wide configuration is correctly overridden by a user-specific configuration file or environment variables like BAT_CONFIG_PATH.

Sources: tests/system_wide_config.rs8-29 tests/examples/bat.conf1-6