The baseline testing system captures compiler output as text files that serve as regression tests. When a test runs, its output is written to tests/baselines/local/ and compared against reference files in tests/baselines/reference/. Any difference causes test failure, making it easy to detect unintended changes to compiler behavior.
For test organization and categories, see page 11.1. For CI validation workflows, see page 13.1.
Baselines are plain-text snapshot files recording expected compiler output. Tests write to local baseline files, which are compared against checked-in reference files. Differences cause test failure, making output changes immediately visible.
Each compiler test case can produce several baseline files, depending on what the test exercises:
| Extension | Contents |
|---|---|
.js | Emitted JavaScript output |
.d.ts | Emitted declaration files |
.js.map | Emitted source maps |
.types | Inferred type of every expression, one per line |
.symbols | Symbol resolution results for every identifier |
.errors.txt | Compiler diagnostic messages |
For example, an API sample test might generate a .js file containing both the input TypeScript source (as comments) and the resulting JavaScript output tests/baselines/reference/APISample_linter.js1-162
tests/baselines/
├── local/ ← Written by test runner; git-ignored
└── reference/ ← Checked into git; source of truth
Baseline path constants are defined in scripts/build/tests.mjs22-23:
The tests/baselines/local/ directory is git-ignored and never committed. The tests/baselines/reference/ directory contains the "golden" results used for comparison.
Sources: scripts/build/tests.mjs22-23
Baseline generation and comparison flow
Sources: scripts/build/tests.mjs34-188 scripts/build/tests.mjs190-193
The cleanTestDirs() function scripts/build/tests.mjs190-193 removes tests/baselines/local/ before each test run, preventing stale output contamination.
The runConsoleTests() function scripts/build/tests.mjs34-188 orchestrates test execution:
cleanTestDirs() scripts/build/tests.mjs58test.config file to pass parameters to the runner scripts/build/tests.mjs88exec scripts/build/tests.mjs152c8 scripts/build/tests.mjs157When test output changes intentionally (e.g., a diagnostic message is updated), local baselines must be promoted to reference baselines. This is typically done via the baseline-accept task defined in the build system.
The acceptance workflow copies files from the git-ignored local directory to the tracked reference directory. Once promoted, these changes must be staged and committed to the repository.
Sources: scripts/build/tests.mjs22-23 Herebyfile.mjs21-24
The CI pipeline validates that checked-in reference baselines are up-to-date by regenerating them and comparing against the repository state. This ensures that no PR is merged with "dirty" or outdated baselines.
CI baseline validation workflow
Sources: azure-pipelines.release.yml52-56
In the release and main CI pipelines, npm test triggers the runtests-parallel task package.json89 which eventually calls runConsoleTests scripts/build/tests.mjs34 If the exec call to Mocha returns a non-zero exit code due to a baseline mismatch, the CI job fails scripts/build/tests.mjs184-185
Baseline system code mapping
Sources: scripts/build/tests.mjs22-23 scripts/build/tests.mjs34 scripts/build/tests.mjs190 scripts/build/utils.mjs25
When a baseline mismatch occurs, developers must investigate the diff between the expected and actual output.
tests/baselines/reference/<test>.js with tests/baselines/local/<test>.js.getDiffTool() function scripts/build/utils.mjs155-162 checks the DIFF environment variable to determine which external program to use for visual comparisons.To speed up the inner loop, developers can run a subset of tests using the --tests (or -t) flag, which is parsed by cmdLineOptions scripts/build/tests.mjs36 and passed to the Mocha runner scripts/build/tests.mjs104
Sources: scripts/build/tests.mjs36-47 scripts/build/utils.mjs155-162