fallow mutation

Check whether your tests fail when the changed code is wrong. fallow mutation makes small changes to your code, runs your Vitest, node:test, or bun test tests against each one, and reports the changes that no test detects.

fallow mutation measures whether your tests fail when the changed code is wrong. It makes small changes to the code (mutants), runs your project tests against each mutant, and reports the mutants that no test detects.

A mutant changes one thing, for example <= to <, + to -, or true to false. When a test fails, the test detected the change and the mutant is killed. When every test passes, the mutant survived. A survivor shows a place where the code can be wrong and your tests still pass.

Fallow reports evidence. It does not write tests and it does not propose code changes. You decide what a survivor means.

fallow mutation

Three commands

Only fallow mutation without --plan runs code. Project config cannot start a run, and the MCP server cannot start a run.

Scope

By default, fallow mutates only the changed code against the base ref. It finds the base ref in the same way as fallow audit.

The changed-code scope, --all, and paths are exclusive. --workspace and --changed-workspaces exit with code 2, because the plan cannot narrow to a workspace yet. To mutate one package, pass its directory as a path:

fallow mutation packages/billing

Source files

The plan, the run, and inspect cover these files:

• JavaScript and TypeScript files: .js, .jsx, .mjs, .cjs, .ts, .tsx, .mts, and .cts.
• Vue, Svelte, and Astro components (.vue, .svelte, .astro). Only the script blocks get mutants: the Vue <script> and <script setup> blocks, the Svelte instance and module scripts, and the Astro frontmatter. The template and the styles do not change, so the framework plugin of your project compiles the mutated component.
• Svelte module files (.svelte.js, .svelte.ts).

The mutant line, column, and span in a component are positions in the whole file, not in the script block. An edit of the template does not change the mutant ids.

The framework compiler moves some code out of the script block. These positions give no mutant:

• A call of a Vue compiler macro (defineProps, defineEmits, defineExpose, defineModel, defineOptions, defineSlots, withDefaults) in a <script setup> block, its arguments, and the destructure defaults of a declaration such as const { size = 1 } = defineProps().
• An expression that holds a Svelte rune call, for example $state(0) or $derived(a + b). The arguments of the rune still get mutants.
• An exported declaration in the Astro frontmatter.

Plan without running code

--plan selects, maps, and ranks the mutants. It runs no code.

fallow mutation --plan
fallow mutation --plan --all

The plan shows each mutant with its location, function, operator, and id:

Mutation plan (no code runs)
Scope: all source files
Mutants: 8 planned, 0 pending, 0 no coverage, 0 pruned
Files: 1 instrumented, 1 test file reaches them

● Planned (8)
  #1 src/pricing.ts:2:7  applyDiscount  relational-operator  c65def5cd9bd279a
  #2 src/pricing.ts:2:15  applyDiscount  relational-operator  72487a15b6adc213
  ...

Use the plan to see the size of a run before you start it. When the changed-code scope is empty, the output suggests fallow mutation --plan --all.

When a mutant is no_coverage and the project has test files, the plan prints one line with the test reach counts, for example No coverage by test reach: 2 mocked-only, 1 none-file.

Run the tests

fallow mutation without --plan runs your project tests against each mutant.

fallow mutation

Runners

Vitest 4.0 is not supported. Fallow stops with an error for Vitest 4.0, because that version ignores the test-id filter and each mutant looks killed.

Bun 1.3 is not supported. Fallow stops with an error for Bun 1.3, because that version gives no test results for a run that stops at the first failure, and it reports each runtime error as an assertion failure.

The default value auto selects the runner from the project, in this order:

1. A vitest dependency in the root package.json selects Vitest.
2. Otherwise, the test module that most test files import: vitest, bun:test, or node:test.
3. Otherwise, a bunfig.toml, bun.lock, or bun.lockb file at the root selects bun test.
4. Otherwise a node --test or borp command in a package.json script selects node:test.
5. Otherwise, Vitest.

borp is a test runner that runs node:test. Fallow reads the test files of a borp command in this order. First, it reads the positional arguments. Fallow ignores a glob that starts with !, so it does not exclude those files. Second, it reads the glob of -p or --pattern. Third, it uses the default glob **/*.test.{js,mjs,cjs,ts,mts,cts}. These files are test entry points, so the plan finds the test files of the project. Fallow does not read a .borp.yaml file.

Fallow finds Vitest in the project node_modules. For node:test and bun test, each test file runs in its own process. The test files are the files that the default patterns of the runner find.

The node:test runner reads the node --test command in your package.json scripts. It reads the test script first, then the other scripts in name order. Each test process gets the module loading flags of that command:

• --experimental-transform-types
• --experimental-strip-types and --no-experimental-strip-types
• --import and --require (-r)
• --experimental-loader (--loader)
• --conditions (-C)

Fallow ignores the other flags of the command. A relative --import, --require or loader path resolves against the project root. Example: a project that uses a TypeScript enum runs its tests with node --experimental-transform-types --test. fallow mutation then uses the same flag, and you do not need to set it.

NODE_OPTIONS of the fallow mutation process also reaches each test process. Use it for a Node flag that is not in the list above.

When a test file does not load, the baseline failure message starts with the error line. Example: SyntaxError [ERR_UNSUPPORTED_TYPESCRIPT_SYNTAX]: TypeScript enum is not supported in strip-only mode.

Node.js and Bun cannot load a Vue, Svelte, or Astro component file. With node-test or bun-test, the plan leaves each component file out. The report lists the file in skipped_files with the reason runner-cannot-load, and the human output prints one Skipped files: line. The mutants of these files do not count in the coverage. To test component files, use Vitest with the framework plugin of your project.

--runtime must name a runtime of the correct kind for the runner. A Bun runtime for node-test, or a Node.js runtime for bun-test, exits with code 2 and FALLOW_MUTATION_RUNTIME_NOT_FOUND. For bun-test, install Bun.

Example output

Mutation testing (ran the project tests)
Scope: all source files
Score 71.4%  killed 5  survived 2  timeout 0  complete
Coverage 87.5%  no coverage 1

● Survived (2)
  src/pricing.ts
    #2 2:15  applyDiscount  relational-operator  if (percent <= 0) return total; -> if (percent < 0) return total;
      id 72487a15b6adc213
      covered by 1 test: src/pricing.test.ts::discount
    #3 2:15  applyDiscount  relational-operator  if (percent <= 0) return total; -> if (percent == 0) return total;
      id 77ac825e81517e35
      covered by 1 test: src/pricing.test.ts::discount

● No coverage (1): no test reaches these mutants
  src/pricing.ts  1 mutant  test reach: 1 none-file

Next
  Inspect a survivor (runs no code): fallow mutation inspect 72487a15b6adc213

The human output shows the 10 best-ranked survivors, grouped by file. The JSON report lists all survivors and all covering tests.

The output can add these lines:

• The score line adds compile error N when the survivor type check found N mutants that do not type-check.
• Survivor type check: 2 checked, 1 does not type-check, 1 not checked (no-project 1) after a run with --typecheck. When the check fails, the line says that the survivors keep the status survived.
• Cache: 2 results came from an earlier run. Pass --no-cache to run each mutant. when the result cache gave at least one result.

Run options

The global flag --no-cache turns off the result cache for one run.

Operators

The regex-literal operator gives one mutant for each of these tokens:

• A ^ anchor at the start of the pattern is removed.
• A $ anchor at the end of the pattern is removed.
• A character class is negated, for example [a-z] to [^a-z], and the reverse. A pattern with the v flag gets no negation.
• A class escape gets the opposite case: \d to \D, \w to \W, and \s to \S, and the reverse.

The flags of the regular expression do not change.

Fallow has no absolute-value operator (x to -x). Without type information, that operator applies to each operand of each arithmetic or relational expression. These expressions already get mutants, so the extra mutants add run time and little new information.

Safety rules

A run executes your project tests, so fallow applies these rules:

• Clean worktree. A run refuses a git worktree with uncommitted changes and exits with code 2. Commit or stash your changes, or pass --allow-dirty. After the run, fallow compares the worktree with the start state and reports each path that changed.
• What the check sees. The check covers tracked and untracked files. It also covers the ignored files that git lists at the start, such as .env, so a write to one of them or a delete is a change. Git lists an ignored directory such as node_modules/ or coverage/ as one entry, so the check does not see the files below it. A new ignored file is not a change, because test runs make such files. The check leaves out mutation/ in the fallow cache directory and the output files of the run.
• Safe cache directory. A run refuses a cache directory that is the project root or a directory above it, or that holds a planned source or test file. It exits with code 2 and FALLOW_MUTATION_UNSAFE_CACHE_DIR. Such a directory would hide the writes of your tests from the check.
• Side effects. Fallow skips the mutants in a function whose own body calls a file system, process, network, or database API. A mutant in top-level code is skipped when the module makes such a call at the top level. These mutants have the status pruned with the reason side-effect-sink. A mutant there can remove a guard of a real side effect. The rule does not follow calls: a mutant in a function that calls a helper, which then makes the call, still runs. --allow-side-effects keeps the skipped mutants. This option is a flag only. The config rejects an allowSideEffects key, so a project config cannot widen what a run executes.
• Project files never change. Fallow does not write to your project files. The changed copies of your files live in a run directory in the fallow cache. Two runs of the same plan share one directory, and a run never reads a partly written one. Fallow keeps the 5 most recently used run directories. It removes older ones that no run used in the last hour. The report lists any path that your tests changed.
• No test process outlives fallow. The test processes run in their own process group on Unix and in a Job Object on Windows. Fallow ends them when the run ends, on SIGINT, SIGTERM, SIGHUP, or SIGQUIT, and on a crash. When fallow starts with SIGHUP ignored, as under nohup, a hangup does not end the run.

Score and coverage

The report gives two separate values. In JSON, both are fractions from 0 to 1.

The score covers only the mutants that ran. A mutant that no test reaches is no_coverage, and it never enters the score. The coverage reports these mutants apart. A run of 200 mutants in a large project therefore gets a score that describes the 200 tested mutants.

Before the mutants run, fallow runs your tests once with no active mutant (the dry run). The dry run decides the coverage of every planned mutant, also of the mutants after the --max-mutants limit. A run that --max-mutants limits therefore reports the coverage of the full plan. A planned mutant that no test covers is no_coverage and does not use a place in the limit. The next covered pending mutant in rank order runs in its place. A cached result uses a place in the limit, so a run with the result cache can execute fewer than N mutants. The --plan output comes before the dry run, so a run can execute a mutant that the plan shows as pending.

A run with no executed mutant has no score (null in JSON).

--min-score and mutation.minScore use percent, not a fraction.

Statuses

A pruned mutant has a reason. One of the reasons:

• framework-config: the mutant changes framework or tool config, not tested logic. Framework config is an export that an active plugin reads, for example export const revalidate = 60 in a Next.js page. It is also the default export of a tool config file, such as vite.config.ts, vitest.config.ts, eslint.config.js, .eslintrc.js or docs/.vitepress/config.ts. A top-level module.exports = ... statement counts as the default export. A mutant inside a function of the config, for example an inline Vite plugin, is not pruned. No test loads a tool config file, so these mutants do not lower the coverage.

Retries

A survived or timeout mutant runs a second time in a new runner process. Host load can make a test time out, so one timeout is not proof that your tests detect the mutant. The second run of a timeout gets twice the time. A mutant with "retried": true in JSON got both runs. A survived or timeout mutant without the field got no second run, because the time budget ended first.

• A killed result gets no retry. A failed test is a fact about the mutant.
• timings.retry_ms is the time of the retries, for the survived and the timed-out mutants.

Completion

completion.status is complete or partial. Only a complete run makes an empty survivor list conclusive. A partial run lists its reasons, for example mutant-limit, time-budget, runtime-error, baseline-failure, or typecheck-failed.

A test that fails in the dry run is a baseline failure. Fallow lists it in baseline_failures, and its result decides no mutant status. Fix the failing test, then run again.

Type tests

When your Vitest config enables the type-check pass (typecheck.enabled), the run does not execute it. A mutant that only a type test detects is not counted as killed. The report adds typecheck-not-run to runner.notes, and the human output prints one line for it. The survivor type check does not run these type tests either.

A type test file (*.test-d.ts, *.spec-d.ts, or a TypeScript file below a test-d/ directory) is not a test file of the mutation plan. No runner executes it. A project whose only test files are type tests has no_test_files: true, and a file that only type tests reach has no coverage.

Test reach of no-coverage mutants

Each no_coverage mutant has a test_reach object in the run report and in the plan. It gives the static reason why no test covers the mutant. Mutants with other statuses do not have the object. The object never changes the score or the coverage.

• none-file: no test loads the file. A project without test files gives none-file to each mutant.
• mocked-only: tests reach the code only through a mock.
• none: a test loads the file, but no test reaches this code.
• direct or transitive on a no_coverage mutant is not a conflict. A test refers to the export, but no test ran the code of the mutant.

In the human output, each file line in the no-coverage section adds the counts by class, for example test reach: 2 mocked-only, 1 none-file.

Type check of survivors

--typecheck, or mutation.typecheck: true in the config, type-checks each survived mutant after the tests. The check uses the same TypeScript companion as --type-aware. Other mutants are not checked, so the cost grows only with the number of survivors.

fallow mutation --typecheck

• The companion applies one mutant at a time to a copy of the file in memory. It never writes a project file.
• It compares the type errors of the mutated file with the type errors of the original file. An error that the original file already has does not count.
• A survivor with a new type error gets the status compile_error. type_error_codes lists the TypeScript error codes, sorted, at most 8. The mutant leaves the score, and it counts as covered.
• The check reads only the mutated file. A mutant that breaks only another file stays survived.
• A survivor that the check cannot check stays survived. The reasons are no-project (no tsconfig.json includes the file), not-type-checked (// @ts-nocheck, or a JavaScript file without checkJs), source-changed, file-missing, and unsupported-file (a Vue, Svelte or Astro component, which the checker cannot read).
• The run looks for the companion before the tests start. When no trusted companion exists, the run exits with code 2 and FALLOW_MUTATION_TYPECHECK_UNAVAILABLE. Fallow finds the companion in the same way as for --type-aware: FALLOW_TYPE_AWARE_BIN, the npm wrapper, or the companion next to the fallow executable.
• When the companion fails after the tests, the survivors keep the status survived. The report then has an error with the code typecheck-failed, and the completion is partial with the reason typecheck-failed.
• --plan --typecheck exits with code 2, because a plan has no survivors. The config key has no effect on a plan.

With the option on, the report has a typecheck object:

The report also has summary.counts.compile_error, functions[].counts.compile_error, and timings.typecheck_ms. Without the option, the typecheck object is absent and no mutant gets compile_error.

Result cache

A run stores the result of each killed mutant and of each survived mutant that a retry confirmed. A later run uses a stored result when the key is the same, and does not run that mutant again. The dry run always runs.

The key covers the mutant, the content of its source file, its covering tests, the content of the covering test files and the files that they import, the runner setup files, the lockfile, the runner config files at the root and the files that they import, the configs that the root tsconfig.json extends, the runner and runtime versions, and the fallow version. A change to one of them runs the mutant again.

• A timeout, flaky, runtime_error, or pending result is never stored.
• A reused survived result also has "retried": true, because only a survivor that a retry confirmed is stored.
• A reused result has "cached": true on its entry in mutants[]. Without a reused result, the key is absent. A reused result counts in the score and the coverage in the same way as a new result.
• The report has a cache object: hits (the reused results), stored (the stored results), and not_keyed (["dynamic-imports", "file-reads"]). The object is absent with --no-cache, and when no test process started.
• The cache file is mutation/results.json in the fallow cache directory. The cache directory has its own .gitignore.
• A failed write of the cache file adds a cache-write-failed entry to errors[]. The run result stays.
• The global --no-cache flag turns off the result cache. The run then reads and writes no entry.

Saved report

Each run saves its full JSON report to .fallow/mutation/last-report.json in the fallow cache directory. The cache directory has its own .gitignore, so the saved report does not make the worktree dirty. Each run replaces the saved report.

For a report file that you keep, use -o:

fallow mutation --format json -o mutation.json

Mutation score in fallow health

fallow health --mutation-report <FILE> reads a saved run report and shows the mutation score of each complexity finding under its CRAP score. It reads files only. It runs no test.

fallow mutation --all
fallow health --mutation-report .fallow/mutation/last-report.json

The mutation score never changes CRAP, the health score, or a gate. A function whose file changed since the run gets no score. See fallow health.

Inspect a mutant

fallow mutation inspect shows one mutant against the current file. It runs no code.

fallow mutation inspect 72487a15b6adc213

Mutant 72487a15b6adc213  fresh
src/pricing.ts:2:15  applyDiscount  relational-operator  (report: survived)
-    2 |   if (percent <= 0) return total;
+    2 |   if (percent < 0) return total;

● Covering tests (1)
  src/pricing.test.ts::discount

Without --report, inspect reads the report that the last run saved. Pass --report <file> to read another saved mutation report or --plan document. Pass the same --root and --config as the run, because the config can move the cache directory. From a subdirectory, inspect finds the report of a run at the project root.

The report holds no source text. Inspect reads the current file and finds the mutant again:

• fresh: the file matches the report. The output shows the original and the mutated line, the covering tests, and the killing test.
• stale: the file changed or is missing. The output shows no code, so a changed file never gives a wrong answer. Run fallow mutation again.

Inspect exits with code 2 when no run saved a report, or when the id is not in the report.

Verdicts for equivalent mutants

Some mutants do not change the behavior of the code. No test can detect them. You can mark them in a verdict document:

{
  "schema_version": "1",
  "verdicts": [
    {
      "review_key": "77ac825e81517e35",
      "verdict": "equivalent",
      "note": "percent is never negative here"
    }
  ]
}

fallow mutation --verdicts verdicts.json

The review_key is the mutant_id. The id stays the same until the enclosing function changes, so one verdict document can serve many runs.

The JSON report lists each group under verdicts, together with a digest of the document. A malformed document, an empty or duplicate review_key, or an unknown field stops the command with exit code 2 before any test runs.

Exit codes

The flag wins over the config key. A run with no score exits with 0, also when every mutant is no_coverage.

fallow mutation --min-score 80

Configuration

Set project defaults in the mutation config section:

{
  "mutation": {
    "runner": "auto",
    "operators": ["arithmetic-operator", "relational-operator", "regex-literal"],
    "minScore": 80,
    "concurrency": 4,
    "typecheck": true
  }
}

A CLI flag wins over the config. No config key starts a run or keeps side-effect mutants.

MCP tools

The MCP server has two read-only tools. Neither tool runs tests.

• mutation_plan returns the plan envelope: each mutant's id, location, operator, disposition, and static test files. It takes changed_since, all, paths, max_mutants, and operators.
• mutation_inspect shows one mutant of a saved report. It takes mutant_id and an optional report. Without report, it reads the report of the last run.

A run needs the CLI command fallow mutation, which executes your tests. An agent should ask you before it starts a run.

Current limits

• The output formats are human and json. There is no SARIF, CodeClimate, LSP, or CI gate output.
• --workspace and --changed-workspaces are not supported. Pass a package directory as a path.
• A run executes the mutants that a verdict marks. Fallow applies the verdicts after the run.
• The in-process mode of node:test (isolation: "none") is not supported.
• With node:test and bun test, fallow can match the code that a test ran to that test only when the tests of a file run in a fixed order. When tests overlap (concurrency in node:test, test.concurrent in Bun) or randomize = true is set in bunfig.toml, each test of the file shares the code that the file ran. A mutant then runs against each test of the file that can kill it.

See also