Worked example

This page builds a small Go package with three guards, writes a spec that removes each guard in turn, finds the one with no test, fixes it, and then breaks the harness on purpose to show what muhar does instead of reporting a clean result. Every output below was captured from a real run. You need muhar on PATH (Install) and a Go toolchain.

The code under test

A withdrawal check with three rules: the amount must be positive, and the day's total must not exceed the limit — where landing exactly on the limit is allowed.

mkdir limits && cd limits
go mod init example.com/limits

limits.go:

package limits

import "errors"

var (
	ErrNonPositive = errors.New("amount must be positive")
	ErrOverLimit   = errors.New("amount exceeds the daily limit")
)

// Check reports whether amount may be withdrawn, given what has
// already been withdrawn today and the daily limit.
func Check(amount, spentToday, limit int64) error {
	if amount <= 0 {
		return ErrNonPositive
	}
	if spentToday+amount > limit {
		return ErrOverLimit
	}
	return nil
}

limits_test.go:

package limits

import (
	"errors"
	"testing"
)

func TestRejectsNonPositive(t *testing.T) {
	for _, amount := range []int64{0, -5} {
		if err := Check(amount, 0, 100); !errors.Is(err, ErrNonPositive) {
			t.Errorf("Check(%d, 0, 100) = %v, want ErrNonPositive", amount, err)
		}
	}
}

func TestRejectsOverLimit(t *testing.T) {
	if err := Check(20, 90, 100); !errors.Is(err, ErrOverLimit) {
		t.Errorf("Check(20, 90, 100) = %v, want ErrOverLimit", err)
	}
}

func TestAllowsWithinLimit(t *testing.T) {
	if err := Check(10, 50, 100); err != nil {
		t.Errorf("Check(10, 50, 100) = %v, want nil", err)
	}
}

go test -cover ./... passes and reports coverage: 100.0% of statements. Neither fact says whether every rule has a test.

The spec

mutations.json names each guard and the edit that removes it. The control is a fault the suite must catch: if the over-limit refusal is replaced by return nil, TestRejectsOverLimit has to fail.

{
  "test": "go test ./... -count=1 -timeout 60s",
  "control": {
    "name": "CONTROL: never refuse an over-limit withdrawal",
    "file": "limits.go",
    "old": "\t\treturn ErrOverLimit",
    "new": "\t\treturn nil"
  },
  "mutations": [
    {
      "name": "zero is accepted as an amount",
      "file": "limits.go",
      "old": "if amount <= 0 {",
      "new": "if amount < 0 {"
    },
    {
      "name": "the limit ignores what was already spent today",
      "file": "limits.go",
      "old": "if spentToday+amount > limit {",
      "new": "if amount > limit {"
    },
    {
      "name": "a withdrawal that lands exactly on the limit is refused",
      "file": "limits.go",
      "old": "spentToday+amount > limit",
      "new": "spentToday+amount >= limit"
    }
  ]
}

Three details worth noticing. -count=1 stops go test from serving a cached pass. The control's old starts with two tab characters, written \t\t in JSON, because the edit is matched byte for byte. The file paths are relative and there is no root, so they resolve against the directory you run muhar from. The spec reference covers each rule.

Run it

Harness healthy: baseline GREEN, positive control CAUGHT.
  CAUGHT         zero is accepted as an amount
  CAUGHT         the limit ignores what was already spent today
  NOT_CAUGHT       a withdrawal that lands exactly on the limit is refused
Totals: 2 caught, 1 NOT CAUGHT, 0 could-not-mutate.

The report goes to stdout. The suite's own output goes to stderr, one block per suite run, in order: the baseline, the control, then each mutation.

ok  	example.com/limits	0.074s
--- FAIL: TestRejectsOverLimit (0.00s)
    limits_test.go:18: Check(20, 90, 100) = <nil>, want ErrOverLimit
FAIL
FAIL	example.com/limits	0.061s
FAIL
--- FAIL: TestRejectsNonPositive (0.00s)
    limits_test.go:11: Check(0, 0, 100) = <nil>, want ErrNonPositive
FAIL
FAIL	example.com/limits	0.068s
FAIL
--- FAIL: TestRejectsOverLimit (0.00s)
    limits_test.go:18: Check(20, 90, 100) = <nil>, want ErrOverLimit
FAIL
FAIL	example.com/limits	0.067s
FAIL
ok  	example.com/limits	0.046s

The last suite run passed with > turned into >=: no test checks that a withdrawal landing exactly on the limit is allowed. Exit status 0 means the harness was healthy, not that every guard was caught. A gate on the result reads the Totals: line; see CI integration.

Fix the gap

Add the missing boundary test to limits_test.go:

func TestAllowsExactlyTheLimit(t *testing.T) {
	if err := Check(10, 90, 100); err != nil {
		t.Errorf("Check(10, 90, 100) = %v, want nil", err)
	}
}

Harness healthy: baseline GREEN, positive control CAUGHT.
  CAUGHT         zero is accepted as an amount
  CAUGHT         the limit ignores what was already spent today
  CAUGHT         a withdrawal that lands exactly on the limit is refused
Totals: 3 caught, 0 NOT CAUGHT, 0 could-not-mutate.

That Totals: line, with 0 NOT CAUGHT, 0 could-not-mutate, is the clean result.

A mutation that does not land

Specs drift. Add two broken entries to the end of mutations — one whose old text is not in the file, one whose file does not exist:

{
  "name": "stale anchor",
  "file": "limits.go",
  "old": "if amount <= 0 {\n\t\treturn ErrNegative",
  "new": "x"
},
{
  "name": "missing file",
  "file": "limit.go",
  "old": "a",
  "new": "b"
}

Run against the original test file (without the boundary test from step 4):

Harness healthy: baseline GREEN, positive control CAUGHT.
  CAUGHT         zero is accepted as an amount
  CAUGHT         the limit ignores what was already spent today
  NOT_CAUGHT       a withdrawal that lands exactly on the limit is refused
  COULD_NOT_MUTATE stale anchor  (mutation "stale anchor": Old text not found in limits.go — edit did not land)
  COULD_NOT_MUTATE missing file  (mutation "missing file": read limit.go: open limit.go: no such file or directory)
Totals: 2 caught, 1 NOT CAUGHT, 2 could-not-mutate.
NOTE: could-not-mutate is NOT a clean gate — those guards were never actually tested.

Neither broken entry ran the suite, so neither can be mistaken for a guard with no test, and the run still exits 0: the harness itself is fine. The NOTE: line and the non-zero could-not-mutate count are what a gate must refuse.

A harness muhar will not trust

Three ways to break the harness, and the run each one produces. Each is exit status 2, with no per-guard verdicts and no Totals: line.

The control's old text is wrong, so the control cannot be planted:

HARNESS BROKEN — run discarded.
  positive control could not be planted (mutation "CONTROL: never refuse an over-limit withdrawal": Old text not found in limits.go — edit did not land) — the harness is not mutating; discard the run

The control is an edit no test can see — renaming the text of an error that the tests compare by identity with errors.Is:

"control": {
  "name": "CONTROL: rename an error message",
  "file": "limits.go",
  "old": "amount must be positive",
  "new": "amount must be > 0"
}

HARNESS BROKEN — run discarded.
  positive control was NOT CAUGHT — a mutation the suite must catch stayed green, so the harness is broken (not mutating, or the suite cannot see the source); discard the run

The suite fails before anything is mutated — here a test command that ends in && false:

HARNESS BROKEN — run discarded.
  baseline suite is RED on pristine source — cannot distinguish a caught mutation from a pre-existing failure; discard the run

Faster runs

-j 2 runs two mutations at a time, each in its own copy of the tree. The report is identical, line for line; muhar names the copied root on stderr.

muhar: 2 mutations in flight, one isolated copy of /home/dev/limits per worker

Harness healthy: baseline GREEN, positive control CAUGHT.
  CAUGHT         zero is accepted as an amount
  CAUGHT         the limit ignores what was already spent today
  NOT_CAUGHT       a withdrawal that lands exactly on the limit is refused
Totals: 2 caught, 1 NOT CAUGHT, 0 could-not-mutate.

-shard i/n splits the mutations across separate invocations, for CI legs on separate machines. Each shard reruns the baseline and the control, and reports only its own mutations:

muhar: shard 0/2 — running 2 of 3 mutations (indices ≡ 0 mod 2); the other shards own the rest

Harness healthy: baseline GREEN, positive control CAUGHT.
  CAUGHT         zero is accepted as an amount
  NOT_CAUGHT       a withdrawal that lands exactly on the limit is refused
Totals: 1 caught, 1 NOT CAUGHT, 0 could-not-mutate.

muhar: shard 1/2 — running 1 of 3 mutations (indices ≡ 1 mod 2); the other shards own the rest

Harness healthy: baseline GREEN, positive control CAUGHT.
  CAUGHT         the limit ignores what was already spent today
Totals: 1 caught, 0 NOT CAUGHT, 0 could-not-mutate.

Next: the concepts behind each line, or straight to CI integration.